Experimental Design

Experimental Design has been an event in Science Olympiad in both divisions for many years. In this event, you will be given several materials and asked to perform an experiment on a certain scientific topic. You will then be required to write-up the experiment in the "lab write-up," which will be used to score you.

Statement of Problem
This is a question posed that will be explored in the experiment. It should not be a simple yes or no question. Try to use the key words, "how" or "why". Remember "KISS", or "Keep It Simple, Stupid!" -and always remember to be neat! One of the formats that can be used for almost any experiment is "How does (Independent Variable) affect (Dependent Variable)?" Ex. How does the height a ball is dropped from (1, 2, 3 meters) affect its rebound height (cm)?

Hypothesis
The most common way a hypothesis is used in scientific research is as a tentative, testable, and falsifiable statement that explains some observed phenomenon in nature. We more specifically call this kind of statement an explanatory hypothesis. However, a hypothesis can also be a statement that describes an observed pattern in nature. In this case we call the statement a generalizing hypothesis. The hypothesis statement can be followed by the specific, measurable prediction you can make if the hypothesis is valid. Thus, we can think of the hypothesis in science as an explanation or generalization on trial.

A prediction in science is a prophecy, a specific and measurable event that is likely to happen in the future as the result of an experiment if the hypothesis is valid.

Teaching the Hypothesis Incorrectly

Many teachers and even many textbooks teach the hypothesis in a way that makes it no different from a prediction. They teach students to write “If – then” statements for their hypotheses. This approach results in the incorrect form: If I do X, then Y will happen. There is no hypothesis here. This is simply a method (if I do X) followed by a prediction (then Y will happen). Some teachers and textbooks add “…because…” at the end of the “If…, then…” statement. The because statement is often close to the hypothesis that is being tested, but it still does not carefully delineate the hypothesis from the prediction. Indeed, even professional scientists can make mistakes.

Variables
There are three different kinds of variables you should define in your lab writeup: the independent variable, the dependent variable, and controlled variables.

Controlled Variables (CV)
Controlled variables are factors which could affect the dependent variable but are kept constant throughout the experiment. Several controlled variables should be listed (usually four is a good amount). For example, if the experiment is testing to see how fast a parachute falls with different mass, a constant variable could be "Height at which parachute is dropped (in meters)". Only four need to be listed to receive full credit for this section.

Independent Variable (IV)
This is the variable that is changed to examine its effect on the dependent variable. There should only be one IV, which should be listed with units. The IV must be operationally defined (in terms of the experiment) and empirically defined (in general for future variations of the experiment), and a minimum of 3 different levels must be listed excluding the control level.

Dependent Variable (DV)
The dependent variable is what is affected by the independent variable. It should be defined in units. Using the same previously given experiment, this would be "Time it takes for parachute to fall (in seconds)." The dependent variable must be operationally and empirically defined, and levels are unnecessary because that is what will be determined through the experiment.

Control or "Standard of Comparison"
A rationale for the control should be included. The SOC is basically the object that is the "normal" one, the one that hasn't been changed at all. For example, if you were doing an experiment on how long a can spins v.s. how many holes it has in it, the SOC would be the can with no holes in it. Changing your IV to zero, or using the highest or lowest possible numeric value makes a good SOC.

Materials
When you list out your materials, be sure to be extremely specific. List the quantity of the materials to be used. It is a good idea to write down brand names or companies next to the material. After all, different companies make different versions of the same thing. When you list your independent variable (say you are testing using rocks with three different weights), write "Rocks (light, medium, heavy). If you aren't sure whether or not to list the materials you used to measure (meter sticks, time pieces, etc.), ask the event supervisor. Some competitions want you to list the measuring devices, while others may take off points for it. Do not list the materials on the front cover of the test; it is unlikely that you will use all of them, and you will get points off for listing unused materials.

Procedure
List the steps in your experiment clearly. Be sure to include labeled diagrams of how your experiment was performed. Three diagrams is typically sufficient, but more may be needed if an experimental apparatus needs to be constructed. To cut time, "Repeat steps X to Y" can be used, but make sure it makes sense. You can check yourself by thinking that if you could show any random person the procedure, would they be able to follow it clearly? Always have three trials for each step. Without this, a single data point may be an outlier—or it may be a real data point and you would never know. Remember to be specific all the time. The last thing on every procedure list should be "Clean up your workspace"- and be sure to do so!

Qualitative Observations
There are three types of observations that must be made to receive full credit: observations about the procedure, results, and anything not related to the DV. Additionally, observations must be made over the course of the experiment about the three.

Typically, observations about the procedure are about noticing flaws in the experiment that went unnoticed before it was performed. These can include flaws in measurement technique, flaws in building/maintaining an experimental unit, and flaws in performing the actual experiment. Observations about the procedure carry over to experimental errors and practical applications, so be sure that you thoroughly explain what went wrong. An example of this type of observation is, "After trials 1 and 2, the penny which was dropped in the oil was not completely cleaned off, and a thin residue was left on the surface for the remainder of the experiment."

Quantitative Data
One way to organize the data is to make two tables. For the first, make a table of four rows and four columns. The first column should consist of, from top to bottom, a blank box, IV 1, IV 2, and IV 3. The second column should be labeled "Trial 1", and following boxes filled accordingly to the data. The next two columns follow the same layout as the Trial 1 box, but with Trials 2 and 3. Title the graph as seen fit for the data. Next to that table, draw a one column, four row condensed table (to the right). Name it "Average" (or AVG for short), and average the data for each IV. Put arrows from the second row of the first table to the second row of the condensed table, and so forth. Give a sample calculation for the average ((Trial 1+Trial 2+Trial 3)/3), located below the table or on one of the arrows. Remember to title both of your tables.

Also be sure use significant figures if you are in C Division, and be sure to keep them consistent and logical. You do not want to have a number down to three decimals when your ruler can only accurately measure to one decimal. See Significant Figures for additional info about significant figures.

Graph
A standard bar, line, or scatter-plot graph works almost universally at any competition level. Even so, always be sure to use the correct type for your data. Always remember to:
 * 1) Label your axis (x+y)
 * 2) Title the graph
 * 3) Use the DV as the y value and IV as the x value
 * 4) Title the individual axis (For example, left of the y axis you would write, "Time taken for parachute to fall (seconds)")
 * 5) Connect the data points or draw a line of best fit
 * 6) Only include the averages of the data for each IV

Statistics
Take the common statistics - mean, mode, range, median. Also include any other relevant statistics and show work. The best idea is to put all statistics in a neat table.

Your table of data should be neat- a ruler helps a lot. Be sure to keep writing your units.

Once your common statistics are done, make sure to do some more. Standard deviation is a very good statistic to include. The equation for calculating standard deviation is: $$\sigma = \sqrt{avg((value - mean)^{2})}$$. For a better visual equation and an explanation of what standard deviation is (which you will need to know to explain the statistic), see Standard Deviation. Actually doing trials is necessary, as a standard deviation of a sample size of 1 is clearly stupid. A key point that is easy to miss is the deviation has to be squared. If you don't, your result will always be 0, and though this may look pretty, it should be obvious that your data does not have a standard deviation of 0.

Graph your data. Make the graph neat, legible. Use a legend if need be. Label the axes (with units) and make a title for the graph (including units here as well is a good idea; for example, "Time in takes a parachute to fall, in seconds, vs the weight, in grams".)

As of 2015, both Division B and Division C will be expected to do more with the data, whereas before only C Division needed to. One essential aspect of the graph will be to create a regression, or line of best fit. Since both divisions are now permitted to bring any type of calculator, this would be a good time to invest in a nice TI-84 or similar graphing calculator because linear, logarithmic, and many other types of regression can be calculated with them. To calculate a linear regression on a TI calculator, start by putting your data in a list. Press STAT and go to the EDIT menu. Press 1 to edit your list. Then, exit and press STAT again. Go to the CALC menu to select the type of regression most suited for your data (which in most cases will be LinReg). Place the list containing your X values in Xlist, and repeat for Y values in Ylist. Scroll down and press Calculate, which will then give you the constants for your regression. Draw in the line on your graph and label it.

If you cannot get a graphing calculator, the best fit line of dubious accuracy is made by drawing a straight line with a ruler that you think seems to go as close to all the points on the graph. Once you do, find the y-intercept, and calculate the slope. Make sure to consider which outliers are significant, and which are experimental errors. If you make these kinds of judgment calls, make sure to point it out and explain it in the Analysis section.

Also, make sure you always have the same units through-out the experiment, if you are using milliseconds in the data table continue using milliseconds for everything else, DO NOT change to seconds or any other units. C Division competitors should remember to use significant figures in their statistics.

Analysis
This section should be one extended paragraph which at least touches on all data points and expands on outliers. Look at the data and draw some reasonable conclusions about the experiment. There should be trends; point them out and explain them. Discuss your statistics and again, explain them. Guesses are okay, even if they're wrong; they show your thought process. If you have any outliers or random "bad" data points, don't ignore them - again, write about them. Was there anything you did wrong that time, or was it just a fluke? Conclude by stating if the IV is directly, inversely, or not clearly proportional to the DV.

Possible Sources of Experimental Error
Look for all the things wrong with your experimental setup. How might they have caused inaccuracies in your data? This is extremely useful, because it can redeem mistakes made earlier in the event by showing that you are aware of them. Sometimes points can even be regained. Try to stay away from human errors and try to focus on experimental sources of errors like you can say things that have to do with temperature. The container of your object may have insulated it. Say any possible thing that could change the outcome of the experiment. Then, explain how the errors are believed to affect the data: increase from normal, decrease from normal, or either.

Also, this section can be written before any data is actually collected, and just added to if there are glaring errors in data collection that you didn't predict. If you're not sure how the experiment is going to turn out, this is a good thing to write first, though any errors that are included in your qualitative observations should be commented on.

Conclusion
DO NOT ever say that your hypothesis was right or wrong. After one experiment, a hypothesis can either be supported or not supported by the data. Restate your hypothesis minus the explanation before concluding in either way. Then, explain why you came to that conclusion with your data as support. Never extrapolate anything; stick to what you observed even if you think the results were wrong. You may attempt to explain why your data varied from your hypothesis using proper scientific terminology that was not considered while writing your hypothesis, but the vast majority of the explanation should be data-based.

Applications and Recommendations for Further Experimentation
When writing this section, consider variations of your experiment that would produce more accurate results. There should be three variations listed: one to improve a certain aspect of your experiment, one to approach the hypothesis in a different way, and one for a future experiment related to the DV. Finally, consider a practical application for the experiment. This section can also be written without any knowledge of how the experiment will turn out, and so it can also be written before data has been collected.

In an experiment on the effect of applied weight on a parachute's descent time, a way to improve the specific experiment would be to use more durable string to avoid breaking the parachute. A way to approach the hypothesis from a different perspective is to use a computer simulation to determine the outcome of the descent. Future experiments could test other IVs, such as air resistance and varying gravity levels on other planets. Practical applications include use in defense and for returning space vehicles safely to Earth.

Common Strategies
Knowing the scoring rubric is the key to success in Experimental Design. The rubricis a set of guidelines used for scoring experiments. When experimenters are aware of what is expected in each section, it becomes much easier to work efficiently.

At the start of the event, start by brainstorming possible experiments. Expect to get a handful of seemingly random items to test with, along with the possibility of a topic or prompt to design the experiment around. If each team member is familiar with general scientific concepts, designing an experiment should not be too difficult. Focus on execution and write-up, not on preparation. Spend no more than 5 minutes on this.

Keep your experiment simple. Too many variables can mean a lot of writing. Consider an example experience from one regional tournament. 3 balls (different colors), 2 rubber bands, a foot of masking tape, a metric stick, and a mini catapult were given. Naturally, one would want to experiment with the fanciest item given (catapult, in this case), but there would be so many variables to consider. Instead, performing a dropping experiment on how a rubber band affects the time it takes for a ball to drop. This is much simpler and, in this scenario, an idea that definitely paid off. Teams that used the catapult had balls flying everywhere throughout the event, and team members had to run around searching for them; wasting time. On the other hand, teams that utilized the other equipment achieved third out of thirty teams. Moral of the story is, ignore the urge to fiddle around with the complex stuff. Keep it simple- experiments will be simpler to write and test, saving time.

However, be sure to have enough trials. Having 3-5 trials for each variable ensures that data is sound and statistics have merit. This way if there is a possibility of strange data (one test being too high/low/fast/slow) there is the "experimental errors" section to comment on that. Only having 1-3 trials means there may not be enough data to show that a data point is strange, because there are not enough points to compare it to.

Know who is doing each section before the competition. All 3 people don't need to be doing the lab; only 1 or 2 people should be experimenting. Don't spend the whole time doing the lab either, 15-20 minutes should be plenty to get a significant amount of data for an experiment. There are many ways to divvy up the work on this event. Find something that works, each group is different.

If the experiment goes horribly wrong and all the data is skewed, focus on the report as much as possible. Make sure to explain why the experiment was bad and where it went wrong. This is where "Possible Experimental Errors" really counts-- be sure to write and explain every error which caused the experiment to go wrong. Having a bad experiment but a very good report  can, in some cases, cancel out the fact that the experiment didn't work.

Just like any other event, practice! Have a fellow teammate or coach gather materials and come up with a possible topic. Spend 50 minutes and come up with, test, and write up a lab report. Have the coach/teammate then grade the lab based on the rubric. This gives great insight into time usage and where improvements need to be made.

How to do your best
In Experimental Design, you should always keep your experiment reasonable. Here are some tips for how to do your best:
 * 1) Come prepared. You should come to the competition equipped with several different writing utensils, a ruler, a stopwatch and any type of calculator as long as it cannot access the Internet and does not have a camera.
 * 2) Study the rubric. You might be able to use it, but just to be safe always look over the rubric before the competition.
 * 3) Be neat. If the judges can't read your experiment, they are not going to take it. You will not have time for one person to write up everything well enough, so try to be neat.
 * 4) Think outside of the box. Don't do what everybody else will. The judges need to see that you are uniquely intelligent.
 * 5) Be efficient. Sometimes speed counts and is important doesn't always apply.
 * 6) Be precise, especially when labeling your list of materials. You can never be too specific in this event!
 * 7) KISS! Keep it simple, stupid!

Practice trials

 * Experimental Design/Practice
 * Test Exchange
 * Sample Experiment from Minnesota Div. C (Note: PDF)
 * Sample Experiment from Minnesota Div. B (Note: PDF)

Links
Experimental Design National Page (Div. C)

The rubric for Experimental Design The rubric rarely varies from year to year but has changed for the 2015 season.