Experimental Design

Experimental Design has been an event in Science Olympiad in both divisions for many years. In this event, you will be given several materials and asked to perform an experiment on a certain science topic. You will then be required to write-up the experiment in the "lab write-up," which will be used to score you.

Statement of Problem
This is a question posed that will be explored in the experiment. It should not be a simple yes or no question. Try to use the key words, "how" or "why". Remember "KISS", or "Keep It Simple Stupid!" -and always remember to be neat! One of the formats that can be used for almost any experiment is "How does (Independant Variable) affect (Dependant Variable)?"

Hypothesis
The hypothesis a guess as to the result of your experiment which relates to the statement of the problem. It should express how the changes to the independent variable will affect the dependent variable. A rationale for the hypothesis should be included. Make sure you have the correct DV (dependent variable) and IV (independent variable). If you have misidentified these two, it's likely you won't get very much else right. "We predict" is a good starter.

Variables
There are three different kinds of variables you should define in your lab writeup. The three different kinds of variables could be, for example, in a lab which uses different masses, a nine gram weight, a twelve gram weight and a fifteen gram weight.

Controlled Variables (CV)
Controlled variables are factors which could effect the dependent variable, but are kept constant throughout the experiment. Several controlled variables should be listed (usually four is a good amount). For example, if the experiment is testing to see how fast a parachute falls with different mass, a constant variable could be "Height at which parachute is dropped (in meters)".

Independent Variable (IV)
This is the variable that is changed to examine its effect on the dependent variable. There should only be one IV, which should be listed with units.

Dependent Variable (DV)
The dependent variable is what is affected by the independent variable. It should be defined in units. Using the same previously given experiment, this would be "Time it takes for parachute to fall (in seconds)."

Control or "Standard of Comparison"
The control, or standard of comparison, should show that the Independent Variable is the variable causing the action or effect in the Dependent Variable. A rationale for the control should be included. The SOC is basically the object that is the "normal" one, the one that hasn't been changed at all. For example, if you were doing an experiment on the time a can spins vs. how many holes it has in it, the SOC would be the can with no holes in it. Changing your DV to zero, or using the highest or lowest possible numeric value makes a good SOC.

Procedure
List the steps in your experiment clearly. Be sure to include diagrams of how your experiment was performed. To cut time, "Repeat steps X to Y" can be used, but make sure it makes sense. One way to try to check your procedure is to imagine if you have not done the experiment; would you be able to run it based only on the procedure? Make sure you have a few trials (at least three) for each step. Without this, a single data point may be an outlier- or it may be a real data point and you would never know. Remember to be specific all the time. The first thing you should put on every procedure list is "Put on Saftey Goggles" or "Gather Materials" The last thing on every procedure list should be "Clean up your workspace"- and be sure to do so!

Observations
List a few quantitative observations and a few qualitative observations. Quantitative observations are those described with numbers- i.e. the mass fell 60 cm. Qualitative observations are those that describe an observation of non-numerical data- i.e. the parachute seemed to open up more with a heavier mass. List anything that you see that might have been a problem ("Parachute did not open on trial three"). If you can't be specific, don't directly tie it into your data, meaning, if you were using a orange you could say "The round orange colored ball rolled the farthest in the fastest amount of time."

Statistics
Take the common statistics - mean, mode, range, median. Explain this data and give an example of a calculation. Also include any other relevant statistics and show work. The best idea is to put all statistics in a neat table.

Once your common statistics are done, make sure to do some more. Standard deviation is a very good statistic to include. The equation for calculating standard deviation is: $$\sigma = \sqrt{avg((value - mean)^{2})}$$. For a better visual equation and an explanation of what standard deviation is (which you will need to know to explain the statistic), see Standard Deviation. Actually doing trials is necessary, as a standard deviation of a sample size of 1 is clearly stupid. A key point that is easy to miss is the deviation has to be squared. If you don't, your result will always be 0, and though this may look pretty, it should be obvious that your data does not have a standard deviation of 0.

Your table of data should be neat- a ruler helps a lot. Be sure to keep writing your units. Also be sure to keep your significant figures consistent and logical. You do not want to have a number down to three decimals when your ruler can only accurately measure to one decimal.

Graph your data. Make the graph neat, legible. Use a legend if need be. Label the axes (with units) and make a title for the graph (including units here as well is a good idea; for example, "Time in takes a parachute to fall, in seconds, vs the weight, in grams".

In Division C, you will be expected to do more with the data. One essential aspect of the graph will be to create a regression, or line of best fit. Basically, since you're not really expected to have all the tools necessary, you won't be expected to make anything but a linear regression. If you can have your calculator create an accurate logarithmic regression. more power to you, but make sure to allot the appropriate amount of time for this section. The best fit line of dubious accuracy is made by drawing a straight line with a ruler that you think seems to go as close to all the points on the graph. Once you do, find the y-intercept, and calculate the slope. Make sure to consider which outliers are significant, and which are experimental errors. If you make these kinds of judgment calls, make sure to point it out and explain it in the Analysis section.

Also, make sure you always have the same units through-out the experiment, if you are using milliseconds in the data table continue using milliseconds for everything else, DO NOT change to seconds or any other units.

Analysis
Look at the data and draw some reasonable conclusions about the experiment. There should be trends; point them out and explain them. Discuss your statistics and again, explain them. Guesses are okay, even if they're wrong; they show your thought process. If you have any outliers or random "bad" data points, don't ignore them - again, write about them. Was there anything you did wrong that time, or was it just a fluke?

Possible Sources of Experimental Error
Look for all the things wrong with your experimental setup. How might they have caused inaccuracies in your data? This is extremely useful, because it can redeem mistakes made earlier in the event by showing that you are aware of them. Sometimes points can even be regained. Human error is a big factor and one you want to write about; many rubrics for the event look for a mention of the specific role of human error in your results. For example, if your experiment involved timing or measuring something, there is always the possibility that the person timing or measuring may not have been consistent in their measurement. In addition, you can say things that have to do with temperature. The container of your object may have insulated it. Say any possible thing that could change the outcome of the experiment.

Also, this section can be written before any data is actually collected, and just added to if there are glaring errors in data collection that you didn't predict. If you're not sure how the experiment is going to turn out, this is a good thing to write first.

Conclusion
Answer your Statement of the Problem. Evaluate your hypothesis. Was it correct? How would you improve the experiment's procedure, if you were given another chance? Would you run a completely different experiment, with a more accurate or visible trend? What problems did you have in the experiment, and how would you fix them?

Applications and Recommendations for Further Experimentation
Consider your experiment and how it may apply to a real-life situation. For instance, the aforementioned parachute experiment could help skydivers know when to unleash their parachute. Then, think of related experiments that could shed more light on the same or a similar topic. How could your results be practically applied? How could the results of another experiment combined with the results of your experiment help? Where is it useful in real life? This section can also be written without any knowledge of how the experiment will turn out, and so it can also be written before data has been collected.

Common Strategies
Know who is doing what before you walk in. To do well, each person should be ready to do their own section of the lab. At the start of the event, discuss possible experiments, but do not waste too much time here. You only have 50 minutes to design, run, and write a lab. Optimally, the task is clear, but don't be surprised if the event supervisors prepare a random set of materials and an unexpected prompt. Focus should be on execution and writeup, not on preparation.

Keep your experiment simple. Too many variables can mean a lot of writing. Consider an example experience from one regional tournament. They gave us three balls (different colors), two rubber bands, a foot of masking tape, a metric stick and a mini catapult. Naturally, you want to do and experiment with the catapult, but there would be lots of variables to consider. So instead, you could perform a dropping experiment on how a rubber band affects the time it takes for a ball to drop. It's simple and in this situation it paid off. For the teams that used the catapult, during the event balls were flying everywhere, and people were running and looking for them. If you lost one, well, too bad. But for the team that used the other experiment it worked out great (they got third out of 30). So, in short, keep it simple- it will make you a lot more likely to win. You just have to ignore your urge to mess around with the catapult.

However, make sure to have enough trials so that your statistics are meaningful. If you have, say, only 2 trials for 4 values of a variable, your mean/median/mode were pretty pointless. It's more useful to have fewer values of your variable — 3 or 4 aren't bad, depending on how much effort per trial your procedure requires- and more trials of each one, since then there's more to talk about with the statistics. Plus, it's much easier to do the same thing a bunch of times and change your setup fewer times than it is to change the setup a bunch of times and just test each one a couple times. It also makes your data more accurate to have a large number of trials for each value.

During the event, know your jobs. Preferably, for running the experiment, there should only be one or two people, and the other(s) can keep up and write the procedure and observations as the experiment is being run. Once the experiment finishes, split up the remaining portions so as to be able to finish within the time limit. A good way to split the work is to have one person start writing the conclusion, which requires no data to write, while another does the statistics. As the statistics and trends become more clear, the third person and the statistics person should work together to write a good analysis of the data.

The above is just an example of a way to split the lab to fit it in the time limit. There are, of course, other ways. However, the point is that knowing your place and your jobs will make the event a whole lot smoother.

To practice, have somebody on your team, be it a coach or another team member, put together a bunch of materials and try to run the event yourself. Do not handicap yourself by always running the same experiment at every practice. Also, try practicing with both very specific and very vague topics, because the topic can range anywhere from "Physics"— extremely vague— to "Do some sort of strength test on a bridge that can span this gap"— about as specific as it can be without them giving you the experiment.

How to do your best
In Experimental Design, you should always keep your experiment reasonable. Here are some tips for how to do your best:
 * 1) Come prepared. You should come to the competition equipped with several different writing utensils, a ruler, a stopwatch and a non-programmable calculator.
 * 2) Study the rubric. You might be able to use it, but just to be safe always, look over the rubric before the competition.
 * 3) Be neat. If the judges can't read your experiment, they are not going to take it. Find someone in your three people group who has neat handwriting.
 * 4) Think outside of the box. Don't do what everybody else will. The judges need to see that you are uniquely intelligent.
 * 5) Be efficient. Sometimes, "slow and steady wins the race" doesn't always apply.
 * 6) Be precise, especially when labeling your list of materials. You can never be too specific in this event!
 * 7) KISS! Keep it simple, stupid!

Practice trials

 * Experimental Design/Practice
 * Test Exchange
 * Sample Experiment from Minnesota Div. C (Note: PDF)
 * Sample Experiment from Minnesota Div. B (Note: PDF)

Links
Experimental Design National Page (Div. C)