Experimental Design

Experimental Design has been an event in Science Olympiad in both divisions for many years. In this event, you will be given several materials and asked to perform an experiment on a certain science topic. You will then be required to write-up the experiment in the "lab write-up," which will be used to score you.

Statement of Problem
This is a question posed that will be explored in the experiment. It should not be a simple yes or no question. Try to use the key words, "how" or "why". Remember "KISS", or "Keep It Simple, Stupid!" -and always remember to be neat! One of the formats that can be used for almost any experiment is "How does (Independent Variable) affect (Dependent Variable)?"

Hypothesis
The hypothesis is a guess as to the result of your experiment, relating to the statement of the problem. It should express how the changes to the independent variable will affect the dependent variable. A rationale for the hypothesis should be included. Make sure you have the correct DV (dependent variable) and IV (independent variable). If you have misidentified these two, it's likely you won't get very much else right. "We predict" is a good starter, but tends not to yield full points. It is best to not include this, but instead act as though you know it is completely true.

Variables
There are three different kinds of variables you should define in your lab writeup. The three different kinds of variables could be, for example, in a lab which uses different masses, a nine gram weight, a twelve gram weight, and a fifteen gram weight.

Controlled Variables (CV)
Controlled variables are factors which could effect the dependent variable, but are kept constant throughout the experiment. Several controlled variables should be listed (usually four is a good amount). For example, if the experiment is testing to see how fast a parachute falls with different mass, a constant variable could be "Height at which parachute is dropped (in meters)".

Independent Variable (IV)
This is the variable that is changed to examine its effect on the dependent variable. There should only be one IV, which should be listed with units.

Dependent Variable (DV)
The dependent variable is what is affected by the independent variable. It should be defined in units. Using the same previously given experiment, this would be "Time it takes for parachute to fall (in seconds)."

Control or "Standard of Comparison"
A rationale for the control should be included. The SOC is basically the object that is the "normal" one, the one that hasn't been changed at all. For example, if you were doing an experiment on how long a can spins v.s. how many holes it has in it, the SOC would be the can with no holes in it. Changing your DV to zero, or using the highest or lowest possible numeric value makes a good SOC.

Materials
When you list out your materials, be sure to be extremely specific. It is a good idea to write down brand names or companies next to the material. After all, different companies make different versions of the same thing. When you list your independent variable (say you are testing using three different weight rocks), write "Rocks (light, medium, heavy). If you aren't sure whether or not to list the materials you used to measure (meter sticks, time pieces, etc.), ask the event supervisor. Some competitions want you to list the measuring devices, while others may take off points for it.

Procedure
List the steps in your experiment clearly. Be sure to include diagrams of how your experiment was performed. To cut time, "Repeat steps X to Y" can be used, but make sure it makes sense. You can check yourself by imagining if you could show any random person the procedure, would they be able to follow it clearly? Always have three trials for each step. Without this, a single data point may be an outlier- or it may be a real data point and you would never know. Remember to be specific all the time. The last thing on every procedure list should be "Clean up your workspace"- and be sure to do so!

Qualitative Observations
List four qualitative observations. Qualitative observations are those that describe an observation of non-numerical data- i.e. the parachute seemed to open up more with a heavier mass. List anything that you see that might have been a problem in the procedures as well ("Parachute did not open on trial three"). Do not list anything that has to do with the dependent variable. Saying "The parachute with the heaviest weight fell the fastest" would be in violation of that rule, thus not earning points.

Quantitative Data
One way to organize the data is to make two tables. For the first, make a table of four rows and four columns. The first column should consist of, from top to bottom, a blank box, IV 1, IV 2, and IV 3. The second column should be labeled "Trial 1", and following boxes filled accordingly to the data. The next two columns follow the same layout as the Trial 1 box, but with Trials 2 and 3. Title the graph as seen fit for the data. Next to that table, draw a one column, four row condensed table (to the right). Name it "Average" (or AVG for short), and average the data for each IV. Put arrows from the second row of the first table to the second row of the condensed table, and so forth. Give a sample calculation for the average ((Trial 1+Trial 2+Trial 3)/3), located below the table or on one of the arrows.

Graph
A standard bar, line, or scatter-plot graph works almost universally at any competition level. Even so, always be sure to use the correct type for your data. Always remember to:
 * 1) Label your axis (x+y)
 * 2) Title the graph
 * 3) Use the DV as the y value and IV as the x value
 * 4) Title the individual axis (For example, left of the y axis you would write, "Time taken for parachute to fall (seconds)")
 * 5) Connect the data points or draw a line of best fit
 * 6) Only include the averages of the data for each IV

Statistics
Take the common statistics - mean, mode, range, median. Also include any other relevant statistics and show work. The best idea is to put all statistics in a neat table.

Once your common statistics are done, make sure to do some more. Standard deviation is a very good statistic to include. The equation for calculating standard deviation is: $$\sigma = \sqrt{avg((value - mean)^{2})}$$. For a better visual equation and an explanation of what standard deviation is (which you will need to know to explain the statistic), see Standard Deviation. Actually doing trials is necessary, as a standard deviation of a sample size of 1 is clearly stupid. A key point that is easy to miss is the deviation has to be squared. If you don't, your result will always be 0, and though this may look pretty, it should be obvious that your data does not have a standard deviation of 0.

Your table of data should be neat- a ruler helps a lot. Be sure to keep writing your units. Also be sure to keep your significant figures consistent and logical. You do not want to have a number down to three decimals when your ruler can only accurately measure to one decimal.

Graph your data. Make the graph neat, legible. Use a legend if need be. Label the axes (with units) and make a title for the graph (including units here as well is a good idea; for example, "Time in takes a parachute to fall, in seconds, vs the weight, in grams".

In Division C, you will be expected to do more with the data. One essential aspect of the graph will be to create a regression, or line of best fit. Basically, since you're not really expected to have all the tools necessary, you won't be expected to make anything but a linear regression. If you can have your calculator create an accurate logarithmic regression. more power to you, but make sure to allot the appropriate amount of time for this section. The best fit line of dubious accuracy is made by drawing a straight line with a ruler that you think seems to go as close to all the points on the graph. Once you do, find the y-intercept, and calculate the slope. Make sure to consider which outliers are significant, and which are experimental errors. If you make these kinds of judgment calls, make sure to point it out and explain it in the Analysis section.

Also, make sure you always have the same units through-out the experiment, if you are using milliseconds in the data table continue using milliseconds for everything else, DO NOT change to seconds or any other units.

Analysis
Look at the data and draw some reasonable conclusions about the experiment. There should be trends; point them out and explain them. Discuss your statistics and again, explain them. Guesses are okay, even if they're wrong; they show your thought process. If you have any outliers or random "bad" data points, don't ignore them - again, write about them. Was there anything you did wrong that time, or was it just a fluke?

Possible Sources of Experimental Error
Look for all the things wrong with your experimental setup. How might they have caused inaccuracies in your data? This is extremely useful, because it can redeem mistakes made earlier in the event by showing that you are aware of them. Sometimes points can even be regained. Try to stay away from human errors and try to focus on experimental sources of errors like you can say things that have to do with temperature. The container of your object may have insulated it. Say any possible thing that could change the outcome of the experiment.

Also, this section can be written before any data is actually collected, and just added to if there are glaring errors in data collection that you didn't predict. If you're not sure how the experiment is going to turn out, this is a good thing to write first.

Conclusion
Answer your Statement of the Problem. Evaluate your hypothesis. Was it correct? How would you improve the experiment's procedure, if you were given another chance? Would you run a completely different experiment, with a more accurate or visible trend? What problems did you have in the experiment, and how would you fix them if the experiment was done again?

Applications and Recommendations for Further Experimentation
Consider your experiment and how it may apply to a real-life situation. For instance, the aforementioned parachute experiment could help skydivers know when to unleash their parachute. Then, think of related experiments that could shed more light on the same or a similar topic. How could your results be practically applied? How could the results of another experiment combined with the results of your experiment help? Where is it useful in real life? This section can also be written without any knowledge of how the experiment will turn out, and so it can also be written before data has been collected.

Common Strategies
Knowing the scoring rubric is the key to success in Experimental Design. The rubricis a set of guidelines used for scoring experiments. When experimenters are aware of what is expected in each section, it becomes much easier to work efficiently.

At the start of the event, the team should spend a few minutes to brainstorm possible experiments. Be prepared for the event supervisors to give a random set of materials and an unexpected prompt. If each team member is familiar with general scientific concepts, designing an experiment should not be too difficult. Focus on execution and write-up, not on preparation.

Keep your experiment simple. Too many variables can mean a lot of writing. Consider an example experience from one regional tournament. 3 balls (different colors), 2 rubber bands, a foot of masking tape, a metric stick, and a mini catapult were given. Naturally, one would want to experiment with the fanciest item given (catapult, in this case), but there would be so many variables to consider. Instead, you could perform a dropping experiment on how a rubber band affects the time it takes for a ball to drop. This is much simpler and, in this scenario, an idea that definitely paid off. The teams that used the catapult had balls flying everywhere throughout the event, and their team members had to run around searching for them; thus wasting time. On the other hand, the team that utilized the other equipment achieved third out of thirty teams. Moral of the story is, ignore your urge to fiddle around with the complex stuff. Keep it simple- you are more likely to come up with a solid experiment that is easy to write about and relieving to scorers.

However, be sure that you have enough trials so that your statistics are meaningful. If you have, say, only 2 trials for 4 values of a variable, your mean/median/mode are much less significant. It's more useful to have fewer values of your variable — 3 or 4 aren't bad, depending on how much effort per trial your procedure requires- and more trials of each one, since then there's more to talk about within the statistics section. Plus, it's much easier to do the same thing a numerous times and change your setup just a few times than it is to do vice versa. It also makes your data more accurate to have a large number of trials for each value.

Know which team member is doing what before you walk in. To do well, each person should be ready to do their own section of the lab. Divide and conquer. Preferably, for running the experiment, there should only be one or two people, and the other(s) can record the procedure and observations as the experiment is being run. Once the experiment is completed, split up the remaining portions so that your team will be able to finish within the time limit. For example, a team could have one person start writing the conclusion, which requires no data to write (as long as your experiment is rather simple and has an obvious result), while another does the statistics. As the statistics and trends become more clear, the third person and the statistics person should work together to write a good analysis of the data. Knowing your place and your duties as a group member will make the event go a whole lot smoother.

If you end up with a horribly executed experiment that yields the least optimal results, focus on the report as much as possible. Make sure you have everything you need to explain why your experiment was bad. This is where "Possible Experimental Errors" really counts-- be sure to write and explain every error which caused you to have a bad experiment. Having a bad experiment but a very good report explaining why you failed can, in some cases, cancel out the fact your experiment was a failure.

To practice for competition day, have somebody on your team (be it a coach or another team member) gather together a bunch of random materials and try to run the event yourself. Do not handicap your group by always running the same experiment at every practice. Also, try practicing with both very specific and very vague topics, because the topic can range anywhere from "Physics"— extremely vague— to "Do some sort of strength test on a bridge that can span this gap"— about as specific as it can be without them giving you the experiment.

How to do your best
In Experimental Design, you should always keep your experiment reasonable. Here are some tips for how to do your best:
 * 1) Come prepared. You should come to the competition equipped with several different writing utensils, a ruler, a stopwatch and a non-programmable calculator.
 * 2) Study the rubric. You might be able to use it, but just to be safe always, look over the rubric before the competition.
 * 3) Be neat. If the judges can't read your experiment, they are not going to take it. Find someone in your three people group who has neat handwriting.
 * 4) Think outside of the box. Don't do what everybody else will. The judges need to see that you are uniquely intelligent.
 * 5) Be efficient. Sometimes, "slow and steady wins the race" doesn't always apply.
 * 6) Be precise, especially when labeling your list of materials. You can never be too specific in this event!
 * 7) KISS! Keep it simple, stupid!

Practice trials

 * Experimental Design/Practice
 * Test Exchange
 * Sample Experiment from Minnesota Div. C (Note: PDF)
 * Sample Experiment from Minnesota Div. B (Note: PDF)

Links
Experimental Design National Page (Div. C)

The rubric for Experimental Design The rubric rarely varies from year to year