Experimental Design

Experimental Design has been a Science Olympiad event for many years in both divisions. In this event, competitors will design, execute, and write-up an experiment based on the topic and materials provided by the event supervisor.

The Lab Write-up

Statement of Problem

The statement of problem is a question posed that will be explored in an experiment. One of the formats that can be used for almost any experiment is "How does (Independent Variable) affect (Dependent Variable)?"

 Ex. How does the height a ball is dropped from (1, 2, 3 meters) affect its rebound height (cm)?

Tips: The statement of problem should not be a simple yes or no question. The key words are "how" and "why".

Hypothesis

The most common way a hypothesis is used in scientific research is as a tentative, testable, and falsifiable statement that explains some observed phenomenon in nature. This kind of statement is more specifically called an explanatory hypothesis. However, a hypothesis can also be a statement that describes an observed pattern in nature. In this case, the statement is called a generalizing hypothesis. The hypothesis statement is followed by a specific, measurable prediction that can can be made if the hypothesis is valid. Thus, in science the hypothesis is thought of as an explanation or generalization on trial.

A prediction in science is a prophecy, a specific and measurable event that is likely to happen in the future as the result of an experiment if the hypothesis is valid.

Teaching the Hypothesis Incorrectly Many teachers and even many textbooks teach the hypothesis in a way that makes it no different from a prediction. They teach students to write “If – then” statements for their hypotheses. This approach results in the incorrect form: If I do X, then Y will happen. There is no hypothesis here. This is simply a method (if I do X) followed by a prediction (then Y will happen). Some teachers and textbooks add “…because…” at the end of the “If…, then…” statement. The because statement is often close to the hypothesis that is being tested, but it still does not carefully delineate the hypothesis from the prediction. Indeed, even professional scientists can make mistakes.

The "because" part of the hypothesis is often referred to as the Rationale.

In short, to receive full points for this section, the hypothesis should be written like this: If I change _____(the IV), then the DV will _____(what changes) because ________(the rationale).

 Ex. If I drop a ball from different heights (1, 2, 3 meters), then the rebound heights (in centimeters) for the higher drop heights will be greater than the lower drop heights because of Issac Newton's 3rd law (For every action, there is an equal and opposite reaction).  This law applies to this experiment because when the drop height is greater, there is more force in the action of the ball falling to the floor, and thus the rebound height (the equal and opposite reaction) will be greater.

Tips: Make sure you have all of the parts for this section complete and explained thoroughly. Sometimes, a rationale may sound complete in a competitor's head, but in reality, it is not fully explained on paper, and the competitor would lose points on the basis of silly mistakes.

Variables

There are three different kinds of variables to be defined in the lab write-up: the independent variable, the dependent variable, and controlled variables.

Tips: Make sure to operationally define (units) all of your variables. Also make them clear and concise to make sure you get all of the points in a point-heavy part like this.

Independent Variable (IV)

The independent variable is the variable that is changed to examine its effect on the dependent variable. There should only be one IV, which should be listed with units. The IV must be operationally defined (in terms of the experiment) and empirically defined (in general for future variations of the experiment). Additionally, a minimum of three different levels of the independent variable must be listed excluding the control level.

 Ex. the drop height (1, 2, 3 meters)

Dependent Variable (DV)

The dependent variable is what is affected by the independent variable. There should only be one IV, which should be listed with units. The dependent variable must be operationally and empirically defined. Do not include levels of the DV as that is what will be determined through the experiment.

 Ex. rebound height (in centimeters)

Controlled Variables (CV)

Controlled variables are factors which could affect the dependent variable but are kept constant throughout the experiment. Multiple controlled variables should be listed, but only four need to be listed to receive full credit for this section.

 Ex. size of ball, material of ball, shape of ball, material of dropping surface, method of releasing ball (dropping, throwing, etc.)

Standard of Comparison or "Control"

The standard of comparison (SOC) is the "normal trial", or the one that hasn't been changed at all (there is only one). It serves as a neutral comparison for the other trials. A rationale for the SOC should be included.

 Ex. The SOC for this experiment would be the 1 meter IV level.  This SOC was chosen because it is the level closest to zero.  Since the hypothesis predicted an increase in rebound heights with increasing drop heights, this IV level would either prove or disprove the hypothesis and would be useful for analyzing the experiment afterwards.

Tips: Changing the IV to zero or using the highest or lowest possible numeric value of the IV make good SOC's

Materials

The materials list is what it sounds like. It's a list of the materials used in the experiment. All materials used in the experiment should be listed (provided materials that are not used should not be included). The materials list should be as specific as possible; the quantity of the materials to be used, as well as brand names should be listed. A person should be able to look at the list and gather the exact materials used in the original experiment. The material of the independent variable should be listed once with the levels after it in parenthesis.

Some competitions want measuring devices to be listed, while others may take off points for it. The event supervisor should distinguish what they want.

 Ex. 3 Penn racquetballs, 3 meter sticks 

Tips: Make sure all materials are listed. Sometimes accidents happen and competitors lose points on what may be the simplest section ever. If time is leftover after finishing an experiment, all competitors should check that all materials are listed.

Procedure

The procedure is a list of the steps in the experiment and includes labeled diagrams (at least three) of how the experiment was performed. The procedure is included in write-ups so that other scientists reproducing the experiment know exactly how it was done the first time. The steps should be listed clearly as well as very specifically, and include three trials for each level of the IV. The very last step of the procedure should be "Clean up your workspace".

Tips: Steps such as, "Repeat steps X to Y", can be used to save time.

Qualitative Observations

There are three types of observations that must be made to receive full credit: observations about the procedure, results, and anything not related to the DV. Additionally, observations must be made over the course of the experiment about the three.

Typically, observations about the procedure are about noticing flaws in the experiment that went unnoticed before it was performed. These can include flaws in measurement technique, flaws in building/maintaining an experimental unit, and flaws in performing the actual experiment. Observations about the procedure carry over to experimental errors and practical applications, so be sure that you thoroughly explain what went wrong. An example of this type of observation is, "After trials 1 and 2, the penny which was dropped in the oil was not completely cleaned off, and a thin residue was left on the surface for the remainder of the experiment."

Quantitative Data

One way to organize the data is to make two tables. For the first, make a table of four rows and four columns. The first column should consist of, from top to bottom, a blank box, IV 1, IV 2, and IV 3. The second column should be labeled "Trial 1", and following boxes filled accordingly to the data. The next two columns follow the same layout as the Trial 1 box, but with Trials 2 and 3. Title the graph as seen fit for the data. Next to that table, draw a one column, four row condensed table (to the right). Name it "Average" (or AVG for short), and average the data for each IV. Put arrows from the second row of the first table to the second row of the condensed table, and so forth. Give a sample calculation for the average ((Trial 1+Trial 2+Trial 3)/3), located below the table or on one of the arrows. Remember to title both of your tables.

Also be sure use significant figures if you are in C Division, and be sure to keep them consistent and logical. You do not want to have a number down to three decimals when your ruler can only accurately measure to one decimal. See Significant Figures for additional info about significant figures.

Graph

A standard bar, line, or scatter-plot graph works almost universally at any competition level. Even so, always be sure to use the correct type for your data. Always remember to:

1. Label your axis (x+y)
2. Title the graph
3. Use the DV as the y value and IV as the x value
4. Title the individual axis (For example, left of the y axis you would write, "Time taken for parachute to fall (seconds)")
5. Connect the data points or draw a line of best fit
6. Only include the averages of the data for each IV

Statistics

Take the common statistics - mean, mode, range, median. Also include any other relevant statistics and show work. The best idea is to put all statistics in a neat table.

Your table of data should be neat- a ruler helps a lot. Be sure to keep writing your units.

Once your common statistics are done, make sure to do some more. Standard deviation is a very good statistic to include. The equation for calculating standard deviation is: [math]\sigma = \sqrt{avg((value - mean)^{2})}[/math]. For a better visual equation and an explanation of what standard deviation is (which you will need to know to explain the statistic), see Standard Deviation. Actually doing trials is necessary, as a standard deviation of a sample size of 1 is clearly stupid. A key point that is easy to miss is the deviation has to be squared. If you don't, your result will always be 0, and though this may look pretty, it should be obvious that your data does not have a standard deviation of 0.

Graph your data. Make the graph neat, legible. Use a legend if need be. Label the axes (with units) and make a title for the graph (including units here as well is a good idea; for example, "Time in takes a parachute to fall, in seconds, vs the weight, in grams".)

As of 2015, both Division B and Division C will be expected to do more with the data, whereas before only C Division needed to. One essential aspect of the graph will be to create a regression, or line of best fit. Since both divisions are now permitted to bring any type of calculator, this would be a good time to invest in a nice TI-84 or similar graphing calculator because linear, logarithmic, and many other types of regression can be calculated with them. To calculate a linear regression on a TI calculator, start by putting your data in a list. Press STAT and go to the EDIT menu. Press 1 to edit your list. Then, exit and press STAT again. Go to the CALC menu to select the type of regression most suited for your data (which in most cases will be LinReg). Place the list containing your X values in Xlist, and repeat for Y values in Ylist. Scroll down and press Calculate, which will then give you the constants for your regression. Draw in the line on your graph and label it.

If you cannot get a graphing calculator, the best fit line of dubious accuracy is made by drawing a straight line with a ruler that you think seems to go as close to all the points on the graph. Once you do, find the y-intercept, and calculate the slope. Make sure to consider which outliers are significant, and which are experimental errors. If you make these kinds of judgment calls, make sure to point it out and explain it in the Analysis section.

Also, make sure you always have the same units through-out the experiment, if you are using milliseconds in the data table continue using milliseconds for everything else, DO NOT change to seconds or any other units. C Division competitors should remember to use significant figures in their statistics.

Analysis

This section should be one extended paragraph which at least touches on all data points and expands on outliers. Look at the data and draw some reasonable conclusions about the experiment. There should be trends; point them out and explain them. Discuss your statistics and again, explain them. Guesses are okay, even if they're wrong; they show your thought process. If you have any outliers or random "bad" data points, don't ignore them - again, write about them. Was there anything you did wrong that time, or was it just a fluke? Conclude by stating if the IV is directly, inversely, or not clearly proportional to the DV.

Possible Sources of Experimental Error

Look for all the things wrong with your experimental setup. How might they have caused inaccuracies in your data? This is extremely useful, because it can redeem mistakes made earlier in the event by showing that you are aware of them. Sometimes points can even be regained. Try to stay away from human errors and try to focus on experimental sources of errors like you can say things that have to do with temperature. The container of your object may have insulated it. Say any possible thing that could change the outcome of the experiment. Then, explain how the errors are believed to affect the data: increase from normal, decrease from normal, or either.

Also, this section can be written before any data is actually collected, and just added to if there are glaring errors in data collection that you didn't predict. If you're not sure how the experiment is going to turn out, this is a good thing to write first, though any errors that are included in your qualitative observations should be commented on.

Conclusion

DO NOT ever say that your hypothesis was right or wrong. After one experiment, a hypothesis can either be supported or not supported by the data. Restate your hypothesis minus the explanation before concluding in either way. Then, explain why you came to that conclusion with your data as support. Never extrapolate anything; stick to what you observed even if you think the results were wrong. You may attempt to explain why your data varied from your hypothesis using proper scientific terminology that was not considered while writing your hypothesis, but the vast majority of the explanation should be data-based.

Applications and Recommendations for Further Experimentation

When writing this section, consider variations of your experiment that would produce more accurate results. There should be three variations listed: one to improve a certain aspect of your experiment, one to approach the hypothesis in a different way, and one for a future experiment related to the DV. Finally, consider a practical application for the experiment. This section can also be written without any knowledge of how the experiment will turn out, and so it can also be written before data has been collected.

In an experiment on the effect of applied weight on a parachute's descent time, a way to improve the specific experiment would be to use more durable string to avoid breaking the parachute. A way to approach the hypothesis from a different perspective is to use a computer simulation to determine the outcome of the descent. Future experiments could test other IVs, such as air resistance and varying gravity levels on other planets. Practical applications include use in defense and for returning space vehicles safely to Earth.

Common Strategies

Knowing the scoring rubric is the key to success in Experimental Design. The rubricis a set of guidelines used for scoring experiments. When experimenters are aware of what is expected in each section, it becomes much easier to work efficiently.

At the start of the event, start by brainstorming possible experiments. Expect to get a handful of seemingly random items to test with, along with the possibility of a topic or prompt to design the experiment around. If each team member is familiar with general scientific concepts, designing an experiment should not be too difficult. Focus on execution and write-up, not on preparation. Spend no more than 5 minutes on this.

Keep your experiment simple. Too many variables can mean a lot of writing. Consider an example experience from one regional tournament. 3 balls (different colors), 2 rubber bands, a foot of masking tape, a metric stick, and a mini catapult were given. Naturally, one would want to experiment with the fanciest item given (catapult, in this case), but there would be so many variables to consider. Instead, performing a dropping experiment on how a rubber band affects the time it takes for a ball to drop. This is much simpler and, in this scenario, an idea that definitely paid off. Teams that used the catapult had balls flying everywhere throughout the event, and team members had to run around searching for them; wasting time. On the other hand, teams that utilized the other equipment achieved third out of thirty teams. Moral of the story is, ignore the urge to fiddle around with the complex stuff. Keep it simple- experiments will be simpler to write and test, saving time.

However, be sure to have enough trials. Having 3-5 trials for each variable ensures that data is sound and statistics have merit. This way if there is a possibility of strange data (one test being too high/low/fast/slow) there is the "experimental errors" section to comment on that. Only having 1-3 trials means there may not be enough data to show that a data point is strange, because there are not enough points to compare it to.

Know who is doing each section before the competition. All 3 people don't need to be doing the lab; only 1 or 2 people should be experimenting. Don't spend the whole time doing the lab either, 15-20 minutes should be plenty to get a significant amount of data for an experiment. There are many ways to divvy up the work on this event. Find something that works, each group is different.

If the experiment goes horribly wrong and all the data is skewed, focus on the report as much as possible. Make sure to explain why the experiment was bad and where it went wrong. This is where "Possible Experimental Errors" really counts-- be sure to write and explain every error which caused the experiment to go wrong. Having a bad experiment but a very good report can, in some cases, cancel out the fact that the experiment didn't work.

Just like any other event, practice! Have a fellow teammate or coach gather materials and come up with a possible topic. Spend 50 minutes and come up with, test, and write up a lab report. Have the coach/teammate then grade the lab based on the rubric. This gives great insight into time usage and where improvements need to be made.

How to do your best

In Experimental Design, you should always keep your experiment reasonable. Here are some tips for how to do your best:

1. Come prepared. You should come to the competition equipped with several different writing utensils, a ruler, a stopwatch and any type of calculator as long as it cannot access the Internet and does not have a camera.
2. Study the rubric. You might be able to use it, but just to be safe always look over the rubric before the competition.
3. Be neat. If the judges can't read your experiment, they are not going to take it. You will not have time for one person to write up everything well enough, so try to be neat.
4. Think outside of the box. Don't do what everybody else will. The judges need to see that you are uniquely intelligent.
5. Be efficient. Sometimes speed is extremely important due to the limited time that you will receive for each test.
6. Be precise Especially when labeling your list of materials. You can never be too specific in this event!
7. KISS! Keep it simple, stupid! (Not literally, you aren't stupid or you wouldn't be in Science Olympiad)

Practice trials

• Experimental Design/Practice
• Test Exchange
• Sample Experiment from Minnesota Div. C (Note: PDF)
• Sample Experiment from Minnesota Div. B (Note: PDF)

Links

Experimental Design National Page (Div. C)

The rubric for Experimental Design The rubric rarely varies from year to year but has changed for the 2015 season.

See Also

• Experimental Design/Practice