Science Olympiad at Penn Invitational 2019

[lumosityfan]

It's how a lot of college exams are written

I think this is a super important point worth mentioning. One of the things that has helped me the most in college is knowing how to take a "bad" and/or "outrageously hard" test through my experiences in science olympiad. I think that even if the only benefit of outrageously long and hard tests is that students are better-prepared for college, there is enough merit to justify writing those kinds of exams. Alot of this boils down to the following question: What is the purpose of Science Olympiad?
If you ask NSO, its to inspire kids to pursue STEM. If you ask parents, its to get into college. If you ask students, youll get some combination of to challenge themselves, explore and learn more about science, or get into college. If you ask alumni, I think many of them will talk about how science olympiad not only helped them get into college, but it prepared them for college. I believe that no matter which of these categories your motivations fall in, the long, difficult tests better fit your own self-defined purpose of science olympiad than an easy, short test.

All test writers have their own personal goals with their exams as primitive_polonium mentioned earlier, but in the end, the goal that they all most definitely have in common is to write an exam that is within the rules. As far as regionals/states practice goes, what else really matters? The real hard part is finding good practice for nationals, as well as finding good ways to compare yourselves to other teams to figure out where you stand. This is where the difficulty and length of exams comes into play. While I understand the futility of writing an exam that the vast majority of teams cant finish more than half of, I think this actually rarely happens. Perhaps teams cant answer half the questions, but It is my experience that teams usually read all of the questions. This argument changes for ID events. I am one of the supervisors that strongly believes in extremely fast paced and challenging ID event exams because, as primitive_polonium suggested, I want to reward the teams that actually know the information and understand how to apply it. I also secretly want to punish the teams with large, cumbersome binders that are inefficient, but that point has become moot with the new 2 inch rule.

One final point I would like to make is that not every team gets to attend nationals, but why should this mean that not every team gets to take a nationals-level test? The new wave of event supervisors has addressed this point and many of them write their exams as similarly to the nationals exams as they can (if the nationals exam for that event is typically written well, which is another story for another thread) to give more teams the opportunity to "experience" what it feels like to compete in a high-quality setting. I think there is great value in this and teams that only attend regionals/states should recognize the value in this.

But the point of invitationals isn't to emulate nationals. The point of invitationals is to give teams that normally wouldn't have the ability to practice the chance to compete at a tournament that's well-run, has challenging tests, and gives those teams the chance to showcase what they have learned about science. The moment we worry about making everything "nationals-prep" is the moment we lose sight of the basics of invitationals: running a tournament well logistically, writing sound challenging tests, proctoring in a fair manner. Those are the kinds of things that teams look for in invitationals. I highly doubt the majority of teams look at Penn and think "ooh nationals-prep". I think they just want preparation in general for future events like regionals and states. In addition, Science Olympiad tests shouldn't be about what's in college. It should be about science in general and how to get people to think further about the topics that are tested. (And trust me some college topics are not always appropriate to get into even the hardest of tests.)

[IcsTam]

In my opinion, Science Olympiad is less about learning about Epidemiology, or Acoustics, or whatever your event is, and more about learning scientific thinking and preparing yourself for a life where you use science. Taking a long test may make it harder to get through all the material on the test, but it a) teaches you to use good test-taking strategies and maximize the points you can get based on what you know and what questions are worth, and b) forces you to learn the initiative to go back over that test, see what you got wrong, and also see what you didn't get to and do those problems on your own time and learn that material. I did SciOly all throughout middle school and high school, and I can confidently say that long tests ended up being much more helpful for me than a short test, whether it be through more effort being put in to even writing a longer test or simply because it can cover a wider array of material. Those skills are much more important than being able to get a perfect score on the test and win the event.

Edit: This thread has also turned into a bunch of grads debating the best way to write an invitational test -- is there a better thread for this discussion?

[primitivepolonium]

While I understand the futility of writing an exam that the vast majority of teams cant finish more than half of, I think this actually rarely happens. Perhaps teams cant answer half the questions, but It is my experience that teams usually read all of the questions.

Wanted to piggyback off Windu's point here.

I wrote the MIT Chem Lab exam, which had a high of 54%. This was way lower than I wanted, but a lot of teams just didn't answer a lot of stuff. Now that's not the point. The point is that people emailed me asking about how to study for those questions, because they either read them while taking the exam or looked over them after competition. They might have scored low, and that's not my goal at all, but if they saw my questions and found a new perspective on physical properties and then went home to get a better and more fundamental understanding of chemistry, I'm glad. That's my goal. I'm more than happy to debate over questions provided the arguments and valid, and to provide textbooks and resources to a reasonable extent.

Similarly, Biophysics the MIT trial event had a very low high score, but there were kids afterwards who asked for the ES' contact info because though the exam destroyed everyone, it presented interesting outlooks and perspectives that showed them how cool Biophys, a field rarely covered in high school, can be.

Are there long exams which are off topic wastes of time that are flimsy veils for the writer's own ego? Are there on topic exams that are just plain excessive? Most certainly! But just because teams didn't do well or struggled doesn't mean the exam wasn't ultimately rewarding. And yes, you can definitely turn high schoolers off STEM, but that happens more when you present off topic and inane problems (cough Chem Lab nationals cough). High schoolers are more resilient than we take them to be, and a lot of them will milk an exam for everything it's worth. Even those from "lower tier schools"; said schools have many indicidual gems in the trough.

Anyway, yes, one certainly needs to scaffold, take into account the level of competition they're writing for. But hard =/= discouraging.

[winchesetr]

It's how a lot of college exams are written

One final point I would like to make is that not every team gets to attend nationals, but why should this mean that not every team gets to take a nationals-level test? The new wave of event supervisors has addressed this point and many of them write their exams as similarly to the nationals exams as they can (if the nationals exam for that event is typically written well, which is another story for another thread) to give more teams the opportunity to "experience" what it feels like to compete in a high-quality setting. I think there is great value in this and teams that only attend regionals/states should recognize the value in this.

Yeah I just want to second all of windu's points here. I also want to thank ScienceTurtle314 for the thoughtful criticism as someone who actually took the exam, and therefor probably has the most validity in commenting on the length or difficulty of it.

I want to point out that the Disease exam was not a ridiculously difficult exam. All of my questions fell within the rules of the event. My exam was just long. In terms of distribution, teams were able to finish or mostly finish the exam (at least the first 3 sections, which are the case studies). The answer key we made delineates the points pretty clearly. Also, even though teams (mostly) finished, they were still getting certain things wrong, and that's where I think learning will happen.

My partner and I discussed at length the difficulty and length of our exam, and consulted with other event supervisors, and friends at different colleges as well in the creation of the exam. As someone who is from PA and competed at all levels of competition in Disease, I think this exam is a useful resource. If anyone would like to see it before criticizing it, I would be happy to send it to them.

I try to make my exams so that top teams can just barely finish. Clearly, that doesn't always work out. Finishing also does not mean scoring highly necessarily. But I do think that the exam is finish-able, for the right pair of people.

In terms of the format of events, I really am a firm believer in the fact that people should know why they are doing something, not just what something is. What's the point in memorizing a bunch of useless formulas or facts if you can't apply the knowledge at some point? Sure facts are important (and my exams test factual knowledge to a large degree!!), but so is application. Certain events are more important to have application in than others in my opinion. Disease, chem lab, and proteins (to some degree), are events that should make you think. I would argue that disease has the most practical application in the real world at the level of a Science Olympiad competition. Case studies are the core of a disease exam.

I would also argue that the purpose of Olympiad is not one strict thing. It should inspire a love of STEM (a close friend of mine is going into public health as a result of disease), it should challenge people to understand difficult concepts and actually learn, it should provide problem solving skills for later on in life, and it should be fun! I try to write every exam to encompass all of those very broad goals.

[GoldenKnight1]

This discussion is great but I apologize for the Penn Invitational thread getting a little of the topic.

I do want to clarify somethings though. My comments about how I think students should be able to separate the test and that I have seen cases where they were not allowed to was not stating that at Penn this was the norm. But it did happen in some (few) events to our team at MIT, Princeton, and Penn. I did not mean to imply it was Penn specific or most of the time.

My further comments on test length may have been taken to sound like I don't like long tests. Nothing can be further from the truth. I want tests that are going to challenge the best teams there. I am okay if they don't finish. I love that my students come away from invitationals with new questions they need to research and additional questions it brings to their minds. If you make a challenging test that follows the rules, and is on topic few will walk away complaining. Make sure you have a progression of easy, medium, hard, and kick-your-butt questions. This way even the students who only prepared a little have something they can answer compared to those who did nothing. And make sure that what stops them from getting a good score is not the logistics on your side. Was the test hard to follow? Was the circuit still wired correctly from the last group that came through? Did where you sat in the room give you a significant advantage over others?

Regarding really, really, long tests I am okay with this for the kids as long as just randomly guessing multiple choice is what determines who wins. My only concern is for the person writing the test. Test writing, when done right, is a long process. A lot of thought and effort should be required to make a good test. Even when I would do what many consider an easy event to make up like Picture This, I would spend a lot of time ensuring the words covered a wide range of sciences, had varying difficulty levels, and that no one student got the same type of word. I am fine if you are okay with your test being so long that no one finishes half of the test and you are okay with those questions not getting answered before you scored it. But if this gives you the feeling that the students did not prepare or that you are giving up too much time for this then I would recommend shortening your test.

Again though, hats off to all of you who have continued in Science Olympiad and share your passion with the next generation.

[syo_astro]

Indeed, sorry my fault, but everyone chill >.>.

[alisulmoh]

Lol you’re right. This thread has been long-jacked. (As in posts about long tests, geddit? NVM)
Could we get feedback on the test content as well etc pretty please.
(Astro ES)

About the paper separating thing, at least in my event we let the students do that and then had the pages staples back afterwards. Writing on the tests when they needed to be reused was an issue though.

[poptrop459]

Just wanted to give my two cents on how SOUP went...

ExD: Well run, but as someone said before, it was a bit cramped. It's a bit of a shame that we had to write our reports on blank sheets of paper instead of actual ExD packets, since it forced us to estimate how long some of our sections were going to be (kind of a pain), instead of simply writing our responses in designated spots. Still, the "blank paper lab report" setup doesn't break any rules and isn't unique to SOUP, so I can't complain - 8/10

Thermo: Another well run event. The short amount of time given for the test portion did a good job of ironing out teams that were too dependent on the binder (like we were, lol). Test questions were of reasonable difficulty, and the event supervisors did a good job of organizing the event in general. 9.0/10

Astro: Again, this was really good. Questions on the exam were doable and long enough to ensure that the best teams would be able to finish on time. The ES made the hardest math question (Tulley-Fisher) doable by splitting the question into multiple parts, which I appreciated. One thing I noticed was that the supervisors never checked our computers for airplane mode. I'm not sure if they just forgot us, or just didn't check anybody's computer during the event (or maybe I just missed them as they walked by, lol). I feel like the majority of the teams would follow the honor code, so it's not that big of a deal. - 9.0/10

Circuit Lab: Holy crap, this one was rough. None of the material on the test was particularly difficult, but the two labs (both of which were tougher than any other I've ever seen) ensured that each team pairing would be split for a big chunk of the time given. My partner and I managed to place 4th in the event, but we walked out feeling like we had bombed since we had run out of time and a lot of the exam blank or with random answers. I personally ended up liking the test itself - it tested on a lot of stuff that's in the rules, but not typically taught in high school classes, ensuring that the highest ranking teams were made up of people who spent time learning the extra stuff instead of just relying on what they learned in school (don't get me wrong, that's still useful!). 8.5/10

Overall, I really liked how SOUP was run. The paper problem is unfortunate, but I understand why the decision was made. I look forward to attending the invitational again next year!

[alisulmoh]

Astro: Again, this was really good. Questions on the exam were doable and long enough to ensure that the best teams would be able to finish on time. The ES made the hardest math question (Tulley-Fisher) doable by splitting the question into multiple parts, which I appreciated. One thing I noticed was that the supervisors never checked our computers for airplane mode. I'm not sure if they just forgot us, or just didn't check anybody's computer during the event (or maybe I just missed them as they walked by, lol). I feel like the majority of the teams would follow the honor code, so it's not that big of a deal. - 9.0/10

thanks for the feedback. Yeah we kinda just peered at screens for any obvious signs of web browser activity. Just felt that doing an actual airplane mode check would be too intrusive. What would you suggest be done next time?
(Astro ES)

[meilingkuo]

Hey everyone! I was one of the event supervisors for Fossils and this was my first time writing a test, so any feedback and criticism would be super helpful. Hope that you guys liked the tournament!

Hi, I am not in your area, but have a general question on fossils. Since you wrote a fossils test, I am wondering where you get the information on temporal range of fossils for the answer key and what is considered main resource for regional/state/national test. We checked several resources and noticed that the range varies from source to source. We consulted a paleontologist and was told that fossilworks database is what scientists in the field use. However, we noticed that answer keys for lots of old test packets are not based on fossilworks database. Our regional is near and we really need to find out.