Science Olympiad at Penn Invitational 2019

[GoldenKnight1]

For context, printing at Penn is 10 cents per page (for a double-sided piece of paper). Say the average exam length is 10 pages for a test an answer key combined, and you print 50 exams, that's ~$1,200. This doesn't include the fact that some things need to be printed in color (more expensive), or the fact that some of the exams are a lot longer than that, or any other things that need to be printed single-sided (more expensive). If this cost was pushed onto teams, that would be ~$23 more for a team to register. So any way possible to minimize the amount needed to print was taken. This clearly doesn't work well for all events (you can't for Disease since all teams take the exam at once), and perhaps code-busters should not have been run that way either.

Printing a single test set with just enough tests for each time block and then separate answer sheets for every team is the system used by nearly every college-run invitational and most national supervisors. Excessive printing should be avoided when possible to do without sacrificing event quality. There will be some events that this is not appropriate for, but for the most part for most events, this is the best practice.

I don't think an additional $25 per team would stop teams from attending. It is hard to imagine that Troy, Boca, New Trier, Brookwood, or Mason to name a few who traveled so far paying for bus, plane, and hotels would have this extra $25 as a deal breaker. Also for most of the non-university held invitationals we have attended it is the expectation that the event supervisor makes 1 copy per team. This cost is something that the supervisor (or their team), in addition to writing the test, has to absorb. If at a university's invitational I don't need to print anything that I am willing to pay more since I am not paying for that on our end. Additionally if I think about all the time and stress that I am saved by not only not having to write a test but also not needing to even judge an event I am happy to pay this extra amount. But that is just me. This could be easily asked of the coaches' attending to get their view of it.

[winchesetr]

For context, printing at Penn is 10 cents per page (for a double-sided piece of paper). Say the average exam length is 10 pages for a test an answer key combined, and you print 50 exams, that's ~$1,200. This doesn't include the fact that some things need to be printed in color (more expensive), or the fact that some of the exams are a lot longer than that, or any other things that need to be printed single-sided (more expensive). If this cost was pushed onto teams, that would be ~$23 more for a team to register. So any way possible to minimize the amount needed to print was taken. This clearly doesn't work well for all events (you can't for Disease since all teams take the exam at once), and perhaps code-busters should not have been run that way either.

Printing a single test set with just enough tests for each time block and then separate answer sheets for every team is the system used by nearly every college-run invitational and most national supervisors. Excessive printing should be avoided when possible to do without sacrificing event quality. There will be some events that this is not appropriate for, but for the most part for most events, this is the best practice.

I don't think an additional $25 per team would stop teams from attending. It is hard to imagine that Troy, Boca, New Trier, Brookwood, or Mason to name a few who traveled so far paying for bus, plane, and hotels would have this extra $25 as a deal breaker. Also for most of the non-university held invitationals we have attended it is the expectation that the event supervisor makes 1 copy per team. This cost is something that the supervisor (or their team), in addition to writing the test, has to absorb. If at a university's invitational I don't need to print anything that I am willing to pay more since I am not paying for that on our end. Additionally if I think about all the time and stress that I am saved by not only not having to write a test but also not needing to even judge an event I am happy to pay this extra amount. But that is just me. This could be easily asked of the coaches' attending to get their view of it.

Sure, perhaps it's not that big of a deal and some teams would be willing to absorb the cost. However I think that one of the benefits of SOUP is that it aims to be accessible to a wide variety of teams, including teams that are not as well funded as some of the Nationals teams. Princeton (if I'm not wrong) has waived the tournament fee entirely, which I think is extremely cool because Science Olympiad should be accessible to everyone. Penn has tried to keep our fees low (in comparison to say, MIT, or even local invitationals such as Battle of Valley Forge). Also I think costs incurred by high schools when they write tests and print them are lower for many schools, because in many districts (at least in this area), printing is free. Again, would some exams benefit from having a copy for every team? Sure, but it's also important to take into consideration the cost impact that has on the tournament, accessibility for non-wealthy schools, and also (as was mentioned earlier), the environmental impact.

I understand the frustration some competitors have with that set-up - it's not ideal. But all teams get the same disadvantage. Also, as a former competitor, for many events this method works pretty well. Tests can still be taken apart and split between competitors. Some events should probably not be designed in such a manner, but I think that there should probably be an Official Rules change for that event, or learned through feedback such as this, rather than printing an excess when an excess is unnecessary. Just my thoughts.

[zcgolf16]

My initial thoughts:

Disease Detectives: It was fine. My one comment would be that I don't see the point of a test that is "not possible to finish". It prevents teams from getting in competition practice on all aspects and topics of the event.

Thermo: Didn't have any problems here. Well-written test and the event supervisors did well handling everyone's box from what I saw.

Fermi: Again, no problems here.

Mission: This is where I had some issues. I unfortunately had to go over the 30 minutes of set up time by about 5 minutes after accidentally triggering my machine and having to set it back up, but the event supervisors started testing other teams' machines, including a team that arrived after I was ready, and it took about 15-20 minutes after I was ready for them to get to me. At this point, my partner had to go to another event so I was the sole representative for my team, which was unfortunate and put us at a slight disadvantage. Everything ultimately worked out, but this was a little annoying.

In all, I thought everyone at SOUP did a good job running the tournament and it was a great day. Thanks to all of the event supervisors!

[GoldenKnight1]

My one comment would be that I don't see the point of a test that is "not possible to finish". It prevents teams from getting in competition practice on all aspects and topics of the event.

This is again why I wish teams had been able to separate the tests. I have read so many times about hardworking division D SO members who work hard on making fantastic test questions but who often don't know how to edit their test. If the top teams are only seeing about half your test because they are running out of time then what is the point of including as much as you did. And if the main reason they are not getting to it is because they were not allowed to separate the test then that should be considered when designing the test. This idea of trying to save money on copying is why we have seen at some competitions the Experiment Design packet not used or, worse still, double sided.

Sure, perhaps it's not that big of a deal and some teams would be willing to absorb the cost. However I think that one of the benefits of SOUP is that it aims to be accessible to a wide variety of teams, including teams that are not as well funded as some of the Nationals teams. Princeton (if I'm not wrong) has waived the tournament fee entirely, which I think is extremely cool because Science Olympiad should be accessible to everyone. Penn has tried to keep our fees low (in comparison to say, MIT, or even local invitationals such as Battle of Valley Forge). Also I think costs incurred by high schools when they write tests and print them are lower for many schools, because in many districts (at least in this area), printing is free. Again, would some exams benefit from having a copy for every team? Sure, but it's also important to take into consideration the cost impact that has on the tournament, accessibility for non-wealthy schools, and also (as was mentioned earlier), the environmental impact.

Free? That is not my school. It comes directly out of our Science Department budget coded as I was the one who made the copies. Many supplies for events also work the same way. Maybe that is just my building though.

Look, I get the desire to keep cost down the Princeton's free registration makes me very reluctant to complain about their copying practices. If you want to keep the cost and environmental impact low why not copy the first half and second half of the test in two separate packets. That keeps the number of copied pages the same but allows the students an easier time working through the test. If there is a reason not to do it this way I would be interested in hearing it but my instinct on this one is just that it has not been considered.

[winchesetr]

My one comment would be that I don't see the point of a test that is "not possible to finish". It prevents teams from getting in competition practice on all aspects and topics of the event.

This is again why I wish teams had been able to separate the tests. I have read so many times about hardworking division D SO members who work hard on making fantastic test questions but who often don't know how to edit their test. If the top teams are only seeing about half your test because they are running out of time then what is the point of including as much as you did. And if the main reason they are not getting to it is because they were not allowed to separate the test then that should be considered when designing the test. This idea of trying to save money on copying is why we have seen at some competitions the Experiment Design packet not used or, worse still, double sided.

Sure, perhaps it's not that big of a deal and some teams would be willing to absorb the cost. However I think that one of the benefits of SOUP is that it aims to be accessible to a wide variety of teams, including teams that are not as well funded as some of the Nationals teams. Princeton (if I'm not wrong) has waived the tournament fee entirely, which I think is extremely cool because Science Olympiad should be accessible to everyone. Penn has tried to keep our fees low (in comparison to say, MIT, or even local invitationals such as Battle of Valley Forge). Also I think costs incurred by high schools when they write tests and print them are lower for many schools, because in many districts (at least in this area), printing is free. Again, would some exams benefit from having a copy for every team? Sure, but it's also important to take into consideration the cost impact that has on the tournament, accessibility for non-wealthy schools, and also (as was mentioned earlier), the environmental impact.

Free? That is not my school. It comes directly out of our Science Department budget coded as I was the one who made the copies. Many supplies for events also work the same way. Maybe that is just my building though.

Look, I get the desire to keep cost down the Princeton's free registration makes me very reluctant to complain about their copying practices. If you want to keep the cost and environmental impact low why not copy the first half and second half of the test in two separate packets. That keeps the number of copied pages the same but allows the students an easier time working through the test. If there is a reason not to do it this way I would be interested in hearing it but my instinct on this one is just that it has not been considered.

Teams were not allowed to separate their exams? That's news to me. We had staplers accessible, so teams should have been allowed to separate the exams even if they were not able to write on them. I'm not sure I understand your criticism entirely, or what you are suggesting. Most invitationals where test packets are reused allow test separation, but just not writing on the exam booklets. That's definitely an issue that can be addressed.

Regardless, to address your second point, all teams in Disease were allowed to separate their tests. We had a stapler and made that clear at the beginning of the exam. The length of the test was intended to be a comprehensive analysis. We have 3 reasons for the length.

1. The nationals test is long. In fact, many teams do not finish the Nats exam. That doesn't make the exam useless. Our test was longer than nationals has been for a few years (though I recall my freshman year of hs being longer) but not severely, and certainly not where it counted. Our case analyses were formatted in the style of a nationals exam, with some personal stylization. Section 4 is a unique throw in that I try to include because it's less analysis and more memorized facts, as some teams might not know how to analyze as well as other teams. This is due to the fact that many ES at other competitions do not format their exams in a Nats style.

2. The test allows for a good separation of scores. Part of Science Olympiad is also learning how to take a test well (which is a very useful skill for college). I could show you the distribution if you are interested, but it's approximately normal. Top teams were able to excel, even if they didn't finish. In my opinion, it is better to have a too long exam, then one in which no score separation is obtained. The length also allows teams that are less familiar with the event to still acquire points and be competitive.

3. The point of the length (and the difficulty) is to also be able to take the exam home and learn from it. Stuff you were not able to finish, you are able to finish later and improve from. Tests are supposed to be a resource in this manner, especially for teams that may not know as much.

I will say this. Despite the length, teams were able to do well and accumulate a good amount of points. The length of the exam was approx. the same as last year's exam, and I doubt that we will be changing the length any time soon, though I appreciate the feedback.

EDIT: Just to clarify, the test was not super difficult, it was just long. There were difficult questions (analyses of why you would do something), and there were easy ones (what is this type of study. benefits, drawbacks. Who popularized the spot map. list the 5 steps of surveillance). I would say that the easy questions outweighed the hard ones by a large amount, and that the easier questions made up 75% of the exam. Again, I reiterate, people were able to get a good amount of points and do well. Disease is one of the events where it is important to test actual science as opposed to memorization, which is done through case-study analysis. That's the way it should be run. My exams are just longer than most, and still attain a good score distribution.

[heterodon]

This is again why I wish teams had been able to separate the tests.

For all the events I competed in, I was able to take apart the tests and staple them back together.

[Synaptic_Cleft]

Just a couple of thoughts.

On printing: GoldenKnight makes an excellent point--printing test packets in two halves is one of those great practical ideas that has been overlooked. However, for what it's worth, as far as I know, most (if not all) events allowed competitors to split the test, just not to write on it. With that said, my perspective on printing tests has changed. I made the unusual transition from being involved with the executive level of SOUP to the event supervisor level this year. As a board member, it made perfect sense to save costs by printing only 20 test packets since our tournament tends to be crazy expensive (another key reason is that we are required to hire security officers on overtime rates, whereas other tournaments are not). However, after being an ES this year, I think that it's probably worth printing one test per team because no matter how often you tell people not to write on the test, they still do it. The added stress in time block transitions from checking test packets to make sure they are blank and the cost of all the extra copies that need to be printed when people inevitably write on them isn't worth the saved printing costs. I do think it's practical to print colored images separately.

On tests that are not possible to finish: To me this is about what you consider to be important about Science Olympiad. If indeed the purpose is just to learn in order to compete, then I agree that there is no purpose to making such a test. However, to me it's about something else. As a test writer, my goal is always two-fold: 1) write a test that is within the rules and that will effectively separate teams for the competition and 2) even more importantly, write a test that is going to challenge students' problem solving abilities and that is based in topics relevant to modern research related to the event, within the rules. This is especially important at an invitational, where students will get the tests back. Almost all of us come into college with the mentality that science is this immense set of "facts" and "truths" that need to be internalized and miss out on the most thrilling part (and really the essence) of doing science, which is using something you know to figure out something you don't. By making tests that are "not possible to finish" which have tough and interesting questions, you give students an opportunity to get excited about a problem they know they can solve using what they know, which hopefully encourages them to read more about it. In my mind, the best tests are the ones that have a solid set of standard Science Olympiad questions, which will help separate teams for the competition, and also have a large set of challenging and stimulating questions which you don't expect students to finish in 50 minutes, but you hope they take the time to work through, to understand, and to get excited about after the fact.

[syo_astro]

My one comment would be that I don't see the point of a test that is "not possible to finish". It prevents teams from getting in competition practice on all aspects and topics of the event.

...I have read so many times about hardworking division D SO members who work hard on making fantastic test questions but who often don't know how to edit their test. If the top teams are only seeing about half your test because they are running out of time then what is the point of including as much as you did...

these points. I always wished to get decent test editors, but it seems like nobody wants to edit much...various invites have boards for this or something, but I find it very tough to edit multiple events even when you have the experience (I've seen this for others too).

I'll also add part of the issue isn't just editing but also intent: most don't know what competitors want to or should get out of tests. Or else they don't even consider the different backgrounds of those taking the tests...or different takes on the issue other than their own and a select few >.>. To be fair, though, I feel like these issues (editing and intent) apply to many MANY test writers.

Let's take

...The nationals test is long. In fact, many teams do not finish the Nats exam. That doesn't make the exam useless

...The test allows for a good separation of scores. Part of Science Olympiad is also learning how to take a test well (which is a very useful skill for college). I could show you the distribution if you are interested, but it's approximately normal...The length also allows teams that are less familiar with the event to still acquire points and be competitive.

...The point of the length (and the difficulty) is to also be able to take the exam home and learn from it...Tests are supposed to be a resource in this manner, especially for teams that may not know as much.

...Despite the length, teams were able to do well and accumulate a good amount of points. The length of the exam was approx. the same as last year's exam...

I'll preface this with saying that I didn't take the test, so I'm not saying strictly "you're wrong" or aiming my issues directly at you (or whoever wrote the test, sorry I'm not sure >.>), but I see some healthy argument based on what you've said. Point by point:
-Just because teams don't finish the nationals exam doesn't mean teams at an invite should have difficulty with finishing. They're different tournaments. Say an invite is a practice for a regional or state competition for some (even many) teams. Does that mean they should get floored with a national level exam? It definitely wouldn't make the exam useless (I don't think / hope nobody thinks that), but it definitely limits the test's usefulness.

-Just because a test is long doesn't mean it helps separate scores. Again, not trying to knock on you specifically, I'm sure you put in a variety of question difficulties and such. But some teams definitely get floored if they see a bunch of difficult questions about stuff they've never seen. Let me put it this way: if students don't even answer questions, then it's as if you can't even evaluate them on that question, which is part of why length doesn't really go with separating scores as much as "long enough" (accounting for question difficulty of course). Science Olympiad test taking is also different from regular test taking because you have a partner, so I don't understand that as a relevant goal to shoot for...I always saw it more like tackling a big project and learning to split the work, it's definitely non-standard.

-So this is an interesting point and one I've been thinking about more, recently (looking at tests from the "learning side" and not just as "diagnostics")! I have read that difficult tests do help learning (that's right, I learn about learning, I'm a nerd:P), but I think it depends on A LOT of factors. Some of the main reasons I've read why is that students have to prepare more broadly for the tests. I've also read that it's important to "make mistakes and learn from them". Usually, students are supposed to give a serious effort on a question or a test first and then learn from those mistakes. But I'm not sure how that applies if you can't even try the questions in the first place. This part is definitely an opinion, but I don't think getting students to make mistakes needs to be taken TOO far or even done all the time to aid in their learning. It's quite the balancing act, it's been interesting to read about!

As for the last point, no comment, good to hear. Anyway, probably a different thread for this, apologies for the interruption.

Edit: One extra point I see a lot based on the last post is that tests should involve challenges or critical analysis similar to doing "real science". While I agree that would be great, practically it can be difficult to execute that on every question, and I have yet to read about how those questions actually do on the diagnostic / learning side of things (I'll get to it soon enough:P). It can sometimes even be difficult to really see how a question on a test relates to...well, "real science". And again, there's a diversity of teams competing anyway, and some don't compete as intensely...just stuff to keep in mind.

[lumosityfan]

My philosophy on test writing is that while the test should be challenging and want teams to learn more, at the same time a test is practice for further competitions. A test is there for competitors to get used to what's up ahead. The point of a good invite test is not to crush the participant before one even begins. That does no one any good if they don't know how to answer any of the questions. If anything it'll scare teams off because they realize that they've got no chance. (Note: this does not mean dumb down tests, but gradate difficulty through the test to have some easy, some medium, and some hard questions, with the ratios adjusted based on type of tournament.) In addition, invites should not be used as a "nationals-prep" tournament. They should be used as a states/regs practice tournament. (That being said I think there should be a big nationals-prep tournament but after all states have finished.) Thus, the difficulty should be adjusted accordingly. syo's right when it doesn't do you any good when you scare off half the teams, most of them are fairly new to Science Olympiad in general. If one of our goals is to spread science and SO in general to more schools, then we can't do them by slamming at them college-level stuff, because then they'll just think that it's not for them and walk away, which does us no good.

[primitivepolonium]

I'll also add part of the issue isn't just editing but also intent: most don't know what competitors want to or should get out of tests. Or else they don't even consider the different backgrounds of those taking the tests...or different takes on the issue other than their own and a select few >.>. To be fair, though, I feel like these issues (editing and intent) apply to many MANY test writers.

There is certainly a school of writers who have the ideology that it's better an invite exam is too long, since it provides teams study content after competition. It's also shaped by the fact that it's still fairly common to get plagiarized, off-topic, and/or insultingly easy exams at official tournaments; people aim to be better than that.

It's a very difficult balancing act. I've seen decently long, very-on-topic but not difficult exams get put down because they had highs of 90% and were expected to be "harder", and I've also seen difficult, long, and educational exams get put down but they're "excessive" and "no one's seen the last half of the exam anyway". It's often a matter of "pigeon if you do, pigeon if you don't" unless you're extremely experienced at exam writing (and a lot of ESs are college freshmen or sophomores) or very lucky. (Though of course, one should always consider accessibility and constructive criticism!)

At the end of the day, the supervisor has to gauge the competition they are writing for (a regionals exam might be easier on average, but really prioritize score separation). One must decide what their goal and priorities is and stick to it as long as they are reasonable and remain within the scope of the rules. For instance, some supervisors prioritize testing breadth of knowledge, and others test how good you are at problem-solving and adapting to challenges. And I'm sure you've all seen ID supervisors who try to wean competitors off binder dependence. More personally, (soapbox time) I, as a Chem Lab writer, try to get kids to use chemical intuition by throwing problems that seem hard but can be solved by an examination of fundamental chemistry concepts taught in high school classes. It's a much better alternative, IMO, to rewarding kids for rote memorization of question formats and information, since there's already a pervasive, false, and harmful perception of chemistry as "memorization" when it really isn't. My aim is to write interesting questions with keywords that kids recognize, so that they 1) are intrigued by the subject and 2) at least have something concrete to study as they prepare for competition.

While I agree that would be great, practically it can be difficult to execute that on every question, and I have yet to read about how those questions actually do on the diagnostic / learning side of things (I'll get to it soon enough:P). It can sometimes even be difficult to really see how a question on a test relates to...well, "real science"

I know what you mean. Sometimes, in the process of trying to test "problem-solving", event supervisors write contrived and ambiguous questions that actively try to lead you towards a "right answer". I do think that for a lot of events, the "case format", where each big FRQ question builds off a scenario, is very helpful. It's how a lot of college exams are written, and as long as the scenarios aren't super dumb, it gives the kids context for how the event fits into real life. That said, I'd much prefer a useful recall question to a leading "problem-solving" question.