Science Olympiad at Penn Invitational 2019

terence.tan
Member
Member
Posts: 175
Joined: September 17th, 2017, 10:40 am
Division: C
State: NY

Re: Science Olympiad at Penn Invitational 2019

Post by terence.tan » February 18th, 2019, 10:10 am

have the coaches received the blank test and keys?
2017 events: Electric Vehicle, Game On, Robot Arm
2018 events: Mouse Trap Vehicle, Game On, Mission Possible, ExpD, Duct Tape Challenge
2019 events: Mouse Trap Vehicle, Sounds of Music, Mission Possible, ExpD, Wright Stuff, WIDI
2020 events: Gravity Vehicle. ExpD, WIDI, Sounds of Music, Machines

JonB
Coach
Coach
Posts: 303
Joined: March 11th, 2014, 12:00 pm
Division: C
State: FL

Re: Science Olympiad at Penn Invitational 2019

Post by JonB » February 18th, 2019, 11:22 am

terence.tan wrote:have the coaches received the blank test and keys?

No, only the test packets that were completed.

User avatar
winchesetr
Member
Member
Posts: 31
Joined: May 6th, 2014, 7:28 am
Division: Grad
State: PA

Re: Science Olympiad at Penn Invitational 2019

Post by winchesetr » February 18th, 2019, 11:33 am

Raleway wrote:
GoldenKnight1 wrote:
Raleway wrote:Thoughts on events:
Codebusters: ...understanding that SOUP does not have full control over printing copies of tests).
I don't understand what you mean by they don't have full control over printing the tests? This issue of only have a set of tests for each time block seems to be a common thing with many university held invitationals. If it is a money thing I am sure most teams would be willing to add a few extra dollars to the registration fee to cover the cost. If it is an environmental thing then why not copy the first half and second half of the test in two separate packets.

And regarding Codebusters specifically, with just a little change in format you could have used the test packet as the answer sheet, not had to print an answer sheet, and thus used got to write on the test with using almost the same amount of paper.
I have no clue; the proctors I spoke with said they didn't hold control over that (might be an oversight issue). However, the second suggestion is quite valid - I hope more people see that and use it.
Yeah, event supervisors did not really have control over the format of the tests, in terms of how printing was run (how many copies of tests were made). I'm not part of finance committee who makes these decisions, but I think it's important to put some things into context because running a tournament is a lot more difficult than it seems as a competitor.

For context, printing at Penn is 10 cents per page (for a double-sided piece of paper). Say the average exam length is 10 pages for a test an answer key combined, and you print 50 exams, that's ~$1,200. This doesn't include the fact that some things need to be printed in color (more expensive), or the fact that some of the exams are a lot longer than that, or any other things that need to be printed single-sided (more expensive). If this cost was pushed onto teams, that would be ~$23 more for a team to register. So any way possible to minimize the amount needed to print was taken. This clearly doesn't work well for all events (you can't for Disease since all teams take the exam at once), and perhaps code-busters should not have been run that way either.

Needless to say, the Exec board reads the forums and are definitely taking these concerns into consideration for next year. As was said last year on the 2018 forums, rooms are also difficult since Penn doesn't give room confirmations until a short time before the tournament. But Exec really does do their best to get the most ideal rooms and they put a lot of work and effort into that.

In terms of inconsistent grading, that's definitely something that should be noted by all SOUP ES for the future. For Disease we tried to have the same people grade the same questions, to avoid that issue. I also tried to make sure that volunteers that were unfamiliar with the event graded the easier parts that had more objective answers (Sections 3 and 4), while my partner and I graded the specific case-studies (Sections 1 and 2). Any questions on specific questions that the volunteers had we were able to answer. Some of the Section 1s were graded by other people, but I tried to check and make sure that they were consistent with my grading pattern. Any teams that were really close together were also regraded and scores re-added for consistency. Did we make some mistakes? Probably. But we tried our best to be fair all around.
I like soup.

Harriton High School Class of 2017


SOUP Disease Detectives 2018-Present
DUSO Disease Detectives 2019-Present

User avatar
windu34
Moderator
Moderator
Posts: 1354
Joined: April 19th, 2015, 6:37 pm
Division: Grad
State: FL
Location: Gainesville, Florida

Re: Science Olympiad at Penn Invitational 2019

Post by windu34 » February 18th, 2019, 11:44 am

winchesetr wrote: For context, printing at Penn is 10 cents per page (for a double-sided piece of paper). Say the average exam length is 10 pages for a test an answer key combined, and you print 50 exams, that's ~$1,200. This doesn't include the fact that some things need to be printed in color (more expensive), or the fact that some of the exams are a lot longer than that, or any other things that need to be printed single-sided (more expensive). If this cost was pushed onto teams, that would be ~$23 more for a team to register. So any way possible to minimize the amount needed to print was taken. This clearly doesn't work well for all events (you can't for Disease since all teams take the exam at once), and perhaps code-busters should not have been run that way either.
Printing a single test set with just enough tests for each time block and then separate answer sheets for every team is the system used by nearly every college-run invitational and most national supervisors. Excessive printing should be avoided when possible to do without sacrificing event quality. There will be some events that this is not appropriate for, but for the most part for most events, this is the best practice.
winchesetr wrote: For Disease we tried to have the same people grade the same questions, to avoid that issue. I also tried to make sure that volunteers that were unfamiliar with the event graded the easier parts that had more objective answers (Sections 3 and 4), while my partner and I graded the specific case-studies (Sections 1 and 2). Any questions on specific questions that the volunteers had we were able to answer. Some of the Section 1s were graded by other people, but I tried to check and make sure that they were consistent with my grading pattern. Any teams that were really close together were also regraded and scores re-added for consistency. Did we make some mistakes? Probably. But we tried our best to be fair all around.
This is perfect and exactly how I always have my exams graded, including rescoring top ~8 if the spread isnt super large between raw scores. There will ALWAYS be grading errors, but this strategy for grading is widely regarded as the most optimal and there is nothing else an ES can do to ensure accurate grading.
Boca Raton Community High School Alumni
Florida State Tournament Director 2020
National Physical Sciences Rules Committee Member
kevin@floridascienceolympiad.org || windu34's Userpage

Circuit Lab Event Supervisor for 2020: UT Austin (B/C), MIT (C), Solon (C), Princeton (C), Golden Gate (C), Nationals (C)

User avatar
klastyioer
Member
Member
Posts: 422
Joined: April 22nd, 2018, 4:46 pm
Division: C
State: PA

Re: Science Olympiad at Penn Invitational 2019

Post by klastyioer » February 18th, 2019, 12:08 pm

i wanna give some feedback on ws

so i think the supervision was okay
there were some things that can def be improved. there were like a lot of supervisors, like a lot. i liked how they gave us time to trim b4 our slot and in between other ppls flights when no one was there, it was rly helpful to get used to that enviroment. i do think that they shouldve taken it more seriously though. they were pretty lenient on a lot of the things. i also think the vents were on slightly in the beg.? but im not sure. they were all v nice and helpful when it came to informing ppl on how much time was left and resolving any conflicts. they did touch our plane however and snapped it sooooooo.... for future references always let the students handle the planes. otherwise, it was a pretty good experience.
it's not about the medals; go out there and have fun. make progress, learn a few things, have one heck of a time, because that's all that matters.

Builder Cult >:)

'17 - Towers, WS, rocks
'18 - Towers, WS, Mystery Arch, road
'19 - WS
'20 - WS, Boomilever, PPP

GoldenKnight1
Coach
Coach
Posts: 216
Joined: May 2nd, 2009, 5:02 pm
Division: Grad
State: PA

Re: Science Olympiad at Penn Invitational 2019

Post by GoldenKnight1 » February 18th, 2019, 12:50 pm

windu34 wrote:
winchesetr wrote: For context, printing at Penn is 10 cents per page (for a double-sided piece of paper). Say the average exam length is 10 pages for a test an answer key combined, and you print 50 exams, that's ~$1,200. This doesn't include the fact that some things need to be printed in color (more expensive), or the fact that some of the exams are a lot longer than that, or any other things that need to be printed single-sided (more expensive). If this cost was pushed onto teams, that would be ~$23 more for a team to register. So any way possible to minimize the amount needed to print was taken. This clearly doesn't work well for all events (you can't for Disease since all teams take the exam at once), and perhaps code-busters should not have been run that way either.
Printing a single test set with just enough tests for each time block and then separate answer sheets for every team is the system used by nearly every college-run invitational and most national supervisors. Excessive printing should be avoided when possible to do without sacrificing event quality. There will be some events that this is not appropriate for, but for the most part for most events, this is the best practice.
I don't think an additional $25 per team would stop teams from attending. It is hard to imagine that Troy, Boca, New Trier, Brookwood, or Mason to name a few who traveled so far paying for bus, plane, and hotels would have this extra $25 as a deal breaker. Also for most of the non-university held invitationals we have attended it is the expectation that the event supervisor makes 1 copy per team. This cost is something that the supervisor (or their team), in addition to writing the test, has to absorb. If at a university's invitational I don't need to print anything that I am willing to pay more since I am not paying for that on our end. Additionally if I think about all the time and stress that I am saved by not only not having to write a test but also not needing to even judge an event I am happy to pay this extra amount. But that is just me. This could be easily asked of the coaches' attending to get their view of it.

User avatar
winchesetr
Member
Member
Posts: 31
Joined: May 6th, 2014, 7:28 am
Division: Grad
State: PA

Re: Science Olympiad at Penn Invitational 2019

Post by winchesetr » February 18th, 2019, 1:47 pm

GoldenKnight1 wrote:
windu34 wrote:
winchesetr wrote: For context, printing at Penn is 10 cents per page (for a double-sided piece of paper). Say the average exam length is 10 pages for a test an answer key combined, and you print 50 exams, that's ~$1,200. This doesn't include the fact that some things need to be printed in color (more expensive), or the fact that some of the exams are a lot longer than that, or any other things that need to be printed single-sided (more expensive). If this cost was pushed onto teams, that would be ~$23 more for a team to register. So any way possible to minimize the amount needed to print was taken. This clearly doesn't work well for all events (you can't for Disease since all teams take the exam at once), and perhaps code-busters should not have been run that way either.
Printing a single test set with just enough tests for each time block and then separate answer sheets for every team is the system used by nearly every college-run invitational and most national supervisors. Excessive printing should be avoided when possible to do without sacrificing event quality. There will be some events that this is not appropriate for, but for the most part for most events, this is the best practice.
I don't think an additional $25 per team would stop teams from attending. It is hard to imagine that Troy, Boca, New Trier, Brookwood, or Mason to name a few who traveled so far paying for bus, plane, and hotels would have this extra $25 as a deal breaker. Also for most of the non-university held invitationals we have attended it is the expectation that the event supervisor makes 1 copy per team. This cost is something that the supervisor (or their team), in addition to writing the test, has to absorb. If at a university's invitational I don't need to print anything that I am willing to pay more since I am not paying for that on our end. Additionally if I think about all the time and stress that I am saved by not only not having to write a test but also not needing to even judge an event I am happy to pay this extra amount. But that is just me. This could be easily asked of the coaches' attending to get their view of it.
Sure, perhaps it's not that big of a deal and some teams would be willing to absorb the cost. However I think that one of the benefits of SOUP is that it aims to be accessible to a wide variety of teams, including teams that are not as well funded as some of the Nationals teams. Princeton (if I'm not wrong) has waived the tournament fee entirely, which I think is extremely cool because Science Olympiad should be accessible to everyone. Penn has tried to keep our fees low (in comparison to say, MIT, or even local invitationals such as Battle of Valley Forge). Also I think costs incurred by high schools when they write tests and print them are lower for many schools, because in many districts (at least in this area), printing is free. Again, would some exams benefit from having a copy for every team? Sure, but it's also important to take into consideration the cost impact that has on the tournament, accessibility for non-wealthy schools, and also (as was mentioned earlier), the environmental impact.

I understand the frustration some competitors have with that set-up - it's not ideal. But all teams get the same disadvantage. Also, as a former competitor, for many events this method works pretty well. Tests can still be taken apart and split between competitors. Some events should probably not be designed in such a manner, but I think that there should probably be an Official Rules change for that event, or learned through feedback such as this, rather than printing an excess when an excess is unnecessary. Just my thoughts.
I like soup.

Harriton High School Class of 2017


SOUP Disease Detectives 2018-Present
DUSO Disease Detectives 2019-Present

zcgolf16
Member
Member
Posts: 1
Joined: February 18th, 2019, 2:00 pm
Division: C
State: PA

Re: Science Olympiad at Penn Invitational 2019

Post by zcgolf16 » February 18th, 2019, 2:11 pm

My initial thoughts:

Disease Detectives: It was fine. My one comment would be that I don't see the point of a test that is "not possible to finish". It prevents teams from getting in competition practice on all aspects and topics of the event.

Thermo: Didn't have any problems here. Well-written test and the event supervisors did well handling everyone's box from what I saw.

Fermi: Again, no problems here.

Mission: This is where I had some issues. I unfortunately had to go over the 30 minutes of set up time by about 5 minutes after accidentally triggering my machine and having to set it back up, but the event supervisors started testing other teams' machines, including a team that arrived after I was ready, and it took about 15-20 minutes after I was ready for them to get to me. At this point, my partner had to go to another event so I was the sole representative for my team, which was unfortunate and put us at a slight disadvantage. Everything ultimately worked out, but this was a little annoying.

In all, I thought everyone at SOUP did a good job running the tournament and it was a great day. Thanks to all of the event supervisors!

GoldenKnight1
Coach
Coach
Posts: 216
Joined: May 2nd, 2009, 5:02 pm
Division: Grad
State: PA

Re: Science Olympiad at Penn Invitational 2019

Post by GoldenKnight1 » February 18th, 2019, 4:25 pm

zcgolf16 wrote:My one comment would be that I don't see the point of a test that is "not possible to finish". It prevents teams from getting in competition practice on all aspects and topics of the event.
This is again why I wish teams had been able to separate the tests. I have read so many times about hardworking division D SO members who work hard on making fantastic test questions but who often don't know how to edit their test. If the top teams are only seeing about half your test because they are running out of time then what is the point of including as much as you did. And if the main reason they are not getting to it is because they were not allowed to separate the test then that should be considered when designing the test. This idea of trying to save money on copying is why we have seen at some competitions the Experiment Design packet not used or, worse still, double sided.
winchesetr wrote:Sure, perhaps it's not that big of a deal and some teams would be willing to absorb the cost. However I think that one of the benefits of SOUP is that it aims to be accessible to a wide variety of teams, including teams that are not as well funded as some of the Nationals teams. Princeton (if I'm not wrong) has waived the tournament fee entirely, which I think is extremely cool because Science Olympiad should be accessible to everyone. Penn has tried to keep our fees low (in comparison to say, MIT, or even local invitationals such as Battle of Valley Forge). Also I think costs incurred by high schools when they write tests and print them are lower for many schools, because in many districts (at least in this area), printing is free. Again, would some exams benefit from having a copy for every team? Sure, but it's also important to take into consideration the cost impact that has on the tournament, accessibility for non-wealthy schools, and also (as was mentioned earlier), the environmental impact.
Free? That is not my school. It comes directly out of our Science Department budget coded as I was the one who made the copies. Many supplies for events also work the same way. Maybe that is just my building though.

Look, I get the desire to keep cost down the Princeton's free registration makes me very reluctant to complain about their copying practices. If you want to keep the cost and environmental impact low why not copy the first half and second half of the test in two separate packets. That keeps the number of copied pages the same but allows the students an easier time working through the test. If there is a reason not to do it this way I would be interested in hearing it but my instinct on this one is just that it has not been considered.

User avatar
winchesetr
Member
Member
Posts: 31
Joined: May 6th, 2014, 7:28 am
Division: Grad
State: PA

Re: Science Olympiad at Penn Invitational 2019

Post by winchesetr » February 18th, 2019, 5:18 pm

GoldenKnight1 wrote:
zcgolf16 wrote:My one comment would be that I don't see the point of a test that is "not possible to finish". It prevents teams from getting in competition practice on all aspects and topics of the event.
This is again why I wish teams had been able to separate the tests. I have read so many times about hardworking division D SO members who work hard on making fantastic test questions but who often don't know how to edit their test. If the top teams are only seeing about half your test because they are running out of time then what is the point of including as much as you did. And if the main reason they are not getting to it is because they were not allowed to separate the test then that should be considered when designing the test. This idea of trying to save money on copying is why we have seen at some competitions the Experiment Design packet not used or, worse still, double sided.
winchesetr wrote:Sure, perhaps it's not that big of a deal and some teams would be willing to absorb the cost. However I think that one of the benefits of SOUP is that it aims to be accessible to a wide variety of teams, including teams that are not as well funded as some of the Nationals teams. Princeton (if I'm not wrong) has waived the tournament fee entirely, which I think is extremely cool because Science Olympiad should be accessible to everyone. Penn has tried to keep our fees low (in comparison to say, MIT, or even local invitationals such as Battle of Valley Forge). Also I think costs incurred by high schools when they write tests and print them are lower for many schools, because in many districts (at least in this area), printing is free. Again, would some exams benefit from having a copy for every team? Sure, but it's also important to take into consideration the cost impact that has on the tournament, accessibility for non-wealthy schools, and also (as was mentioned earlier), the environmental impact.
Free? That is not my school. It comes directly out of our Science Department budget coded as I was the one who made the copies. Many supplies for events also work the same way. Maybe that is just my building though.

Look, I get the desire to keep cost down the Princeton's free registration makes me very reluctant to complain about their copying practices. If you want to keep the cost and environmental impact low why not copy the first half and second half of the test in two separate packets. That keeps the number of copied pages the same but allows the students an easier time working through the test. If there is a reason not to do it this way I would be interested in hearing it but my instinct on this one is just that it has not been considered.
Teams were not allowed to separate their exams? That's news to me. We had staplers accessible, so teams should have been allowed to separate the exams even if they were not able to write on them. I'm not sure I understand your criticism entirely, or what you are suggesting. Most invitationals where test packets are reused allow test separation, but just not writing on the exam booklets. That's definitely an issue that can be addressed.

Regardless, to address your second point, all teams in Disease were allowed to separate their tests. We had a stapler and made that clear at the beginning of the exam. The length of the test was intended to be a comprehensive analysis. We have 3 reasons for the length.

1. The nationals test is long. In fact, many teams do not finish the Nats exam. That doesn't make the exam useless. Our test was longer than nationals has been for a few years (though I recall my freshman year of hs being longer) but not severely, and certainly not where it counted. Our case analyses were formatted in the style of a nationals exam, with some personal stylization. Section 4 is a unique throw in that I try to include because it's less analysis and more memorized facts, as some teams might not know how to analyze as well as other teams. This is due to the fact that many ES at other competitions do not format their exams in a Nats style.

2. The test allows for a good separation of scores. Part of Science Olympiad is also learning how to take a test well (which is a very useful skill for college). I could show you the distribution if you are interested, but it's approximately normal. Top teams were able to excel, even if they didn't finish. In my opinion, it is better to have a too long exam, then one in which no score separation is obtained. The length also allows teams that are less familiar with the event to still acquire points and be competitive.

3. The point of the length (and the difficulty) is to also be able to take the exam home and learn from it. Stuff you were not able to finish, you are able to finish later and improve from. Tests are supposed to be a resource in this manner, especially for teams that may not know as much.

I will say this. Despite the length, teams were able to do well and accumulate a good amount of points. The length of the exam was approx. the same as last year's exam, and I doubt that we will be changing the length any time soon, though I appreciate the feedback.

EDIT: Just to clarify, the test was not super difficult, it was just long. There were difficult questions (analyses of why you would do something), and there were easy ones (what is this type of study. benefits, drawbacks. Who popularized the spot map. list the 5 steps of surveillance). I would say that the easy questions outweighed the hard ones by a large amount, and that the easier questions made up 75% of the exam. Again, I reiterate, people were able to get a good amount of points and do well. Disease is one of the events where it is important to test actual science as opposed to memorization, which is done through case-study analysis. That's the way it should be run. My exams are just longer than most, and still attain a good score distribution.
Last edited by winchesetr on February 18th, 2019, 6:22 pm, edited 1 time in total.
I like soup.

Harriton High School Class of 2017


SOUP Disease Detectives 2018-Present
DUSO Disease Detectives 2019-Present

Post Reply

Return to “2019 Invitationals”

Who is online

Users browsing this forum: No registered users and 1 guest