Tournaments Run with Test Writing Teams

For anything Science Olympiad-related that might not fall under a specific event or competition.
Post Reply
User avatar
Umaroth
Exalted Member
Exalted Member
Posts: 398
Joined: February 10th, 2018, 8:51 pm
Division: C
State: CA
Pronouns: He/Him/His
Has thanked: 167 times
Been thanked: 325 times

Tournaments Run with Test Writing Teams

Post by Umaroth »

Invitational tests have always been a hot topic around here, whether it's quality, how well-run the event is, or the t-word of the forums. There has been a lot of discussion about how everything should be run, so I am going to offer my observations from helping run Jeffrey Trail B Invitational and try to generate some discussion about having test-writing teams for invitationals.

Currently, the most popular options are as follows:
  • Having each team at an invitational run an event is by far the most common method of running invitational events. It makes it easier logistically for the invitational host, but does put some burden on attending teams. It is also common to see low quality tests where the team writing ends up winning or placing highly in the event, or the point differences between the top positions are mere decimals. While it is not the most ideal method, it is certainly much easier to accomplish than the other two I will list.
  • The invitational hosts write all the tests. This is done at some invitationals; from personal experience, many of the tests at Churchill B and SOLVI C were written by their students or their alumni. It tends to improve upon the issue of low quality tests because the writers are more knowledgeable in the event, but comes with the downside of the students not being able to compete in the events. Of course it is not a problem if it is alumni writing like Div C kids writing for a B invitational, but at SOLVI, the invitational was run by the student organization (very well if I may add), thus preventing them, or at least their main teams, from competing themselves. It also does provide a bit more burden to the hosts having to prepare everything themselves.
  • Most commonly used by university undergrad-led invitationals is the method of having a designated team of test-writers who will prepare each event. For this, most invitationals have a committee for selecting writers and a review committee to ensure that each test is written according to standards. This, of course, puts a lot more in the hands of the tournament directors to ensure that everything runs smoothly and the tests get in on time. This typically leads to fairly high quality events with the slight risk of it being too hard or too off-topic if the event isn't vetted well. This will be the focus of this post.
Having a test-writing committee was mostly reserved for university invitationals like MIT, GGSO, or UT with plenty of undergrad students who may or may not have had SciOly experience who know what a good test looks like. This year, with the increase in active Div D squads and even Div C volunteers, a lot more non-university invitationals began to experiment with having events completely organized by separate writing groups. From my knowledge, the main ones this year were Eagle View B, Jeffrey Trail B, Raymond Park BC, Rustin B, and Solon C. Since I was in charge of this process for Jeffrey Trail B, that will be the basis of the rest of this post.

For Jeffrey Trail, we definitely lucked out on many things. I described a few of them in my post on the Jeffrey Trail/Kraemer Invitational thread, but I will restate the most relevant ones here:
  • Our original plan was to have a team of test writers, many of whom based out-of-state, to prepare tests and communicate with us for what they need. Each school was assigned an event and the coaches communicated with the writers so the writers could give them instructions for getting materials, printing the tests, and grading them at the competition. We got lucky that one of the team member's parents offered to sponsor us by printing the tests at their print office, meaning that the "host schools" would only need to purchase any materials that were specifically requested by the writers.
  • The materials costs for the host schools was cut down by being able to use materials from Portola HS, where the invitational was hosted. We were able to use some of their equipment for the lab events when needed and also reprint a page on each of the Dynamic Planet test packets because there was a big mistake on it (for some reason the answers on the MC were bolded).
  • Jeffrey Trail was one of the biggest invitationals, with 73 teams registered and 69 there (one team pulled out last minute because of COVID-19 concerns). It was difficult fitting so many teams into rooms for events, but luckily all the rooms in each building were connected, allowing for proctors to roam around more easily.
  • We had a ton of volunteers, probably upwards of 40 including many Div C competitors from various local schools and community volunteers such as PacificGoldenPlover with his amazing Ornithology test. These volunteers really came through, helping set up and run each event much more smoothly than I have seen at many invitationals and helping get grading done so fast that we had to cut the planned stalling short for the awards ceremony. We also had a lot of contacts to help write the tests courtesy of Mr. Evola and myself.
Our main goals in this experiment were to improve the experience for attending coaches since they don't have to do much in the way of preparing events, maintain high quality for our events and tests, and make sure that the students get the experience of having a fairly challenging test to give good practice for regional and state level tournaments. From my point of view it worked very well as a proof of concept. The tests were definitely challenging, although some may have found them absurd (sorry for that super troll test for Codebusters lol). We were able to get many state and nationals medalists as well as professionals to write the tests, including the national champion for Heredity, national champion and runner-up for Meteorology, and a member of the A-Team and a IOAA medalist. There were some hiccups with some teams not checking in with us in the morning and some misplaced packets of tests, but overall it went much more smoothly than previous years.

Obviously, this model may not be easily achieved by all tournaments because of a lack of one of those key bullet points mentioned above, particularly contact with volunteers to write and run events. However, I do foresee this issue getting a bit easier to handle with the advent of many alumni test writing teams like the A-Team, pika's 4N6 team, heck even DMAH at this point is becoming a test writing team having written a lot of tests for many of the invitationals I listed above. At that point after finding people from these groups to help write, the only part left is to get volunteers to help run the events.

There are two main reasons I bring this up right now:
  1. We are still unsure of what next season will bring to the table, but we can be certain that if this pandemic persists, there will be a great impact on SciOly tournaments. With this experience of having had tests written through online communication, we open up many doors that could potentially help in our search for a viable solution that will allow us to have a 2021 season. Beyond next season, the amount of students entering Div D increases each year, resulting in more opportunities for alumni to help out with local competitions. Their experience from competing would greatly benefit their SciOly community and could help with invitationals all over the place.
  2. There are many discussions about private test-trading (ooooh I said the t-word), particularly about whether or not invitationals should release tests publicly. With these newly founded groups of test writers, many of the tests they write are publicized on their websites or scioly wiki pages. This makes it much easier for tournaments to release materials as all they have to do is give a link to the writer's website or something of the sort. While many tournament directors see these tests as the reason that teams want to come to the invitational, having good quality tests that will later be released can change the priority of attendees from getting the tests to getting a good competition experience. Call me an idealist, but I feel like it would attract more teams than it would repel, as seen by MIT Invitational or GGSO.
These are just my thoughts on this subject that I think could be of higher importance in the coming years. Please let me know all of your thoughts on it, this thread could potentially give inspiration and guidance for many tournament hosts who want to experiment with this idea. I would love to hear the perspectives of others who have hosted tournaments with various types of organization.
These users thanked the author Umaroth for the post:
Unome (May 31st, 2020, 4:03 pm)
Cal 2026
Troy SciOly 2021 Co-Captain
Proud Padre of the Evola SciOly Program 2018-now
Dank Memes Area Homeschool Juggernaut 2018-now
Sierra Vista SciOly Co-Head Coach 2020-now

Umaroth's Userpage
User avatar
SilverBreeze
Exalted Member
Exalted Member
Posts: 306
Joined: November 28th, 2019, 3:42 pm
Division: C
State: CA
Has thanked: 156 times
Been thanked: 289 times

Re: Tournaments Run with Test Writing Teams

Post by SilverBreeze »

On the topic of access to tests:
In middle school, one year we registered for an invitational for the sake of obtaining a set of tests. We told them beforehand we would not be attending. However, I can't vouch for how commonplace this practice is for a few reasons:
  • We were not a powerhouse and did not have anything resembling a test bank, and coaches were not familiar with public sources of tests
  • Our team participated in SciOly mostly for fun, and many team members lacked the dedication to attend more than one invitational a year(hey, SciOly is time-consuming), which was one of the reasons the coaches pursued that option in the first place
  • That was our first year experimenting with non-fixed teams, i.e., A and B teams were not assigned until it was closer to regionals, and coaches needed a way of differentiating teams that had scored closely at the invitational we attended
My personal take, based on my own experience, is that publicly-released tests provide a way for dedicated students that are part of non-powerhouses to access resources. Additionally, I've found attending an invitational to be a far better method of preparing for competitions than taking practice tests at home, since you can practice testing in an unfamiliar setting and learn to work around obstacles and mishaps that occur right before or during your event. Personally I have also found practice tests to provide an advantage, but often not nearly as much as people tend to imply. Speaking as someone who has competed both with no access to a test bank (nor knowledge of public test sets) and with access to one, I have found that the time you choose to put into studying from randomly googled sources makes a far greater impact on results than whatever resources your team provides you(though if your team does, you should definitely use them).
Troy SciOly 2019 - 2023
Captain 2021-2023
Former Events: Ecology, Water Quality, Green Gen, Ornithology, Forestry, Disease Detectives, Forensics, Chem Lab, Env Chem, Sounds, Dynamic Planet, Crime Busters, Potions & Poisons, Exp Design, Towers, Mystery Arch, Reach for the Stars, Mission Possible
User avatar
sciolyperson1
Exalted Member
Exalted Member
Posts: 1074
Joined: April 23rd, 2018, 7:13 pm
Division: C
State: NJ
Pronouns: He/Him/His
Has thanked: 529 times
Been thanked: 601 times
Contact:

Re: Tournaments Run with Test Writing Teams

Post by sciolyperson1 »

Umaroth wrote: May 31st, 2020, 1:55 am There are many discussions about private test-trading (ooooh I said the t-word), particularly about whether or not invitationals should release tests publicly. With these newly founded groups of test writers, many of the tests they write are publicized on their websites or scioly wiki pages. This makes it much easier for tournaments to release materials as all they have to do is give a link to the writer's website or something of the sort. While many tournament directors see these tests as the reason that teams want to come to the invitational, having good quality tests that will later be released can change the priority of attendees from getting the tests to getting a good competition experience. Call me an idealist, but I feel like it would attract more teams than it would repel, as seen by MIT Invitational or GGSO.[/list]
I agree with you that the invitationals that publicly release their test sets attract more teams than they repel; they make the invitational more well known to teams that are unable to attend and travel to invitationals (many people from small teams pop in in IRC or discord to ask for public test sets to practice with). Even an invitational that just started, Langley, has been getting a bit more attention than other invitationals around the region, due to them releasing their tests (although I don't know of their quality).

Releasing tests worked especially well for MIT, where tests are known to be extremely hard and long, which teams use as practice tests. Not only do they now have one of the only lottery systems running for an invitational due to 70 teams registering in just 2 minutes, their invitational is seen as a national level tournament with teams travelling from California, Texas, and Washington.

I do think an invitational like Jeffrey Trail would benefit from publicly releasing tests. Not only are there a huge amount of Californian teams that do not have access to invitationals and invitational released test sets, but the test sets that they would be able to get would be of higher quality than most other tournaments around the region.
SoCal Planning Team & BirdSO Tournament Director
WW-P HSN '22, Community MS '18
Sciolyperson1's Userpage
pb5754
Exalted Member
Exalted Member
Posts: 518
Joined: March 5th, 2017, 7:49 pm
Division: C
State: NJ
Pronouns: He/Him/His
Has thanked: 45 times
Been thanked: 85 times

Re: Tournaments Run with Test Writing Teams

Post by pb5754 »

sciolyperson1 wrote: May 31st, 2020, 8:10 am
Umaroth wrote: May 31st, 2020, 1:55 am There are many discussions about private test-trading (ooooh I said the t-word), particularly about whether or not invitationals should release tests publicly. With these newly founded groups of test writers, many of the tests they write are publicized on their websites or scioly wiki pages. This makes it much easier for tournaments to release materials as all they have to do is give a link to the writer's website or something of the sort. While many tournament directors see these tests as the reason that teams want to come to the invitational, having good quality tests that will later be released can change the priority of attendees from getting the tests to getting a good competition experience. Call me an idealist, but I feel like it would attract more teams than it would repel, as seen by MIT Invitational or GGSO.[/list]
Releasing tests worked especially well for MIT, where tests are known to be extremely hard and long, which teams use as practice tests. Not only do they now have one of the only lottery systems running for an invitational due to 70 teams registering in just 2 minutes, their invitational is seen as a national level tournament with teams travelling from California, Texas, and Washington.
To be fair, that was 2 years before MIT adopted its current test release policy. I wouldn't say MIT's policy made its tournament any more popular, as it was already a very popular tournament.
West Windsor-Plainsboro High School South '21
2021 Nationals: Astronomy - 1st, Geologic Mapping - 1st, Team - 6th
knightmoves
Member
Member
Posts: 637
Joined: April 26th, 2018, 6:40 pm
Has thanked: 4 times
Been thanked: 110 times

Re: Tournaments Run with Test Writing Teams

Post by knightmoves »

I know our team finds coming up with enough volunteers to proctor and grade all the tests at an invitational to be a significantly bigger burden than writing the tests.
syo_astro
Exalted Member
Exalted Member
Posts: 621
Joined: December 3rd, 2011, 9:45 pm
Division: Grad
State: NY
Has thanked: 3 times
Been thanked: 20 times
Contact:

Re: Tournaments Run with Test Writing Teams

Post by syo_astro »

I appreciate you bringing this up! One thing I did at UMSO (I was that test organizing type of person too) was to have people elsewhere (other states, another scioly org at MSU) write tests in addition to our own team. It worked out alright for paper and pen ones (we weren't in good contact with MSU, so it's recommended to have better contact developed first if you do collabs like that)...but for labs and IDs, it's probably better to have someone locally write it (that's obviously a privilege not every tournament has, and we struggled getting lab access on campus). I can't comment on other tournies, but I can share about that, A-team, and SSSS if people were starting anything:).
Umaroth wrote: May 31st, 2020, 1:55 am There are many discussions about private test-trading (ooooh I said the t-word)...
Probably only need a brief explanation: Where did the "t-word curse" come from? Not meaning this personally, maybe I don't visit enough (:/)...test trading isn't a secret, and I don't think there's any issue discussing the *reasons*, etc. Though, it tends to lead to threads in excess of 30 pages with little progress so far...

I think the "ban" (that I believe we all even agreed on) was *active* private test trading/promoting. I think this was in a thread with gz839918, and there was one extra aspect I personally consider toxic competitiveness (which I'm aware people disagree, maybe I explain badly/rudely, I'd have to check). It was suggested focus trading outside your state so you aren't in direct competition. I think SSSS was brought up as an example, but SSSS is open to all states. Don't limit collaborations because of competition! (was my point)

And see^, I'm ranting, why I don't typically get into it XD. Your actual question was on test release, and there has been threads about that too! Might take a bit to find, but I recall nicholasmaurer explaining that multiple invites hosted by teams commonly want to retain competitive advantage, and no amount of arguing is likely to change that aside from simply attracting teams to the university-run invites. The plus of this is we get our test release, the downside is that some of these invites are in well-established locations, which means it can make it harder on some teams if moved...whether that's good or bad for teams that are relatively underserved, I don't know, depends locally I'm sure.

I have no clue whether releasing tests promotes your invite (I'm sure there's some teams that don't check and that you'd need to reach through other means), but it definitely doesn't seem to *hurt* it (I'm not there now...but we ran UMSO that way from the *start*, and they now not only retained teams but also *expanded* to div B). I can't comment on actual impacts on competition, but I assume there are numerous other factors than test stockpiles that contribute to team success...that's an interesting study if anyone has data on it, I could be wrong, but again, I don't think it would change the minds of some.
Last edited by syo_astro on June 30th, 2020, 6:34 pm, edited 3 times in total.
These users thanked the author syo_astro for the post:
Umaroth (June 30th, 2020, 7:45 pm)
B: Crave the Wave, Environmental Chemistry, Robo-Cross, Meteo, Phys Sci Lab, Solar System, DyPlan (E and V), Shock Value
C: Microbe Mission, DyPlan (Fresh Waters), Fermi Questions, GeoMaps, Grav Vehicle, Scrambler, Rocks, Astro
Grad: Writing Tests/Supervising (NY/MI)
Post Reply

Return to “General Competition”

Who is online

Users browsing this forum: No registered users and 11 guests