Test Set Exchange Update

User avatar
EastStroudsburg13
Administrator
Administrator
Posts: 3000
Joined: January 17th, 2009, 7:32 am
Division: Grad
State: MD
Location: At work trying to be a real adult
Contact:

Re: Test Set Exchange Update

Postby EastStroudsburg13 » February 21st, 2019, 8:10 pm

windu34 wrote:
ScottMaurer19 wrote:
lumosityfan wrote:As I mentioned earlier, teams don't go to invitationals for the tests. They go there for the experience. The tests simply happen to be a good practice platform that the teams can use for further analysis and reference. Also just because it's unrealistic for every tournament to release tests doesn't mean that we can't do our part. It will hopefully start a precedent that will eventually reach us to that point. And those teams that couldn't attend tournaments would love to pay for those teams. However, they simply can't because they're so new that they realistically can't reach that point in which they can go to invies constantly until maybe Year 3 or 4. Until then those tests will give them a nice starting point so that they don't go into regionals and states totally unprepared.

I know I'm biased, but I'm with Nick. Releasing tests has positive consequences for new teams, but also realize SO is a competition. Yes it's about learning, exploring, and building connections within science. That's great and SO has given me lots of opportunities and knowledge that I wouldn't have had without it. That being said I'm not only here to play nice with other teams and help them. As Nick said Solon fully supports new teams wishing to attend our invitational, but you must also think about experienced teams. You are neglecting their perspective and assume that the only effect of releasing all tests would be for new teams to gain resources whereas in reality experienced teams are losing an edge on competition against their primary competitors as well (ignoring the new teams as competition in the short run). How would people feel if Troy stopped attending invitationals and just asked teams for tests from their invitationals? I guarentee you there would be few teams willing to just hand them tests.

I know this sounds cut throat and maybe even a bit mean, but this is the reality of the program.

Im with Nick and Scott on this one. Teams are free to go to whichever tournaments they want. If you dont like the way one tournament is with their tests/principles, go to a different tournament. I think tournament directors should be free to do whatever they want with their exams and resources.
As a regional tournament director myself, I make all of our exams public immediately after the tournament, but i have an additional motive - I WANT other teams to take our exams, especially those in Florida, realize that our tournament can actually follow the rules for each event, and then choose to come attend here. When I supervise ANYWHERE (including Florida regionals and states, MIT, Princeton, etc), I make certain that my test is made public whether or not the tournament venue wants that or not. I wrote the test - it is my intellectual property and I can choose to share it if I want to.

I can't say that I think it is a GOOD thing for invites to not release exams publically, but I also believe in free-market capitalism and forcing tournaments to share their exams so that "every team has an equal advantage" sounds awfully socialist to me...

Trying to compare SO to a free capitalist market is a very dangerous road to go down. This is, at its foremost, an educational activity for teenagers.
East Stroudsburg South Class of 2012, Alumnus of JT Lambert, Drexel University Class of 2017

Helpful Links
Wiki
Wiki Pages that Need Work
FAQ and SciOly FAQ Wiki
Chat (See IRC Wiki for more info)
BBCode Wiki


If you have any questions for me, always feel free to shoot me a PM.

syo_astro
Exalted Member
Exalted Member
Posts: 591
Joined: December 3rd, 2011, 9:45 pm
Division: Grad
State: NY
Contact:

Re: Test Set Exchange Update

Postby syo_astro » February 21st, 2019, 9:40 pm

I don't think a capitalism vs. socialism debate should be had on this thread unless that's REALLY necessary...I don't think it would end well. Anyway, I don't think the sides will see eye to eye exactly without being more informed. For me, it's more fruitful to then ask the benefits of competitiveness with tests.

What follows from here is me honestly being confused. I'm aware there are other benefits to invites, but clearly I don't get the importance of these tests. Hopefully not dumb to ask...I'm going to type a lot, but to nicholasmaurer, etc feel free to give a brief answer if it's obvious (a lot of these questions are similar):
People can trade digital copies on a whim, which probably (easily) gets back to the big competitive teams. Here, I'm assuming random "big teams of concern" work hard at test collecting, have a wide enough network, etc...fair?. Is this the biggest fear among the most competitive teams running / attending invites? Here, I'm ignoring fundraising or benefits to the host since that's not an incentive to all participants...meanwhile tests are one of the bigger / biggest incentives to all attendees. Did I get that right? Why not then ONLY give back the paper copies to curb other teams getting your tests? I am aware this has been done in the past. I'm sure teams could still take pictures or make scans, but it would at least slow things down since the pictures would need to be retyped or reformatted to make them clean copies.

How do you draw the line between what helps "other teams" less and what helps "our team" more for either teams that host OR attend? Moreover, if tournaments give people a website, google drive, or flashdrive of tests (maybe not everyone does this...but many do something like this now, right?), then what are you actually doing to curb "helping the competition"? Maybe there's an extra factor here, like those are things that attract good teams, but don't those teams want a competitive edge too?

People have said they basically want to help their team more than other ones, is that right? It sounds like people agree that invites are ran for reasons other than tests like seeing how you stack up vs. other teams, etc etc (am I wrong?). Like, if the only reason for invites were tests I'm sure students could come up with some alternative complex "black market" test trading network between individual coaches (can't believe I typed that lol). In fact, why not do that anyway with or without invites?

To summarize, how do you balance everything above in your value "equation" or whatever metric you use? I'm going to go on loop here, but if invites have to be ran, but you're afraid of tests being traded, why isn't more done to curb test trading? If the argument goes back to "because we want more tests than other teams" or "attendees don't like that", then see above...does test trading make your tests not help the competition at least equally? Is test trading not that widespread among teams you care about as I thought in my first point? Hopefully you see why this at least appears contradictory (or if you prefer extremely convoluted) to me.

OK, now my briefer two cents:
I find it hard to believe anyone can practice or make use of 60 tests. Like, just as far as I know about the practical usefulness of tests from a learning / education perspective or even for notesheets...I guess anything is possible. Maybe I don't have a good estimate of this "test value" considering I haven't competed recently.
B: Crave the Wave, Environmental Chemistry, Robo-Cross, Meteorology, Physical Science Lab, Solar System, DyPlan (E and V), Shock Value
C: Microbe Mission, DyPlan (Earth's Fresh Waters), Fermi Questions, GeoMaps, Gravity Vehicle, Scrambler, Rocks, Astronomy
Grad: Writing Tests/Supervising (NY/MI)

nicholasmaurer
Coach
Coach
Posts: 356
Joined: May 19th, 2017, 10:55 am
Division: Grad
State: OH
Location: Solon, OH

Re: Test Set Exchange Update

Postby nicholasmaurer » February 21st, 2019, 9:53 pm

I agree that there’s a quickly diminishing return from acquiring more tests. If you have a few good ones, you can prepare well. There’s no reason to hoard 60 test sets a year.

To answer your broader question, I don’t think there’s any objection to test trading on our part. We’re not trying to prevent trading. I expect that many teams who attend our invitational will trade those tests for other sets. My point is that if we posted our tests publicly, the teams who attend would have nothing to trade. This seems unfair since those teams also wrote many of the tests and volunteered to run events.

In essence, we’d be saying to participating teams: “Thanks for paying to attend, writing these tests, and grading them for us. We’re now going to share them with everyone regardless of whether they paid or contributed.”
Assistant Coach and Alumni ('14) - Solon High School Science Olympiad
Tournament Director - Northeast Ohio Regional Tournament

Opinions expressed on this site are not official; the only place for official rules changes and FAQs is soinc.org.

syo_astro
Exalted Member
Exalted Member
Posts: 591
Joined: December 3rd, 2011, 9:45 pm
Division: Grad
State: NY
Contact:

Re: Test Set Exchange Update

Postby syo_astro » February 21st, 2019, 10:27 pm

Thank you very much for your answer, sorry that was repetitive. I can see why you won't want to directly post them online if a school wants to host their own invite, has attendees paying, etc.

How are you or the attendees not against test trading even a little bit? (that one is a little rhetorical...read on) I understand the point earlier by you and others that trading gives your test "continued value". But if teams don't like the competition getting hold of their work when someone shares it online, why do they not fear that (or consider it a non-issue) while trading tests? Note: I repeat, I'm absolutely not saying you should post your tests or be in favor of your tests being posted, hopefully this last question is clear then.

But now say 5 invites are leaked online. Aren't the tests overvalued if they can be leaked or obtained so easily? It sounds like it's acknowledged the big teams will get all the tests anyway...? Again, apologies if I'm being ignorant here.
B: Crave the Wave, Environmental Chemistry, Robo-Cross, Meteorology, Physical Science Lab, Solar System, DyPlan (E and V), Shock Value
C: Microbe Mission, DyPlan (Earth's Fresh Waters), Fermi Questions, GeoMaps, Gravity Vehicle, Scrambler, Rocks, Astronomy
Grad: Writing Tests/Supervising (NY/MI)

nicholasmaurer
Coach
Coach
Posts: 356
Joined: May 19th, 2017, 10:55 am
Division: Grad
State: OH
Location: Solon, OH

Re: Test Set Exchange Update

Postby nicholasmaurer » February 22nd, 2019, 6:15 am

syo_astro wrote:Thank you very much for your answer, sorry that was repetitive. I can see why you won't want to directly post them online if a school wants to host their own invite, has attendees paying, etc.

How are you or the attendees not against test trading even a little bit? (that one is a little rhetorical...read on) I understand the point earlier by you and others that trading gives your test "continued value". But if teams don't like the competition getting hold of their work when someone shares it online, why do they not fear that (or consider it a non-issue) while trading tests? Note: I repeat, I'm absolutely not saying you should post your tests or be in favor of your tests being posted, hopefully this last question is clear then.

But now say 5 invites are leaked online. Aren't the tests overvalued if they can be leaked or obtained so easily? It sounds like it's acknowledged the big teams will get all the tests anyway...? Again, apologies if I'm being ignorant here.


I don't think schools expect the tests they contribute to be kept confidential - they know that when you distribute them to the 70 other teams they will be shared, traded, etc. I think the objection is to tests being given away for nothing. If the tests are leaked and posted online as you state in your hypothetical, that is obviously not ideal. But I can tell you based on conversations I have had with coaches, deliberately posting them publicly as the original intent would raise a lot of objections from our participants.
Assistant Coach and Alumni ('14) - Solon High School Science Olympiad
Tournament Director - Northeast Ohio Regional Tournament

Opinions expressed on this site are not official; the only place for official rules changes and FAQs is soinc.org.

Raleway
Exalted Member
Exalted Member
Posts: 228
Joined: March 12th, 2017, 7:19 pm
Division: C
State: -

Re: Test Set Exchange Update

Postby Raleway » February 22nd, 2019, 7:00 am

Just my two cents:

This feels very much like the prisoner's dilemma, especially from the view of the tournament. In the event of publicly releasing tests, should some invitationals strive to keep an advantage over other invitationals, the dominant strategy is to withhold publically releasing tests. This exists for both sides, so the Nash equilibrium (generalizing the many invitationals) is to simply withhold tests and not publically release them. What I'm seeing, and very much enjoying, is that invitationals are realizing that their tests will be traded (even the watermarking was ineffective). Even if teams attend to get exclusive high-level tests, it makes no sense as a trading tool if every single test becomes publically released. At that point, teams will attend invitationals for quality, competition experience, and seeing how teams do against each other. As a competitor (soon to be a Division D member), I would very much want tournament quality to be the foremost priority. The issue of test trading is inherently the fault of invitationals; if every test was made public a week after the invitational, there would be no test trading.

None. Let that sink in. The onus is not on teams. Invitationals exist to allow teams a chance to compete against other teams and promote the spirit of collaborative science in a competitive setting.

I'm going to conclude with my final thoughts that SOINC should step in and require all invitationals register with SOINC and release tests and keys promptly to centralize these resources. The recent addition of "Invitational Ruleset" has been a foray into this but I hope it becomes more useful later. I applaud Princeton for being the first high-level invitational to publicly release tests and set off the next wave of invitationals to publically release tests (MIT, SOUP, etc.). Kudos to the aforementioned invitationals. Please make all tests public. It would save teams countless hours not worrying about tests to get and get rid of this issue once and for all.
Sleep is for the week; one only needs it once a week :!: :geek: :roll: :?: :idea:

God bless Len Joeris | Balsaman

KC1FVS
Member
Member
Posts: 6
Joined: August 18th, 2017, 7:23 am
State: -
Contact:

Re: Test Set Exchange Update

Postby KC1FVS » February 22nd, 2019, 7:33 am

I am glad to hear everyone's take on this; thanks for explaining your thinking.

I think that for those of us who act as both a tournament director and a coach, then our obligation to be impartial should in fact inform the rationale of our test release policies. When deciding who should get our tests after the tournament, we are still acting as directors, not as coaches. Coaches who host or run invitationals should not be using the privilege of that opportunity to wield the advantage of proprietary information for one or more teams. If we are withholding them solely in order to incentivize registration and attendance at our tournament, then I suppose that’s justifiable, but I will explain in a moment why I have concluded that even this rationale is flawed. If the decision to withhold invitational tests from public release is explicitly founded in our desire to disadvantage non-attending teams, then I think we have mixed up our coaching hat with our directing hat.

Our tournament has itself been withholding tests in order to incentivize tournament registration, but I have conlcuded that this practice is misguided. Consider the response I just received from the student whose email initiated my post. When I expressed my hesitation to share the data from our invitational, they stated,

"...nothing can stop me from going to the other schools that attended your invitational and offer them the same deal and I am sure someone will accept it...”

So, we are not in possession of a valuable commodity. Our attempts to limit access will likely be undermined by students such as these. Getting the tests without attending is arguably easier than attending. If we all release our tests, there will be no need for this ridiculousness. The only reason I can see to limit test access is to incentivize tournament attendance, which frankly needs no such incentive. Teams and coaches will still want to attend as many invites as they can.

I am dismayed to find that there is an insider market of test exchanges taking place between teams who are lucky enough to attend several invitationals. Don’t we just want to help all students to learn to do science well and have the best students win on their own merits? Do we want them learning science or hawking information in elaborate schemes to leverage advantage? As coaches, we may be reluctant to surrender the advantage that these test exchanges can give our teams, but as leaders in SO (coaches and directors), we should model good sportsmanship, fair practice, and equity so that the best students can win. The best way to do this is to share information freely. If this results in another team winning, then they have earned their victory and we should congratulate them.

Our tournament has now removed the password protection on our tests, and I hope that other invitationals decide to do the same, regardless of whether they are run by coaches or alumni, for the benefit of ALL students in SO. They are on our site here https://sites.google.com/site/dartmouthscioly/.

User avatar
EastStroudsburg13
Administrator
Administrator
Posts: 3000
Joined: January 17th, 2009, 7:32 am
Division: Grad
State: MD
Location: At work trying to be a real adult
Contact:

Re: Test Set Exchange Update

Postby EastStroudsburg13 » February 22nd, 2019, 8:19 am

Raleway wrote:I'm going to conclude with my final thoughts that SOINC should step in and require all invitationals register with SOINC and release tests and keys promptly to centralize these resources. The recent addition of "Invitational Ruleset" has been a foray into this but I hope it becomes more useful later. I applaud Princeton for being the first high-level invitational to publicly release tests and set off the next wave of invitationals to publically release tests (MIT, SOUP, etc.). Kudos to the aforementioned invitationals. Please make all tests public. It would save teams countless hours not worrying about tests to get and get rid of this issue once and for all.

If I could thumbs up a post, I would. I think they should also do this with state tournament results, but that's just for personal convenience. This test trading issue, in my opinion, is eating away at the spirit and values of Science Olympiad.
East Stroudsburg South Class of 2012, Alumnus of JT Lambert, Drexel University Class of 2017

Helpful Links
Wiki
Wiki Pages that Need Work
FAQ and SciOly FAQ Wiki
Chat (See IRC Wiki for more info)
BBCode Wiki


If you have any questions for me, always feel free to shoot me a PM.

syo_astro
Exalted Member
Exalted Member
Posts: 591
Joined: December 3rd, 2011, 9:45 pm
Division: Grad
State: NY
Contact:

Re: Test Set Exchange Update

Postby syo_astro » February 22nd, 2019, 10:09 am

nicholasmaurer wrote:
syo_astro wrote:How are you or the attendees not against test trading even a little bit? (that one is a little rhetorical...read on) I understand the point earlier by you and others that trading gives your test "continued value". But if teams don't like the competition getting hold of their work when someone shares it online, why do they not fear that (or consider it a non-issue) while trading tests? Note: I repeat, I'm absolutely not saying you should post your tests or be in favor of your tests being posted, hopefully this last question is clear then.

Aren't the tests overvalued if they can be leaked or obtained so easily? It sounds like it's acknowledged the big teams will get all the tests anyway...?

I don't think schools expect the tests they contribute to be kept confidential - they know that when you distribute them to the 70 other teams they will be shared, traded, etc. I think the objection is to tests being given away for nothing.

So to the others above, I don't think nicholasmaurer, etc (sorry, not just directing this to you...just convenient you know >.>) would agree at least as scioly stands right now. If they can get a competitive advantage, they will. And fine, coaches will object for similar reasons as rules stand right now. If you read his posts, I think he makes this clearer than I could say, thank you again! In fact, it's clear we're talking at different issues.

Okay, paraphrasing the above conversation and repeating some questions because I'm not sure I totally get the "why" part of this...Why do schools expect tests not to be confidential and are okay with trading?
B: Crave the Wave, Environmental Chemistry, Robo-Cross, Meteorology, Physical Science Lab, Solar System, DyPlan (E and V), Shock Value
C: Microbe Mission, DyPlan (Earth's Fresh Waters), Fermi Questions, GeoMaps, Gravity Vehicle, Scrambler, Rocks, Astronomy
Grad: Writing Tests/Supervising (NY/MI)

User avatar
Unome
Moderator
Moderator
Posts: 4036
Joined: January 26th, 2014, 12:48 pm
Division: Grad
State: GA
Location: somewhere in the sciolyverse

Re: Test Set Exchange Update

Postby Unome » February 22nd, 2019, 1:40 pm

Raleway wrote:I'm going to conclude with my final thoughts that SOINC should step in and require all invitationals register with SOINC and release tests and keys promptly to centralize these resources. The recent addition of "Invitational Ruleset" has been a foray into this but I hope it becomes more useful later. I applaud Princeton for being the first high-level invitational to publicly release tests and set off the next wave of invitationals to publically release tests (MIT, SOUP, etc.). Kudos to the aforementioned invitationals. Please make all tests public. It would save teams countless hours not worrying about tests to get and get rid of this issue once and for all.

This is exactly what I don't want. I'm certain NSO will mess things up the more involved they get. It's a natural consequence of being distant from the ground level.
Userpage
Chattahoochee High School Class of 2018
Georgia Tech Class of 2022

Opinions expressed on this site are not official; the only place for official rules changes and FAQs is soinc.org.

Schrodingerscat
Administrator
Administrator
Posts: 331
Joined: March 2nd, 2011, 7:10 pm
Division: Grad
State: KS
Location: Kansas City metro

Re: Test Set Exchange Update

Postby Schrodingerscat » February 22nd, 2019, 3:30 pm

Raleway wrote:I'm going to conclude with my final thoughts that SOINC should step in and require all invitationals register with SOINC and release tests and keys promptly to centralize these resources. The recent addition of "Invitational Ruleset" has been a foray into this but I hope it becomes more useful later. I applaud Princeton for being the first high-level invitational to publicly release tests and set off the next wave of invitationals to publically release tests (MIT, SOUP, etc.). Kudos to the aforementioned invitationals. Please make all tests public. It would save teams countless hours not worrying about tests to get and get rid of this issue once and for all.


Personally from a national policy perspective I'd rather see a "right to redistribute" rather than an "obligation to distribute" even if the latter would result in more tests available. In effect to say that if a sanctioned tournament decides to release tests to teams, any single recipient of the tests has the right to upload the tests to a site like this, and that sites like us could return to the old days of not worrying so much about permission to host invite tests (at least to my knowledge then). I think it might be a step too far to require tournament directors do anything specific to distribute tests though.

Raleway
Exalted Member
Exalted Member
Posts: 228
Joined: March 12th, 2017, 7:19 pm
Division: C
State: -

Re: Test Set Exchange Update

Postby Raleway » February 22nd, 2019, 8:57 pm

@Unome: Unfortunately, the consequence of NSO not being involved in this and soliciting advice and/or adding people to the steering committee with adequate knowledge is exactly the status quo. And it's not pretty. Sure, the first year or even subsequent years could be tough as NSO and invitationals work to get a balance, but in the end, it will be much more beneficial. There is no other oversight anything that has the power besides NSO. The problem with NSO being distant is also a larger issue in of itself. It just means they are not staying current, which is a dire problem that needs to be solved as soon as possible.

@Schrodingerscat: The issue with the right to redistribute is that what is currently happening occurs. Like in game theory, there is a much stronger incentive to not allow sharing of tests. Obviously, the incentives and intangibles at play have caused the current status quo; an external force will be required to create a paradigm shift (namely, NSO stepping in). I had a hope of Princeton, MIT, and SOUP being leading examples to other invitationals to publicly release tests, but it seems it is not the case (Cornell this year and whichever other invitationals).

It's unfortunate that things are like this, but I know teams that have dedicated test traders and individuals spending countless hours trying to get tests and make deals. That shouldn't be a thing in Science Olympiad.
Sleep is for the week; one only needs it once a week :!: :geek: :roll: :?: :idea:

God bless Len Joeris | Balsaman

User avatar
Unome
Moderator
Moderator
Posts: 4036
Joined: January 26th, 2014, 12:48 pm
Division: Grad
State: GA
Location: somewhere in the sciolyverse

Re: Test Set Exchange Update

Postby Unome » February 23rd, 2019, 5:02 am

Raleway wrote:@Unome: Unfortunately, the consequence of NSO not being involved in this and soliciting advice and/or adding people to the steering committee with adequate knowledge is exactly the status quo. And it's not pretty. Sure, the first year or even subsequent years could be tough as NSO and invitationals work to get a balance, but in the end, it will be much more beneficial. There is no other oversight anything that has the power besides NSO. The problem with NSO being distant is also a larger issue in of itself. It just means they are not staying current, which is a dire problem that needs to be solved as soon as possible.

@Schrodingerscat: The issue with the right to redistribute is that what is currently happening occurs. Like in game theory, there is a much stronger incentive to not allow sharing of tests. Obviously, the incentives and intangibles at play have caused the current status quo; an external force will be required to create a paradigm shift (namely, NSO stepping in). I had a hope of Princeton, MIT, and SOUP being leading examples to other invitationals to publicly release tests, but it seems it is not the case (Cornell this year and whichever other invitationals).

It's unfortunate that things are like this, but I know teams that have dedicated test traders and individuals spending countless hours trying to get tests and make deals. That shouldn't be a thing in Science Olympiad.

Consider how many invitationals do you think would lose significant numbers of teams and/or just stop running altogether as a result of something like what you propose.
Userpage
Chattahoochee High School Class of 2018
Georgia Tech Class of 2022

Opinions expressed on this site are not official; the only place for official rules changes and FAQs is soinc.org.

drcubbin
Member
Member
Posts: 157
Joined: January 9th, 2015, 6:28 pm
Division: B
State: NY
Location: Bay Academy, Brooklyn
Contact:

Re: Test Set Exchange Update

Postby drcubbin » February 23rd, 2019, 5:55 am

I could be wrong, but I thought (or maybe it's just old age setting in) that when we began almost 5 years ago, someone told us that there are certain test-writers who were adamantly opposed to releasing their tests and that you never saw their tests. And I vaguely remember never seeing those tests. But I suppose, like most teams, we do have somewhat of a small "repository" of tests that we use as a means to see the direction an event will take during competition. That being said, I believe the overall "pros" of test trading (getting a glimpse into what to study) are far overshadowed by the "cons", such as a) students taking the same test from last year (since test-writers have the same access to past tests), b) students using valuable study time to accumulate past tests (which in spite of this practice, currently appears to be a necessary evil) or at the very least c) getting into a rut of studying to the test (which I think we can all agree is not good). The bottom line is... there are a mountain of past tests currently in circulation. The question is, "What do we do now?", because as test-writers and event supervisors - and educated adults - I think we can all agree something needs to be done... and rather quickly, either yea or nay.

Most people do not know, but I had a previous - very successful and highly lucrative - former life as a chiropractor treating more than 300 patients/week (that's a LOT!) for almost 20 years. And if it weren't for my desire to teach, I might still be there, but with a crippling "SciOly void" in my life. I graduated in 86' (pre-computer lunacy) and had to study for everything I learned. While you might have seen some SciOly-level test questions (particularly A&P or Thermo) on our National and State boards, those examinations were infinitely more difficult. The teachers taught, we took notes, read books and took tests. No "previous tests" were ever available (though we did hear urban legends that ONE of Dr. Bogin's Physiology finals were floating around somewhere, but I never saw it). There were no "SAT prep" classes using previous tests either. We studied, learned a boatload of information and were set loose into the world. Trying to think back, I cannot even imagine what it would have been like if we had access to prior exams. My guess is that many - not all - would have been studying to the tests, and studying the information less. Like where we are now, it is just the nature of the beast. This is why I am in favor of halting the release of all future exams - a total moratorium on releasing any and all tests. Here's why.

There is currently a massive amount of "competitive direction and support" being given through a variety of venues for students to amply prepare for the rigors of competition: SciOly Wiki, scioly.org forums, etc... In fact, I would guess that many competitors have read old tests more completely than they have read the Rules manual for their events or the Wiki site (and this is often how they end up with construction violations). I believe these resources could be further developed so competitors would have even more resources at their disposal - but no more tests.

For those students complaining that event supervisors often include questions that are "way out there", they are right. We often do stretch the limits of test writing because so many of the reasonable and excellent questions have been rehashed over and over again and then given to students - through test-trading - to memorize. Imagine if there were no "previous" tests? You would never again hear a competitor say, "We took 3rd Place because it was the same test from two years ago". Is this really what we want? To the test-writers who write multiple tests for the same event, imagine being able to write one kick-butt test and simply modify it between events, instead of having to come up with unique and deal-breaker questions that even the best students are often only guessing at? On that note, thank you Alice Kasten for never releasing any NYC Regional exams. And while I am sure the NYC test-writers do write new tests every year, I am sure it would be nice to "rehash" test great questions or labs that students should be expected to know and not just because they have seen them on a dozen previous test versions. What test writer would love to use certain questions again - because they were well-developed and caused the competitors to think - but did not because the kids already had them in their hands?

It seems to me we are at a crossroads. Do we create a widespread underground/open-market for past exams for students to value over the actual information they need to know? (*Note - I do believe the schools that are crushing it at competitions really do know the information, are studying diligently and are very well prepared, but some in the middle range may be weighing the value of tests too heavily). Or do we stop the practice of test-trading here and now. Create an "across the board" end to releasing tests (leaving those tests that are out there to do what they may) and possibly create a more challenging approach to competitive events that the competitors may actually like. Having spent this past summer with so many great National Supervisors at the SOSI, I am guessing most would be in favor of the latter.

Disclaimer - the opinions of this contributor are sure to raise the ire of some, but he does not really care. He loves SciOly too much and believes that if the US is to ever claw its way back to the top of international strength in science SciOly is the only viable means to that end.

User avatar
lumosityfan
Exalted Member
Exalted Member
Posts: 308
Joined: July 14th, 2012, 7:00 pm
Division: Grad
State: NY
Location: New York, NY
Contact:

Re: Test Set Exchange Update

Postby lumosityfan » February 23rd, 2019, 9:51 am

drcubbin wrote:I could be wrong, but I thought (or maybe it's just old age setting in) that when we began almost 5 years ago, someone told us that there are certain test-writers who were adamantly opposed to releasing their tests and that you never saw their tests. And I vaguely remember never seeing those tests. But I suppose, like most teams, we do have somewhat of a small "repository" of tests that we use as a means to see the direction an event will take during competition. That being said, I believe the overall "pros" of test trading (getting a glimpse into what to study) are far overshadowed by the "cons", such as a) students taking the same test from last year (since test-writers have the same access to past tests), b) students using valuable study time to accumulate past tests (which in spite of this practice, currently appears to be a necessary evil) or at the very least c) getting into a rut of studying to the test (which I think we can all agree is not good). The bottom line is... there are a mountain of past tests currently in circulation. The question is, "What do we do now?", because as test-writers and event supervisors - and educated adults - I think we can all agree something needs to be done... and rather quickly, either yea or nay.

Most people do not know, but I had a previous - very successful and highly lucrative - former life as a chiropractor treating more than 300 patients/week (that's a LOT!) for almost 20 years. And if it weren't for my desire to teach, I might still be there, but with a crippling "SciOly void" in my life. I graduated in 86' (pre-computer lunacy) and had to study for everything I learned. While you might have seen some SciOly-level test questions (particularly A&P or Thermo) on our National and State boards, those examinations were infinitely more difficult. The teachers taught, we took notes, read books and took tests. No "previous tests" were ever available (though we did hear urban legends that ONE of Dr. Bogin's Physiology finals were floating around somewhere, but I never saw it). There were no "SAT prep" classes using previous tests either. We studied, learned a boatload of information and were set loose into the world. Trying to think back, I cannot even imagine what it would have been like if we had access to prior exams. My guess is that many - not all - would have been studying to the tests, and studying the information less. Like where we are now, it is just the nature of the beast. This is why I am in favor of halting the release of all future exams - a total moratorium on releasing any and all tests. Here's why.

There is currently a massive amount of "competitive direction and support" being given through a variety of venues for students to amply prepare for the rigors of competition: SciOly Wiki, scioly.org forums, etc... In fact, I would guess that many competitors have read old tests more completely than they have read the Rules manual for their events or the Wiki site (and this is often how they end up with construction violations). I believe these resources could be further developed so competitors would have even more resources at their disposal - but no more tests.

For those students complaining that event supervisors often include questions that are "way out there", they are right. We often do stretch the limits of test writing because so many of the reasonable and excellent questions have been rehashed over and over again and then given to students - through test-trading - to memorize. Imagine if there were no "previous" tests? You would never again hear a competitor say, "We took 3rd Place because it was the same test from two years ago". Is this really what we want? To the test-writers who write multiple tests for the same event, imagine being able to write one kick-butt test and simply modify it between events, instead of having to come up with unique and deal-breaker questions that even the best students are often only guessing at? On that note, thank you Alice Kasten for never releasing any NYC Regional exams. And while I am sure the NYC test-writers do write new tests every year, I am sure it would be nice to "rehash" test great questions or labs that students should be expected to know and not just because they have seen them on a dozen previous test versions. What test writer would love to use certain questions again - because they were well-developed and caused the competitors to think - but did not because the kids already had them in their hands?

It seems to me we are at a crossroads. Do we create a widespread underground/open-market for past exams for students to value over the actual information they need to know? (*Note - I do believe the schools that are crushing it at competitions really do know the information, are studying diligently and are very well prepared, but some in the middle range may be weighing the value of tests too heavily). Or do we stop the practice of test-trading here and now. Create an "across the board" end to releasing tests (leaving those tests that are out there to do what they may) and possibly create a more challenging approach to competitive events that the competitors may actually like. Having spent this past summer with so many great National Supervisors at the SOSI, I am guessing most would be in favor of the latter.

Disclaimer - the opinions of this contributor are sure to raise the ire of some, but he does not really care. He loves SciOly too much and believes that if the US is to ever claw its way back to the top of international strength in science SciOly is the only viable means to that end.


While in theory having no access to any prior resources would be nice for SciOly, the problem is that people aren't perfect. Competitors will naturally ask supervisors for old tests and supervisors will inevitably feel compelled to give the tests to them, despite their best efforts. Also the reason why competitions like USABO, the psychology tests, and others are not released is because they're propriety. SciOly tests aren't propriety, meaning there's nothing inherently wrong with releasing tests. Also if we are to allow event supervisors to repeat tests, that gives so much more advantages to competitors who have actually competed in previous year's event. That prevents new teams from having an even keel when it comes to understanding event supervisors' tendencies (and tests should not be taken based on knowing event supervisors' tendencies - why I feel irked by the fact that those habits make a difference). Also it provides no new accountability as there would be no way of figuring out if an event supervisor reuses a test.
JP Stevens 2015, Columbia University 2019
See my favorite teams' event history: https://docs.google.com/spreadsheets/d/ ... sp=sharing
2016-19 UCC Regionals Astronomy ES, 2017 Princeton Invitational Helicopters ES


Return to “Announcements”

Who is online

Users browsing this forum: No registered users and 2 guests