Test Set Exchange Update

User avatar
lumosityfan
Exalted Member
Exalted Member
Posts: 320
Joined: July 14th, 2012, 7:00 pm
Division: Grad
State: TX
Location: Houston, TX
Contact:

Re: Test Set Exchange Update

Postby lumosityfan » February 23rd, 2019, 9:52 am

@Unome: Unfortunately, the consequence of NSO not being involved in this and soliciting advice and/or adding people to the steering committee with adequate knowledge is exactly the status quo. And it's not pretty. Sure, the first year or even subsequent years could be tough as NSO and invitationals work to get a balance, but in the end, it will be much more beneficial. There is no other oversight anything that has the power besides NSO. The problem with NSO being distant is also a larger issue in of itself. It just means they are not staying current, which is a dire problem that needs to be solved as soon as possible.

@Schrodingerscat: The issue with the right to redistribute is that what is currently happening occurs. Like in game theory, there is a much stronger incentive to not allow sharing of tests. Obviously, the incentives and intangibles at play have caused the current status quo; an external force will be required to create a paradigm shift (namely, NSO stepping in). I had a hope of Princeton, MIT, and SOUP being leading examples to other invitationals to publicly release tests, but it seems it is not the case (Cornell this year and whichever other invitationals).

It's unfortunate that things are like this, but I know teams that have dedicated test traders and individuals spending countless hours trying to get tests and make deals. That shouldn't be a thing in Science Olympiad.
Consider how many invitationals do you think would lose significant numbers of teams and/or just stop running altogether as a result of something like what you propose.
The thing is though competitions like Princeton, MIT, and UT-Austin do it and as far as I know, their attendance has not suffered. (Indeed Princeton's attendance has gone up and UT-Austin and MIT have stayed in-demand as ever.)
John P. Stevens Class of 2015 (Go Hawks!)
Columbia University Class of 2019 (Go Lions!)
2016-19 UCC Regs Astronomy ES, 2018 NJ States Astronomy ES, 2017 Princeton Helicopters ES, 2018 Princeton WGYN ES

syo_astro
Exalted Member
Exalted Member
Posts: 592
Joined: December 3rd, 2011, 9:45 pm
Division: Grad
State: NY
Contact:

Re: Test Set Exchange Update

Postby syo_astro » February 23rd, 2019, 12:08 pm

The thing is though competitions like Princeton, MIT, and UT-Austin do it and as far as I know, their attendance has not suffered. (Indeed Princeton's attendance has gone up and UT-Austin and MIT have stayed in-demand as ever.)
And UMSO;P. And Golden Gate I thought? And Dartmouth now! Anyway, Lumo, that wasn't the point of what Unome was saying. Can you name non-university invites that stayed in demand and compare it with the other mass amount of invites? I think cat has the most reasonable approach in line with how scioly usually runs things from the ground up anyway. If tests are even just allowed to be more openly shared, maybe we'll realize how this all shouldn't be such a big deal...

To drcubbin: Seems like it would be interesting discussing with you! (Nobody answered my question on the last page *why* people expect tests to "not be confidential" and are okay with test trading with the context and questions that you can see last page...would like it if someone could...). My opinion summarized in response to your issues: If we are indeed a science competition, then our questions should have plenty of concepts, math, etc. These questions SHOULD be constantly modified and also emphasize applying the basics. The events all change in different ways to reflect this, which is one thing I love! In fact, I think this is the best thing, science questions CAN be modified, which means you have to connect ideas from studies to do well. Some factual questions don't need to be, that's not the point of those. I know I don't have infinite experience, but I also find my past tests need significant modifying anyway based on answers I get (I sometimes even use invites to test question ideas I have...). Therefore, the issues of test content and test release can be separated a little more.

Now para by para (not quoting that all...you also seem to have a lot of different issues here that I try to separate):
1) You also have a repository of tests...so having some tests is a necessary evil? Just something to keep in mind. In response to your cons...point (a)...test writers reusing tests sort of goes against my opinion. I know it takes more time, but I think the better tests are usually rewritten anyway. Is your issue that we don't have enough writers? Why not take from someone who is writing in another state, which I imagine has to be done anyway? As for students taking study time to accumulate tests (b), yup fully agree, a waste of time, but I don't think we can totally control that. The internet exists, they'll figure out a way to make "test banks". Is it worse to be more open with people (students, coaches), not release essential or regionals test (fair enough), and only release practice tests (invites)? Your point (c) I also agree with. But the mountain of tests aren't all very good...it sounds like most I've talked to respect that 60 tests are a waste of time anyway.

2) Useful to know where you're coming from. Basically: What Lumo said. From my perspective, you're comparing different tests in different contexts. The internet exists, and the onus is on us to make use of it, not to say "just ignore it". I'm not saying the way you're talking about learning is "wrong". But the teachers I learned best from made their own questions that connected ideas you just can't google and don't really want to anyway. Your last few sentences I'd agree with. From here you assert whys!

On posted resources and reading the rules:
3) Perhaps competitors read tests more than the rules manual. Is that the fault of the tests, though? You said there's already so many online that we can't do anything about...won't they just keep paying more attention to those then? I know this isn't your point, but should the question marathons go too since they're easy to reuse questions? If you're saying we should update the site more, why not give better sources for practice with tests too? Practice tests (though I scorn tests) do have value for learning. If you have issue with students not reading the rules, then isn't that a more complicated and separate issue? For example, some may not read the rules because of last minute preparations as opposed to pure ignorance. Posting tests or not that can happen anyway, no?

On types of questions and question reuse:
4) This depends on the type of "out there" question. There are those "out there" questions that aren't justified by the rules. Other times students rightly dislike getting tested on literally everything (especially trivia), but people are more okay with that since it's in the rules. On the other hand, those questions may do us writers no justice: it makes it seem like the event should focus on memorizing or cramming as much into your notes as possible. A few of those questions can be fun or give a test quirks, though, so I don't see this as a huge deal except for writers that do it way too much.

Here's where I really get confused. "So many of the reasonable and excellent questions have been rehashed over and over again through test trading"...huh? Is this the main reason? All my tests are posted online, but nobody seems to do well on them:P. Maybe this is more specific to certain events than a universal issue? This just goes back to my prior opinion that updating questions or writing new questions can be (and certainly is!) a good thing. If a team can easily get 3rd place because it was the same test as two years ago, how can you fault the students? Wouldn't this happen anyway? Of many examples, one I like: they could have written the questions down on paper just after the test. I know this because I used to do that...more because a few questions bothered the HECK out of me. Even ignoring the internet, there's plenty of ways to circumvent regionals not getting posted.

Also, I don't know how much you write tests, but I need to write multiple tests anyway because rules constantly change and invite, regional, and state exams should NOT look the same. That has nothing to do with my tests being posted. I need to adjust the difficulties and question types anyway...if it's a question so trivial it could be answered with notes, isn't that my fault for giving a literally trivial question? There is a place for having some of those, and I think test writers (or at least I) reuse some of those questions anyway (yet students still mess them up...-_-).

5) This already exists. We can't undo that. If we give a system that can be gamed, students (people in general) WILL game it difficult or not. I can come up with a long list of ways to game testing if you like, but I hope you get the point. I also don't get what you mean by that last part: aren't making tests (or rules) more difficult and releasing tests not mutually exclusive?
Disclaimer - the opinions of this contributor are sure to raise the ire of some, but he does not really care. He loves SciOly too much and believes that if the US is to ever claw its way back to the top of international strength in science SciOly is the only viable means to that end.
Disclaimer: I'm not mad, thanks for sharing...know that others love scioly too;). I also feel like there are fundamental problems with education that need to be addressed beyond just scioly, but that's a story for another day and not this thread...
B: Crave the Wave, Environmental Chemistry, Robo-Cross, Meteorology, Physical Science Lab, Solar System, DyPlan (E and V), Shock Value
C: Microbe Mission, DyPlan (Earth's Fresh Waters), Fermi Questions, GeoMaps, Gravity Vehicle, Scrambler, Rocks, Astronomy
Grad: Writing Tests/Supervising (NY/MI)

drcubbin
Member
Member
Posts: 162
Joined: January 9th, 2015, 6:28 pm
Division: B
State: NY
Location: Bay Academy, Brooklyn
Contact:

Re: Test Set Exchange Update

Postby drcubbin » February 23rd, 2019, 2:13 pm

I appreciate the response syo_astro, but please understand that not being a child of the digital age, I find corresponding over the internet always loses something in translation, which it obviously did, and which is why I hesitated posting my thoughts in the first place as I had a feeling they would be "examined" under a microscope. I am sure if we were to sit and chat face to face, we would get much more accomplished. I paint with a broad brush, and though verbose, my intentions were not complicated, so allow me a little clarifying:

a) I believe we all love SciOly. That is why we are here. I never meant otherwise.

b) When I first learned about old tests being used for studying purposes, my first instinct was that it was somehow wrong. I still feel that way. Yes, we (our teams) do have tests. Yes, we use them to prepare. Call me a hypocrite, but no one is perfect. But I'm sure most would agree, our top students can live without these tests, so long as they are provided a little more information on what can, should and will be asked of them in Invitationals, Regionals and Nationals (I am just guessing about the Nationals part). Many students are not using tests to find out the basics. They can find this anywhere. They are looking to see how far astray the upcoming test will go. Take Potions and Poisons. I have no problem holding teams accountable for knowing molarity or even intermediate dilution calculations. But when the manual calls for no more than "balancing simple chemical equations", and the invy test asks for detailed understanding understanding of molarity... well then, yes, having that test to show them that it might come up sometime in the future is a definite asset - in spite of what limits the manual states.

c) I completely agree. Tests should always be modified. I never said we should make a test and use it over and over. I think if the test parameters could be expanded - and explained in the manual - to include higher level chemistry (for example with P&P), it would negate the need for reviewing tests, and require more learning take place. It would also make writing the test easier. Just one simple man's opinion, so please do not dissect this idea.

d) Yes, there are lots and lots of online resources. And it never ceases to amaze me how difficult it is for "children of the computer age" to find all of them, but easy to want to review another test. I like question marathons and I love the Summer Test exchange. Students making student tests for review is an absolute winner.

e) I apologize for the confusion behind "So many of the reasonable and excellent questions have been rehashed over and over again through test trading". What I am saying is that students are not using tests to study the basics, but to find the exception, rather than the rule. I think this is best illustrated by Section 7, page 7, question #52 of the 2019 Cornell DD test. Tell me the number of students who answered this correctly and I will admit I am wrong. In hindsight, I can see what they were looking for, but after speaking with other schools, many lost all points on this. There is so much more relevant information on microbiology and public health that can be asked. Again, having studied this for years, it is just my opinion.

f) You write, "I don't know how much you write tests, but I need to write multiple tests anyway because rules constantly change and invite, regional, and state exams should NOT look the same." Come on. I know that, but there is so much to ask in so many areas that when the competitors are given such limited parameters, the result is sure to make them look anywhere to find what will be asked that is not being told they will be asked.

g) "If a team can easily get 3rd place because it was the same test as two years ago, how can you fault the students?" I am not faulting the students. They did not write the test, but I thought my point here was obvious.

I am just sending this off in a flurry, so if I have not responded to all points, so be it. It is, after all, just a forum for ideas. And I will leave with a line from luminosityfan, "While in theory having no access to any prior resources would be nice for SciOly, the problem is that people aren't perfect". Yes, it would be nice.

syo_astro
Exalted Member
Exalted Member
Posts: 592
Joined: December 3rd, 2011, 9:45 pm
Division: Grad
State: NY
Contact:

Re: Test Set Exchange Update

Postby syo_astro » February 23rd, 2019, 2:27 pm

...I had a feeling they would be "examined" under a microscope. I am sure if we were to sit and chat face to face, we would get much more accomplished. I paint with a broad brush, and though verbose, my intentions were not complicated, so allow me a little clarifying:
...a) I believe we all love SciOly. That is why we are here. I never meant otherwise.
...I am just sending this off in a flurry, so if I have not responded to all points, so be it. It is, after all, just a forum for ideas.
Probably not worth extending that discussion then, but just wanted to say thanks for the thoughts and clarifications (really, I mean that). Sorry, that was a lot, I'm just trying to get at what people mean exactly, which I agree confuses me too on the internet (why I try to toss in lots of questions and examples). @a) If I said or implied something that indicated someone didn't love scioly, really didn't mean to...I'm aware we're on a forum dedicated to this...
B: Crave the Wave, Environmental Chemistry, Robo-Cross, Meteorology, Physical Science Lab, Solar System, DyPlan (E and V), Shock Value
C: Microbe Mission, DyPlan (Earth's Fresh Waters), Fermi Questions, GeoMaps, Gravity Vehicle, Scrambler, Rocks, Astronomy
Grad: Writing Tests/Supervising (NY/MI)

User avatar
Unome
Moderator
Moderator
Posts: 4120
Joined: January 26th, 2014, 12:48 pm
Division: Grad
State: GA
Location: somewhere in the sciolyverse

Re: Test Set Exchange Update

Postby Unome » February 23rd, 2019, 3:29 pm

I appreciate the response syo_astro, but please understand that not being a child of the digital age, I find corresponding over the internet always loses something in translation, which it obviously did, and which is why I hesitated posting my thoughts in the first place as I had a feeling they would be "examined" under a microscope. I am sure if we were to sit and chat face to face, we would get much more accomplished. I paint with a broad brush, and though verbose, my intentions were not complicated, so allow me a little clarifying:

a) I believe we all love SciOly. That is why we are here. I never meant otherwise.

b) When I first learned about old tests being used for studying purposes, my first instinct was that it was somehow wrong. I still feel that way. Yes, we (our teams) do have tests. Yes, we use them to prepare. Call me a hypocrite, but no one is perfect. But I'm sure most would agree, our top students can live without these tests, so long as they are provided a little more information on what can, should and will be asked of them in Invitationals, Regionals and Nationals (I am just guessing about the Nationals part). Many students are not using tests to find out the basics. They can find this anywhere. They are looking to see how far astray the upcoming test will go. Take Potions and Poisons. I have no problem holding teams accountable for knowing molarity or even intermediate dilution calculations. But when the manual calls for no more than "balancing simple chemical equations", and the invy test asks for detailed understanding understanding of molarity... well then, yes, having that test to show them that it might come up sometime in the future is a definite asset - in spite of what limits the manual states.

c) I completely agree. Tests should always be modified. I never said we should make a test and use it over and over. I think if the test parameters could be expanded - and explained in the manual - to include higher level chemistry (for example with P&P), it would negate the need for reviewing tests, and require more learning take place. It would also make writing the test easier. Just one simple man's opinion, so please do not dissect this idea.

d) Yes, there are lots and lots of online resources. And it never ceases to amaze me how difficult it is for "children of the computer age" to find all of them, but easy to want to review another test. I like question marathons and I love the Summer Test exchange. Students making student tests for review is an absolute winner.

e) I apologize for the confusion behind "So many of the reasonable and excellent questions have been rehashed over and over again through test trading". What I am saying is that students are not using tests to study the basics, but to find the exception, rather than the rule. I think this is best illustrated by Section 7, page 7, question #52 of the 2019 Cornell DD test. Tell me the number of students who answered this correctly and I will admit I am wrong. In hindsight, I can see what they were looking for, but after speaking with other schools, many lost all points on this. There is so much more relevant information on microbiology and public health that can be asked. Again, having studied this for years, it is just my opinion.

f) You write, "I don't know how much you write tests, but I need to write multiple tests anyway because rules constantly change and invite, regional, and state exams should NOT look the same." Come on. I know that, but there is so much to ask in so many areas that when the competitors are given such limited parameters, the result is sure to make them look anywhere to find what will be asked that is not being told they will be asked.

g) "If a team can easily get 3rd place because it was the same test as two years ago, how can you fault the students?" I am not faulting the students. They did not write the test, but I thought my point here was obvious.

I am just sending this off in a flurry, so if I have not responded to all points, so be it. It is, after all, just a forum for ideas. And I will leave with a line from luminosityfan, "While in theory having no access to any prior resources would be nice for SciOly, the problem is that people aren't perfect". Yes, it would be nice.
b and e) Interesting point on the main reason for tes-gathering being to identify areas not explicitly outlined in the rules but frequently covered on tests. I rather prefer that competitors have access to past tests though, because otherwise this just means that competitors with more competition experience have a vastly better understanding of what is likely to be asked, which is not currently the case to such a large degree (or at least, does not have to be the case for those willing to put in some effort).

c) I think most of us agree on this point. Some of the events, Div B in particular, are woefully restricted to the point where it's very difficult to make new questions (Density Lab comes to mind).
Userpage
Chattahoochee High School Class of 2018
Georgia Tech Class of 2022

Opinions expressed on this site are not official; the only place for official rules changes and FAQs is soinc.org.

drcubbin
Member
Member
Posts: 162
Joined: January 9th, 2015, 6:28 pm
Division: B
State: NY
Location: Bay Academy, Brooklyn
Contact:

Re: Test Set Exchange Update

Postby drcubbin » February 23rd, 2019, 4:48 pm

Thank you both syo_astro and unome for your responses. I am glad we can agree about our love of SciOly and the need to expand what students need to know on tests. After writing my first exam a few years back, another coach was a bit peeved that I didn't include all components of the test that were in the manual. I didn't make that mistake again, But as you agree, unome, how nice it would be to double - or triple! - the number of topics and have students be prepared to respond to say, 10 out of the 15 topics (or 15 out of 20 or more)! So to you both, I see your points about releasing the tests, but I still vote no, but yes for expanding the parameters of each event in manual and yes to halting all test-sharing. And while we are on "little things old guys like me don't understand" ;) I know many people in this forum know each other. It would be nice to know with whom I am speaking. I like others to know who I am. I believe one of the administrators told me recently that scioly.org does not require members to do this, but (unless you are a student) it would just "be nice" to know who we are. Even a school would be nice. (I guess this shows my age as well).

User avatar
Unome
Moderator
Moderator
Posts: 4120
Joined: January 26th, 2014, 12:48 pm
Division: Grad
State: GA
Location: somewhere in the sciolyverse

Re: Test Set Exchange Update

Postby Unome » February 23rd, 2019, 4:54 pm

Thank you both syo_astro and unome for your responses. I am glad we can agree about our love of SciOly and the need to expand what students need to know on tests. After writing my first exam a few years back, another coach was a bit peeved that I didn't include all components of the test that were in the manual. I didn't make that mistake again, But as you agree, unome, how nice it would be to double - or triple! - the number of topics and have students be prepared to respond to say, 10 out of the 15 topics (or 15 out of 20 or more)! So to you both, I see your points about releasing the tests, but I still vote no, but yes for expanding the parameters of each event in manual and yes to halting all test-sharing. And while we are on "little things old guys like me don't understand" ;) I know many people in this forum know each other. It would be nice to know with whom I am speaking. I like others to know who I am. I believe one of the administrators told me recently that scioly.org does not require members to do this, but (unless you are a student) it would just "be nice" to know who we are. Even a school would be nice. (I guess this shows my age as well).
One of the more interesting trends I've noticed is the major shift over the last 15-20 years away from "internet identity is separate from real-life identify". That still seems to hold true in some areas (e.g. Reddit) but overall it seems like the concept is less common. The OBB archive was really interesting to read, given that it spans roughly 10-20 years ago - the shift is very noticeable.

I didn't really make my identity known on here until everyone already kind of knew. That said, on here I think it's pretty easy to find out who people are, or at the very least what school they're from.
Userpage
Chattahoochee High School Class of 2018
Georgia Tech Class of 2022

Opinions expressed on this site are not official; the only place for official rules changes and FAQs is soinc.org.

User avatar
TheChiScientist
Member
Member
Posts: 676
Joined: March 11th, 2018, 11:25 am
Division: Grad
State: IL
Location: Suffering in a college class

Re: Test Set Exchange Update

Postby TheChiScientist » February 23rd, 2019, 7:32 pm

Thank you both syo_astro and unome for your responses. I am glad we can agree about our love of SciOly and the need to expand what students need to know on tests. After writing my first exam a few years back, another coach was a bit peeved that I didn't include all components of the test that were in the manual. I didn't make that mistake again, But as you agree, unome, how nice it would be to double - or triple! - the number of topics and have students be prepared to respond to say, 10 out of the 15 topics (or 15 out of 20 or more)! So to you both, I see your points about releasing the tests, but I still vote no, but yes for expanding the parameters of each event in manual and yes to halting all test-sharing. And while we are on "little things old guys like me don't understand" ;) I know many people in this forum know each other. It would be nice to know with whom I am speaking. I like others to know who I am. I believe one of the administrators told me recently that scioly.org does not require members to do this, but (unless you are a student) it would just "be nice" to know who we are. Even a school would be nice. (I guess this shows my age as well).
One of the more interesting trends I've noticed is the major shift over the last 15-20 years away from "internet identity is separate from real-life identify". That still seems to hold true in some areas (e.g. Reddit) but overall it seems like the concept is less common. The OBB archive was really interesting to read, given that it spans roughly 10-20 years ago - the shift is very noticeable.

I didn't really make my identity known on here until everyone already kind of knew. That said, on here I think it's pretty easy to find out who people are, or at the very least what school they're from.
Well, I was hesitant to mark what school I came from at first but then I met quite a few others from SciOly that love SciOly as much as me and it kinda changed my view on the whole identity thing by seeing others I could relate to. Besides, it's not too hard to figure out who I am if you meet me in person at an invite. :P (Intentionally or by mistake :lol: )
A Science Olympian from 2015 - 2019
Medal Count:30 8-)
School:Crystal Lake Central High School Wiki
Assassinator #119 and Co-Conspirator in #120
President of The Builder Cult. Builders rise up!

syo_astro
Exalted Member
Exalted Member
Posts: 592
Joined: December 3rd, 2011, 9:45 pm
Division: Grad
State: NY
Contact:

Re: Test Set Exchange Update

Postby syo_astro » February 23rd, 2019, 9:06 pm

...I know many people in this forum know each other. It would be nice to know with whom I am speaking. I like others to know who I am. I believe one of the administrators told me recently that scioly.org does not require members to do this, but (unless you are a student) it would just "be nice" to know who we are...
That's fair, I just don't feel like updating my signature and figured people ask if they want to know. I think it comes from the (illusion of) anonymity online, which is why I hesitate to fuss about it. As Unome said, probably people can figure out stuff about me searching my posts or asking around, but it's true asking directly is way more straightforward.

Answer (hopefully sufficient): I'm a former competitor -> graduate student studying astrophysics at University of Rochester now (NYS wooh!). I help with Astro in various ways, help with SSSS and check the Astro thread on the forums. Helped with other events in the past...but now am busy and Astro is the easiest for me to give help with.
B: Crave the Wave, Environmental Chemistry, Robo-Cross, Meteorology, Physical Science Lab, Solar System, DyPlan (E and V), Shock Value
C: Microbe Mission, DyPlan (Earth's Fresh Waters), Fermi Questions, GeoMaps, Gravity Vehicle, Scrambler, Rocks, Astronomy
Grad: Writing Tests/Supervising (NY/MI)

drcubbin
Member
Member
Posts: 162
Joined: January 9th, 2015, 6:28 pm
Division: B
State: NY
Location: Bay Academy, Brooklyn
Contact:

Re: Test Set Exchange Update

Postby drcubbin » February 24th, 2019, 3:03 am

...I know many people in this forum know each other. It would be nice to know with whom I am speaking. I like others to know who I am. I believe one of the administrators told me recently that scioly.org does not require members to do this, but (unless you are a student) it would just "be nice" to know who we are...
That's fair, I just don't feel like updating my signature and figured people ask if they want to know. I think it comes from the (illusion of) anonymity online, which is why I hesitate to fuss about it. As Unome said, probably people can figure out stuff about me searching my posts or asking around, but it's true asking directly is way more straightforward.

Answer (hopefully sufficient): I'm a former competitor -> graduate student studying astrophysics at University of Rochester now (NYS wooh!). I help with Astro in various ways, help with SSSS and check the Astro thread on the forums. Helped with other events in the past...but now am busy and Astro is the easiest for me to give help with.
Thank you, Astro. It does help me to know if I am speaking with an 11 year-old 6th grader, a HS competitor or a knowledgeable graduate such as yourself. Much appreciated. School affiliation helps to let me know if the person is coming from some level of success (and understanding what it takes to be successful) or if they are first year competitors. The anonymity part? I have never understood the need for it. And maybe it's just an old man thing, but I do speak differently with a middle-school student than I do with a college grad... and listen differently, too ;)


Return to “Announcements”

Who is online

Users browsing this forum: No registered users and 0 guests