Releasing graded tests at Regionals and States

[Unome]

There are only so many invitationals near these particular event supervisors. Wright State/Centerville tends to be a good one to check because a lot of the National Event Supervisors from Ohio live near Dayton (i.e. within a few hundred miles or so - Dayton is nearer than the Cleveland area or Sylvania for most of them). I haven't ever seen WL-S tests, but those might also be useful on the same basis (not certain).

In my estimation, the high ratio of NES at Centerville/Wright State is less a matter of location (although this certainly helps), and more that CeAnn Chalker and Alan Chalker are involved in this tournament.

That sounds about right, but it's also true that a lot of the NESes are in the area. Perhaps the Chalkers are the confounding factor

[knottingpurple]

As am I. All of the supervisors I have spoken to are always in favor of releasing tests to students. This is mostly because they are all alumni and remember a time when they were competitors and wished they could see their tests...

For graded tests, personally, no (even as a competitor). I opt for only putting "standard" tests and invites online. Releasing all tests can saturate one with resources and can narrow studying to be on specific tests. Tests help, etc, but tests aren't automatically going to be helpful (especially with too many).

For obvious reasons, if states are undermanned, have to use the same ES, etc...tests probably won't get released. Still, here's a list of reasons (and counters) I can think of to release graded tests (note: I assume the main goal here is to improve learning / tests...hopefully not mistaken):
-"It helps students to identify their own specific mistakes"...Call me skeptical, but, from what I've seen, releasing graded tests mostly leads to scoring complaints. Yes, some use them well for studying / give helpful feedback, but this is not the majority.
-"More practice is needed to do well at events"...Invitationals seem to be good for that (I am a fan of invites releasing tests online, which is why prestigious invites don't make much sense to me as opposed to invites that give good practice for regionals / states...a rant for another day).
-"Practice needs to be relevant to the specific ES / region / state"...Then maybe something like NC's system would work best. At least it'd inform everyone uniformly. Some tests aren't even good practice, especially if ESs change.

I think it's worth investigating what's good practice as well as teaching better question making / scoring practices. But I can't think of how giving back scored tests from regionals / states necessarily helps with this (...other than, sure, learning you got a specific question wrong...). The only other issue I can think of would be about accountability, but that is very much a rabbit hole that people can't seem to agree about.

Not for Science Olympiad, but for some other competitions we've done where we didn't get exams back, we got back from competition and sat and wrote down everything we remembered being asked. So like, if there's a whole general topic you didn't understand, even on a test which isn't returned from you, you can still remember it immediately after the end of the event and learn from it...

[knightmoves]

Who has ever come out of a test (any kind of test) thinking they've done well, and then been surprised when they've got their grade back?

So why did you do badly? Do you not know as much as you think? Did you misjudge the level of rigor required in the answers? Or was the ES just "unusual"?

As knottingpurple says, if you know you bombed a test, you can write down the things you couldn't do and study them. But if you don't know, you don't know how to improve.

My friend did bridge building last time it was an event. His team was not historically strong at building events, but he built a bridge that he was proud of, that held the full weight, and took it to an invitational. He placed near the bottom. So he built another bridge which was a bit lighter, and again placed near the bottom. Eventually, a coach from another team spoke to his coach, and pointed out that they were in completely the wrong ballpark, and that improving a bridge from 70g to 40g wasn't going to help his score. The second year, he placed at regionals. If nobody had told him what he was doing wrong, he'd have spent the whole second year also building heavy things and scoring badly.

[bearasauras]

But wouldn't the student or the coach know that there's still room for improvement when they're placed near the bottom?

[TheChiScientist]

Yes but some coaches are not as aware of what wins SciOly competitions so they don't know what to improve upon.

[knightmoves]

But wouldn't the student or the coach know that there's still room for improvement when they're placed near the bottom?

They knew they had to do better, but didn't understand how much better. Without the heads-up from the other coach, they'd have been shaving single grams off a 40g bridge.

[bearasauras]

Ah that makes sense. So with the written events, would it be helpful to get the raw score and the range of scores back?

[windu34]

Who has ever come out of a test (any kind of test) thinking they've done well, and then been surprised when they've got their grade back?

So why did you do badly? Do you not know as much as you think? Did you misjudge the level of rigor required in the answers? Or was the ES just "unusual"?

As knottingpurple says, if you know you bombed a test, you can write down the things you couldn't do and study them. But if you don't know, you don't know how to improve.

My friend did bridge building last time it was an event. His team was not historically strong at building events, but he built a bridge that he was proud of, that held the full weight, and took it to an invitational. He placed near the bottom. So he built another bridge which was a bit lighter, and again placed near the bottom. Eventually, a coach from another team spoke to his coach, and pointed out that they were in completely the wrong ballpark, and that improving a bridge from 70g to 40g wasn't going to help his score. The second year, he placed at regionals. If nobody had told him what he was doing wrong, he'd have spent the whole second year also building heavy things and scoring badly.

These are some good points. Additionally, there are certainly competitions (MIT comes to mind) where you can think you did badly and still win an event. For example, Rock and Minerals last year had the highest score at a 30-40% (forgive me I forget the exact number) and a student accustomed to taking tests at school where everyone usually has enough time to finish and at least a few people get A's could be horrified by their raw score. However, if they had access to full raw scores of all the teams, they could see how they actually performed comparatively. Also teams may not have had time to look at all the questions and therefore would not be able to remember what the questions were and therefore not be able to improve unless they got a copy of the test. I know this is an extreme example, but there is no way to say that this doesn't occur at regionals/states across the nation every year. In fact, it probably does happen to many of the teams just starting out. Not releasing the tests could be another barrier to success for many new/inexperienced teams.

[0ddrenaline]

I'll share what Region 12 of Michigan does, since I think it might resolve some problems.

At a set time after regionals (before state), up to three coaches are allowed to view each team's test. They may not take the tests with them after viewing; the tests are returned to the tournament organizers. Coaches are allowed to take notes, but not take pictures. Event supervisors can opt out of their tests being shared. For building events, all teams' scores are available for viewing. The coaches have a couple hours maximum, so they don't have enough time to write every detail of the test. It's only enough time to identify which topics each person needs to improve on.

I like this method, but it's not perfect. It introduces some other possible problems. For example, I remember seeing both the test score and the build scores for every Thermodynamics team. Other building events provided similar information. I personally think that is too much information to share with us.

[bearasauras]

That's great! That's what we used to do in one of my Regionals, but as the number of teams increased, it has become more challenging to do. We have 40 B and 40 C teams on average per tournament, how do you make sure you have the time and personnel to support this effort?