Amended Nationals Appeals Policy

chalker
Member
Member
Posts: 2107
Joined: January 9th, 2009, 7:30 pm
Division: Grad
State: OH
Has thanked: 1 time
Been thanked: 56 times

Re: Amended Nationals Appeals Policy

Post by chalker »

Ashernoel wrote:
I can confirm that the 53rd in EV was due to a "construction violation" that was appealed but rejected.

The ruling was disgusting, relying not on proof but the word of the ES, and was based on an incredibly unclear section of the rules that had gone uncalled all year.

The run was also one of their best all year, and would have resulted in New Trier placing 6th or 7th overall.

The arbitration committee completely missed the ball and the ruling shows a failure in the national arbitration system.
Just want to chime in a bit here. I don't know any of the details of the specific situation you mention, however I do know a LOT of details of the national arbitration system. While I can understand you are upset due to the impact this situation had on your team, I have some general observations:

1. The National Event Supervisors (and in particular the EV ones you refer to) don't have any skin in the game with regards to treating one team differently from another (i.e. there is no reason to NOT 'take them at their word').
2. The EV supervisors in question have a LOT of experience with Science Olympiad, and in particular the EV event. They work closely on the rules and help out at the National and State level in a LOT of ways (i.e. they are 'ideal' supervisors that really take their role seriously)
3. Likewise, the National arbitration committee has pretty much the most extensive experience in Science Olympiad as anyone could have. They take their job VERY seriously and focus on nothing but arbitrations during SONT. (i.e. they are 'ideal' arbitrators that don't have any reason to be biased).
4. The arbitrators have an explicit policy of 'erring on the side of the competitors' whenever possible. I've heard them say time and again that they won't use the 'intent' of the rules writers to make a ruling - rather the actual event rules and the general rules. They often do rule against National Event Supervisors as a result of this.
5. We have an FAQ process available throughout the year to help clarify what might be 'incredibly unclear sections of the rules'. The onus is on the competitor to seek clarifications in advance rather than to try to 'sneak something past the supervisors' in a grey area of the rules.

So, if the situation was as you generally describe (an unclear to you section of the rules, which the supervisors ruled one way on, but then you appealed, and the arbitration committee seriously investigated via discussions with both the team coach and the event supervisors), then just because they didn't rule in your favor doesn't mean there was a failure in the system. Rather, the system sounds like it worked exactly as anyone would hope it would: with all available evidence being heard and evaluated against a consistent set of policies by a qualified set of unbiased, engaged arbitrators.

Student Alumni
National Event Supervisor
National Physical Sciences Rules Committee Chair
hippo9
Member
Member
Posts: 271
Joined: March 12th, 2018, 9:35 am
Division: C
State: IN
Has thanked: 1 time
Been thanked: 6 times

Re: Amended Nationals Appeals Policy

Post by hippo9 »

Ashernoel wrote:
I can confirm that the 53rd in EV was due to a "construction violation" that was appealed but rejected.

The ruling was disgusting, relying not on proof but the word of the ES, and was based on an incredibly unclear section of the rules that had gone uncalled all year.

The run was also one of their best all year, and would have resulted in New Trier placing 6th or 7th overall.

The arbitration committee completely missed the ball and the ruling shows a failure in the national arbitration system.
What even was the violation? As chalker said there probably was a reason behind the decision.
2018: Battery Buggy, Road Scholar, Roller Coaster
2019: Chem Lab, Code, Disease, Fossils, Geo Maps, Sounds
2020 and 2021: Astro, Chem Lab, Code, Fossils, Geo Maps, Sounds

When you miss nats twice by a combined two points :|
chalker
Member
Member
Posts: 2107
Joined: January 9th, 2009, 7:30 pm
Division: Grad
State: OH
Has thanked: 1 time
Been thanked: 56 times

Re: Amended Nationals Appeals Policy

Post by chalker »

nicholasmaurer wrote:
In terms of calculation, I did all of my work in Excel. The formula is relatively straightforward. Here is a link to the math.
Nick: thanks for providing this, however the process you outline in the PDF doesn't quite match what you described. The 'big' issue I see is that you only calculate the mean of the original set of 23 events, but use that same value for the calculation of the modified set of 22 events. Technically, you should recalculate the mean for the 22 events prior to doing the standard deviation. The 'small' issue is I think you needed to multiple by 23 and divide by 22 in the formula, not 22 and 21.

Addressing the 'big issue' is the one I'm struggling with. I can't seem to come up with an easy way in excel to calculate the standard deviation of a set range of numbers yet ignoring the biggest one. Let's say we have a range of numbers in cells A1:A23 (but we don't apriori know which is the maximum). We can easily do the following in a formula:
mean23=AVERAGE(A1:A23)
stdev23=STDEVPA(A1:A23)
mean22=(SUM(A1:A23)-MAX(A1:A23))/22

What I'm missing is a reasonable way to do the next step and calculate the standard deviation for the 22 'best' events, not knowing which element of the 23 is the max one.

Student Alumni
National Event Supervisor
National Physical Sciences Rules Committee Chair
chalker
Member
Member
Posts: 2107
Joined: January 9th, 2009, 7:30 pm
Division: Grad
State: OH
Has thanked: 1 time
Been thanked: 56 times

Re: Amended Nationals Appeals Policy

Post by chalker »

chalker wrote:
nicholasmaurer wrote:
In terms of calculation, I did all of my work in Excel. The formula is relatively straightforward. Here is a link to the math.
Nick: thanks for providing this, however the process you outline in the PDF doesn't quite match what you described. The 'big' issue I see is that you only calculate the mean of the original set of 23 events, but use that same value for the calculation of the modified set of 22 events. Technically, you should recalculate the mean for the 22 events prior to doing the standard deviation. The 'small' issue is I think you needed to multiple by 23 and divide by 22 in the formula, not 22 and 21.

Addressing the 'big issue' is the one I'm struggling with. I can't seem to come up with an easy way in excel to calculate the standard deviation of a set range of numbers yet ignoring the biggest one. Let's say we have a range of numbers in cells A1:A23 (but we don't apriori know which is the maximum). We can easily do the following in a formula:
mean23=AVERAGE(A1:A23)
stdev23=STDEVPA(A1:A23)
mean22=(SUM(A1:A23)-MAX(A1:A23))/22

What I'm missing is a reasonable way to do the next step and calculate the standard deviation for the 22 'best' events, not knowing which element of the 23 is the max one.
Although maybe I'm being too technical and the formula you provided is a close enough approximation to what you want to do.

Student Alumni
National Event Supervisor
National Physical Sciences Rules Committee Chair
GoldenKnight1
Coach
Coach
Posts: 225
Joined: May 2nd, 2009, 5:02 pm
Division: Grad
State: PA
Pronouns: He/Him/His
Has thanked: 0
Been thanked: 18 times

Re: Amended Nationals Appeals Policy

Post by GoldenKnight1 »

chalker wrote: mean23=AVERAGE(A1:A23)
stdev23=STDEVPA(A1:A23)
mean22=(SUM(A1:A23)-MAX(A1:A23))/22

What I'm missing is a reasonable way to do the next step and calculate the standard deviation for the 22 'best' events, not knowing which element of the 23 is the max one.

What about a formula in a separate column (or sheet) that removes that max value or any value outside of a range you determine by that threshold factor?

=if(ABS(A1-mean23)>threshold factor*stdev23, “”, A1)

Then calculate the standard deviation of that group.
nicholasmaurer
Coach
Coach
Posts: 422
Joined: May 19th, 2017, 10:55 am
Division: Grad
State: OH
Has thanked: 1 time
Been thanked: 22 times

Re: Amended Nationals Appeals Policy

Post by nicholasmaurer »

chalker wrote:
nicholasmaurer wrote:
In terms of calculation, I did all of my work in Excel. The formula is relatively straightforward. Here is a link to the math.
Nick: thanks for providing this, however the process you outline in the PDF doesn't quite match what you described. The 'big' issue I see is that you only calculate the mean of the original set of 23 events, but use that same value for the calculation of the modified set of 22 events. Technically, you should recalculate the mean for the 22 events prior to doing the standard deviation. The 'small' issue is I think you needed to multiple by 23 and divide by 22 in the formula, not 22 and 21.

Addressing the 'big issue' is the one I'm struggling with. I can't seem to come up with an easy way in excel to calculate the standard deviation of a set range of numbers yet ignoring the biggest one. Let's say we have a range of numbers in cells A1:A23 (but we don't apriori know which is the maximum). We can easily do the following in a formula:
mean23=AVERAGE(A1:A23)
stdev23=STDEVPA(A1:A23)
mean22=(SUM(A1:A23)-MAX(A1:A23))/22

What I'm missing is a reasonable way to do the next step and calculate the standard deviation for the 22 'best' events, not knowing which element of the 23 is the max one.
It might be simpler to explain/show you in person what I am doing at some point this weekend. Your point about recalculating the mean is valid, although I don’t know if it makes a difference from a practical standpoint. Regarding your “small issue” about whether to divide by 22 vs 23, I believe I am correct because standard deviation typically relies on a divisor of n-1 to reflect the degrees of freedom.
Assistant Coach and Alumnus ('14) - Solon High School Science Olympiad
Tournament Director - Northeast Ohio Regional Tournament
Tournament Director - Solon High School Science Olympiad Invitational

Opinions expressed on this site are not official; the only place for official rules changes and FAQs is soinc.org.
knightmoves
Member
Member
Posts: 632
Joined: April 26th, 2018, 6:40 pm
Has thanked: 4 times
Been thanked: 108 times

Re: Amended Nationals Appeals Policy

Post by knightmoves »

chalker wrote: Let's say we have a range of numbers in cells A1:A23 (but we don't apriori know which is the maximum). We can easily do the following in a formula:
mean23=AVERAGE(A1:A23)
stdev23=STDEVPA(A1:A23)
mean22=(SUM(A1:A23)-MAX(A1:A23))/22

What I'm missing is a reasonable way to do the next step and calculate the standard deviation for the 22 'best' events, not knowing which element of the 23 is the max one.
sum23 = sum(a1:a23)
sumsq23 = sumsq(a1:a23)
sum22 = sum23-max(a1:a23)
sumsq22 = sumsq23-max(a1:a23)*max(a1:a23)

mean23 = sum23/23
stdev23 = sqrt(sumsq23/23 - mean23*mean23)

mean22 = sum22/22
stdev22 = sqrt(sumsq22/22 - mean22*mean22)
chalker
Member
Member
Posts: 2107
Joined: January 9th, 2009, 7:30 pm
Division: Grad
State: OH
Has thanked: 1 time
Been thanked: 56 times

Re: Amended Nationals Appeals Policy

Post by chalker »

knightmoves wrote: sum23 = sum(a1:a23)
sumsq23 = sumsq(a1:a23)
sum22 = sum23-max(a1:a23)
sumsq22 = sumsq23-max(a1:a23)*max(a1:a23)

mean23 = sum23/23
stdev23 = sqrt(sumsq23/23 - mean23*mean23)

mean22 = sum22/22
stdev22 = sqrt(sumsq22/22 - mean22*mean22)
Cool, I didn't know about the sumsq function. I'm still a little tired from the Ohio State Tournament yesterday, but I don't see how your stdev formulas actually equal stdev? Fundamentally it seems like you are saying that (x-y)^2 is the same as x^2 - y^2, which isn't true. Am I missing something in your algebraic transformation?

Student Alumni
National Event Supervisor
National Physical Sciences Rules Committee Chair
knightmoves
Member
Member
Posts: 632
Joined: April 26th, 2018, 6:40 pm
Has thanked: 4 times
Been thanked: 108 times

Re: Amended Nationals Appeals Policy

Post by knightmoves »

chalker wrote: Cool, I didn't know about the sumsq function. I'm still a little tired from the Ohio State Tournament yesterday, but I don't see how your stdev formulas actually equal stdev? Fundamentally it seems like you are saying that (x-y)^2 is the same as x^2 - y^2, which isn't true. Am I missing something in your algebraic transformation?
Let's drop the square roots and deal with variance.

Var(x) = (1/n)sum((x-xmean)^2)
= (1/n)sum(x^2 -2*x*xmean + xmean*xmean)
Push the xmeans out of the sum, because they're constants, and you get
Var(x) = (1/n)sum(x^2) - 2*xmean*(1/n)sum(x) + xmean*xmean
= meansq(x) - 2*xmean*xmean + xmean*xmean
= meansq(x) - xmean*xmean

My notation is horrible here. I hope you get the point.
User avatar
bernard
Administrator
Administrator
Posts: 2498
Joined: January 5th, 2014, 3:12 pm
Division: Grad
State: WA
Pronouns: He/Him/His
Has thanked: 186 times
Been thanked: 789 times
Contact:

Re: Amended Nationals Appeals Policy

Post by bernard »

hippo9 wrote:
Ashernoel wrote:
I can confirm that the 53rd in EV was due to a "construction violation" that was appealed but rejected.

The ruling was disgusting, relying not on proof but the word of the ES, and was based on an incredibly unclear section of the rules that had gone uncalled all year.

The run was also one of their best all year, and would have resulted in New Trier placing 6th or 7th overall.

The arbitration committee completely missed the ball and the ruling shows a failure in the national arbitration system.
What even was the violation? As chalker said there probably was a reason behind the decision.
The device's dowel was not within 1.0 cm of the track's surface for the entire duration of each run. This was observed on the first run and the team was notified so they could adjust their device before the second run. For both runs, the dowel was in an allowable position before the run but measured as too high above the track after the run.
"One of the ways that I believe people express their appreciation to the rest of humanity is to make something wonderful and put it out there." – Steve Jobs
Locked

Return to “2018 Nationals”

Who is online

Users browsing this forum: No registered users and 2 guests