Amended Nationals Appeals Policy

chalker
Member
Member
Posts: 2086
Joined: January 9th, 2009, 7:30 pm
Division: Grad
State: OH

Re: Amended Nationals Appeals Policy

Postby chalker » April 26th, 2018, 7:18 am

Ashernoel wrote:
I can confirm that the 53rd in EV was due to a "construction violation" that was appealed but rejected.

The ruling was disgusting, relying not on proof but the word of the ES, and was based on an incredibly unclear section of the rules that had gone uncalled all year.

The run was also one of their best all year, and would have resulted in New Trier placing 6th or 7th overall.

The arbitration committee completely missed the ball and the ruling shows a failure in the national arbitration system.


Just want to chime in a bit here. I don't know any of the details of the specific situation you mention, however I do know a LOT of details of the national arbitration system. While I can understand you are upset due to the impact this situation had on your team, I have some general observations:

1. The National Event Supervisors (and in particular the EV ones you refer to) don't have any skin in the game with regards to treating one team differently from another (i.e. there is no reason to NOT 'take them at their word').
2. The EV supervisors in question have a LOT of experience with Science Olympiad, and in particular the EV event. They work closely on the rules and help out at the National and State level in a LOT of ways (i.e. they are 'ideal' supervisors that really take their role seriously)
3. Likewise, the National arbitration committee has pretty much the most extensive experience in Science Olympiad as anyone could have. They take their job VERY seriously and focus on nothing but arbitrations during SONT. (i.e. they are 'ideal' arbitrators that don't have any reason to be biased).
4. The arbitrators have an explicit policy of 'erring on the side of the competitors' whenever possible. I've heard them say time and again that they won't use the 'intent' of the rules writers to make a ruling - rather the actual event rules and the general rules. They often do rule against National Event Supervisors as a result of this.
5. We have an FAQ process available throughout the year to help clarify what might be 'incredibly unclear sections of the rules'. The onus is on the competitor to seek clarifications in advance rather than to try to 'sneak something past the supervisors' in a grey area of the rules.

So, if the situation was as you generally describe (an unclear to you section of the rules, which the supervisors ruled one way on, but then you appealed, and the arbitration committee seriously investigated via discussions with both the team coach and the event supervisors), then just because they didn't rule in your favor doesn't mean there was a failure in the system. Rather, the system sounds like it worked exactly as anyone would hope it would: with all available evidence being heard and evaluated against a consistent set of policies by a qualified set of unbiased, engaged arbitrators.

Student Alumni
National Event Supervisor
National Physical Sciences Rules Committee Chair

hippo9
Member
Member
Posts: 242
Joined: March 12th, 2018, 9:35 am
Division: C
State: IN

Re: Amended Nationals Appeals Policy

Postby hippo9 » April 26th, 2018, 7:45 am

Ashernoel wrote:
I can confirm that the 53rd in EV was due to a "construction violation" that was appealed but rejected.

The ruling was disgusting, relying not on proof but the word of the ES, and was based on an incredibly unclear section of the rules that had gone uncalled all year.

The run was also one of their best all year, and would have resulted in New Trier placing 6th or 7th overall.

The arbitration committee completely missed the ball and the ruling shows a failure in the national arbitration system.

What even was the violation? As chalker said there probably was a reason behind the decision.
"If we knew what we were doing, it wouldn’t be called research." - Albert Einstein

2018: Battery Buggy, Road Scholar, Roller Coaster
2019: Chemistry Lab, Codebusters, Disease Detectives, Fossils, Geologic Mapping, Sounds of Music

chalker
Member
Member
Posts: 2086
Joined: January 9th, 2009, 7:30 pm
Division: Grad
State: OH

Re: Amended Nationals Appeals Policy

Postby chalker » April 26th, 2018, 8:06 am

nicholasmaurer wrote:
In terms of calculation, I did all of my work in Excel. The formula is relatively straightforward. Here is a link to the math.


Nick: thanks for providing this, however the process you outline in the PDF doesn't quite match what you described. The 'big' issue I see is that you only calculate the mean of the original set of 23 events, but use that same value for the calculation of the modified set of 22 events. Technically, you should recalculate the mean for the 22 events prior to doing the standard deviation. The 'small' issue is I think you needed to multiple by 23 and divide by 22 in the formula, not 22 and 21.

Addressing the 'big issue' is the one I'm struggling with. I can't seem to come up with an easy way in excel to calculate the standard deviation of a set range of numbers yet ignoring the biggest one. Let's say we have a range of numbers in cells A1:A23 (but we don't apriori know which is the maximum). We can easily do the following in a formula:
mean23=AVERAGE(A1:A23)
stdev23=STDEVPA(A1:A23)
mean22=(SUM(A1:A23)-MAX(A1:A23))/22

What I'm missing is a reasonable way to do the next step and calculate the standard deviation for the 22 'best' events, not knowing which element of the 23 is the max one.

Student Alumni
National Event Supervisor
National Physical Sciences Rules Committee Chair

chalker
Member
Member
Posts: 2086
Joined: January 9th, 2009, 7:30 pm
Division: Grad
State: OH

Re: Amended Nationals Appeals Policy

Postby chalker » April 26th, 2018, 8:08 am

chalker wrote:
nicholasmaurer wrote:
In terms of calculation, I did all of my work in Excel. The formula is relatively straightforward. Here is a link to the math.


Nick: thanks for providing this, however the process you outline in the PDF doesn't quite match what you described. The 'big' issue I see is that you only calculate the mean of the original set of 23 events, but use that same value for the calculation of the modified set of 22 events. Technically, you should recalculate the mean for the 22 events prior to doing the standard deviation. The 'small' issue is I think you needed to multiple by 23 and divide by 22 in the formula, not 22 and 21.

Addressing the 'big issue' is the one I'm struggling with. I can't seem to come up with an easy way in excel to calculate the standard deviation of a set range of numbers yet ignoring the biggest one. Let's say we have a range of numbers in cells A1:A23 (but we don't apriori know which is the maximum). We can easily do the following in a formula:
mean23=AVERAGE(A1:A23)
stdev23=STDEVPA(A1:A23)
mean22=(SUM(A1:A23)-MAX(A1:A23))/22

What I'm missing is a reasonable way to do the next step and calculate the standard deviation for the 22 'best' events, not knowing which element of the 23 is the max one.

Although maybe I'm being too technical and the formula you provided is a close enough approximation to what you want to do.

Student Alumni
National Event Supervisor
National Physical Sciences Rules Committee Chair

GoldenKnight1
Coach
Coach
Posts: 197
Joined: May 2nd, 2009, 5:02 pm
Division: Grad
State: PA

Re: Amended Nationals Appeals Policy

Postby GoldenKnight1 » April 26th, 2018, 8:43 am

chalker wrote:mean23=AVERAGE(A1:A23)
stdev23=STDEVPA(A1:A23)
mean22=(SUM(A1:A23)-MAX(A1:A23))/22

What I'm missing is a reasonable way to do the next step and calculate the standard deviation for the 22 'best' events, not knowing which element of the 23 is the max one.



What about a formula in a separate column (or sheet) that removes that max value or any value outside of a range you determine by that threshold factor?

=if(ABS(A1-mean23)>threshold factor*stdev23, “”, A1)

Then calculate the standard deviation of that group.

nicholasmaurer
Coach
Coach
Posts: 354
Joined: May 19th, 2017, 10:55 am
Division: Grad
State: OH
Location: Solon, OH

Re: Amended Nationals Appeals Policy

Postby nicholasmaurer » April 26th, 2018, 3:12 pm

chalker wrote:
nicholasmaurer wrote:
In terms of calculation, I did all of my work in Excel. The formula is relatively straightforward. Here is a link to the math.


Nick: thanks for providing this, however the process you outline in the PDF doesn't quite match what you described. The 'big' issue I see is that you only calculate the mean of the original set of 23 events, but use that same value for the calculation of the modified set of 22 events. Technically, you should recalculate the mean for the 22 events prior to doing the standard deviation. The 'small' issue is I think you needed to multiple by 23 and divide by 22 in the formula, not 22 and 21.

Addressing the 'big issue' is the one I'm struggling with. I can't seem to come up with an easy way in excel to calculate the standard deviation of a set range of numbers yet ignoring the biggest one. Let's say we have a range of numbers in cells A1:A23 (but we don't apriori know which is the maximum). We can easily do the following in a formula:
mean23=AVERAGE(A1:A23)
stdev23=STDEVPA(A1:A23)
mean22=(SUM(A1:A23)-MAX(A1:A23))/22

What I'm missing is a reasonable way to do the next step and calculate the standard deviation for the 22 'best' events, not knowing which element of the 23 is the max one.


It might be simpler to explain/show you in person what I am doing at some point this weekend. Your point about recalculating the mean is valid, although I don’t know if it makes a difference from a practical standpoint. Regarding your “small issue” about whether to divide by 22 vs 23, I believe I am correct because standard deviation typically relies on a divisor of n-1 to reflect the degrees of freedom.
Assistant Coach and Alumni ('14) - Solon High School Science Olympiad
Tournament Director - Northeast Ohio Regional Tournament

Opinions expressed on this site are not official; the only place for official rules changes and FAQs is soinc.org.

knightmoves
Member
Member
Posts: 99
Joined: April 26th, 2018, 6:40 pm
State: -

Re: Amended Nationals Appeals Policy

Postby knightmoves » April 26th, 2018, 8:40 pm

chalker wrote: Let's say we have a range of numbers in cells A1:A23 (but we don't apriori know which is the maximum). We can easily do the following in a formula:
mean23=AVERAGE(A1:A23)
stdev23=STDEVPA(A1:A23)
mean22=(SUM(A1:A23)-MAX(A1:A23))/22

What I'm missing is a reasonable way to do the next step and calculate the standard deviation for the 22 'best' events, not knowing which element of the 23 is the max one.


sum23 = sum(a1:a23)
sumsq23 = sumsq(a1:a23)
sum22 = sum23-max(a1:a23)
sumsq22 = sumsq23-max(a1:a23)*max(a1:a23)

mean23 = sum23/23
stdev23 = sqrt(sumsq23/23 - mean23*mean23)

mean22 = sum22/22
stdev22 = sqrt(sumsq22/22 - mean22*mean22)

chalker
Member
Member
Posts: 2086
Joined: January 9th, 2009, 7:30 pm
Division: Grad
State: OH

Re: Amended Nationals Appeals Policy

Postby chalker » April 29th, 2018, 8:17 am

knightmoves wrote:sum23 = sum(a1:a23)
sumsq23 = sumsq(a1:a23)
sum22 = sum23-max(a1:a23)
sumsq22 = sumsq23-max(a1:a23)*max(a1:a23)

mean23 = sum23/23
stdev23 = sqrt(sumsq23/23 - mean23*mean23)

mean22 = sum22/22
stdev22 = sqrt(sumsq22/22 - mean22*mean22)


Cool, I didn't know about the sumsq function. I'm still a little tired from the Ohio State Tournament yesterday, but I don't see how your stdev formulas actually equal stdev? Fundamentally it seems like you are saying that (x-y)^2 is the same as x^2 - y^2, which isn't true. Am I missing something in your algebraic transformation?

Student Alumni
National Event Supervisor
National Physical Sciences Rules Committee Chair

knightmoves
Member
Member
Posts: 99
Joined: April 26th, 2018, 6:40 pm
State: -

Re: Amended Nationals Appeals Policy

Postby knightmoves » April 29th, 2018, 1:03 pm

chalker wrote:Cool, I didn't know about the sumsq function. I'm still a little tired from the Ohio State Tournament yesterday, but I don't see how your stdev formulas actually equal stdev? Fundamentally it seems like you are saying that (x-y)^2 is the same as x^2 - y^2, which isn't true. Am I missing something in your algebraic transformation?


Let's drop the square roots and deal with variance.

Var(x) = (1/n)sum((x-xmean)^2)
= (1/n)sum(x^2 -2*x*xmean + xmean*xmean)
Push the xmeans out of the sum, because they're constants, and you get
Var(x) = (1/n)sum(x^2) - 2*xmean*(1/n)sum(x) + xmean*xmean
= meansq(x) - 2*xmean*xmean + xmean*xmean
= meansq(x) - xmean*xmean

My notation is horrible here. I hope you get the point.

User avatar
bernard
Administrator
Administrator
Posts: 1964
Joined: January 5th, 2014, 3:12 pm
Division: Grad
State: WA
Location: Seattle, WA
Contact:

Re: Amended Nationals Appeals Policy

Postby bernard » April 29th, 2018, 2:16 pm

hippo9 wrote:
Ashernoel wrote:
I can confirm that the 53rd in EV was due to a "construction violation" that was appealed but rejected.

The ruling was disgusting, relying not on proof but the word of the ES, and was based on an incredibly unclear section of the rules that had gone uncalled all year.

The run was also one of their best all year, and would have resulted in New Trier placing 6th or 7th overall.

The arbitration committee completely missed the ball and the ruling shows a failure in the national arbitration system.

What even was the violation? As chalker said there probably was a reason behind the decision.

The device's dowel was not within 1.0 cm of the track's surface for the entire duration of each run. This was observed on the first run and the team was notified so they could adjust their device before the second run. For both runs, the dowel was in an allowable position before the run but measured as too high above the track after the run.
"One of the ways that I believe people express their appreciation to the rest of humanity is to make something wonderful and put it out there."

hippo9
Member
Member
Posts: 242
Joined: March 12th, 2018, 9:35 am
Division: C
State: IN

Re: Amended Nationals Appeals Policy

Postby hippo9 » April 29th, 2018, 3:46 pm

bernard wrote:
hippo9 wrote:
Ashernoel wrote:
I can confirm that the 53rd in EV was due to a "construction violation" that was appealed but rejected.

The ruling was disgusting, relying not on proof but the word of the ES, and was based on an incredibly unclear section of the rules that had gone uncalled all year.

The run was also one of their best all year, and would have resulted in New Trier placing 6th or 7th overall.

The arbitration committee completely missed the ball and the ruling shows a failure in the national arbitration system.

What even was the violation? As chalker said there probably was a reason behind the decision.

The device's dowel was not within 1.0 cm of the track's surface for the entire duration of each run. This was observed on the first run and the team was notified so they could adjust their device before the second run. For both runs, the dowel was in an allowable position before the run but measured as too high above the track after the run.

How does that happen? Was it touching the axle or something?
"If we knew what we were doing, it wouldn’t be called research." - Albert Einstein

2018: Battery Buggy, Road Scholar, Roller Coaster
2019: Chemistry Lab, Codebusters, Disease Detectives, Fossils, Geologic Mapping, Sounds of Music

User avatar
bernard
Administrator
Administrator
Posts: 1964
Joined: January 5th, 2014, 3:12 pm
Division: Grad
State: WA
Location: Seattle, WA
Contact:

Re: Amended Nationals Appeals Policy

Postby bernard » April 29th, 2018, 4:16 pm

hippo9 wrote:
bernard wrote:
hippo9 wrote:What even was the violation? As chalker said there probably was a reason behind the decision.

The device's dowel was not within 1.0 cm of the track's surface for the entire duration of each run. This was observed on the first run and the team was notified so they could adjust their device before the second run. For both runs, the dowel was in an allowable position before the run but measured as too high above the track after the run.

How does that happen? Was it touching the axle or something?

For both runs, the team's device stopped abruptly, tilting forward and pushing the vertical dowel up.
"One of the ways that I believe people express their appreciation to the rest of humanity is to make something wonderful and put it out there."

chalker
Member
Member
Posts: 2086
Joined: January 9th, 2009, 7:30 pm
Division: Grad
State: OH

Re: Amended Nationals Appeals Policy

Postby chalker » April 29th, 2018, 5:48 pm

knightmoves wrote:
Let's drop the square roots and deal with variance.

Var(x) = (1/n)sum((x-xmean)^2)
= (1/n)sum(x^2 -2*x*xmean + xmean*xmean)
Push the xmeans out of the sum, because they're constants, and you get
Var(x) = (1/n)sum(x^2) - 2*xmean*(1/n)sum(x) + xmean*xmean
= meansq(x) - 2*xmean*xmean + xmean*xmean
= meansq(x) - xmean*xmean

My notation is horrible here. I hope you get the point.


Yep.. that did it for me. Thanks.

Student Alumni
National Event Supervisor
National Physical Sciences Rules Committee Chair

hippo9
Member
Member
Posts: 242
Joined: March 12th, 2018, 9:35 am
Division: C
State: IN

Re: Amended Nationals Appeals Policy

Postby hippo9 » April 29th, 2018, 6:08 pm

bernard wrote:
hippo9 wrote:
bernard wrote:The device's dowel was not within 1.0 cm of the track's surface for the entire duration of each run. This was observed on the first run and the team was notified so they could adjust their device before the second run. For both runs, the dowel was in an allowable position before the run but measured as too high above the track after the run.

How does that happen? Was it touching the axle or something?

For both runs, the team's device stopped abruptly, tilting forward and pushing the vertical dowel up.

Ok makes sense, but being able to stop that abruptly is a feat in and of itself.
"If we knew what we were doing, it wouldn’t be called research." - Albert Einstein

2018: Battery Buggy, Road Scholar, Roller Coaster
2019: Chemistry Lab, Codebusters, Disease Detectives, Fossils, Geologic Mapping, Sounds of Music

User avatar
Unome
Moderator
Moderator
Posts: 4036
Joined: January 26th, 2014, 12:48 pm
Division: Grad
State: GA
Location: somewhere in the sciolyverse

Re: Amended Nationals Appeals Policy

Postby Unome » April 29th, 2018, 7:07 pm

hippo9 wrote:
bernard wrote:
hippo9 wrote:How does that happen? Was it touching the axle or something?

For both runs, the team's device stopped abruptly, tilting forward and pushing the vertical dowel up.

Ok makes sense, but being able to stop that abruptly is a feat in and of itself.

The abruptness of stopping is not the concern.
Userpage
Chattahoochee High School Class of 2018
Georgia Tech Class of 2022

Opinions expressed on this site are not official; the only place for official rules changes and FAQs is soinc.org.


Return to “2018 Nationals”

Who is online

Users browsing this forum: No registered users and 1 guest