National Test Discussion

chalker
Member
Member
Posts: 2052
Joined: Sat Jan 10, 2009 3:30 am
Division: Grad
State: OH
Contact:

Re: National Test Discussion

Postby chalker » Tue May 23, 2017 6:36 pm

Uber wrote:.......Ecology is an extremely variable event where almost every tournament has a different style and focus. Yes, the style is somewhat different from other tournaments, but the content it tested was overly basic, with some random sections mixed in. The random sections involved guessing what the proctor wanted. Since most of the test was too easy, the series of fill in the blank biodiversity and random vocab was what determined most of the final rankings, and we ended up on the wrong end. When 25% of the teams score within 3% of each other, there's something wrong with the test.


Just wanted to chime in with one minor thing to contribute to this conversation. But I'd like to preempt any detailed questions about specifics to the Ecology event. I don't know any details of how the event was run or specific team answers, nor do I have easy access to them. I'm also not likely going to share scoring information beyond what's already publicly available.

One of the numerous checks and balances we have as part of the scoring process is a cross-correlation between each events rankings and the overall team rankings. This is built into my Excel scoring system and primarily used to catch major errors like having low score wins instead of high score wins. The math involved is pretty simple, and anyone with a copy of the final score sheet that is posted on the website could generate these correlation coefficient numbers.

However, it's trivial easy for me to just share these with you for a bit of perspective (note these include ONLY the competition events, not the trial events):

Max event correlation: 0.52
Average event correlation: 0.39
Standard deviation: 0.09
Min event correlation: 0.20

Ecology had a correlation coefficient of 0.28. This means it was about 1 standard deviation below the average correlation for events. There were several events with lower correlations.

In essence, what this means is that statistically, the resulting ranks in Ecology are reasonably well aligned across all teams with the overall team ranks.

Student Alumni
National Event Supervisor
National Physical Sciences Rules Committee Chair

maxxxxx
Member
Member
Posts: 281
Joined: Mon Nov 30, 2015 4:11 pm
Division: Grad
State: PA
Location: CA
Contact:

Re: National Test Discussion

Postby maxxxxx » Tue May 23, 2017 6:54 pm

chalker wrote:However, it's trivial easy for me to just share these with you for a bit of perspective (note these include ONLY the competition events, not the trial events):

Max event correlation: 0.52
Average event correlation: 0.39
Standard deviation: 0.09
Min event correlation: 0.20


Would you be willing to share how many events were over and under the mean and which events had the max and min correlation, or is that secret committee info?
Lower Merion Class Of 2017

KoolKalvin
Member
Member
Posts: 4
Joined: Wed May 03, 2017 8:37 pm
Division: B
State: IN
Contact:

Re: National Test Discussion

Postby KoolKalvin » Tue May 23, 2017 7:08 pm

The48thYoshi wrote:
efeng wrote:
KoolKalvin wrote:That's true, but those types of questions don't show up much on nationals tests.

Yeah; I've never seen a trivia question on a nationals disease test.


I believe they asked about spot maps on 2012 at UCF?

Yeah, I remember doing that test. That's why I remembered it.

lastTime
Member
Member
Posts: 2
Joined: Tue May 23, 2017 12:59 pm
Division: C
State: VT
Contact:

Re: National Test Discussion

Postby lastTime » Tue May 23, 2017 8:01 pm

Tailsfan101 wrote:
Ouch! I feel so sorry for you, lastTime! My team's best finish was 16th in Disease Detectives. (Which me and Nerd_Bunny competed in :D)

+1, +1, (I described it as hectic mayhem) and +∞∞∞∞∞∞infinity! (I was backstage at the time, I was doing the student pledge! :))

Thanks. Congrats on making top 20, that certainly is an achievement.

And yeah, I was expecting it to be more organized, like a table for each state or something. Then again, we only had pins to trade, which was certainly not great.

It just seems like the professor didn't realize how unexciting his lecture was. At MIT, the speaker was really passionate about what she did and threw a few jokes in, and at UPenn, there was engagement from both speakers. However, here, there was no mention of application and seemed to flit from topic to topic.
Fairfax HS (Virginia)
Cornell/MIT/Regs/State/UPenn/Nats

ED: 19/--/--/--/--/--
Ecology: 5/38/1/2/20/19
Dynamic Planet 2/14/4/2/7/14
Remote Sensing --/6/1/4/16/36
Hydrogeology 5/22/1/1/1/7 :(

User avatar
Unome
Moderator
Moderator
Posts: 3348
Joined: Sun Jan 26, 2014 8:48 pm
Division: Grad
State: GA
Location: somewhere in the sciolyverse
Contact:

Re: National Test Discussion

Postby Unome » Tue May 23, 2017 8:08 pm

lastTime wrote:
Tailsfan101 wrote:
Ouch! I feel so sorry for you, lastTime! My team's best finish was 16th in Disease Detectives. (Which me and Nerd_Bunny competed in :D)

+1, +1, (I described it as hectic mayhem) and +∞∞∞∞∞∞infinity! (I was backstage at the time, I was doing the student pledge! :))

Thanks. Congrats on making top 20, that certainly is an achievement.

And yeah, I was expecting it to be more organized, like a table for each state or something. Then again, we only had pins to trade, which was certainly not great.

It just seems like the professor didn't realize how unexciting his lecture was. At MIT, the speaker was really passionate about what she did and threw a few jokes in, and at UPenn, there was engagement from both speakers. However, here, there was no mention of application and seemed to flit from topic to topic.

Hm personally I found the MIT speaker to be very dull as well. As for Nationals I'm sure anyone who hasn't done Astro or Remote would be rather confused as to what he was talking about.
Userpage
Chattahoochee High School Class of 2018
Georgia Tech Class of 2022

User avatar
daydreamer0023
Member
Member
Posts: 181
Joined: Fri Jan 30, 2015 1:44 am
Division: C
State: -
Location: I have no idea where I am, but I can tell you exactly how fast I'm going ;)
Contact:

Re: National Test Discussion

Postby daydreamer0023 » Tue May 23, 2017 9:39 pm

Unome wrote:
lastTime wrote:
Tailsfan101 wrote:
Ouch! I feel so sorry for you, lastTime! My team's best finish was 16th in Disease Detectives. (Which me and Nerd_Bunny competed in :D)

+1, +1, (I described it as hectic mayhem) and +∞∞∞∞∞∞infinity! (I was backstage at the time, I was doing the student pledge! :))

Thanks. Congrats on making top 20, that certainly is an achievement.

And yeah, I was expecting it to be more organized, like a table for each state or something. Then again, we only had pins to trade, which was certainly not great.

It just seems like the professor didn't realize how unexciting his lecture was. At MIT, the speaker was really passionate about what she did and threw a few jokes in, and at UPenn, there was engagement from both speakers. However, here, there was no mention of application and seemed to flit from topic to topic.

Hm personally I found the MIT speaker to be very dull as well. As for Nationals I'm sure anyone who hasn't done Astro or Remote would be rather confused as to what he was talking about.


Don't forget to mention the spelling errors...*cough* Adeninie *cough*
"I am among those who think that science has great beauty. A scientist in his laboratory is not only a technician: he is also a child placed before natural phenomena which impress him like a fairy tale." - Marie Curie

fast__facts
Member
Member
Posts: 18
Joined: Tue Mar 21, 2017 10:49 pm
Division: C
State: -
Contact:

Re: National Test Discussion

Postby fast__facts » Tue May 23, 2017 9:57 pm

Anybody know when the awards ceremony and opening ceremony videos will be up? and where will they be on to find?

kenniky
Member
Member
Posts: 282
Joined: Fri Jan 22, 2016 2:16 am
Division: Grad
State: MA
Location: ABRHS --> UMichigan
Contact:

Re: National Test Discussion

Postby kenniky » Tue May 23, 2017 10:05 pm

maxxxxx wrote:
chalker wrote:However, it's trivial easy for me to just share these with you for a bit of perspective (note these include ONLY the competition events, not the trial events):

Max event correlation: 0.52
Average event correlation: 0.39
Standard deviation: 0.09
Min event correlation: 0.20


Would you be willing to share how many events were over and under the mean and which events had the max and min correlation, or is that secret committee info?

You can calculate this yourself lol.... all the data is there for you

User avatar
bernard
Administrator
Administrator
Posts: 1856
Joined: Sun Jan 05, 2014 11:12 pm
Division: Grad
State: WA
Location: Seattle, WA
Contact:

Re: National Test Discussion

Postby bernard » Tue May 23, 2017 11:22 pm

Here's some data you can play with. I've included the Pearson correlation coefficient comparing each event to overall team rank. This can be done for any tournament's results.
Attachments
sont-2017-correlation.xlsx
(69.28 KiB) Downloaded 119 times
"One of the ways that I believe people express their appreciation to the rest of humanity is to make something wonderful and put it out there."

ScottMaurer19
Member
Member
Posts: 498
Joined: Thu Jan 05, 2017 5:39 pm
Division: C
State: OH
Location: Solon, OH
Contact:

Re: National Test Discussion

Postby ScottMaurer19 » Tue May 23, 2017 11:26 pm

bernard wrote:Here's some data you can play with. I've included the Pearson correlation coefficient comparing each event to overall team rank. This can be done for any tournament's results.

Based on these results how is WIDI still an event (only partially joking)
SHS '19 Co-Captain
Placements:
2017 (r/s/n):
Hydro: 3/5/18
Robot Arm: na/1/1
Rocks: 1/1/1

2018 (r/s/n):
Heli: 2/1/7 :?
Herp: 1/4/4
Mission: 1/1/6 :cry:
Rocks: 1/1/1
Eco: 6/3/9

2019?
Dynamic
Fossils
GLM
Herp
Mission
WQ
WS

Total Top 3 Placements: 100

User avatar
windu34
Moderator
Moderator
Posts: 1125
Joined: Mon Apr 20, 2015 1:37 am
Division: Grad
State: FL
Location: Gainesville, Florida
Contact:

Re: National Test Discussion

Postby windu34 » Tue May 23, 2017 11:48 pm

bernard wrote:Here's some data you can play with. I've included the Pearson correlation coefficient comparing each event to overall team rank. This can be done for any tournament's results.

Schools arent matched with scores
President of Science Olympiad at the University of Florida || Boca Raton Community High School Alumni
SONT 2016: 2nd place in Electric Vehicle || 3rd place in Robot Arm
SONT 2017: 2nd place in Electric Vehicle || 5th place in Invasive Species
Jazzin710@gmail.com
windu34's Userpage

User avatar
Unome
Moderator
Moderator
Posts: 3348
Joined: Sun Jan 26, 2014 8:48 pm
Division: Grad
State: GA
Location: somewhere in the sciolyverse
Contact:

Re: National Test Discussion

Postby Unome » Tue May 23, 2017 11:58 pm

windu34 wrote:
bernard wrote:Here's some data you can play with. I've included the Pearson correlation coefficient comparing each event to overall team rank. This can be done for any tournament's results.

Schools arent matched with scores

Looks like this is only in Div C though (via spellchecking a few medalists that I remember in B).
Userpage
Chattahoochee High School Class of 2018
Georgia Tech Class of 2022

User avatar
bernard
Administrator
Administrator
Posts: 1856
Joined: Sun Jan 05, 2014 11:12 pm
Division: Grad
State: WA
Location: Seattle, WA
Contact:

Re: National Test Discussion

Postby bernard » Wed May 24, 2017 12:54 am

Unome wrote:
windu34 wrote:
bernard wrote:Here's some data you can play with. I've included the Pearson correlation coefficient comparing each event to overall team rank. This can be done for any tournament's results.

Schools arent matched with scores

Looks like this is only in Div C though (via spellchecking a few medalists that I remember in B).

Thanks, these scores were copied directly from the spreadsheet I use for calculating prediction contest scores but thankfully prediction contest scoring is correct as it only uses team number. I've updated the file in the original post.
"One of the ways that I believe people express their appreciation to the rest of humanity is to make something wonderful and put it out there."

chalker
Member
Member
Posts: 2052
Joined: Sat Jan 10, 2009 3:30 am
Division: Grad
State: OH
Contact:

Re: National Test Discussion

Postby chalker » Wed May 24, 2017 1:50 am

chalker wrote:
Max event correlation: 0.52
Average event correlation: 0.39
Standard deviation: 0.09
Min event correlation: 0.20

Ecology had a correlation coefficient of 0.28. This means it was about 1 standard deviation below the average correlation for events. There were several events with lower correlations.

In essence, what this means is that statistically, the resulting ranks in Ecology are reasonably well aligned across all teams with the overall team ranks.


Oops.. I did this during lunch and thanks to Bernard posting his sheet I realized I did the wrong ranges. Below are the actual numbers, but the general conclusion is the same:

Max: 0.85
Average: 0.75
Std. Dev.: 0.10
Min: 0.42
Ecology: 0.57

WIDI and EV are lower than Ecology.
8 of the 23 events are under the average.

Student Alumni
National Event Supervisor
National Physical Sciences Rules Committee Chair

User avatar
windu34
Moderator
Moderator
Posts: 1125
Joined: Mon Apr 20, 2015 1:37 am
Division: Grad
State: FL
Location: Gainesville, Florida
Contact:

Re: National Test Discussion

Postby windu34 » Wed May 24, 2017 2:42 am

chalker wrote:
chalker wrote:
Max event correlation: 0.52
Average event correlation: 0.39
Standard deviation: 0.09
Min event correlation: 0.20

Ecology had a correlation coefficient of 0.28. This means it was about 1 standard deviation below the average correlation for events. There were several events with lower correlations.

In essence, what this means is that statistically, the resulting ranks in Ecology are reasonably well aligned across all teams with the overall team ranks.


Oops.. I did this during lunch and thanks to Bernard posting his sheet I realized I did the wrong ranges. Below are the actual numbers, but the general conclusion is the same:

Max: 0.85
Average: 0.75
Std. Dev.: 0.10
Min: 0.42
Ecology: 0.57

WIDI and EV are lower than Ecology.
8 of the 23 events are under the average.

To be fair, I don't think build events and events such as WIDI can reasonably grouped together with Ecology. I have medalled at nationals twice in EV now and if you look at my history in EV at regionals and various invites, you would never even think there would be a chance that I would rank above 20th at nationals. The smallest falter in a vehicle can mean the difference between 1st and last in this event. Ecology is a study event - if you don't know one small detail, it wont significantly inhibit you from doing well. Correlation coefficients for study events should be much higher than those for builds and inherently inconsistent events such as Widi thus if you are going to compare Ecology to anything, it should only be to other study events of a similar nature (Dynamic Planet, Remote Sensing, etc)
President of Science Olympiad at the University of Florida || Boca Raton Community High School Alumni
SONT 2016: 2nd place in Electric Vehicle || 3rd place in Robot Arm
SONT 2017: 2nd place in Electric Vehicle || 5th place in Invasive Species
Jazzin710@gmail.com
windu34's Userpage


Return to “2017 Nationals”

Who is online

Users browsing this forum: No registered users and 1 guest