National Test Discussion

Locked
kenniky
Member
Member
Posts: 283
Joined: January 21st, 2016, 6:16 pm
Division: Grad
State: MA
Has thanked: 0
Been thanked: 0

Re: National Test Discussion

Post by kenniky » May 24th, 2017, 6:01 am

pikachu4919 wrote: Wait kenniky why were you at home during nationals this year...?
check the 2017 MA thread
Automated Event Assigner!
UMich 2018: Chem Lab, Fermi

[url=http://tinyurl.com/kenniky-so-test]Rate my tests![/url]
[url]https://scioly.org/wiki/index.php/User:Kenniky[/url]

[url=https://scioly.org/forums/viewtopic.php?t=10008&start=34]2017 Nats = rip[/url]
[url=https://youtu.be/MCo8IAovjfw]ABRHS 2016[/url]

ScottMaurer19
Member
Member
Posts: 592
Joined: January 5th, 2017, 9:39 am
Division: Grad
State: OH
Has thanked: 0
Been thanked: 1 time

Re: National Test Discussion

Post by ScottMaurer19 » May 24th, 2017, 6:23 am

Did anyone else take to Div C Rocks & Minerals test?
Solon '19 Captain, CWRU '23
2017 (r/s/n):
Hydro: 3/5/18
Robot Arm: na/1/1
Rocks: 1/1/1

2018 (r/s/n):
Heli: 2/1/7 
Herp: 1/4/4
Mission: 1/1/6
Rocks: 1/1/1
Eco: 6/3/9

2019 (r/s/n):
Fossils: 1/1/1
GLM: 1/1/1
Herp: 1/1/5
Mission: 1/1/3
WS: 4/1/10

Top 3 Medals: 144
Golds: 80

User avatar
Tailsfan101
Wiki Moderator
Wiki Moderator
Posts: 813
Joined: April 14th, 2017, 4:33 pm
Division: C
State: ID
Pronouns: He/Him/His
Has thanked: 119 times
Been thanked: 106 times
Contact:

Re: National Test Discussion

Post by Tailsfan101 » May 24th, 2017, 7:47 am

bernard wrote:Here's some data you can play with. I've included the Pearson correlation coefficient comparing each event to overall team rank. This can be done for any tournament's results.
So I'm not really understanding this. Do the numbers get higher if more good teams succeed in a certain event? (If so, WIDI is almost like the lottery :lol:)
2016-2021 Treasure Valley Homeschool, 2020-2021 Div C Captain
2021 events: Disease Detectives, Designer Genes, Geologic Mapping, Write It Do It, Codebusters, Anatomy and Physiology

Assassinator for games 118: New Life and 136: A SciOly Pandemic
Selected Assassinators for games 121 and 148

Uber
Member
Member
Posts: 58
Joined: October 1st, 2015, 4:33 pm
Division: C
State: TX
Has thanked: 0
Been thanked: 0

Re: National Test Discussion

Post by Uber » May 24th, 2017, 7:59 am

Tailsfan101 wrote: So I'm not really understanding this. Do the numbers get higher if more good teams succeed in a certain event? (If so, WIDI is almost like the lottery :lol:)
Coefficient of correlation tests the correlation between two variables, in this case overall team ranking and team ranking in a certain event. The closer the coefficient of correlation gets to +1.00, the closer to a perfect correlation between overall team rank and the team rank in a certain event.
You're right about WIDI, and it's likely because it's difficult to perform consistently, so alot of teams just gave up. Ours definitely did.
Harvard '22
Liberal Arts and Science Academy '18

User avatar
bernard
Administrator
Administrator
Posts: 2248
Joined: January 5th, 2014, 3:12 pm
Division: Grad
State: WA
Pronouns: He/Him/His
Has thanked: 138 times
Been thanked: 608 times
Contact:

Re: National Test Discussion

Post by bernard » May 24th, 2017, 9:06 am

Uber wrote:
Tailsfan101 wrote: So I'm not really understanding this. Do the numbers get higher if more good teams succeed in a certain event? (If so, WIDI is almost like the lottery :lol:)
Coefficient of correlation tests the correlation between two variables, in this case overall team ranking and team ranking in a certain event. The closer the coefficient of correlation gets to +1.00, the closer to a perfect correlation between overall team rank and the team rank in a certain event.
You're right about WIDI, and it's likely because it's difficult to perform consistently, so alot of teams just gave up. Ours definitely did.
Pearson correlation coefficient ranges from -1 to +1, where -1 indicates negative correlation (teams that do well overall do poorly in this event), 0 indicates no correlation, and +1 indicates positive correlation (teams that do well overall do well in this event). Note PCC does not tell us the slope of a regression.

When the team rankings are inverted, the correlations for each event become negative. However, when you're in the scoring room the effect of an event scored backwards with the best teams ranked last may be more subtle because the overall team rank is calculated with the error so it's not as simple as an opposite sign.
"One of the ways that I believe people express their appreciation to the rest of humanity is to make something wonderful and put it out there."
Rate my tests!

jander14indoor
Member
Member
Posts: 1587
Joined: April 30th, 2007, 7:54 am
Has thanked: 0
Been thanked: 2 times

Re: National Test Discussion

Post by jander14indoor » May 24th, 2017, 11:30 am

Keep in mind, absolute correlation between individual events and the overall team ranking isn't really desirable. That would say there is no way for individuals on a team to outperform the overall team. You have to expect some variation. Low correlation may just say the talent for that event is more widely spread, or less dependent on team support. Not that the test isn't good.

In general correlation ONLY tells you things move together, NOT why.

Jeff Anderson
Livonia, MI

sciolyFTW_aku
Member
Member
Posts: 135
Joined: June 10th, 2015, 11:48 am
Division: C
State: MO
Has thanked: 0
Been thanked: 0

Re: National Test Discussion

Post by sciolyFTW_aku » May 24th, 2017, 12:05 pm

Hi jander14indoor,

However, I would argue for study events, a lower value might mean an easier test in general. For example, look at Anatomy and Physiology for both divisions. In Division B, the test was mostly on the easier side, allowing people from teams that don't do so well to have a chance to perform well in this event. However, in Division C, the test was mostly hard and well-written, which means the good teams would do well (just look at the top 6 in C; there are all from the top 10 overall, except for (me :P) Ladue, who came in at 11th).

Based on this, I would expect the 2015-2016 Division B A&P value to be on the lower side as well, if anyone would care to prove my hypothesis right/wrong?

Thanks,
sciolyFTW_aku
B-)

uictoria1
Member
Member
Posts: 20
Joined: May 21st, 2016, 10:23 am
Has thanked: 0
Been thanked: 0

Re: National Test Discussion

Post by uictoria1 » May 24th, 2017, 12:29 pm

From what I have gathered, the common thread for Anatomy is that ES Patty Palmietto writes tests that are generally too easy and fail to separate out strong teams from weak teams. I know she was the Div B ES at nationals this year and I have taken tests from her in the past. They were generally focused on very basic anatomy or process skills, which therefore means in-depth knowledge of Anatomy is of limited benefit. The only area she tends to go into detail on is disease identification, often based off of images.

nicholasmaurer
Coach
Coach
Posts: 422
Joined: May 19th, 2017, 10:55 am
Division: Grad
State: OH
Has thanked: 1 time
Been thanked: 21 times

Re: National Test Discussion

Post by nicholasmaurer » May 24th, 2017, 12:32 pm

chalker wrote:
chalker wrote:
Max event correlation: 0.52
Average event correlation: 0.39
Standard deviation: 0.09
Min event correlation: 0.20

Ecology had a correlation coefficient of 0.28. This means it was about 1 standard deviation below the average correlation for events. There were several events with lower correlations.

In essence, what this means is that statistically, the resulting ranks in Ecology are reasonably well aligned across all teams with the overall team ranks.
Oops.. I did this during lunch and thanks to Bernard posting his sheet I realized I did the wrong ranges. Below are the actual numbers, but the general conclusion is the same:

Max: 0.85
Average: 0.75
Std. Dev.: 0.10
Min: 0.42
Ecology: 0.57

WIDI and EV are lower than Ecology.
8 of the 23 events are under the average.
Chalker, is there ever any discussion of removing or refining events if they consistently show a poor correlation with team scores? If WIDI is so variable and a poor predictor of team outcome, why has its grading/scope not been better standardized or controlled?
Assistant Coach and Alumnus ('14) - Solon High School Science Olympiad
Tournament Director - Northeast Ohio Regional Tournament
Tournament Director - Solon High School Science Olympiad Invitational

Opinions expressed on this site are not official; the only place for official rules changes and FAQs is soinc.org.

nicholasmaurer
Coach
Coach
Posts: 422
Joined: May 19th, 2017, 10:55 am
Division: Grad
State: OH
Has thanked: 1 time
Been thanked: 21 times

Re: National Test Discussion

Post by nicholasmaurer » May 24th, 2017, 1:14 pm

Uber wrote:
5. MIT ecology was hands down the most brutal test I've ever taken. Golden Gate also had a much more difficult test than nationals, with more critical thinking involved. We won both. The national test came nowhere close. We finished most stations and double-checked with time to spare.

.
I'm glad you liked my MIT test. It was supposed to be brutal and make sure to separate the good teams from the weak ones. I remember you won first by a significant margin, so congrats!
Assistant Coach and Alumnus ('14) - Solon High School Science Olympiad
Tournament Director - Northeast Ohio Regional Tournament
Tournament Director - Solon High School Science Olympiad Invitational

Opinions expressed on this site are not official; the only place for official rules changes and FAQs is soinc.org.

Locked

Return to “2017 Nationals”

Who is online

Users browsing this forum: No registered users and 3 guests