National Test Discussion

Locked
kenniky
Member
Member
Posts: 283
Joined: January 21st, 2016, 6:16 pm
Division: Grad
State: MA
Has thanked: 0
Been thanked: 0

Re: National Test Discussion

Post by kenniky »

pikachu4919 wrote: Wait kenniky why were you at home during nationals this year...?
check the 2017 MA thread
Automated Event Assigner!
UMich 2018: Chem Lab, Fermi

[url=http://tinyurl.com/kenniky-so-test]Rate my tests![/url]
[url]https://scioly.org/wiki/index.php/User:Kenniky[/url]

[url=https://scioly.org/forums/viewtopic.php?t=10008&start=34]2017 Nats = rip[/url]
[url=https://youtu.be/MCo8IAovjfw]ABRHS 2016[/url]
ScottMaurer19
Member
Member
Posts: 592
Joined: January 5th, 2017, 9:39 am
Division: Grad
State: OH
Has thanked: 0
Been thanked: 1 time

Re: National Test Discussion

Post by ScottMaurer19 »

Did anyone else take to Div C Rocks & Minerals test?
Solon '19 Captain, CWRU '23
2017 (r/s/n):
Hydro: 3/5/18
Robot Arm: na/1/1
Rocks: 1/1/1

2018 (r/s/n):
Heli: 2/1/7 
Herp: 1/4/4
Mission: 1/1/6
Rocks: 1/1/1
Eco: 6/3/9

2019 (r/s/n):
Fossils: 1/1/1
GLM: 1/1/1
Herp: 1/1/5
Mission: 1/1/3
WS: 4/1/10

Top 3 Medals: 144
Golds: 80
User avatar
Tailsfan101
Member
Member
Posts: 829
Joined: April 14th, 2017, 4:33 pm
Division: Grad
State: ID
Has thanked: 130 times
Been thanked: 117 times
Contact:

Re: National Test Discussion

Post by Tailsfan101 »

bernard wrote:Here's some data you can play with. I've included the Pearson correlation coefficient comparing each event to overall team rank. This can be done for any tournament's results.
So I'm not really understanding this. Do the numbers get higher if more good teams succeed in a certain event? (If so, WIDI is almost like the lottery :lol:)
"Blessed are you when people insult you and persecute you, and falsely say all kinds of evil against you because of Me. Rejoice and be glad, for your reward in heaven is great; for in the same way they persecuted the prophets who were before you." Matthew 5:11-12

I have no regrets.
Uber
Member
Member
Posts: 58
Joined: October 1st, 2015, 4:33 pm
Division: C
State: TX
Has thanked: 0
Been thanked: 0

Re: National Test Discussion

Post by Uber »

Tailsfan101 wrote: So I'm not really understanding this. Do the numbers get higher if more good teams succeed in a certain event? (If so, WIDI is almost like the lottery :lol:)
Coefficient of correlation tests the correlation between two variables, in this case overall team ranking and team ranking in a certain event. The closer the coefficient of correlation gets to +1.00, the closer to a perfect correlation between overall team rank and the team rank in a certain event.
You're right about WIDI, and it's likely because it's difficult to perform consistently, so alot of teams just gave up. Ours definitely did.
Harvard '22
Liberal Arts and Science Academy '18
User avatar
bernard
Administrator
Administrator
Posts: 2498
Joined: January 5th, 2014, 3:12 pm
Division: Grad
State: WA
Pronouns: He/Him/His
Has thanked: 186 times
Been thanked: 789 times
Contact:

Re: National Test Discussion

Post by bernard »

Uber wrote:
Tailsfan101 wrote: So I'm not really understanding this. Do the numbers get higher if more good teams succeed in a certain event? (If so, WIDI is almost like the lottery :lol:)
Coefficient of correlation tests the correlation between two variables, in this case overall team ranking and team ranking in a certain event. The closer the coefficient of correlation gets to +1.00, the closer to a perfect correlation between overall team rank and the team rank in a certain event.
You're right about WIDI, and it's likely because it's difficult to perform consistently, so alot of teams just gave up. Ours definitely did.
Pearson correlation coefficient ranges from -1 to +1, where -1 indicates negative correlation (teams that do well overall do poorly in this event), 0 indicates no correlation, and +1 indicates positive correlation (teams that do well overall do well in this event). Note PCC does not tell us the slope of a regression.

When the team rankings are inverted, the correlations for each event become negative. However, when you're in the scoring room the effect of an event scored backwards with the best teams ranked last may be more subtle because the overall team rank is calculated with the error so it's not as simple as an opposite sign.
"One of the ways that I believe people express their appreciation to the rest of humanity is to make something wonderful and put it out there." – Steve Jobs
jander14indoor
Member
Member
Posts: 1653
Joined: April 30th, 2007, 7:54 am
Has thanked: 0
Been thanked: 29 times

Re: National Test Discussion

Post by jander14indoor »

Keep in mind, absolute correlation between individual events and the overall team ranking isn't really desirable. That would say there is no way for individuals on a team to outperform the overall team. You have to expect some variation. Low correlation may just say the talent for that event is more widely spread, or less dependent on team support. Not that the test isn't good.

In general correlation ONLY tells you things move together, NOT why.

Jeff Anderson
Livonia, MI
sciolyFTW_aku
Member
Member
Posts: 135
Joined: June 10th, 2015, 11:48 am
Division: C
State: MO
Has thanked: 0
Been thanked: 0

Re: National Test Discussion

Post by sciolyFTW_aku »

Hi jander14indoor,

However, I would argue for study events, a lower value might mean an easier test in general. For example, look at Anatomy and Physiology for both divisions. In Division B, the test was mostly on the easier side, allowing people from teams that don't do so well to have a chance to perform well in this event. However, in Division C, the test was mostly hard and well-written, which means the good teams would do well (just look at the top 6 in C; there are all from the top 10 overall, except for (me :P) Ladue, who came in at 11th).

Based on this, I would expect the 2015-2016 Division B A&P value to be on the lower side as well, if anyone would care to prove my hypothesis right/wrong?

Thanks,
sciolyFTW_aku
B-)
uictoria1
Member
Member
Posts: 20
Joined: May 21st, 2016, 10:23 am
Has thanked: 0
Been thanked: 0

Re: National Test Discussion

Post by uictoria1 »

From what I have gathered, the common thread for Anatomy is that ES Patty Palmietto writes tests that are generally too easy and fail to separate out strong teams from weak teams. I know she was the Div B ES at nationals this year and I have taken tests from her in the past. They were generally focused on very basic anatomy or process skills, which therefore means in-depth knowledge of Anatomy is of limited benefit. The only area she tends to go into detail on is disease identification, often based off of images.
nicholasmaurer
Coach
Coach
Posts: 422
Joined: May 19th, 2017, 10:55 am
Division: Grad
State: OH
Has thanked: 1 time
Been thanked: 22 times

Re: National Test Discussion

Post by nicholasmaurer »

chalker wrote:
chalker wrote:
Max event correlation: 0.52
Average event correlation: 0.39
Standard deviation: 0.09
Min event correlation: 0.20

Ecology had a correlation coefficient of 0.28. This means it was about 1 standard deviation below the average correlation for events. There were several events with lower correlations.

In essence, what this means is that statistically, the resulting ranks in Ecology are reasonably well aligned across all teams with the overall team ranks.
Oops.. I did this during lunch and thanks to Bernard posting his sheet I realized I did the wrong ranges. Below are the actual numbers, but the general conclusion is the same:

Max: 0.85
Average: 0.75
Std. Dev.: 0.10
Min: 0.42
Ecology: 0.57

WIDI and EV are lower than Ecology.
8 of the 23 events are under the average.
Chalker, is there ever any discussion of removing or refining events if they consistently show a poor correlation with team scores? If WIDI is so variable and a poor predictor of team outcome, why has its grading/scope not been better standardized or controlled?
Assistant Coach and Alumnus ('14) - Solon High School Science Olympiad
Tournament Director - Northeast Ohio Regional Tournament
Tournament Director - Solon High School Science Olympiad Invitational

Opinions expressed on this site are not official; the only place for official rules changes and FAQs is soinc.org.
nicholasmaurer
Coach
Coach
Posts: 422
Joined: May 19th, 2017, 10:55 am
Division: Grad
State: OH
Has thanked: 1 time
Been thanked: 22 times

Re: National Test Discussion

Post by nicholasmaurer »

Uber wrote:
5. MIT ecology was hands down the most brutal test I've ever taken. Golden Gate also had a much more difficult test than nationals, with more critical thinking involved. We won both. The national test came nowhere close. We finished most stations and double-checked with time to spare.

.
I'm glad you liked my MIT test. It was supposed to be brutal and make sure to separate the good teams from the weak ones. I remember you won first by a significant margin, so congrats!
Assistant Coach and Alumnus ('14) - Solon High School Science Olympiad
Tournament Director - Northeast Ohio Regional Tournament
Tournament Director - Solon High School Science Olympiad Invitational

Opinions expressed on this site are not official; the only place for official rules changes and FAQs is soinc.org.
Locked

Return to “2017 Nationals”

Who is online

Users browsing this forum: No registered users and 2 guests