Tournaments Releasing Raw Scores

For anything Science Olympiad-related that might not fall under a specific event or competition.
Post Reply
RoboMarth
Member
Member
Posts: 71
Joined: August 7th, 2014, 7:51 pm
Division: Grad
State: OH
Has thanked: 20 times
Been thanked: 32 times

Tournaments Releasing Raw Scores

Post by RoboMarth » June 24th, 2020, 1:04 pm

With SciolyFF gaining the ability to produce results pages that include raw scores, I was wondering about the current state of SO Tournaments releasing of raw scores. Mainly:
  • Why do tournaments generally not release raw scores? (Or, has this changed?)
  • Would making raw scores easier to release encourage more tournaments to release them?
  • Would the typical competitor find raw scores useful/motivating?
This is mostly out of curiosity and deciding what the default setting should be in the new Unosmium scoring system (which can support hiding raw scores by converting them to just places if desired).
These users thanked the author RoboMarth for the post (total 4):
sciolyperson1 (June 24th, 2020, 6:04 pm) • sneepity (June 24th, 2020, 6:36 pm) • lumosityfan (June 25th, 2020, 6:50 am) • MadCow2357 (July 1st, 2020, 9:05 am)

User avatar
EastStroudsburg13
Administrator
Administrator
Posts: 3156
Joined: January 17th, 2009, 7:32 am
Division: Grad
State: MD
Pronouns: He/Him/His
Location: At work trying to be a real adult
Has thanked: 20 times
Been thanked: 126 times
Contact:

Re: Tournaments Releasing Raw Scores

Post by EastStroudsburg13 » June 24th, 2020, 3:18 pm

RoboMarth wrote:
June 24th, 2020, 1:04 pm
With SciolyFF gaining the ability to produce results pages that include raw scores, I was wondering about the current state of SO Tournaments releasing of raw scores. Mainly:
  • Why do tournaments generally not release raw scores? (Or, has this changed?)
  • Would making raw scores easier to release encourage more tournaments to release them?
  • Would the typical competitor find raw scores useful/motivating?
This is mostly out of curiosity and deciding what the default setting should be in the new Unosmium scoring system (which can support hiding raw scores by converting them to just places if desired).
Generally, my impression has been that tournaments that don't release raw scores publicly avoid it because doing so opens up a lot of possibilities for post-tournament appeals and such. By keeping raw scores internal, they avoid having to deal with small point-related appeals. It's also a way to maintain the privacy of the competitors in build events especially, because builds are generally standard across any tournament. With only a couple tournaments releasing raw scores, I could see teams not wanting other teams to know what their scores in build events are.

I don't think it's a problem of raw scores being easy to release or not; rather, I think tournaments generally make the conscious decision to keep raw scores private. I think competitors could find raw scores useful if they are kept private to the teams that attended the tournament, because it allows them to see how they did on a test versus the other teams at the tournament. But on the outside, I think their usefulness is limited to build events.

Ultimately, while raw scores might be nice to have, I don't personally see a need to encourage tournaments to release them. Contrast this to test sets, where I think there is a need to encourage tournaments to publicly release partial or complete test sets in order to curb the level of private trading that exists, and promote a more open sharing of resources.
These users thanked the author EastStroudsburg13 for the post (total 4):
sneepity (June 24th, 2020, 6:36 pm) • RoboMarth (June 24th, 2020, 7:59 pm) • lumosityfan (June 25th, 2020, 6:50 am) • MadCow2357 (July 1st, 2020, 9:05 am)
East Stroudsburg South Class of 2012, Alumnus of JT Lambert, Drexel University Class of 2017

Helpful Links
Wiki
Wiki Pages that Need Work
FAQ and SciOly FAQ Wiki
Chat (See IRC Wiki for more info)
BBCode Wiki


So long, and thanks for all the Future Dictator titles!

sciolyperson1
Member
Member
Posts: 685
Joined: April 23rd, 2018, 7:13 pm
Division: C
State: NJ
Pronouns: He/Him/His
Location: WW-P HSN
Has thanked: 52 times
Been thanked: 127 times

Re: Tournaments Releasing Raw Scores

Post by sciolyperson1 » June 24th, 2020, 6:01 pm

EastStroudsburg13 wrote:
June 24th, 2020, 3:18 pm
RoboMarth wrote:
June 24th, 2020, 1:04 pm
With SciolyFF gaining the ability to produce results pages that include raw scores, I was wondering about the current state of SO Tournaments releasing of raw scores. Mainly:
  • Why do tournaments generally not release raw scores? (Or, has this changed?)
  • Would making raw scores easier to release encourage more tournaments to release them?
  • Would the typical competitor find raw scores useful/motivating?
This is mostly out of curiosity and deciding what the default setting should be in the new Unosmium scoring system (which can support hiding raw scores by converting them to just places if desired).
Generally, my impression has been that tournaments that don't release raw scores publicly avoid it because doing so opens up a lot of possibilities for post-tournament appeals and such. By keeping raw scores internal, they avoid having to deal with small point-related appeals. It's also a way to maintain the privacy of the competitors in build events especially, because builds are generally standard across any tournament. With only a couple tournaments releasing raw scores, I could see teams not wanting other teams to know what their scores in build events are.

I don't think it's a problem of raw scores being easy to release or not; rather, I think tournaments generally make the conscious decision to keep raw scores private. I think competitors could find raw scores useful if they are kept private to the teams that attended the tournament, because it allows them to see how they did on a test versus the other teams at the tournament. But on the outside, I think their usefulness is limited to build events.

Ultimately, while raw scores might be nice to have, I don't personally see a need to encourage tournaments to release them. Contrast this to test sets, where I think there is a need to encourage tournaments to publicly release partial or complete test sets in order to curb the level of private trading that exists, and promote a more open sharing of resources.
Check out FERPA, a law that protects the privacy of students' academic records:

https://nces.ed.gov/pubs97/web/97859.asp
FERPA Defines an Education Record
Education records include a range of information about a student that is maintained in schools in any recorded way, such as handwriting, print, computer media, video or audio tape, film, microfilm, and microfiche. Examples are:

Date and place of birth, parent(s) and/or guardian addresses, and where parents can be contacted in emergencies;
Grades, test scores, courses taken, academic specializations and activities, and official letters regarding a student's status in school;
Special education records;
Disciplinary records;
Medical and health records that the school creates or collects and maintains;
Documentation of attendance, schools attended, courses taken, awards conferred, and degrees earned;
Personal information such as a student's identification code, social security number,picture, or other information that would make it easy to identify or locate a student.
Personal notes made by teachers and other school officials that are not shared with others are not considered education records. Additionally, law enforcement records created and maintained by a school or district's law enforcement unit are not education records.
An explanation about why UMich 2018 didn't release raw scores, relating to FERPA, can be found here: viewtopic.php?f=267&t=11006&start=50#p333694
FERPA is kind of iffy. While I highly doubt we would get in any trouble (nobody else has yet), we have to be careful to play it safe with the University. I know Chalker has discussed this, and it is inconclusive. Whether or not Science Olympiad tests are student records is up for debate... again, I doubt anyone would sue us, but crazier things have happened! Basically, we just decided as a group after weighing different possibilities that histograms would be the best option. It is possible to generally discern where teams were at, just not down to the exact decimal place. But like you said, we really hope that the experience is the most important and best part.
Although some tournaments (like Solon, for example) release raw score spreadsheets, usually a histogram can pretty accurately depict the scores at a tournament, without revealing too much information about building event and other study event scores.
These users thanked the author sciolyperson1 for the post (total 6):
sneepity (June 24th, 2020, 6:36 pm) • RoboMarth (June 24th, 2020, 7:59 pm) • EastStroudsburg13 (June 24th, 2020, 8:34 pm) • lumosityfan (June 25th, 2020, 6:50 am) • gz839918 (June 25th, 2020, 7:16 am) • MadCow2357 (July 1st, 2020, 9:05 am)
WW-P HSN '22, Community '18
'20: Gravity - Boomi - WIDI - PPP

Nats:
Team - 3rd '18, '19
Roller - 1st '19
Mystery - 3rd '19
Buggy - 4th '19, 5th '18
Mission - 3rd '17


MIT: Gravity - 1st '20
PUSO: PPP - 1st '20, WIDI -1st '20

Rate my tests!

User avatar
Unome
Moderator
Moderator
Posts: 4254
Joined: January 26th, 2014, 12:48 pm
Division: Grad
State: GA
Location: somewhere in the sciolyverse
Has thanked: 105 times
Been thanked: 36 times

Re: Tournaments Releasing Raw Scores

Post by Unome » June 25th, 2020, 7:25 am

While the more prominent tournaments may be thinking about it actively, I think the biggest reason why most tournaments don't do so is simply because they never have and don't consider the possibility.
These users thanked the author Unome for the post:
RoboMarth (June 25th, 2020, 8:55 am)
Userpage
Chattahoochee High School Class of 2018
Georgia Tech Class of 2022

Opinions expressed on this site are not official; the only place for official rules changes and FAQs is soinc.org.

User avatar
jaspattack
Exalted Member
Exalted Member
Posts: 79
Joined: April 30th, 2018, 8:29 am
Division: C
State: MO
Location: within cyberspace
Has thanked: 46 times
Been thanked: 47 times
Contact:

Re: Tournaments Releasing Raw Scores

Post by jaspattack » June 25th, 2020, 9:01 am

I had a really big, long post typed up about the reasons tournament directors might not want to release scores but it just boils down to what sciolyperson1 and Unome said: tournament directors either think it's a FERPA violation, or don't know that people care about raw scores. A lot of tournament directors don't release any scores at all, let alone raw scores, and I don't know how much that's going to change in the next few years.

As for the actual usefulness of raw scores, I think it's limited to either people who want to do some sort of statistical analysis, or the competitors themselves. Raw scores in events with test components are kind of silly because it doesn't tell you what you missed, it just tells you that you missed points. Tests can vary wildly in terms of quality and raw scores do little to indicate basically anything about a competitor's performance unless you look at the test. Raw scores are mostly of use to builders (like East mentioned) because that actually tells you how your build was scored and what could potentially be improved based on what you saw at the competition.

TL;DR: I actually think having raw scores as a viewable option on Unosmium would be nice, assuming the information is out there. However, I don't really think a big push for releasing raw scores is that necessary. Let's focus on getting tournaments to release basic scoresheets in the first place, yeah?
These users thanked the author jaspattack for the post (total 2):
RoboMarth (June 25th, 2020, 9:13 am) • MadCow2357 (July 1st, 2020, 9:05 am)
2019-20 Events (C): Designer Genes, Forensics, Fossils
2018-19 Events (C): Designer Genes, Fossils, Write It Do It (Mousetrap Vehicle and Experimental Design @ Regionals)
2017-18 Events (B): Rocks and Minerals, Wright Stuff, Write It Do It
I edit the wiki sometimes.

sciolycoach
Coach
Coach
Posts: 37
Joined: July 21st, 2008, 1:37 pm
Division: Grad
State: WI
Has thanked: 0
Been thanked: 8 times

Re: Tournaments Releasing Raw Scores

Post by sciolycoach » June 25th, 2020, 9:55 am

Hello,

I am going to bring up something slightly different to think about that as a tournament director I have encountered. At the Boyceville Invitational we have released raw scores to the teams who have competed, but do not release them publicly. Many coaches and tournament personnel over the years have encouraged us to keep going this way, as many haven't deemed it to be fair to teams that didn't participate in a tournament to see the raw scores. I know some people will get into debates here on this policy, but it has worked for many years with strong support from participating coaches.

Another issue to consider...we always strongly encourage coaches and teams to use raw scores to help drive their growth and development. Unfortunately, some teams in the past have seen that they are very close to have earned a medal and/or lost a tie-breaker after looking at the raw scores and they will immediately begin questioning many different areas of the event/test, trying to "fish" for the points they missed. Now, we will make changes if a legitimate scoring error is found (often times an addition error on total points, etc.)...and before you way we could stop this with better score counseling, with the number of teams and events an even very small error rate will result in many errors. We do our best to continually educate our coaches and event supervisors to check scores many times with many eyes, but sometimes errors get through...we are only human. Of course there aren't many coaches/students who do this but it is very disheartening when it happens and we always reconsider the release of raw scores after this....but we agree that the raw scores are important for the participating teams.

I guess I think releasing raw scores to participating teams is very reasonable but I'm not a big fan of releasing them online for the entire public to see.
These users thanked the author sciolycoach for the post (total 4):
RoboMarth (June 25th, 2020, 10:01 am) • EastStroudsburg13 (June 25th, 2020, 2:57 pm) • pepperonipi (June 27th, 2020, 3:07 pm) • MadCow2357 (July 1st, 2020, 9:05 am)
Andy Hamm
Boyceville Science Olympiad
Boyceville, WI

knightmoves
Member
Member
Posts: 280
Joined: April 26th, 2018, 6:40 pm
Has thanked: 2 times
Been thanked: 8 times

Re: Tournaments Releasing Raw Scores

Post by knightmoves » June 25th, 2020, 1:04 pm

sciolycoach wrote:
June 25th, 2020, 9:55 am
I am going to bring up something slightly different to think about that as a tournament director I have encountered. At the Boyceville Invitational we have released raw scores to the teams who have competed, but do not release them publicly. Many coaches and tournament personnel over the years have encouraged us to keep going this way, as many haven't deemed it to be fair to teams that didn't participate in a tournament to see the raw scores. I know some people will get into debates here on this policy, but it has worked for many years with strong support from participating coaches.
My team certainly found raw scores very useful, from the places we've had them. So you ranked somewhere in the middle of the field in an event - were the scores close together, and you were only a couple of points off the medals (so if you hadn't made that boneheaded error / had brought the right version of your resource sheet / whatever, you'd have placed quite well), or are the better teams getting two or three times your score?

sciolyperson1
Member
Member
Posts: 685
Joined: April 23rd, 2018, 7:13 pm
Division: C
State: NJ
Pronouns: He/Him/His
Location: WW-P HSN
Has thanked: 52 times
Been thanked: 127 times

Re: Tournaments Releasing Raw Scores

Post by sciolyperson1 » June 25th, 2020, 1:41 pm

knightmoves wrote:
June 25th, 2020, 1:04 pm
sciolycoach wrote:
June 25th, 2020, 9:55 am
I am going to bring up something slightly different to think about that as a tournament director I have encountered. At the Boyceville Invitational we have released raw scores to the teams who have competed, but do not release them publicly. Many coaches and tournament personnel over the years have encouraged us to keep going this way, as many haven't deemed it to be fair to teams that didn't participate in a tournament to see the raw scores. I know some people will get into debates here on this policy, but it has worked for many years with strong support from participating coaches.
My team certainly found raw scores very useful, from the places we've had them. So you ranked somewhere in the middle of the field in an event - were the scores close together, and you were only a couple of points off the medals (so if you hadn't made that boneheaded error / had brought the right version of your resource sheet / whatever, you'd have placed quite well), or are the better teams getting two or three times your score?
Couldn't this be done just as easily with a score histogram or distribution?
WW-P HSN '22, Community '18
'20: Gravity - Boomi - WIDI - PPP

Nats:
Team - 3rd '18, '19
Roller - 1st '19
Mystery - 3rd '19
Buggy - 4th '19, 5th '18
Mission - 3rd '17


MIT: Gravity - 1st '20
PUSO: PPP - 1st '20, WIDI -1st '20

Rate my tests!

sciolycoach
Coach
Coach
Posts: 37
Joined: July 21st, 2008, 1:37 pm
Division: Grad
State: WI
Has thanked: 0
Been thanked: 8 times

Re: Tournaments Releasing Raw Scores

Post by sciolycoach » June 25th, 2020, 6:05 pm

Yes they can, but sometimes, depending on the size of the graph it can be very difficult to tell really how far away you were from the top or bottom, especially if a really high or low outlier skews the distribution. This was an issue years ago in Scrambler when top scores were around 5000 but some teams were below zero. For example in this situation it can be difficult on some graphs to tell the difference from 4700 and 5000 vs. 4950 and 5000. In particular I have noticed this in Avogadro. The software is really great and I really like it but sometimes the graph is just a bit too small.
These users thanked the author sciolycoach for the post (total 4):
sciolyperson1 (June 25th, 2020, 6:33 pm) • Unome (June 26th, 2020, 11:03 am) • knightmoves (June 29th, 2020, 8:00 pm) • MadCow2357 (July 1st, 2020, 9:05 am)
Andy Hamm
Boyceville Science Olympiad
Boyceville, WI

Post Reply

Return to “General Competition”

Who is online

Users browsing this forum: No registered users and 1 guest