Mis Graded event tests

Shoot the breeze with other Olympians.
JoeyC
Member
Member
Posts: 1
Joined: Tue Nov 07, 2017 9:43 pm
Division: B
State: TX
Contact:

Mis Graded event tests

Postby JoeyC » Tue Nov 07, 2017 9:56 pm

So I've comeback from an invitational and found out that some of the tests are misgraded (fast facts and meteorology), what do I do? (or what can I do?) I'm relatively sure that I could've placed if they weren't misgraded.

User avatar
Unome
Moderator
Moderator
Posts: 2494
Joined: Sun Jan 26, 2014 8:48 pm
Division: C
State: GA
Location: somewhere in the sciolyverse
Contact:

Re: Mis Graded event tests

Postby Unome » Tue Nov 07, 2017 10:10 pm

JoeyC wrote:So I've comeback from an invitational and found out that some of the tests are misgraded (fast facts and meteorology), what do I do? (or what can I do?) I'm relatively sure that I could've placed if they weren't misgraded.

It depends on the tournament. Invitationals have various policies on this, but chances are you will be unable to get any formal correction. It's very uncommon for any event to end up perfectly graded, since there's a lot of time pressure and significant room for error.
Userpage
Chattahoochee High School

2018 Events: Anatomy and Physiology, Astronomy, Disease Detectives, Dynamic Planet, Herpetology, Materials Science, Microbe Mission, Remote Sensing

JoeyC
Member
Member
Posts: 1
Joined: Tue Nov 07, 2017 9:43 pm
Division: B
State: TX
Contact:

Re: Mis Graded event tests

Postby JoeyC » Wed Nov 08, 2017 12:56 pm

Ok, thanks

User avatar
whythelongface
Exalted Member
Exalted Member
Posts: 167
Joined: Mon Mar 13, 2017 2:42 am
Division: C
State: NJ
Location: At some point (x,y,z) at some time t
Contact:

Re: Mis Graded event tests

Postby whythelongface » Sun Nov 12, 2017 10:09 pm

Question: what counts as a misgraded test? Is it a test that is graded according to the key, when the key is wrong? Or is it test that just does not follow the key?
Who lives, who dies, who tells your story?
West Windsor Plainsboro High School South '18

User avatar
Unome
Moderator
Moderator
Posts: 2494
Joined: Sun Jan 26, 2014 8:48 pm
Division: C
State: GA
Location: somewhere in the sciolyverse
Contact:

Re: Mis Graded event tests

Postby Unome » Sun Nov 12, 2017 10:35 pm

whythelongface wrote:Question: what counts as a misgraded test? Is it a test that is graded according to the key, when the key is wrong? Or is it test that just does not follow the key?

The most common mistakes are either incorrect summation (happened to me yesterday in Materials Science) or a question marked incorrectly (just random mistakes when grading quickly).
Userpage
Chattahoochee High School

2018 Events: Anatomy and Physiology, Astronomy, Disease Detectives, Dynamic Planet, Herpetology, Materials Science, Microbe Mission, Remote Sensing

SPP SciO
Member
Member
Posts: 189
Joined: Tue Mar 24, 2015 3:21 pm
Division: B
State: NY
Location: Brooklyn
Contact:

Re: Mis Graded event tests

Postby SPP SciO » Mon Nov 13, 2017 11:30 pm

On one hand, realize that misgraded tests are equally likely to help you as they are to hurt you. Of course, human error that undoes the result of months of hard work is heartbreaking - which is why you won’t see the scores tests from qualifying tournaments as opposed to invitationals, which are learning experiences.

Supervisors know this and (should always) try to make the tests as easy to grade as possible - think multiple choice over short answers, etc. And as a competitor you can do your part - Write clear, capital letters and unambiguous numbers. If you need to write sentences, get right to the point (test graders not impressed by your beautiful prose). Follow all the directions and make sure you’re recording everything where you’re supposed to. If you’re not sure, ask! The easier you make it on the graders, the more likely you are to get a correct score.
Coach
MS 821 Sunset Park Prep
http://www.sppscio.com

AlphaTauri
Moderator
Moderator
Posts: 824
Joined: Fri Sep 11, 2009 8:41 pm
Division: Grad
State: PA
Location: 04h 35m 55.239s, +16° 30′ 33.49″
Contact:

Re: Mis Graded event tests

Postby AlphaTauri » Tue Nov 14, 2017 3:50 am

SPP SciO wrote:Supervisors know this and (should always) try to make the tests as easy to grade as possible - think multiple choice over short answers, etc.

Hahaha no.

Admittedly, multiple choice is the fastest to grade, but as a testwriter it's absolutely terrible. Not only do I have to come up with (N-1) wrong but reasonable answers, it means that people guessing randomly have a 1 in N chance of being correct based off nothing but blind luck. Also, it's faster for competitors to go through (much easier to pick out the right answer from a list of options than to think of the answer), so I actually have to write more questions to make the test an appropriate length.

Conversely, with short answer I can focus on just writing the question and people are highly unlikely to guess right by random chance, which in turn makes me more confident that my final team rankings are reflective of the amount of knowledge they had and not how much the RNGods were on their side.

Also, my policy on bad handwriting -- though I've rarely had to use it -- is that if I can't read it within a few seconds and neither can any of the other graders, it's wrong.
Hershey Science Olympiad 2009 - 2014
Volunteer for Michigan SO 2015 -

]\/[ Go Blue!

User avatar
Unome
Moderator
Moderator
Posts: 2494
Joined: Sun Jan 26, 2014 8:48 pm
Division: C
State: GA
Location: somewhere in the sciolyverse
Contact:

Re: Mis Graded event tests

Postby Unome » Tue Nov 14, 2017 1:35 pm

AlphaTauri wrote:Admittedly, multiple choice is the fastest to grade, but as a testwriter it's absolutely terrible. Not only do I have to come up with (N-1) wrong but reasonable answers, it means that people guessing randomly have a 1 in N chance of being correct based off nothing but blind luck. Also, it's faster for competitors to go through (much easier to pick out the right answer from a list of options than to think of the answer), so I actually have to write more questions to make the test an appropriate length.

This, definitely. Were it not that I rarely am able to proctor my own tests, I would be able to write, say, 40 good short answer questions instead of 80 multiple choice, and not end up with a bunch of people clustered right above 25% (lol 7-way ties...).
Userpage
Chattahoochee High School

2018 Events: Anatomy and Physiology, Astronomy, Disease Detectives, Dynamic Planet, Herpetology, Materials Science, Microbe Mission, Remote Sensing

SPP SciO
Member
Member
Posts: 189
Joined: Tue Mar 24, 2015 3:21 pm
Division: B
State: NY
Location: Brooklyn
Contact:

Re: Mis Graded event tests

Postby SPP SciO » Tue Nov 14, 2017 3:33 pm

AlphaTauri wrote:
SPP SciO wrote:Supervisors know this and (should always) try to make the tests as easy to grade as possible - think multiple choice over short answers, etc.

Hahaha no.

Admittedly, multiple choice is the fastest to grade, but as a testwriter it's absolutely terrible. Not only do I have to come up with (N-1) wrong but reasonable answers, it means that people guessing randomly have a 1 in N chance of being correct based off nothing but blind luck. Also, it's faster for competitors to go through (much easier to pick out the right answer from a list of options than to think of the answer), so I actually have to write more questions to make the test an appropriate length.

Conversely, with short answer I can focus on just writing the question and people are highly unlikely to guess right by random chance, which in turn makes me more confident that my final team rankings are reflective of the amount of knowledge they had and not how much the RNGods were on their side.

Also, my policy on bad handwriting -- though I've rarely had to use it -- is that if I can't read it within a few seconds and neither can any of the other graders, it's wrong.


Wow, I didn't realize that advice was laughably bad! I guess I was only extrapolating from my own experience. I agree with you - I shouldn't have valued multiple choice over short answers but rather over extended response questions. The issue I have with short answers is that sometimes, a perfectly reasonable answer would be marked incorrect, because maybe the grader didn't anticipate it, or the rubric didn't address it explicitly - for example, if a rubric said "respiration" and the competitor wrote "cellular respiration" it's possible that a quickly scanning grader (not trained in the content) could mark it wrong. Questions that have explicit, unambiguous short answers tend to require less critical thinking. Multiple choice tests are a lot tougher to write but when they're well constructed they can require just as much knowledge to answer correctly, and the "blind luck" factor can eliminated by penalizing incorrect answers a fraction of a point, but not penalizing blank answers.
Coach
MS 821 Sunset Park Prep
http://www.sppscio.com

User avatar
Unome
Moderator
Moderator
Posts: 2494
Joined: Sun Jan 26, 2014 8:48 pm
Division: C
State: GA
Location: somewhere in the sciolyverse
Contact:

Re: Mis Graded event tests

Postby Unome » Tue Nov 14, 2017 4:47 pm

SPP SciO wrote:The issue I have with short answers is that sometimes, a perfectly reasonable answer would be marked incorrect, because maybe the grader didn't anticipate it, or the rubric didn't address it explicitly - for example, if a rubric said "respiration" and the competitor wrote "cellular respiration" it's possible that a quickly scanning grader (not trained in the content) could mark it wrong.

Agreed, which is why I've been using mainly multiple choice for tests I'm not proctoring (and when I don't, I take time to be very clear with what qualifies as a correct answer).
Userpage
Chattahoochee High School

2018 Events: Anatomy and Physiology, Astronomy, Disease Detectives, Dynamic Planet, Herpetology, Materials Science, Microbe Mission, Remote Sensing

User avatar
EastStroudsburg13
Administrator
Administrator
Posts: 2321
Joined: Sat Jan 17, 2009 3:32 pm
Division: Grad
State: MD
Location: At work trying to be a real adult
Contact:

Re: Mis Graded event tests

Postby EastStroudsburg13 » Tue Nov 14, 2017 8:09 pm

For my tests I generally attempt to include a variety of question types; having just one multiple choice section can really help decrease the time of grading, even if I'm the one doing the grading. However, I definitely try to have multiple sections of short answer or other similar formats, as they do help with rank differentiation.

An important factor to consider is when in the day the event is. If it's in the first half of the day, then you can get away with including more short answer and open-ended questions. If you've got people taking tests in the last slot of the day, you're going to want to make sure that there are quick ways to grade, otherwise you're going to be very crunched at the end, and you risk being that event that delays awards. ;)
East Stroudsburg South Class of 2012, Alumnus of JT Lambert, Drexel University Class of 2017

Helpful Links
Wiki
Wiki Pages that Need Work
FAQ and SciOly FAQ Wiki
Chat (See IRC Wiki for more info)
BBCode Wiki


If you have any questions for me, always feel free to shoot me a PM.

nicholasmaurer
Coach
Coach
Posts: 42
Joined: Fri May 19, 2017 5:55 pm
Division: Grad
State: OH
Location: Solon, OH
Contact:

Re: Mis Graded event tests

Postby nicholasmaurer » Tue Nov 14, 2017 8:31 pm

EastStroudsburg13 wrote:For my tests I generally attempt to include a variety of question types; having just one multiple choice section can really help decrease the time of grading, even if I'm the one doing the grading. However, I definitely try to have multiple sections of short answer or other similar formats, as they do help with rank differentiation.

An important factor to consider is when in the day the event is. If it's in the first half of the day, then you can get away with including more short answer and open-ended questions. If you've got people taking tests in the last slot of the day, you're going to want to make sure that there are quick ways to grade, otherwise you're going to be very crunched at the end, and you risk being that event that delays awards. ;)


In the interest of simplifying grading and ensuring objective scoring, I tend to stick to questions that have a clear right-or-wrong answer: multiple choice, fill-in-the-blank, true/false, etc. If you are careful to write these with a varying level of difficulty, it is still possible to get effective differentiation between teams. This is especially true if you have at least 2 questions per team competing (e.g. 90 questions if 45 teams are competing that day).
Assistant Coach and Alumni - Solon High School Science Olympiad
Co-Director - Northeast Ohio Region

User avatar
Unome
Moderator
Moderator
Posts: 2494
Joined: Sun Jan 26, 2014 8:48 pm
Division: C
State: GA
Location: somewhere in the sciolyverse
Contact:

Re: Mis Graded event tests

Postby Unome » Tue Nov 14, 2017 9:13 pm

nicholasmaurer wrote:
EastStroudsburg13 wrote:For my tests I generally attempt to include a variety of question types; having just one multiple choice section can really help decrease the time of grading, even if I'm the one doing the grading. However, I definitely try to have multiple sections of short answer or other similar formats, as they do help with rank differentiation.

An important factor to consider is when in the day the event is. If it's in the first half of the day, then you can get away with including more short answer and open-ended questions. If you've got people taking tests in the last slot of the day, you're going to want to make sure that there are quick ways to grade, otherwise you're going to be very crunched at the end, and you risk being that event that delays awards. ;)


In the interest of simplifying grading and ensuring objective scoring, I tend to stick to questions that have a clear right-or-wrong answer: multiple choice, fill-in-the-blank, true/false, etc. If you are careful to write these with a varying level of difficulty, it is still possible to get effective differentiation between teams. This is especially true if you have at least 2 questions per team competing (e.g. 90 questions if 45 teams are competing that day).

I try... although despite 80 MC and 20 points of free-response, there was still a 7-way tie at 31 points on Remote last weekend. I've yet to master the art of writing medium-difficulty questions - although that could just be because there's no real difference between the 10th place and 30th place teams anyway.
Userpage
Chattahoochee High School

2018 Events: Anatomy and Physiology, Astronomy, Disease Detectives, Dynamic Planet, Herpetology, Materials Science, Microbe Mission, Remote Sensing

User avatar
whythelongface
Exalted Member
Exalted Member
Posts: 167
Joined: Mon Mar 13, 2017 2:42 am
Division: C
State: NJ
Location: At some point (x,y,z) at some time t
Contact:

Re: Mis Graded event tests

Postby whythelongface » Tue Nov 14, 2017 9:54 pm

Unome wrote:
nicholasmaurer wrote:
EastStroudsburg13 wrote:For my tests I generally attempt to include a variety of question types; having just one multiple choice section can really help decrease the time of grading, even if I'm the one doing the grading. However, I definitely try to have multiple sections of short answer or other similar formats, as they do help with rank differentiation.

An important factor to consider is when in the day the event is. If it's in the first half of the day, then you can get away with including more short answer and open-ended questions. If you've got people taking tests in the last slot of the day, you're going to want to make sure that there are quick ways to grade, otherwise you're going to be very crunched at the end, and you risk being that event that delays awards. ;)


In the interest of simplifying grading and ensuring objective scoring, I tend to stick to questions that have a clear right-or-wrong answer: multiple choice, fill-in-the-blank, true/false, etc. If you are careful to write these with a varying level of difficulty, it is still possible to get effective differentiation between teams. This is especially true if you have at least 2 questions per team competing (e.g. 90 questions if 45 teams are competing that day).

I try... although despite 80 MC and 20 points of free-response, there was still a 7-way tie at 31 points on Remote last weekend. I've yet to master the art of writing medium-difficulty questions - although that could just be because there's no real difference between the 10th place and 30th place teams anyway.

To be fair, Remote is one of those events which, especially early in the season, nobody has experience doing and usually get drafted into. The understanding comes after one or two more invitationals.
Who lives, who dies, who tells your story?
West Windsor Plainsboro High School South '18

knottingpurple
Member
Member
Posts: 10
Joined: Sun Apr 10, 2016 12:44 pm
Division: C
State: -
Contact:

Re: Mis Graded event tests

Postby knottingpurple » Wed Nov 15, 2017 3:02 am

Unome wrote:
nicholasmaurer wrote:
EastStroudsburg13 wrote:For my tests I generally attempt to include a variety of question types; having just one multiple choice section can really help decrease the time of grading, even if I'm the one doing the grading. However, I definitely try to have multiple sections of short answer or other similar formats, as they do help with rank differentiation.

An important factor to consider is when in the day the event is. If it's in the first half of the day, then you can get away with including more short answer and open-ended questions. If you've got people taking tests in the last slot of the day, you're going to want to make sure that there are quick ways to grade, otherwise you're going to be very crunched at the end, and you risk being that event that delays awards. ;)


In the interest of simplifying grading and ensuring objective scoring, I tend to stick to questions that have a clear right-or-wrong answer: multiple choice, fill-in-the-blank, true/false, etc. If you are careful to write these with a varying level of difficulty, it is still possible to get effective differentiation between teams. This is especially true if you have at least 2 questions per team competing (e.g. 90 questions if 45 teams are competing that day).

I try... although despite 80 MC and 20 points of free-response, there was still a 7-way tie at 31 points on Remote last weekend. I've yet to master the art of writing medium-difficulty questions - although that could just be because there's no real difference between the 10th place and 30th place teams anyway.


I think free-response can be made more easy to grade - by being super super specific in the answer key, especially for math problems requiring certain formulas written out and so on, by identifying multiple different ways to explain something as long as certain parts were included for either explanation - but the way to make grading easier is to put more and more and more time into answer keys, so I guess at some point it is a question of how much can you reasonably ask of event supervisors who have other things going on in their lives?
"She was determined to drop his acquaintance immediately, and she was very thankful that she had never been acquainted with him at all... she hated him so much that she was resolved never to mention his name again, and she should tell everybody she saw, how good-for-nothing he was."


Return to “General Chat”

Who is online

Users browsing this forum: No registered users and 1 guest