Page 5 of 7

Re: University of Michigan SciOly Invitational!!!

Posted: February 20th, 2018, 5:48 pm
by syo_astro
This is an invitational, raw scores are generally released at the least the ones our team has gone to. Isn't that a known coming in? If you want anonymity, don't go to invitationals.
Raw scores are rarely released for many invitationals. At most you know your own score from the test packets they usually give you at the end of the day.
Actually, it's both. Most invites *in the country* don't release anything related to scores. It's more a Michigan invite thing (not sure about every one, but I'll assume you know better than me), along with maybe a few other states (as Unome indicated) and one or two university invites off the top of my head.

Releasing histograms wasn't my decision, but I don't feel the need to protest it. Score distributions are often informative and focus on what matters. Among other things, they indicate how you do relatively, give a measure of how bad, hard, etc a test was, and tell you the max if you care (like with builds). Overall, they tell just as much as raw scores and help you to focus on relative results (which is what matters really).

I am unsure of any issues with this, unless you think distributions or histograms were done incorrectly. If something sounds wrong with this, please say. Otherwise, hope to update soon!

Re: University of Michigan SciOly Invitational!!!

Posted: February 20th, 2018, 5:52 pm
by dcambrid
This is an invitational, raw scores are generally released at the least the ones our team has gone to. Isn't that a known coming in? If you want anonymity, don't go to invitationals.
Raw scores are rarely released for many invitationals. At most you know your own score from the test packets they usually give you at the end of the day.
Actually, it's both. Most invites *in the country* don't release anything related to scores. It's more a Michigan invite thing (not sure about every one, but I'll assume you know better than me), along with maybe a few other states (as Unome indicated) and one or two university invites off the top of my head.

Releasing histograms wasn't my decision, but I don't feel the need to protest it. Score distributions are often informative and focus on what matters. Among other things, they indicate how you do relatively, give a measure of how bad, hard, etc a test was, and tell you the max if you care (like with builds). Overall, they tell just as much as raw scores and help you to focus on relative results (which is what matters really).

I am unsure of any issues with this, unless you think distributions or histograms were done incorrectly. If something sounds wrong with this, please say. Otherwise, hope to update soon!
But this is Michigan and that is the standard in Michigan. Just keep in mind if teams feel like they are not getting the full benefit of coming to the tournament, they may not return. I don't understand the secrecy, that is one of the big takeaways we get from invitationals.

Re: University of Michigan SciOly Invitational!!!

Posted: February 20th, 2018, 6:11 pm
by kenniky
Releasing raw scores violates FERPA.

Re: University of Michigan SciOly Invitational!!!

Posted: February 20th, 2018, 6:12 pm
by MIScioly1

Raw scores are rarely released for many invitationals. At most you know your own score from the test packets they usually give you at the end of the day.
Actually, it's both. Most invites *in the country* don't release anything related to scores. It's more a Michigan invite thing (not sure about every one, but I'll assume you know better than me), along with maybe a few other states (as Unome indicated) and one or two university invites off the top of my head.

Releasing histograms wasn't my decision, but I don't feel the need to protest it. Score distributions are often informative and focus on what matters. Among other things, they indicate how you do relatively, give a measure of how bad, hard, etc a test was, and tell you the max if you care (like with builds). Overall, they tell just as much as raw scores and help you to focus on relative results (which is what matters really).

I am unsure of any issues with this, unless you think distributions or histograms were done incorrectly. If something sounds wrong with this, please say. Otherwise, hope to update soon!
But this is Michigan and that is the standard in Michigan. Just keep in mind if teams feel like they are not getting the full benefit of coming to the tournament, they may not return. I don't understand the secrecy, that is one of the big takeaways we get from invitationals.
We might have to agree to disagree on this one. As a student who competed in MI for six years, I totally understand that most tournaments here release their raw scores to teams. Several members of our board raised valid concerns to releasing them, and releasing the histograms was a compromise that we all liked. We 100% feel that you will attain all of the same information from the histograms that you would from the pure raw scores. In fact, in some cases, you may get more information. For example, raw scores in Hovercraft don't tell you much, since they are curved. Four of the histograms are in the test folder already (Mat sci, Chem lab, Fermi, and Hovercraft) with more to come when I finish studying tonight or tomorrow. With Chem Lab and Fermi, the distribution is also done question-by-question, so teams know how hard each individual question was (thanks kenniky!). Unfortunately, I did not collect such data on Hovercraft and Mat Sci, and obviously, we do not have the tests anymore!

If you have concerns, or specific requests for information to be placed on the histogram, please feel free to message me, although I cannot guarantee I'll respond tonight. Midterms... *sigh*.

Also, kenniky is correct saying that the raw scores release violates FERPA, and we did not collect waivers from students on that. Among other things, that was one of the concerns of our board. As we are a University group, we have to be very careful to comply with University policies (as you know from all of the paperwork you had to bring for your students!).

Re: University of Michigan SciOly Invitational!!!

Posted: February 20th, 2018, 6:26 pm
by Tesel

Raw scores are rarely released for many invitationals. At most you know your own score from the test packets they usually give you at the end of the day.
Actually, it's both. Most invites *in the country* don't release anything related to scores. It's more a Michigan invite thing (not sure about every one, but I'll assume you know better than me), along with maybe a few other states (as Unome indicated) and one or two university invites off the top of my head.

Releasing histograms wasn't my decision, but I don't feel the need to protest it. Score distributions are often informative and focus on what matters. Among other things, they indicate how you do relatively, give a measure of how bad, hard, etc a test was, and tell you the max if you care (like with builds). Overall, they tell just as much as raw scores and help you to focus on relative results (which is what matters really).

I am unsure of any issues with this, unless you think distributions or histograms were done incorrectly. If something sounds wrong with this, please say. Otherwise, hope to update soon!
But this is Michigan and that is the standard in Michigan. Just keep in mind if teams feel like they are not getting the full benefit of coming to the tournament, they may not return. I don't understand the secrecy, that is one of the big takeaways we get from invitationals.
The full benefit of coming to the tournament was well-written tests (by professors or committed students) and a competitive group of teams. As such, it follows the standards of national invitationals. Although raw scores were not released, teams had the opportunity to see their own personal results and find areas of improvement.

Also, it is certainly possible to use the histograms as general guidelines of comparisons without violating FERPA.

Re: University of Michigan SciOly Invitational!!!

Posted: February 20th, 2018, 7:27 pm
by EastStroudsburg13
Forgive me if I'm not grasping the full situation, but can't you match up the histograms with the event placements for the most part, to see roughly what raw scores other teams received?

I don't think there's any problem with an invitational releasing full raw scores, at least to the individual teams that competed (though I'm not super well-versed on FERPA law), but I also don't think it's an outlandish decision to release raw scores in histogram form, either. That just comes down to tournament preference, and I would hope that teams would value the experience enough to offset any concerns about not receiving raw scores.

Final point, each state has their own quirks that have good and not-so-good aspects. Michigan, while having a lot of invitationals and presumably being open within the Michigan community, is rather notorious for having scores that are impossible to find online, which helps nobody. There's a give and take for almost every state in the country (yes, even Pennsylvania).

Re: University of Michigan SciOly Invitational!!!

Posted: February 20th, 2018, 7:38 pm
by MIScioly1
Forgive me if I'm not grasping the full situation, but can't you match up the histograms with the event placements for the most part, to see roughly what raw scores other teams received?

I don't think there's any problem with an invitational releasing full raw scores, at least to the individual teams that competed (though I'm not super well-versed on FERPA law), but I also don't think it's an outlandish decision to release raw scores in histogram form, either. That just comes down to tournament preference, and I would hope that teams would value the experience enough to offset any concerns about not receiving raw scores.

Final point, each state has their own quirks that have good and not-so-good aspects. Michigan, while having a lot of invitationals and presumably being open within the Michigan community, is rather notorious for having scores that are impossible to find online, which helps nobody. There's a give and take for almost every state in the country (yes, even Pennsylvania).
FERPA is kind of iffy. While I highly doubt we would get in any trouble (nobody else has yet), we have to be careful to play it safe with the University. I know Chalker has discussed this, and it is inconclusive. Whether or not Science Olympiad tests are student records is up for debate... again, I doubt anyone would sue us, but crazier things have happened! Basically, we just decided as a group after weighing different possibilities that histograms would be the best option. It is possible to generally discern where teams were at, just not down to the exact decimal place. But like you said, we really hope that the experience is the most important and best part.

Re: University of Michigan SciOly Invitational!!!

Posted: February 20th, 2018, 7:48 pm
by dcambrid
Forgive me if I'm not grasping the full situation, but can't you match up the histograms with the event placements for the most part, to see roughly what raw scores other teams received?

I don't think there's any problem with an invitational releasing full raw scores, at least to the individual teams that competed (though I'm not super well-versed on FERPA law), but I also don't think it's an outlandish decision to release raw scores in histogram form, either. That just comes down to tournament preference, and I would hope that teams would value the experience enough to offset any concerns about not receiving raw scores.

Final point, each state has their own quirks that have good and not-so-good aspects. Michigan, while having a lot of invitationals and presumably being open within the Michigan community, is rather notorious for having scores that are impossible to find online, which helps nobody. There's a give and take for almost every state in the country (yes, even Pennsylvania).
FERPA is kind of iffy. While I highly doubt we would get in any trouble (nobody else has yet), we have to be careful to play it safe with the University. I know Chalker has discussed this, and it is inconclusive. Whether or not Science Olympiad tests are student records is up for debate... again, I doubt anyone would sue us, but crazier things have happened! Basically, we just decided as a group after weighing different possibilities that histograms would be the best option. It is possible to generally discern where teams were at, just not down to the exact decimal place. But like you said, we really hope that the experience is the most important and best part.
I did not realize that there could be any legal ramifications. The last thing you want to do is expose the University to any legal issues. I completely understand!

Re: University of Michigan SciOly Invitational!!!

Posted: February 20th, 2018, 8:03 pm
by MIScioly1
I did not realize that there could be any legal ramifications. The last thing you want to do is expose the University to any legal issues. I completely understand!
Yeah, like most things in law, it's complicated :?: It never really crossed our minds to look into it this year until after the competition, but since we have a lot more time before next year, we can look and see what any potential issues would be with the University. None of us are law students, so we are just trying to stay out of trouble :) We hope you guys had fun and that we'll see you next year!

On a totally separate note, in my own personal non-educated legal opinion, releasing raw scores would be a violation of FERPA unless there is explicit student consent. But really, who is going to sue anybody about that? Anyhow, FERPA is a discussion for a totally separate forum category! This was Chalker's opinion of the law: "FERPA. This is a federal law that restricts the release of personal student information. There are a lot of nuances involved in the law, but I know that several lawyers have looked at it and indicated that various SO related institutions might be at risk were they to release detailed information on competitor performances. There is some ambiguity here, but some of the guidance that's been provided is to NOT release raw data / tests."

Re: University of Michigan SciOly Invitational!!!

Posted: February 23rd, 2018, 7:24 pm
by syo_astro
And another update...on our website we have posted tests (except ecology and with pending updates on dynamic)!

https://www.umichscioly.org/ -> 2018 Results -> Tests (or Final Rankings) -> https://drive.google.com/drive/u/0/fold ... A8ovBVlIU-

Please share with others! I'll also try to get them posted to the test exchange so the site can more centrally have tests (in case people don't know to stumble to certain invite websites). Note this also includes build scores (as histograms...which again, if you look at the histograms I really think you'll see what I mean by they're actually very useful presentations of the data).