Musings on Test Length

[SilverBreeze]

However, there are benefits to a short test as well,,
you can make sure everyone can have ample time to think out each question,
and you can make sure everyone tries their best to answer every question and gets credit.
This is just what my opinion is, I'm curios to know what you think!

Hmm that's fair, but I think "try to answer every question so they can answer every question" can be a bit circular. Sorry, I'm still not seeing the benefit specific to answering every question versus leaving some questions blank.
I agree that feeling less rushed can give a better test experience. My concern is that ample time to think it out is also ample time to google search.

I think that everyone should at least have a basic understanding of the whole event and every topic and subtopic.

But not everyone does, and the people who don't still deserve some credit on the test. (you might not meet this goal even if you don't split up the sections)

If you were to split up the studying, it seems to me more like self study, or just understanding a topic a lot so you can do well on the test.

In my experience, the main motivation for splitting is to save work. A lot of students do SciOly for fun, or to hang out with friends; they feel bad if they bomb the event and drop the team score, but want to spend minimal time studying (after all, studying the same thing as your partner is a bit of a waste of time).

I dislike the implication that they only want to do well without learning, as they are still learning the material - the way their knowledge is distributed between them is just different. In 6th grade, I split up RFTS with my partner - she would learn the constellations, and I would learn facts about stars. (people with RFTS experience can probably guess we didn't do very well) We didn't split it up "just so we could do well." Part of it was preference/interest. The main reason, however, was simply how much content the event seemed to have. As a 6th grader new to SciOly, learning the whole event by myself seemed hopelessly overwhelming.

And a huge part of my reasoning for organizing by sections is testing experience. Skipping back and forth and leaving stuff blank/overriding answers is extremely frustrating. I think the debate of whether teams should split sections is productive, but realistically, there's no way to punish splitting topics except by making the test frustrating to take(which is not the point).

But people can do this because of many reasons, one being extreme interest in one aspect, or they (like I said) want to do well in the event. I'm debated on this too, I'm trying to decide whether it's putting teams that went into decent depth in a disadvantage or not. I do know that strategy is a part of the competition, but it's also important to keep in mind that overall knowledge and ability should be tested primarily. Maybe it's all just a part of the competition? Not sure.

From my perspective, they are still learning the material, even if they don't have a holistic understanding of the event. Knowing moray eel ecology is knowing moray eel ecology, even if you know nothing about aragonite saturation. I think part of what I'm chewing over is: what defines knowing the whole event? Many events are split by rotating topics: I wouldn't test freshwater on a marine WQ test with the justification that everyone should have basic understanding of Water Quality, no matter which year. NSO splits these topics using their best judgement; there's nothing inherent about the set of topics in the rules that make them a "complete" event.

This is very interesting- now that I think about it, I find it very hard to actually test what the pair knows collectively. We can't stop them from dividing the test or etc. But I don't get what you mean by the goal of a test writer is to separate each team- sorry if I'm getting this wrong, but does it mean to evaluate each team independently?

Why is it hard? When I say collectively, I mean I'm trying to test what they know together - if I wanted to test what each of them knew by themselves, I would be writing for an individual competition. From this perspective, whether person A or person B or both knows the answer doesn't matter: as long as at least one of them knows it, they should get the points. It's actually much harder to test what they know individually- after all, they're working on the same test.
I can't stop them from dividing the test, and I don't want to. I don't need to tell them which questions they can work on at what time. It's very hard and frustrating to try to do the same problem at the same time, speaking from experience. (only one person can write at a time, so the most that I'm doing is verbally giving the answer and sitting there twiddling my thumbs)

When I say separate each team, I mean separate them by score - since SciOly is a competition, it is bad to have very close scores or lots of ties. That doesn't mean evaluating each team one-on-one, just writing a test that lets teams demonstrate their abilities the best. I say take away advantages in the sense of taking away unfair advantages - if I wanted to take away all advantages, I would just give them all the same score. In my opinion, with two teams with the same amount of knowledge, the faster team should still get a higher score - they put time and effort into getting faster (this is assuming I'm writing questions that use critical thinking instead of "how many dumb recall questions can you blitz through").

[SilverBreeze]

But on a project, you still need to talk about how the parts are going to join together to make a coherent whole. If we're working on a history project, say, each of us might research a different part of the topic, but what we produce at the end is one report - not two short reports stuck together.

Two short reports stuck together have redundancies - that's more like both studying the same thing to less depth, if anything. If you each research a different part of the topic, I assume you will each write a different paragraph of the same report. (unless you're telepathic and can write the second half of a sentence while your partner writes the first)

Whereas what happens when you split the test is more like doing math homework together, and deciding that one of you will answer the odd questions and the other will answer the even questions. There's no actual working together involved - even if you agree that once you've done your questions, you'll work on the ones your partner skipped or whatever.

What do you propose instead? The only feasible alternative to this I see is one of us doing the whole problem, while the other person only checks for mistakes and helps if they get stuck, then switch. That's not really fun or efficient - the person checking doesn't get to actually do a problem.
I see one other alternative - we both do the same math homework, but we each solve every problem, just asking each other for help if we get stuck. To me, that's even worse, and doesn't require more teamwork.

[knightmoves]

I agree that feeling less rushed can give a better test experience. My concern is that ample time to think it out is also ample time to google search.

True. And you know that some people will cheat.

So perhaps the right answer for this year is to make all the tests explicitly "open google". Many of the test subjects allow binders of any length as it is - it's not so much of a stretch to also allow unrestricted googling. It does change the questions that you can reasonably ask, though - you really can't ask straightforward factual questions, as they're easy to google. But whilst googling can support someone in answering critical thinking questions, google can't do your thinking for you.

[sneepity]

From my perspective, they are still learning the material, even if they don't have a holistic understanding of the event. Knowing moray eel ecology is knowing moray eel ecology, even if you know nothing about aragonite saturation. I think part of what I'm chewing over is: what defines knowing the whole event? Many events are split by rotating topics: I wouldn't test freshwater on a marine WQ test with the justification that everyone should have basic understanding of Water Quality, no matter which year. NSO splits these topics using their best judgement; there's nothing inherent about the set of topics in the rules that make them a "complete" event.

Ohh- okay, I think I understand what you're trying to get at. Maybe I'm going too into the specifics, instead looking at it overall!
I guess I'm more for testing that gets at the knowledge of the test takers instead of strategy, and etc. But yeah, what you say is right, there's no way of defining anything. We're trying to accentuate it as something concrete when it's more abstract- there's many factors to the words "hard", "easy", "strategic", and other words thrown around. And I feel like that's something to consider as well, not all of us think the same! So it's interesting what the viewpoints are :o)

[sneepity]

I agree that feeling less rushed can give a better test experience. My concern is that ample time to think it out is also ample time to google search.

True. And you know that some people will cheat.

So perhaps the right answer for this year is to make all the tests explicitly "open google". Many of the test subjects allow binders of any length as it is - it's not so much of a stretch to also allow unrestricted googling. It does change the questions that you can reasonably ask, though - you really can't ask straightforward factual questions, as they're easy to google. But whilst googling can support someone in answering critical thinking questions, google can't do your thinking for you.

There's many, many problems that come with googling though. Bad wifi, chats with someone experienced, some people may have better resources. It's just not, "ethical" to do this on events that require you to know all the information.
(sorry for the double post!)

[SilverBreeze]

There's many, many problems that come with googling though. Bad wifi, chats with someone experienced, some people may have better resources. It's just not, "ethical" to do this on events that require you to know all the information.
(sorry for the double post!)

The general rule I've seen with open-internet stuff is no asking for help from other people. I think the idea is that people with those resources will use them whether you allow it or not, so better make it somewhat even between cheaters and noncheaters.

What do you mean by unethical? With normal tests, there are inequalities too, but fewer - some people can't afford textbooks, some people don't have internet connection to do research, some people can't afford to print that much, some people have experts to talk to/ask for help. There's inequality in every competition - the decision is where to operate within the balance of inequality and testing experience.
I ran Chicken Breeds open-internet, as I wanted to get people interested in chickens, and I don't expect people to study chickens. I don't think open-internet is superior to closed-internet in every way, and it's situational. However, I wouldn't argue that open internet is unethical.

EDIT: just to be clear, I do agree internet speed makes open-internet more unfair to disadvantaged teams

[sneepity]

There's many, many problems that come with googling though. Bad wifi, chats with someone experienced, some people may have better resources. It's just not, "ethical" to do this on events that require you to know all the information.
(sorry for the double post!)

The general rule I've seen with open-internet stuff is no asking for help from other people. I think the idea is that people with those resources will use them whether you allow it or not, so better make it somewhat even between cheaters and noncheaters.

What do you mean by unethical? With normal tests, there are inequalities too, but fewer - some people can't afford textbooks, some people don't have internet connection to do research, some people can't afford to print that much, some people have experts to talk to/ask for help. There's inequality in every competition - the decision is where to operate within the balance of inequality and testing experience.
I ran Chicken Breeds open-internet, as I wanted to get people interested in chickens, and I don't expect people to study chickens. I don't think open-internet is superior to closed-internet in every way, and it's situational. However, I wouldn't argue that open internet is unethical.

EDIT: just to be clear, I do agree internet speed makes open-internet more unfair to disadvantaged teams

Hmm.. I guess if an open- internet test was devised in such a way that you would have to use critical thinking and do a bit more research, it would be fun. But like Knightmoves said, if it was just asking you for straight up facts, that's where more of the inequalities come in. What do you think? Would it be beneficial?

[AstroClarinet]

I agree that feeling less rushed can give a better test experience. My concern is that ample time to think it out is also ample time to google search.

True. And you know that some people will cheat.

So perhaps the right answer for this year is to make all the tests explicitly "open google". Many of the test subjects allow binders of any length as it is - it's not so much of a stretch to also allow unrestricted googling. It does change the questions that you can reasonably ask, though - you really can't ask straightforward factual questions, as they're easy to google. But whilst googling can support someone in answering critical thinking questions, google can't do your thinking for you.

There's many, many problems that come with googling though. Bad wifi, chats with someone experienced, some people may have better resources. It's just not, "ethical" to do this on events that require you to know all the information.
(sorry for the double post!)

I will jump in and offer my point of view:
My state is essentially allowing open book/internet for about half of the events with tests. It is also using Nearpod for most of these events rather than Scilympiad, meaning questions are timed (as in 30 sec - 1.5 min per question, plus blocks of questions where you get 5 - 10 min total) and only one partner can enter in answers.

I think when you have tests timed like this, open internet seems to be okay because there is not, in fact, time to use Google. But neither is there time to use binders/note sheets, especially since splitting up the test is not possible (and of course this is also an issue with station tests).

When it comes to the types of questions given, I do want to point out that the "googeable" answers are very similar to note sheet answers, which from my experience rarely make up the majority of questions on tests anyway (at least, on the better tests I've taken).

Simply put, my opinion is that there's not much reason for allowing open internet if you're just going to make the test ungoogeable (or at least nearly so) anyways. On the flip side, it can indeed be argued that by that same reasoning that it would be fine to allow open internet.

[sneepity]

Hmm.. I guess if an open- internet test was devised in such a way that you would have to use critical thinking and do a bit more research, it would be fun. But like Knightmoves said, if it was just asking you for straight up facts, that's where more of the inequalities come in. What do you think? Would it be beneficial?

If an open-internet test is written to be very critical-thinking heavy, where googling wouldn't help much, I think that's a great way to both give a good testing experience and pretty much prevent cheating. My main concern would be for lower/middle teams - questions designed to be easy are almost always recall or common sense.
An open-internet test with mostly recall is a a googling competition.

I think open-internet is better than closed if you can pull it off well. If you can't, it will be worse than a closed-internet test.

I agree with AstroClarinet's last paragraph, but think it's an argument slightly in favor of open-internet: if you accidentally put on a few trivia questions, teams can google those and get those right with a clean conscience. Otherwise, cheaters would benefit slightly.
I still believe long tests accomplish the same anti-cheating goal as giving one question at a time - they just also give students the ability to manage their own time and do the questions they want to do.

[knightmoves]

I still believe long tests accomplish the same anti-cheating goal as giving one question at a time - they just also give students the ability to manage their own time and do the questions they want to do.

I suppose whether you like long tests or not depends on whether you think getting the correct answer 10s faster than someone else is a particularly valuable thing. Like East said earlier, long tests emphasize speed. I think it's a good thing to get answers fast, but it's not the only thing. I'd rather have a test with progressively more difficult questions that tested whether people could answer the hard ones than a test that tested how fast they could answer easier questions.

One question at a time with a fixed time-per-question is almost the opposite of long tests. As long as you can complete the question in the time available, it doesn't reward speed at all, but does reward accuracy and being able to answer hard questions.

(On the subject of anti-cheating, there are plenty of technological ways to prevent people from cheating in online tests, but none of them are compatible with the goal of allowing people to compete from whatever device they have (which might be a school-issue chromebook, or might be a personally-owned device).