Lexington B Invitational

Area to advertise for your competitions!
User avatar
Umaroth
Exalted Member
Exalted Member
Posts: 398
Joined: February 10th, 2018, 8:51 pm
Division: C
State: CA
Pronouns: He/Him/His
Has thanked: 167 times
Been thanked: 325 times

Re: Lexington B Invitational

Post by Umaroth »

Codebusters ES Review

Okay so I totally didn't forget to make an ES review but here it is now! This is my fourth year doing Codebusters and I will be going into my fifth next year, so I have quite a bit to say about this test and also about making good code tests in general. I've experienced the good, the bad, the atrociously bad, and the really really amazing, so let's jump in here and see my final thoughts at the end!

Stats
Image

Max: 8504

Thoughts
I was asked by some people to get Lexington to run Code after several subpar Code tests at the regional and state level that left students unsatisfied, and I think the test ended up fairly well. The timed question had a straightforward solve path (which I walked through in the awards video, so go check it out), but teams still seemed to struggle with it. The fastest time was the first-place team with 5 minutes 24 seconds, and the next fastest was the second-place team just behind them with 5 minutes 26 seconds (for reference, I had written the timed question with the estimate that it would be a sub-3-minute solve for the top couple teams). In total, only 4 teams out of 14 solved the timed question within the first 10 minutes and an additional 4 solved it for no bonus.

The question type distribution for the rest of the test was as follows:
Aristo 400
Aristo 375
Aristo 400
Erristo 475
Aristo 300
Aristo 350
Aristo 450
Atbash decode 200
"Affine" encode (really just atbash disguised as a 25/25 affine) 225
Affine decode 325
Affine with crib 375
Morbit 450
Pollux 425
Xeno 525
Pat with word hint 675
Pat with 4 letters hint 525
Pat with no hint 625
Baconian 325
Baconian with crib 450
Railfence 225
Vig decrypt300
Vig encrypt 325
Vig with crib 375
Caesar decrypt 175
Caesar decrypt 225
"Vig decrypt" freebie with key AAAA 25

As expected, the questions with the highest number of solves besides the freebie were the two Caesars and the normal Atbash with 7 solves each, and surprisingly the first Baconian had the second-most with 6 solves. The Aristocrat with the highest number of solves was #3 (or #2 if you're looking at the Toebes file) with 6 solves. Unsurprisingly, the Erroristo, Pollux, Xeno, harder Baconian, and cribbed Vig had no solves, likely because teams did not attempt them.

Overall I was satisfied with the results, and I hope that people can use this test as good practice (or a model for what a decent test should look like). Remember, if you are writing a code test, the keys for a good competitor experience is:
1) KEEP THE QUESTIONS SHORT
2) Stay away from movie or book or otherwise well-known quotes unless it's for encryption
3) KEEP THE QUESTIONS SHORT KEEP THE QUESTIONS SHORT KEEP THE QUESTIONS SHORT ESPECIALLY THE MORSE CODE QUESTIONS
4) Include frequency tables, just because you can solve it without a frequency table doesn't mean it's not an unpleasant experience that will leave competitors hating you!!!!!!!!!
5) Please, for the love of whatever fake deity you believe in, keep the questions short. Elaborating, not so short that they are unsolvable, but short enough that the time accrued is mostly time figuring out letters and not time filling them in. This event is about logic, not typing speed!!!
6) Do not undervalue questions. Right now, the time bonus is MUCH too high, so it is better to inflate point values to minimize the effects of the time bonus on the score. With a badly valued test, the timed question will make or break a team's score, and if the timed question is either extremely long or a recognizable quote, say goodbye to all fairness.

So there is my advice/rant about Code test quality, hope this helps anyone who sees it! Linked below is the folder with the test and key, as well as the json file for those who want to practice it with their teammates while socially distanced through the Toebes website:

https://drive.google.com/drive/folders/ ... sp=sharing

- Tim
These users thanked the author Umaroth for the post (total 4):
Hoatzin (May 15th, 2021, 10:01 am) • ArchdragoonKaela (May 15th, 2021, 11:25 am) • azboy1910 (May 15th, 2021, 2:15 pm) • sciolyperson1 (May 15th, 2021, 5:22 pm)
Cal 2026
Troy SciOly 2021 Co-Captain
Proud Padre of the Evola SciOly Program 2018-now
Dank Memes Area Homeschool Juggernaut 2018-now
Sierra Vista SciOly Co-Head Coach 2020-now

Umaroth's Userpage
User avatar
builderguy135
Exalted Member
Exalted Member
Posts: 736
Joined: September 8th, 2018, 12:24 pm
Division: C
State: NJ
Pronouns: He/Him/His
Has thanked: 191 times
Been thanked: 143 times
Contact:

Re: Lexington B Invitational

Post by builderguy135 »

Umaroth wrote: May 14th, 2021, 9:57 pm Codebusters ES Review

Okay so I totally didn't forget to make an ES review but here it is now! This is my fourth year doing Codebusters and I will be going into my fifth next year, so I have quite a bit to say about this test and also about making good code tests in general. I've experienced the good, the bad, the atrociously bad, and the really really amazing, so let's jump in here and see my final thoughts at the end!

Stats
Image

Max: 8504

Thoughts
I was asked by some people to get Lexington to run Code after several subpar Code tests at the regional and state level that left students unsatisfied, and I think the test ended up fairly well. The timed question had a straightforward solve path (which I walked through in the awards video, so go check it out), but teams still seemed to struggle with it. The fastest time was the first-place team with 5 minutes 24 seconds, and the next fastest was the second-place team just behind them with 5 minutes 26 seconds (for reference, I had written the timed question with the estimate that it would be a sub-3-minute solve for the top couple teams). In total, only 4 teams out of 14 solved the timed question within the first 10 minutes and an additional 4 solved it for no bonus.

The question type distribution for the rest of the test was as follows:
Aristo 400
Aristo 375
Aristo 400
Erristo 475
Aristo 300
Aristo 350
Aristo 450
Atbash decode 200
"Affine" encode (really just atbash disguised as a 25/25 affine) 225
Affine decode 325
Affine with crib 375
Morbit 450
Pollux 425
Xeno 525
Pat with word hint 675
Pat with 4 letters hint 525
Pat with no hint 625
Baconian 325
Baconian with crib 450
Railfence 225
Vig decrypt300
Vig encrypt 325
Vig with crib 375
Caesar decrypt 175
Caesar decrypt 225
"Vig decrypt" freebie with key AAAA 25

As expected, the questions with the highest number of solves besides the freebie were the two Caesars and the normal Atbash with 7 solves each, and surprisingly the first Baconian had the second-most with 6 solves. The Aristocrat with the highest number of solves was #3 (or #2 if you're looking at the Toebes file) with 6 solves. Unsurprisingly, the Erroristo, Pollux, Xeno, harder Baconian, and cribbed Vig had no solves, likely because teams did not attempt them.

Overall I was satisfied with the results, and I hope that people can use this test as good practice (or a model for what a decent test should look like). Remember, if you are writing a code test, the keys for a good competitor experience is:
1) KEEP THE QUESTIONS SHORT
2) Stay away from movie or book or otherwise well-known quotes unless it's for encryption
3) KEEP THE QUESTIONS SHORT KEEP THE QUESTIONS SHORT KEEP THE QUESTIONS SHORT ESPECIALLY THE MORSE CODE QUESTIONS
4) Include frequency tables, just because you can solve it without a frequency table doesn't mean it's not an unpleasant experience that will leave competitors hating you!!!!!!!!!
5) Please, for the love of whatever fake deity you believe in, keep the questions short. Elaborating, not so short that they are unsolvable, but short enough that the time accrued is mostly time figuring out letters and not time filling them in. This event is about logic, not typing speed!!!
6) Do not undervalue questions. Right now, the time bonus is MUCH too high, so it is better to inflate point values to minimize the effects of the time bonus on the score. With a badly valued test, the timed question will make or break a team's score, and if the timed question is either extremely long or a recognizable quote, say goodbye to all fairness.

So there is my advice/rant about Code test quality, hope this helps anyone who sees it! Linked below is the folder with the test and key, as well as the json file for those who want to practice it with their teammates while socially distanced through the Toebes website:

https://drive.google.com/drive/folders/ ... sp=sharing

- Tim
Unfortunately, while I completely agree with what everything Umaroth said in his post, I think he missed out on a key detail to pay attention to when writing Codebusters tests: keep the questions short.
These users thanked the author builderguy135 for the post (total 5):
Umaroth (May 15th, 2021, 7:31 pm) • kh.aotic (May 15th, 2021, 8:27 pm) • Klebb (May 15th, 2021, 8:28 pm) • azboy1910 (May 16th, 2021, 12:50 pm) • club21ids (October 8th, 2021, 2:04 am)
West Windsor-Plainsboro High School North '22
BirdSO Co-Director
My Userpage
Locked

Return to “2021 Invitationals”

Who is online

Users browsing this forum: No registered users and 0 guests