The 2008 North Central North America
Regional Programming Contest
Sponsored by IBM
Judging Tips
The notes given below provide observations about the problems, the test data
sets, and suggestions for judging the submitted solutions. Please read through all
of the material provided here, and possibly discuss it during the judge’s
meeting.
 Pat yourself on the back for helping out! This contest is a volunteer effort,
and without lots of you volunteering, the contest wouldn’t happen — or it would
have to be hosted in a single site (with attendant long and potentially difficult
travel) and with registration fees.
 Verify the compilation and execution environments work, and are identical.
While this task is most appropriate for the Site Director, as judges you are
also responsible for the correct and timely judging of solution attempts by the
teams. Try to compile and execute a simple "Hello, world" program on a
contestant's machine, then do the same on the judging machine. Look at the
version numbers for compilers and verify that they are identical.
 Prepare the initial Dialogue Request form and distribute it to all teams. If
you haven’t already done so, make a decision as to how teams are to read the
judge’s input data set for each problem. There are two obvious choices. One is
to expect the team’s program to read from standard input and write to
standard output. I/O redirection for standard input and output (using < and > on
the command line) make it almost impossible for an executing program to
modify the standard input file. The other scenario (frequently used) is to place
the judge’s test data set in a named file (e.g. prob01.txt) and require team
programs to open and read from that file (and write their output to a similarly
named file, like prob1.out). If you choose to have team solutions read from a
data file, you must give names to the data files and inform the teams at the
beginning of the contest. The data files in the judge’s data directory
(JudgingData) are named judge01.in, judge02.in and so forth. Whatever names
2008 North Central North America Regional Programming Contest
Judging Tips, Page 1 of 6
you select for the data files must be advertised to the teams through the
initial Dialogue Request (which has already been provided to the Site Director).
Also inform the teams as to where their output should be directed — to the
display or to a file. A copy of the Dialogue Request should be provided to each
team, either through the message board or in paper.
 Become familiar with the problem set. Read through the problem statements
as soon as possible and become familiar with them. Work through, where
possible, some of the test data shown in the problem statement. If you find
something that doesn’t look quite right, discuss it with the other judges. If
after discussion you still can’t discover how a particular output was obtained,
send e-mail to stanw@unomaha.edu or call (402) 554-3583. I’ll be available
during the entire contest.
 Plan for a consistent, professional judging effort. As much as possible, try to
be consistent in how you judge runs. If possible, it is ideal for the same judge
to be responsible for (that is, to judge) all of the submissions of a given
problem. If there are eight judges, then each judge can be responsible for a
single problem. At some sites this won’t be possible, but each judge could be
responsible for several problems. Try to avoid the “let’s see what we find
wrong with this submission” — or the “gee, look what a silly mistake these folks
made!” syndromes. Remember that this contest is supposed to be fun for the
contestants, not a grueling test of legalese and arcane contest procedures. Also
remember that there are many other sites in this region that are hosting this
contest, and you — the judges — are responsible for determining the winners
across the entire set of sites!
 Become familiar with the judging and logging procedures. As a judge, you
have two tasks: evaluate submissions by teams, and log submissions and the
result of your judging. Prior to reaching you, each submission should have been
logged—that is, the identity of the team and problem, and the time it was
submitted should be recorded. Some sites may choose to have you perform this
task, and the Site Director will inform you of the procedure for doing this.
Other sites may use a networked facility (like PC^2). In any case, you – as
judges – should be aware of this logging procedure, since you are responsible
for the overall correctness of the judging.
If a networked procedure is not used, the forms provided to you by the Site
Director can be used for logging. A judging log sheet should be prepared for
each team at the site. After judging a run, enter in the appropriate column
2008 North Central North America Regional Programming Contest
Judging Tips, Page 2 of 6
(headed by the problem number) the time when the run was submitted, a slash,
and the number of minutes since the start of the contest (12:30 PM CST). If
the run was correct, enter this information at the bottom of the column. If
the run was incorrect for any reason, enter the information in the next unused
entry at the top part of the column. (Multiple log sheets can be used in case a
team should submit a run more than 10 times.) A sample Judging Log Sheet for
a (hopefully) fictitious school has been provided. You need not worry about
completing the last two rows on the Judging Log Sheet until the contest is
complete.
Judging data is also to be entered on the regional score system. This allows
automatic updates of the scoreboard, and guarantees that the required data
(which must be submitted to the ICPC headquarters to guarantee we get our
share of the “wild card” slots for the finals) will be available. Details on the
regional scoring system are provided in a separate document.
 Verify the time-stamping procedure for judged run submissions. Each
solution is to be time-stamped as soon as it is submitted by a team. Please make
certain that all those individuals responsible for assisting in the transport of
runs from teams to judges are aware of the procedure for marking the Time In
field on the Judging Request sheet that accompanies each problem.
 Review the possible responses to a judged run. Each submitted run is either
judged as a Correct Solution, or one of the following messages should be
indicated. DO NOT GIVE ANY RESPONSE TO A JUDGED RUN EXCEPT ONE
OF THESE. DO NOT GIVE ANY ADDITIONAL COMMENTS, REMARKS, OR
SUGGESTIONS TO THE TEAMS. FAILURE TO FOLLOW THIS PROCEDURE
MAY RESULT IN APPEALS.
Correct Solution
The run produces the correct output for the judge’s input data set,
produces no extraneous output, and terminates properly.
Output
formats may vary slightly from those shown in the “Expected Results”
document, within reason, unless the output formatting is a significant
component of the problem.
Wrong answer
This is the canonical incorrect solution message. As soon as any result
produced by the program is incorrect, mark this message.
Run-time error
Examples include floating point overflow, division by zero, domain error
(e.g. square root of a negative number), input error (reading text when a
numeric was expected) and attempt to read past end of file. This will
2008 North Central North America Regional Programming Contest
Judging Tips, Page 3 of 6
normally be marked only when the execution environment produces a
termination message.
Time-Limit exceeded When it is obvious that a program has entered an infinite loop, or the
algorithm used to solve a program is grossly inefficient, then this
message should be indicated. There is no explicit time limit for any
problem. Still, no reasonable solution to any of the problems in the set
should require longer than 60 seconds, and that’s being very, very
generous!
Presentation error
If the output format produced by a program is wrong, or varies greatly
from that shown in the example provided in each problem statement,
then this message will should convey that information to the team.
Likewise, if a problem requires a certain degree of numerical precision,
and the program does not producing that precision, then this message can
also be used. If specific formatting is not required by the problem
statement, then some latitude in the output format can be allowed.
The official rules for the contest are
csalpha.ist.unomaha.edu/acmregn/rules.html.
available
2008 North Central North America Regional Programming Contest
on
the
regional
web
site
at
Judging Tips, Page 4 of 6
Notes on the Judging the Problem Set
Problem 1: Prime Tracks
The soundex idea has been around for a long time, and this variant is right out of Knuth
volume 3. But it does not seem to have been used as contest fodder for a while. This
shouldn’t be much of a challenge for the best teams, but will still provide some challenge for
most. There shouldn’t be much trouble in the judging of this problem since the output
format is simple. Do verify the output format, however, since it’s explicitly stated in the
problem that there must be four spaces (or a tab) between the soundex code and the input
name. The names should logically appear in the same case used in the input, but if it’s been
converted to all one case, that’s not a problem (that is, no presentation error should be
reported). However the first character of the code must be upper case. If it’s correct, but
has the wrong case,then a presentation error should be reported.
Problem 2: Election Results
This problem has a number of potential “gotchas” but is still pretty reasonable. Enumerating
all the possible cases is essential if a team is to deal with the varied input permitted. Note
that there is no limit on the length of a candidate’s name (except that it must be at least
two characters long), and there’s no limit on the length of the name of an office. There is
also no restriction on what single characters (those that are ostensibly used by a voter to
select a candidate) may be used on a ballot. There is a requirement that at least one
whitespace character will appear on each side of each voter mark (otherwise the multiple
marks would be interpreted as a candidate’s name, and one case tests this). The order in
which candidate’s names appear for a given office in the input is not required to be fixed,
and the judge’s input data tests this, too. The problem statement makes it clear that
candidates elected to or tied for a “high” office are ineligible for election to a “low” office,
and this can potentially eliminate all candidates for an office.
Please make certain that the teams (and the judges!) are aware of a change to the problem
statement after they were printed. The printed problem statement does not specify the
order in which the names of tied candidates are to be displayed; in fact, it specifically says
they may be displayed in any order. To make judging easier, the problem (and the expected
output) has been changed to require that these names be sorted prior to their display. So
the tied candidates for election 2, office of ChiefCook, should be displayed in the order
“Bob Fred” and not “Fred Bob” as is printed.
Still, judging the problem should be reasonably simple. Some output lines will be very long
since the data tests cases with 100 tied candidates, and has one case with an extremely long
(but real – see http://everything2.com/index.pl?node_id=1534419) name.
Problem 3: Cannonball Pyramids
This is intended to be a problem that all teams will solve. The output format is explicitly
specified, so presentation errors are possible. A limit on the number of cannonballs on an
edge of the base is given, so a team should be able to manually verify their solution will run
for all possible cases.
2008 North Central North America Regional Programming Contest
Judging Tips, Page 5 of 6
Problem 4: Pencils from the Nineteenth Century
This little problem shouldn’tchallenge the best teams, but a lot of teams may “wander
around” for a while trying to develop a solution approach. Judging shouldn’t be too difficult,
but tedium may set in if you’re using manual judging. The output format is pretty explicit, so
“diff-based” judging is possible, especially since the order in which multiple solutions are
presented is specified. If any of the blank lines are missing, but everything else is correct, a
presentation error should be reported. If any of the solutions are missing or incorrect, then
wrong answer is appropriate.
Problem 5: Fibonacci Period
This problem looks innocent, but may well be one of the most difficult in this year’s contest.
One of the test cases requires a relatively long time to run (both Java and C solutions run a
bit longer than a minute on a variety of machines), so it may take a judge some time to learn
how to distinguish between correct solutions and those that are running way too long. The
output itself is trivial to judge. The likely judging responses are time limit exceeded, wrong
answer, and correct (of course).
Problem 6: Painted Calculator
The challenge here is organizing the combinatorial search so it doesn’t run too long. Correct
solutions might run tens of seconds, but much longer than this probably indicates poor
algorithm choice and should get time limit exceeded. Other than time, the output should be
very easy to judge.
Problem 7: 3-Trees
This little problem should be another one that good teams get quickly, but most teams
shouldn’t have too much difficulty with it at all. The expected approach is to build the 3tree data structure, then do the appropriate traversal and count the nodes visited. Obiously
snafus could result in time limit exceeded, but it should normally run very quickly. Another
potential problem could be a memory fault, but the output format should be easy to judge if
a program completes in a timely manner.
Problem 8: Egyptian Multipllication
The idea behind this problem has been around for quite a while, and it shouldn’t be too tough
to solve. The potential for presentation errors is obvious, but time shouldn’t be a concern.
Problem 9: Binary Clock
The binary clock idea isn’t new, but may be new to many of the teams. Still it’s pretty easy to
understand. What makes this problem different is its use of a text representation of the
time on such a clock. The problem was made a bit more interesting by asking teams to verify
the time is legal and the syntax of the input is correct. (No LALR parsers are required :-)
2008 North Central North America Regional Programming Contest
Judging Tips, Page 6 of 6