JazzeeB
Berkeley’s customizable admissions system
Title:
JazzeeB: Berkeley’s New Graduate Admissions System
A highly customizable admissions system built by and for campus cooperation.
Submitter:
Andrew Szeri, Dean
University of California, Berkeley, Graduate Division
Email: graddean@berkeley.edu
Phone: 510 642 5472
Project Team:
All team members work for the UCB Graduate Division unless otherwise noted.
Leadership
Corinne Kosmitzki
Shelly Kleinschrodt
Judy Dobry
Technical
Jon Johnson
Yu-Hung Lin
Marvin Plettner
Bill Clark
Wayne Chan
Jason Murray
Patrick McMahon
Assistant Dean Graduate Services, Interim CAO
Senior Project Manager, TPO
Director, IT Systems
Chief Architect for Jazzee
Director of IT and Institutional Research, UCSF Graduate
Division
Software Engineer
Software Engineer
Software Engineer
Software Engineer
Software Engineer
Designer
URLs:
For videos describing JazzeeB, please go to the application and applicant review
training sites.
Summary
JazzeeB, Berkeley’s new online graduate admissions system, allows each
department to create its own online application for university admissions. Using
easy-to-use online forms, the department enters the questions it wants to ask from
applicants and requests the materials it wants uploaded. These departmentspecific questions and upload requests are seamlessly folded into the questions
asked of all Berkeley graduate applicants. This creates an admissions application
that meets the data needs of all Berkley graduate programs as well as the campus
as a whole.
Once applicants complete and submit these applications, JazzeeB allows
departments to set up customized systems to review them. Once again using
easy-to-use online forms, a department can set up as many review groups as it
likes, create customized scoring categories for each group, and create a custom
scoring system for each category.
All of this customization has one goal, helping departments to make more
informed decisions about each applicant by putting IT in the service of their
needs. Feedback from departments tells us that JazzeeB is meeting this goal and
it is doing it while saving money for Berkeley.
This new system saves each of Berkeley’s 100+ departments from $5,000 to
$50,000 per year. It also saves the central campus $200,000 per year. Adding the
departmental and central campus savings together creates a total savings of at
least $700,000 per year.
Additional savings resulted from building this system using “recycled” parts. The
online admissions application came from UCSF and was built out collaboratively.
The plumbing for the online review system came from another UCB Graduate
Division system, GLOW, and the joint build out of these two systems means we
save money on both.
Given this flexibility and savings, it is not surprising that other UC campuses and
even campuses outside the UC system have expressed a strong interest in using
Jazzee. Because Jazzee’s customization features make it highly scalable, we
expect to further drive down costs by working with these schools to both develop
new features and to offer Jazzee as a software–as-a-service product.
Project Description
Berkeley’s old online admissions application began showing its age about
the time the fiscal crisis hit. To find a new system to fit its reduced resources,
UCB published an RFP. Responses to this RFP all involved commercial systems
costing more than $500,000 per year.
The cost alone raised a red flag for UCB but something else troubled our RFP
reviewers: the offered systems failed to meet Berkeley’s fundamental needs.
Because Berkeley offers over 100 very varied graduate programs, it needs to be
able to customize both its admissions application and its review system on a
program by program basis. Unfortunately, all of the RFP systems required
custom programming to make this possible – a costly and unwieldy approach.
Fortunately, we knew about a system that did meet Berkeley’s needs for a
customizable online admissions application. At a Council of Graduate Dean’s
meeting we had seen the UCSF Graduate Division Director of IT and Institutional
Research, Jon Johnson, demonstrate what is now called the Jazzee system.
We contacted Jon and the then Dean of the UCSF Graduate Division, Patricia
Calarco, and they very graciously agreed to work with us. In the best University
of California tradition, UCSF open sourced their code and helped us to get it up
and running. They then worked with us to add additional functionality to the
system.
While we worked with UCSF on the admissions application code, we also began
working on a customizable review system. Because we needed to quickly gather
requirements for this system, we invited representatives from Berkeley’s 100+
campus departments to a two day workshop. At this workshop, we used the
agile/lean technique of process mapping to create the high level systems
requirements.
To flesh out these requirements, we asked for volunteers from departments. A
group of 12 volunteers came forward. Over the next six months they met every
two weeks to determine every aspect of our new system. Working with the
Graduate Division staff they wrote user stories, created a feature lists and made
the hard choices necessary to rank that list. Features below a certain rank were
not programmed because we followed the Facebook maxim “Done is better than
perfect.” We expected push back from the committee on this but it never
happened. Instead committee members embraced this approach and went out and
sold it to their colleagues and to the faculty on campus.
As JazzeeB neared completion, we began a campus-wide educational effort to
teach departments how to use this new system. We created videos which we
posted online to allow department staff and faculty to learn this new system. We
also offered in person training to both faculty and staff. As a result of this
training, most programs created custom admissions applications and customized
review systems and used them successfully.
While this approach worked for most departments, JazzeeB also adapted to
special needs. For example, some departments must use centralized systems that
allow applicants to fill out one application and apply to the same type of program
at schools nationwide. Other departments wanted to use an admissions review
system they had previously developed. To accommodate these needs, JazzeeB
defined user interfaces that allowed departments the freedom they needed.
After a little over a year of development work, JazzeeB went live on September 1,
2011. Over the course of the 2012-13 admissions season, 40,000+ applicants;
100,000+ reference writers and reviewers; and the staff of 100+ departments
successfully used this system. Over 90% of applicants surveyed rated Jazzee as
better than other online admissions systems. Faculty in the many departments
that previously reviewed applications on paper loved the convenience and time
savings provided by online review. Most importantly, by allowing each
department to create a customized application JazzeeB helped reviewers to better
assess the qualifications of each candidate for admission.
Financial Impact
We estimate that JazzeeB will save Berkeley a minimum of $700,000 per year
and could save more than $1 million per year, once fully implemented.
These savings come from two sources, money saved by the central campus and
money saved by departments. JazzeeB’s total cost for the central campus comes
to approximately $300,000 per year. If Berkeley had purchased a commercial
system the cost to central campus would have started at $500,000 and likely
would have been much higher. Consequently, just by using JazzeeB, instead of a
commercial system, central campus saves over $200,000 per year.
Departments also save money by using JazzeeB. By eliminating the need to
collect supplementary materials and to review on paper, Jazzee eliminates
significant clerical costs for all departments. By eliminating the need for
homegrown computer systems to help with this clerical work, Jazzee lowers costs
for departments that have them. Because departments have varied procedures and
receive very different numbers of applications, their savings vary. Some save as
little as $5,000 per year and some save as much as $50,000 per year. Since
Berkeley has 100+ departments this means that Jazzee saves Berkeley’s
departments at least $500,000 per year and almost certainly saves much more.
While our current savings amount to at least $700,000 per year, we expect to save
even more as Jazzee expands to serve other campuses. We are in contact with at
least four other UC campuses interested in using some or all of our code. In
addition, we have received enquiries from campuses outside of the UC system.
Some of these schools want to use the system by installing it on their campus and
others are interested in using it in software as a service mode. Either way
everyone will save because coding for one campus can easily be shared with all
campuses.
Applicant Impact
Applicants log into JazzeeB and fill out information required of all Berkeley
applicants. Most are also asked to provide program specific information.
Previously this additional information (writing samples, additional questionnaires,
work portfolios, etc.) was collected by each department on its own. This required
applicants to get a list of a department’s requirements and then to supply them via
mail, email or a department’s homegrown online system.
This was a logistical nightmare for departments that collected this material via
mail or email. And even departments that collected this material via homegrown
computer systems found this process difficult. With JazzeeB all of this work is
eliminated providing a more convenient process for both applicants and staff.
Administrator Impact
Departmental administrators create their admissions application pages using an
easy to use user interface. They then use JazzeeB’s administrative functions to
manage the applications that come in.
Before JazzeeB, managing these applications presented logistical and security
problems because most Berkeley departments reviewed on paper. In departments
that reviewed on paper the process involved:
 downloading the applications from the online admissions system and
printing them out;
 combining each printed application with materials the applicant was
required to send directly to the department;
 making a Xerox of the combined application for each reviewer; and
 keeping track of all this confidential information.
With JazzeB in place this whole convoluted process went away. Now all
materials are entered via the central online admissions application and can be
reviewed via the online review module. To customize this review module the
department administrator authorizes reviewers, sets up review groups and
recommends applicants for admissions and fellowships.
To authorize a reviewer, the administrator begins entering information about the
reviewer (first name, last name, etc.) in an online form. As the information is
entered the administrator is shown matching records in the Berkeley LDAP
system. When the correct record is displayed, the administrator can easily
authorize that person to review applications. Access is CAS authenticated.
Most departments review applications by group. Authorized administrators set
these groups up using four simple tabs on an online page. The first tab lets the
administrators name the group. The second tab lets them check off the names of
reviewers assigned to the group. The third tab lets them check off the names of
applicants assigned to the group. And the fourth tab allows them to enter the
scoring categories, the scoring method (1 to 10 or A, B, C, etc.) and scoring
instructions for each category.
Once the review groups are set up the reviewers determine which students to
admit. After this determination is made, the administrator uses a simple user
interface to officially convey the admissions recommendations to the Admissions
Office. Using this same interface, an administrator can nominate applicants for
the university fellowships.
Using the information provided by departments, the Admissions Office uses
administrative screens to officially admit or deny applicants. Applicants receive
this decision via email. Those who wish to accept the offer of admission go to
the “Statement of Intent to Register” system. This system gives the student an id
number and passes their information on to the registrar.
Reviewer Impact
Faculty and student reviewers who come to the JazzeeB review site find an
admissions application and review system customized to their needs. The
applications they review include questions requested by their department. The
scoring categories and scoring scales are also determined by their department. In
addition, the review module provides powerful tools for comparing, evaluating
and discussing applicants.
Comparing applicants is easy because information on all applicants is available in
an online spreadsheet. This spreadsheet provides a variety of searching and
sorting functions, but alternatively all applicant information can be downloaded to
Excel with the push of a button.
When the reviewer wants to evaluate an individual applicant, a link on the online
spreadsheet brings up the applicant’s complete record. Since reviewers look
primarily at transcripts, statements of purpose, writing samples and other
materials stored as PDFs, a link takes the reviewer to a single PDF containing all
of these materials. There is no need to open five or six PDF links.
As reviewers evaluate each candidate, they enter comments and scores using a tab
on the applicant information page. Comments and scores can be hidden from
other reviewers during the initial phase of the evaluation process if a department
wants to conduct a “blind” review. During the later parts of the evaluation
process, when reviewer groups meet to make final decisions, summary score and
comment reports make it easy for review groups to quickly see all evaluations in
one place.
At the graduate level, a good fit between the applicant and the program
immensely benefits both parties. By providing reviewers with the data they need
and with good tools for analyzing the data, JazzeeB increases the odds of a good
fit between each department and its accepted applicants.
Technical Information
Server-side technologies:
Java/J2EE, PHP
Client-side technologies:
Testing:
Database:
Infrastructure:
Monitoring:
Project Management:
Agile Methods:
JavaScript, JQuery, HTML, CSS
AppScan, LoadRunner, JUnit, Selenium
Oracle
Linux, Apache
Google Analytics, ChartBeat
SVN, GoogleDocs, Jira, Confluence, Pencil
Scrum, Kanban, User Stories, Effort Points
The JazzeeB online admissions application and administrative modules are written in
PHP. The JazzeeB admissions review module is a Java/J2EE application that makes
extensive use of JavaScript and JQuery.
All modules share a common Oracle database. Application data in this database is stored
primarily as name-value pairs. This NoSQL approach allows database backed forms to
be created using a simple user interface. While many commercial products offer this
capability (see SurveyMonkey, Woofu or Google Forms), they only allow the creation of
one form at a time. Jazzee uniquely allows the creation of a complex multi-form
application using this method.
It is because of this capability that Jazzee is so scalable. Any campus can install Jazzee
and configure an entire admissions application without any custom programming. Just as
important this capability makes Jazzee a prime candidate for use as a software as a
service application with all of the economies of scale that implies.
JazzeeB Benefits
The JazzeeB system offers many benefits:

A Better First Impression. The admissions application is often an
applicant’s first and only contact with Berkeley before she is admitted. By
providing an application described by 90% of applicants surveyed as
“better than other online applications”, JazzeeB makes sure this first
contact is a good one. In competing for the best candidates, this certainly
gives Berkeley an edge.

Better Admissions Decisions. JazzeeB’s customization features provide
reviewers with more germane information. This improved information
increases the applicant’s chance of being admitted to a program where he
can succeed. It also increases a department’s chance of admitting students
who can make the most of what Berkeley has to offer.

IT Costs. JazzeeB saves Berkeley a minimum of $700,000 per year and
could save the campus as much as a $1,000,000 per year. Because Jazzee
can be used by any campus without code changes, it offers economies of
scale that can continue to drive down costs as more schools use the
system.

Security. The pre-JazzeeB paper-based review systems used by many
departments required staff and faculty to keep track of hundreds of
admissions applications containing highly confidential information. The
new online system relieves them of this burden and provides far more
protection against inadvertent or malicious information release.

Building Community. In creating the JazzeeB system, we sought
community involvement throughout our planning process. We asked
representatives from all departments to come to a two day workshop to
create an initial system plan. We asked a group of 12 department staff
members to meet regularly for six months to help us to fine tune the
design. And when the admissions season ended we surveyed all
participating groups and asked them to attend a workshop to plan for next
year.
Each time we asked for input we got well thought out, intelligent
responses. Most of all we got tremendous support for our system. We
discovered that people were willing to accept the tough tradeoffs required
by any system just as long as they had a voice in the process.
Because of the success of this community involvement model, we have
extended it to other systems. And we have had many people on campus
express an interest in using it on their projects..
Reviews
JazzeeB has received an enthusiastic response from all users – applicants,
reviewers and staff.
Over 90% of applicants surveyed said that “compared to online applications of
other universities, the Berkeley application was more user-friendly and easier to
complete”. What they did not know, but departmental staff noticed, was how
much easier it was to complete the application process now that all data is
collected in the same place.
While few applicants remembered our previous admissions process, reviewers
certainly did. For reviewers used to reviewing on paper, JazzeeB was a
revelation. For the first time, they could review from anywhere. Faculty on
vacation or at a conference could participate even if they were a continent away.
And faculty abroad or at home, appreciated not having to keep tracks of stacks of
admissions applications.
Perhaps staff appreciated JazzeeB most of all. Many people in charge of
departmental admissions had gotten used to using their winter break to handle
incoming supplementary materials and to prepare for admissions review. All of
the scanning, emailing, collating and copying required for even a small
department kept them away from their families and cut into their time for more
important things like student advising. With the new system they suddenly had
free time and they let us know how much they appreciated it.
Finally, we are proud that JazzeeB is not perfect. When we started, we knew that
if with our limited resources we produced all of the requested features the system
would go up in September, 2012 instead of September, 2011. We and our campus
advisory group decided the pain and cost of keeping the old system for another
year made this unacceptable. We refused to ask our colleagues to collect
supplementary materials or conduct paper-based review for another year. In
addition, in a time of tight budgets we wanted to do everything we could to help
departments save money.
To get things done in the time we had with our limited resources, we adopted the
agile motto, “Done is better than perfect.” This meant our campus advisory
group ruthlessly cut “nice to have” features and stuck with “must haves”. It also
meant we prioritized the applicant experience sometimes at the expense of other
stakeholders. We knew we would get some pushback on this. What surprised us
is that our campus advisory group members were the ones who pushed back
against the pushback. It was they, more than anyone else, who went out and sold
our approach to both faculty and staff.
Another agile principle we adopted was the idea of continuous improvement.
And our campus advisory committee is ready to do just that. Already they are
asking us when we will start meeting again because they are eager to use their
experience from last admissions season to improve the next iteration of JazzeeB.
With that kind of enthusiasm, we expect JazzeeB to continuously improve for
many years to come.