The Joel Test: 12 Steps to Better Code
By Joel Spolsky
Have you ever heard of SEMA? It's a fairly esoteric system for measuring how good a software team is. No, wait!
Don't follow that link! It will take you about six years just to understand that stuff. So I've come up with my own,
highly irresponsible, sloppy test to rate the quality of a software team. The great part about it is that it takes about 3
minutes. With all the time you save, you can go to medical school.
The Joel Test
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
Do you use source control?
Can you make a build in one step?
Do you make daily builds?
Do you have a bug database?
Do you fix bugs before writing new code?
Do you have an up-to-date schedule?
Do you have a spec?
Do programmers have quiet working conditions?
Do you use the best tools money can buy?
Do you have testers?
Do new candidates write code during their interview?
Do you do hallway usability testing?
The neat thing about The Joel Test is that it's easy to get a quick yes or no to each question. You don't have to figure
out lines-of-code-per-day or average-bugs-per-inflection-point. Give your team 1 point for each "yes" answer. The
bummer about The Joel Test is that you really shouldn't use it to make sure that your nuclear power plant software is
safe.
A score of 12 is perfect, 11 is tolerable, but 10 or lower and you've got serious problems. The truth is that
most software organizations are running with a score of 2 or 3, and they need serious help, because companies like
Microsoft run at 12 full-time.
Of course, these are not the only factors that determine success or failure: in particular, if you have a great software
team working on a product that nobody wants, well, people aren't going to want it. And it's possible to imagine a
team of "gunslingers" that doesn't do any of this stuff that still manages to produce incredible software that changes
the world. But, all else being equal, if you get these 12 things right, you'll have a disciplined team that can
consistently deliver.
1. Do you use source control?
I've used commercial source control packages, and I've used CVS, which is free, and let me tell you, CVS is fine. But
if you don't have source control, you're going to stress out trying to get programmers to work together. Programmers
have no way to know what other people did. Mistakes can't be rolled back easily. The other neat thing about source
control systems is that the source code itself is checked out on every programmer's hard drive -- I've never heard of
a project using source control that lost a lot of code.
2. Can you make a build in one step?
By this I mean: how many steps does it take to make a shipping build from the latest source snapshot? On good
teams, there's a single script you can run that does a full checkout from scratch, rebuilds every line of code, makes
the EXEs, in all their various versions, languages, and #ifdef combinations, creates the installation package, and
creates the final media -- CDROM layout, download website, whatever.
If the process takes any more than one step, it is prone to errors. And when you get closer to shipping, you want to
have a very fast cycle of fixing the "last" bug, making the final EXEs, etc. If it takes 20 steps to compile the code, run
the installation builder, etc., you're going to go crazy and you're going to make silly mistakes.
For this very reason, the last company I worked at switched from WISE to InstallShield: we required that the
installation process be able to run, from a script, automatically, overnight, using the NT scheduler, and WISE couldn't
run from the scheduler overnight, so we threw it out. (The kind folks at WISE assure me that their latest version does
support nightly builds.)
3. Do you make daily builds?
When you're using source control, sometimes one programmer accidentally checks in something that breaks the
build. For example, they've added a new source file, and everything compiles fine on their machine, but they forgot to
add the source file to the code repository. So they lock their machine and go home, oblivious and happy. But nobody
else can work, so they have to go home too, unhappy.
Breaking the build is so bad (and so common) that it helps to make daily builds, to insure that no breakage goes
unnoticed. On large teams, one good way to insure that breakages are fixed right away is to do the daily build every
afternoon at, say, lunchtime. Everyone does as many checkins as possible before lunch. When they come back, the
build is done. If it worked, great! Everybody checks out the latest version of the source and goes on working. If the
build failed, you fix it, but everybody can keep on working with the pre-build, unbroken version of the source.
On the Excel team we had a rule that whoever broke the build, as their "punishment", was responsible for babysitting
the builds until someone else broke it. This was a good incentive not to break the build, and a good way to rotate
everyone through the build process so that everyone learned how it worked.
Read more about daily builds in my article Daily Builds are Your Friend.
4. Do you have a bug database?
I don't care what you say. If you are developing code, even on a team of one, without an organized database listing
all known bugs in the code, you are going to ship low quality code. Lots of programmers think they can hold the bug
list in their heads. Nonsense. I can't remember more than two or three bugs at a time, and the next morning, or in
the rush of shipping, they are forgotten. You absolutely have to keep track of bugs formally.
Bug databases can be complicated or simple. A minimal useful bug database must include the following data for every
bug:

complete steps to reproduce the bug

expected behavior

observed (buggy) behavior

who it's assigned to

whether it has been fixed or not
If the complexity of bug tracking software is the only thing stopping you from tracking your bugs, just make a simple
5 column table with these crucial fields and start using it.
For more on bug tracking, read Painless Bug Tracking.
5. Do you fix bugs before writing new code?
The very first version of Microsoft Word for Windows was considered a "death march" project. It took forever. It kept
slipping. The whole team was working ridiculous hours, the project was delayed again, and again, and again, and the
stress was incredible. When the dang thing finally shipped, years late, Microsoft sent the whole team off to Cancun for
a vacation, then sat down for some serious soul-searching.
What they realized was that the project managers had been so insistent on keeping to the "schedule" that
programmers simply rushed through the coding process, writing extremely bad code, because the bug fixing phase
was not a part of the formal schedule. There was no attempt to keep the bug-count down. Quite the opposite. The
story goes that one programmer, who had to write the code to calculate the height of a line of text, simply wrote
"return 12;" and waited for the bug report to come in about how his function is not always correct. The schedule was
merely a checklist of features waiting to be turned into bugs. In the post-mortem, this was referred to as "infinite
defects methodology".
To correct the problem, Microsoft universally adopted something called a "zero defects methodology". Many of the
programmers in the company giggled, since it sounded like management thought they could reduce the bug count by
executive fiat. Actually, "zero defects" meant that at any given time, the highest priority is to eliminate bugs before
writing any new code. Here's why.
In general, the longer you wait before fixing a bug, the costlier (in time and money) it is to fix.
For example, when you make a typo or syntax error that the compiler catches, fixing it is basically trivial.
When you have a bug in your code that you see the first time you try to run it, you will be able to fix it in no time at
all, because all the code is still fresh in your mind.
If you find a bug in some code that you wrote a few days ago, it will take you a while to hunt it down, but when you
reread the code you wrote, you'll remember everything and you'll be able to fix the bug in a reasonable amount of
time.
But if you find a bug in code that you wrote a few months ago, you'll probably have forgotten a lot of things about
that code, and it's much harder to fix. By that time you may be fixing somebody else's code, and they may be in
Aruba on vacation, in which case, fixing the bug is like science: you have to be slow, methodical, and meticulous, and
you can't be sure how long it will take to discover the cure.
And if you find a bug in code that has already shipped, you're going to incur incredible expense getting it fixed.
That's one reason to fix bugs right away: because it takes less time. There's another reason, which relates to the fact
that it's easier to predict how long it will take to write new code than to fix an existing bug. For example, if I asked
you to predict how long it would take to write the code to sort a list, you could give me a pretty good estimate. But if
I asked you how to predict how long it would take to fix that bug where your code doesn't work if Internet Explorer
5.5 is installed, you can't even guess, because you don't know (by definition) what's causing the bug. It could take 3
days to track it down, or it could take 2 minutes.
What this means is that if you have a schedule with a lot of bugs remaining to be fixed, the schedule is unreliable. But
if you've fixed all the known bugs, and all that's left is new code, then your schedule will be stunningly more accurate.
Another great thing about keeping the bug count at zero is that you can respond much faster to competition. Some
programmers think of this as keeping the product ready to ship at all times. Then if your competitor introduces a killer
new feature that is stealing your customers, you can implement just that feature and ship on the spot, without having
to fix a large number of accumulated bugs.
6. Do you have an up-to-date schedule?
Which brings us to schedules. If your code is at all important to the business, there are lots of reasons why it's
important to the business to know when the code is going to be done. Programmers are notoriously crabby about
making schedules. "It will be done when it's done!" they scream at the business people.
Unfortunately, that just doesn't cut it. There are too many planning decisions that the business needs to make well in
advance of shipping the code: demos, trade shows, advertising, etc. And the only way to do this is to have a
schedule, and to keep it up to date.
The other crucial thing about having a schedule is that it forces you to decide what features you are going to do, and
then it forces you to pick the least important features and cut them rather than slipping into featuritis (a.k.a. scope
creep).
Keeping schedules does not have to be hard. Read my article Painless Software Schedules, which describes a simple
way to make great schedules.
7. Do you have a spec?
Writing specs is like flossing: everybody agrees that it's a good thing, but nobody does it.
I'm not sure why this is, but it's probably because most programmers hate writing documents. As a result, when
teams consisting solely of programmers attack a problem, they prefer to express their solution in code, rather than in
documents. They would much rather dive in and write code than produce a spec first.
At the design stage, when you discover problems, you can fix them easily by editing a few lines of text. Once the code
is written, the cost of fixing problems is dramatically higher, both emotionally (people hate to throw away code) and
in terms of time, so there's resistance to actually fixing the problems. Software that wasn't built from a spec usually
winds up badly designed and the schedule gets out of control. This seems to have been the problem at Netscape,
where the first four versions grew into such a mess that management stupidly decided to throw out the code and
start over. And then they made this mistake all over again with Mozilla, creating a monster that spun out of control
and took several years to get to alpha stage.
My pet theory is that this problem can be fixed by teaching programmers to be less reluctant writers by sending them
off to take an intensive course in writing. Another solution is to hire smart program managers who produce the
written spec. In either case, you should enforce the simple rule "no code without spec".
Learn all about writing specs by reading my 4-part series.
8. Do programmers have quiet working conditions?
There are extensively documented productivity gains provided by giving knowledge workers space, quiet, and privacy.
The classic software management book Peopleware documents these productivity benefits extensively.
Here's the trouble. We all know that knowledge workers work best by getting into "flow", also known as being "in the
zone", where they are fully concentrated on their work and fully tuned out of their environment. They lose track of
time and produce great stuff through absolute concentration. This is when they get all of their productive work done.
Writers, programmers, scientists, and even basketball players will tell you about being in the zone.
The trouble is, getting into "the zone" is not easy. When you try to measure it, it looks like it takes an average of 15
minutes to start working at maximum productivity. Sometimes, if you're tired or have already done a lot of creative
work that day, you just can't get into the zone and you spend the rest of your work day fiddling around, reading the
web, playing Tetris.
The other trouble is that it's so easy to get knocked out of the zone. Noise, phone calls, going out for lunch, having to
drive 5 minutes to Starbucks for coffee, and interruptions by coworkers -- especially interruptions by coworkers -- all
knock you out of the zone. If a coworker asks you a question, causing a 1 minute interruption, but this knocks you
out of the zone badly enough that it takes you half an hour to get productive again, your overall productivity is in
serious trouble. If you're in a noisy bullpen environment like the type that caffeinated dotcoms love to create, with
marketing guys screaming on the phone next to programmers, your productivity will plunge as knowledge workers
get interrupted time after time and never get into the zone.
With programmers, it's especially hard. Productivity depends on being able to juggle a lot of little details in short term
memory all at once. Any kind of interruption can cause these details to come crashing down. When you resume work,
you can't remember any of the details (like local variable names you were using, or where you were up to in
implementing that search algorithm) and you have to keep looking these things up, which slows you down a lot until
you get back up to speed.
Here's the simple algebra. Let's say (as the evidence seems to suggest) that if we interrupt a programmer, even for a
minute, we're really blowing away 15 minutes of productivity. For this example, lets put two programmers, Jeff and
Mutt, in open cubicles next to each other in a standard Dilbert veal-fattening farm. Mutt can't remember the name of
the Unicode version of the strcpy function. He could look it up, which takes 30 seconds, or he could ask Jeff, which
takes 15 seconds. Since he's sitting right next to Jeff, he asks Jeff. Jeff gets distracted and loses 15 minutes of
productivity (to save Mutt 15 seconds).
Now let's move them into separate offices with walls and doors. Now when Mutt can't remember the name of that
function, he could look it up, which still takes 30 seconds, or he could ask Jeff, which now takes 45 seconds and
involves standing up (not an easy task given the average physical fitness of programmers!). So he looks it up. So now
Mutt loses 30 seconds of productivity, but we save 15 minutes for Jeff. Ahhh!
9. Do you use the best tools money can buy?
Writing code in a compiled language is one of the last things that still can't be done instantly on a garden variety
home computer. If your compilation process takes more than a few seconds, getting the latest and greatest computer
is going to save you time. If compiling takes even 15 seconds, programmers will get bored while the compiler runs
and switch over to reading The Onion, which will suck them in and kill hours of productivity.
Debugging GUI code with a single monitor system is painful if not impossible. If you're writing GUI code, two monitors
will make things much easier.
Most programmers eventually have to manipulate bitmaps for icons or toolbars, and most programmers don't have a
good bitmap editor available. Trying to use Microsoft Paint to manipulate bitmaps is a joke, but that's what most
programmers have to do.
At my last job, the system administrator kept sending me automated spam complaining that I was using more than
... get this ... 220 megabytes of hard drive space on the server. I pointed out that given the price of hard drives these
days, the cost of this space was significantly less than the cost of the toilet paper I used. Spending even 10 minutes
cleaning up my directory would be a fabulous waste of productivity.
Top notch development teams don't torture their programmers. Even minor frustrations caused by using
underpowered tools add up, making programmers grumpy and unhappy. And a grumpy programmer is an
unproductive programmer.
To add to all this... programmers are easily bribed by giving them the coolest, latest stuff. This is a far cheaper way
to get them to work for you than actually paying competitive salaries!
10. Do you have testers?
If your team doesn't have dedicated testers, at least one for every two or three programmers, you are either shipping
buggy products, or you're wasting money by having $100/hour programmers do work that can be done by $30/hour
testers. Skimping on testers is such an outrageous false economy that I'm simply blown away that more people don't
recognize it.
Read Top Five (Wrong) Reasons You Don't Have Testers, an article I wrote about this subject.
11. Do new candidates write code during their interview?
Would you hire a magician without asking them to show you some magic tricks? Of course not.
Would you hire a caterer for your wedding without tasting their food? I doubt it. (Unless it's Aunt Marge, and she
would hate you forever if you didn't let her make her "famous" chopped liver cake).
Yet, every day, programmers are hired on the basis of an impressive resumé or because the interviewer enjoyed
chatting with them. Or they are asked trivia questions ("what's the difference between CreateDialog() and
DialogBox()?") which could be answered by looking at the documentation. You don't care if they have memorized
thousands of trivia about programming, you care if they are able to produce code. Or, even worse, they are asked
"AHA!" questions: the kind of questions that seem easy when you know the answer, but if you don't know the
answer, they are impossible.
Please, just stop doing this. Do whatever you want during interviews, but make the candidate write some code. (For
more advice, read my Guerrilla Guide to Interviewing.)
12. Do you do hallway usability testing?
A hallway usability test is where you grab the next person that passes by in the hallway and force them to try to use
the code you just wrote. If you do this to five people, you will learn 95% of what there is to learn about usability
problems in your code.
Good user interface design is not as hard as you would think, and it's crucial if you want customers to love and buy
your product. You can read my free online book on UI design, a short primer for programmers.
But the most important thing about user interfaces is that if you show your program to a handful of people, (in fact,
five or six is enough) you will quickly discover the biggest problems people are having. Read Jakob Nielsen's article
explaining why. Even if your UI design skills are lacking, as long as you force yourself to do hallway usability tests,
which cost nothing, your UI will be much, much better.
Four Ways To Use The Joel Test
1. Rate your own software organization, and tell me how it rates, so I can gossip.
2. If you're the manager of a programming team, use this as a checklist to make sure your team is working as
well as possible. When you start rating a 12, you can leave your programmers alone and focus full time on
keeping the business people from bothering them.
3. If you're trying to decide whether to take a programming job, ask your prospective employer how they rate
on this test. If it's too low, make sure that you'll have the authority to fix these things. Otherwise you're going
to be frustrated and unproductive.
4. If you're an investor doing due diligence to judge the value of a programming team, or if your software
company is considering merging with another, this test can provide a quick rule of thumb.
The Joel Test for Web Development
21 September 2004
Previously, I have asserted that web development is software development and that the lessons learned by the
software industry can and should be applied to our own work. One good source of commentary on software
development is Joel on Software. Joel takes a down-to-earth approach and communicates lucidly (his books come
highly recommended, too). Based on his experience as a developer and running his own software company, Joel
devised a quick test to help anyone rate the quality of their development team. You can read about it – in fact please
go do that now. I’ll wait.
As the Joel Test is based on traditional software development, I though it would be interesting to try and apply it to a
web development team (to my web development team), and see how we come out. Of course, some of the concepts
need to be tweaked to apply more succinctly to typical web development, but hopefully between us (I welcome your
feedback) we can come up with something close the The Joel Test for Web Development.
This is going to be lengthy, so I’ll run it over a few days. Here goes.
Do you use source control?
So we know about source control – I’ve written about it a few times before. Source control for web development is
typically done using Microsoft’s Visual Source Safe or either CVS or Subversion (SVN). From experience you tend to
find teams pick a platform and run with it (as is sensible), so ASP and ASP.NET shops tend to run VSS and developers
working in PHP/Python/Perl/Ruby/whatever with go with either CVS or SVN.
That said, I suspect you’ll find that most smaller development teams cannot answer yes to this question. Whereas
source control is pretty commonplace in software development, it’s less common for web development. This is
probably because no-one’s figured out they’re writing software yet. We used to work like this, and figured that as
there were only two of us doing the majority of the development we’d be ok. We’re using SVN on our current project.
It works really well.
Can you make a build in one step?
This is one of those points that feels like it doesn’t apply to web development. However, it helps to read Joel’s point of
clarification:
By this I mean: how many steps does it take to make a shipping build from the latest source snapshot?
This applies – the ‘shipping build’ in our case is a deployment of the app to a server. This could be for any number of
reasons – you might need to deploy the app so that it can be tested, or so the client can demo it to investors, or
whatever, the fact remains that deploying a web app is a pretty involved process. Steps for this generally include:

Taking a source snapshot

Taking a copy of the database, free from any junk data

Tweaking of configuration files so that file paths are correct for the new location

Moving the source to its new location

Importing the database dump into a clean database

Setting up a site or virtual site on the web server

Configuring any required DNS entries
Only when all that is done can you begin debugging any differences between the two, and if the servers aren’t
specifically configured to be identical there’s bound to be a few differences. This process can be lengthy – for
deploying to a new server for the first time I typically set aside a whole day. Subsequent deployments probably only
take an hour or two. If this could be reduced to a one-step process, I’d save literally days of effort on each project. If
I could get it down to a three-step process, even that would be supremely beneficial. Most importantly, the mere fact
that there’s so much scope to make mistakes and miss out a step or two means that it really needs to be streamlined
into byte-sized chunks.
Do you make daily builds?
In porting this test to a web development context, it’s important to get behind the reasoning for the question. The
point of carrying out daily builds is to quickly spot bugs as they creep in. Fresh bugs are easier to fix than old ones, so
it’s really important to be able to spot new bugs as they arise. The importance of this is directionally proportional to
the severity of the bug, too – if someone’s broken a major piece of the application, you need to know ASAP.
What we’re talking about here is not the ability to deploy the app to a server every day, but the ability to check the
integrity of the code at any point. If you’re using source control (yes, that old chestnut) this is surprisingly easy to do.
You will invariably end up with a site for each developer, serving the files from their working copy so that they can
test their changes before eventually checking them back into the source control. I use a naming scheme of developername.project.dev for this. The simple solution is to set up yet another site (I use just project.dev) running a ‘clean’
checkout of the files. Update the files daily from the source control, and you have yourself a testable daily build.
Comments
1. Jesse Sep 22, 01:06 AM
Ok ok.. you have inspired me. Since we are just moving towards more serious PHP app development at my
place of work I have wondered at what point do we want to manage this different? I’ll start thinking more
about it next week ;) To provide my excuse – I have never had the support to do much but little hacks and
such, now I find myself managing small application development.
I think I am at the point where I need to do things a bit differently. At a guess, others creep into full blown
app development the same way. First off make something like a form/mailer/feedback thing.. and then
maybe an RSS creator/reader or two… now I want a system that pulls content from a DB into static pages and
back again once edited with blah blah… It just snowballs as you discover PHP (or asp? nooo).
2. Peter Mount Sep 22, 01:50 AM
Hi
I’m glad this subject is being talked about. At TAFE (“Technical and Further Education” – a name for some
tertiary schools in Australia) I had two semesters of classes in web development. I studied both ColdFusion
and ASP.
It really helped me that I’d already studied programming languages like Java and Visual Basic. Just having
some background in programming logic was really appropriate. I’m really interested in the multi step process
you’ve described. There are so many things mentioned here that I didn’t learn in my studies.
The biggest point that hits me is “Web development is software development”. That sentence alone really
summarises everything for me. Just as important, it’s the catch phrase I can use to explain web development
to the people I know who wouldn’t otherwise understand.
3. Ivan Kurmanov Sep 22, 09:05 PM
Web development is more than just software development and more that just web applications. Just let’s not
mix different things. Web development may be design for web, it may be hand-coding HTML & CSS, it may be
creating web standards, it may be creating publishing systems, which are not necessarily web-applications.
Otherwise, isn’t it obvious that web application is software?
The Joel Test for Web Development - Part 2
22 September 2004
This is the second part in a series looking at applying The Joel Test to a web development setting. You should read
the opening part before continuing, else you’ll think I’m a crazy loon.
Do you have a bug database?
So, once you’ve got your daily builds going and are hopefully performing daily testing on any new code, you’ll be
spotting new bugs. There’s no shame in having bugs. They’re a daily part of development work and merely a
symptom of the human tendency not to be perfect. The shame is in not documenting them sufficiently so that they
can be fixed without hassle. For this, you need a bug database.
Bug databases have to work on a couple of levels. Firstly they have to cater to the needs of the developer. A
developer should be able to log bugs in his own or a colleagues work with comprehensive technical detail. On the
other hand you have your regular testers. Typically these folk don’t have the technical expertise to explain why
something has gone wrong, only that they tried xyz and it didn’t work. Your bug database has to be easy to use and
able to tease as much information out of these non-technical users as possible. It’s actually not so hard.
We use a web-based tool called Mantis, but there are lots of options out there. Joel has one of his own, Trac is
another excellent option (I’m looking at switching to this), and there are plenty more besides.
Do you fix bugs before writing new code?
I asserted in the first part of this article that new bugs are easier to fix than old ones. This is a Joel-ism which I won’t
try to re-explain, just go read the corresponding item on The Joel Test. This has to apply to web development pretty
well – all the points about forgetting how code works and having to relearn the process before beginning to look for
the bug all ring true.
Personally, I can’t say that I adhere to this rule 100% at the moment, but my tendency is to fix bugs as soon as I
spot them. That is, unless they look tricky when I’ll leave them for Some Better Day.
Do you have an up-to-date schedule?
Ohboy, schedules. Estimating development hours is always impossibly difficult. I find that breaking the schedule up
into broad chunks of functionality rather than anything too specific helps to both keep the project on track and stop
me getting depressed whenever something small runs over, even though there may be no impact to the overall
timeline. Estimating hours for ‘user authentication system’ is a lot easier than ‘forgotten password script’, for
example.
The key point here is making sure the schedule is kept up to date. An out of date schedule is about as useful as
having no schedule at all, and if things genuinely take longer than you estimated, it’s amazing how little people mind
sometimes when they have plenty of notice.
Do you have a spec?
One great aspect about working on regular web site projects is that changes are often very easy and quick to make.
Of course, this doesn’t necessarily hold true for application development, where a change that may be perceived as
small from the outside and involve significant restructuring and hours or wasted work. The danger being that those
you’re working with might not recognise that they’re commissioning software development – they may well still be in
a standard web site mindset.
It is here that the spec is your friend. By documenting the whole application before you start writing any code, you
give yourself the best possible chance of not having to refactor the entire application midway through. Naturally,
when working on larger projects it’s common to think of things you’d not previously considered once you get into the
nitty-gritty of the app. Getting into the nitty-gritty on paper first enables those changes to be made where they are
cheap to make.
We tend to run a couple of different of spec documents on projects at work. The first is a Functional Specification
which describes all the features the application will have, how each of those features works and what tasks the user
can achieve with them. This is the document that enables the client to see that the proposed application will meet the
needs and requirements they are trying to address. The second is a Screen Specification. This details literally each
screen of the application, the data that screen collects and the data it displays. This document is reenforced with a
flat graphic or HTML-only build of each main screen so that the client can see what their getting. Once all that is
signed off, we begin the build, updating the documents as we go to reflect any minor tweaks and changes so that the
documents reflect the application at the end as well as they did in the beginning.
To be continued …
The story so far
The Joel Test for Web Development - Part 3
26 September 2004
This is the third part in a series looking at applying The Joel Test to a web development setting. You should read the
first two before continuing, else you’ll think I’ve lost the plot and am about to start eating your rabbit.
Do programmers have quiet working conditions?
So, you’re now working to an up-to-date spec, you’re logging bugs effectively and fixing them as you go, you’re
running to schedule and your source control system is keeping everything in check. If you could only hear yourself
think you might be able to get some work done! This point really seams to go against the grain if your development
environment is the general studio space of a creative company. Your co-workers may work best in a ‘lively’
atmosphere with music, chatting, phone conversations, maybe even video games (we try to keep ours in a separate
room!), but it’s often very difficult to program in that environment. And that’s okay – it’s not your fault.
It’s a point that Joel raises, but something I’ve personally found to be true for the whole time I’ve been coding is that
once I’ve got my head down and am in the zone, World War III (or even Burnout 3) could be going on behind me and
I wouldn’t notice. However, the zone is a very difficult place to get to. If it’s noisy by the time I start work, there’s
little hope of ever getting set in, and once I’m there, any small distraction like a phone call can knock me out.
Rule #1: never phone a developer when an email will do. Developers are not easily distracted by incoming email, yet
a ringing phone demands attention NOW. Besides, developers don’t know how to operate office phones anyway.
There aren’t always easy solutions to conflicting needs in a working environment, however, if you find yourself
wishing you could just work at home for a few days for the peace-and-quiet, you probably need to flag the issue with
your boss sharpish. It’s his problem, not yours.
Do you use the best tools money can buy?
Traditional software development usually requires some pretty meaty tools. Last time I installed Visual Studio, for
example, it came on about 308 DVDs and took around a week to complete. Compiling code takes processing power,
and the more you’ve got the less time it takes so the more productive you will be. Us web developers, on the other
hand, work mainly with text files. So who needs a quick computer for editing text files?
Us! The above would be a valid argument if web developers sat around writing a novel in a single text file all day, but
that is obviously not the case. What we actually do a lot of is flicking around between applications. Your typical web
dev set up will include a text editor with about 30 documents open, at least a couple of browsers with multiple tabs
and windows, an email client, a database tool, either a shell or a Remote Deskop session to a development server,
and optionally a graphics editor and perhaps a copy of Word with spec documents or source content. And what are we
doing? Edit, save, switch, reload, switch, edit, save, switch, reload .. you know the drill. I don’t know about you, but
the very last thing I try to do on a computer that is struggling under its own weight is switch between applications. To
be productive, you need a machine that not only will happily run all of the above at once, but will be really bloody
quick at switching between them. This means lots of memory, plenty of spare CPU, and a reasonable 2D graphics
card (so that screen redraws are snappy). Disc space isn’t an issue.
The other factor that is really critical is what I like to call elbow room. Smarmy people call it ‘screen real estate’ and
deserve a kicking, but the fact remains that when working with lots of documents, tools and browsers you need a lot
of room on your screen. We’re talking high-res, baby. If anyone tells you that a single screen at 1024×768 is big
enough for modern web development they are quite literally having a laugh. Doing so is both a productivity killer and
an exercise in pure frustration. I’d suggest at least one screen at 1280×1024, and if you can add a second screen to
the system, that’s even better. A second screen at 1024×786 will happily accommodate both your various browsers
and your email client, and will keep your neck and shoulders from getting too stiff (seriously).
The last point on this topic is to say a computer that does this is so insanely cheap these days that nothing else
makes sense. All of the above can be purchased from the retailer of your choice for less than 1000 units of the
currency of your choice. (okay, smartarse, pounds/euros/dollars). The last Windows-based box I built as a web dev
platform cost around 400 GBP, including tax. Making developers suffer with sub-standard technology is so much of a
false economy that perpetrators should be stripped of their businesses on the spot.
Do you have testers?
Testing is one of those things that applies in pretty much the same way to all types of software development. I guess
for web development a little more of the initial testing is carried out by the developer, due to the nature of the
medium. However, don’t mistake “I wrote it, tried it and it worked” for testing on anything more than a superficial
level. That’s not proper testing – for that you need someone who doesn’t know how the code works to try it under
multiple circumstances, platforms, browsers, connection speeds, you know the drill. It’s like proof-reading your own
work, you can’t do it accurately. Get testers.
Do new candidates write code during their interview?
A practical test at the interview stage is a really good way to assess someone’s competence and ability to do the job.
However, the trick to getting good results (at least in my experience) is to make the situation as close to a genuine
working situation as possible. Allow the candidate to look stuff up on the web. Leave some reference books with them
too, if you like, but then get the hell out of there and give them a reasonable time to get the task done.
On returning, you can assess how far they’ve got through the task, and then sit down and ask them to talk you
through what they’ve done. Ask them about the decisions they’ve made along the way. Ask if anything was
problematic. Ask how they’d do it better if they did it again. Suggest improvements and see how they react to them.
Be nice, it’s not a sport.
Do you do hallway usability testing?
Hallway usability testing is a term used to describe quick, informal usability tests. The idea is to grab the next person
who walks by, show them what you’re working on and get feedback on how they think the interface works, or most
importantly, should work. The point is that usability testing needn’t be an overly formal drawn-out process. Brief, loweffort testing as you go along can be extremely beneficial, and ultimately is so much more beneficial than forgetting
about usability testing altogether.
In practise this is very easy to implement. The way we work it is that we bounce our ideas off each other, within the
team. As there are only two of us on a project, we typically are working on very disparate parts of an app, and so
neither of us is too familiar with the other’s area that our view of it is coloured. In other circumstances this wouldn’t
work so well – but the crucial thing is to find some way to make quick usability tests easy to do, and then do them.
The Joel Test for Web Development - Conclusions
4 October 2004
These are the closing notes of a three part series looking at applying The Joel Test to a web development setting. You
should really read the whole thing through from the first post before continuing, else your eyes will glaze over,
catapulting you into a trance not seen since the finale of the American/Pop Idol was aired.
The point of translating The Joel Test into web development terms was to find out if there are any useful lessons web
developers can learn from the traditional software industry. By asking if the test easily translates into terms and
principals that are meaningful to us, we are not only forced to analyse the questions, but also look long and hard at
the way we work and identify where improvements can be made.
Rate Your Team
The first thing to do is to take the test yourself for the team you work in (even if that’s a team of one) and see how
you score. If you score 12 you get a lollipop. If you score less you get some fun new challenges to think about.
Here’s how I score:
1. Do you use source control? Yes
2. Can you make a build in one step? Not quite!
3. Do you make daily builds? No
4. Do you have a bug database? Yes
5. Do you fix bugs before writing new code? From today, Yes. See, improvement already!
6. Do you have an up-to-date schedule? Yes
7. Do you have a spec? Yes
8. Do programmers have quiet working conditions? No, at least not consistently.
9. Do you use the best tools money can buy? No
10. Do you have testers? Yes but I think we need more.
11. Do new candidates write code during their interview? Yes
12. Do you do hallway usability testing? Yes
So that’s 8/12 which isn’t too shabby, but we have room for improvement. It’s dirty-laundry-in-public time, leave
your scores in the comments below. Be honest.
What can we learn?
Well, aside from the twelve questions in the test, one of the important points this highlights is the fact that these
principals do translate easily to web development. And if this little test can produce so much helpful wisdom, imagine
what can be learned from all the rest of the stuff on software development out there.
Comments
1. Matt Oct 5, 12:49 AM
Scored 8 here too, but it was a different 8. Thank you for writing this series, I enjoyed it quite a bit.
2. Mark Rowe Oct 5, 12:59 AM
When I started working for my current employer, their score on this test would have been near zero. Since I
have been here I have gone out of my way to try and improve the situation. I started by using Subversion
and Trac for source control and bug tracking. On several occasions the source control has saved a work-mate
several hours of ‘unbreaking’ code. The other more managerial aspects of the test are a little more difficult to
change, but over time even those will improve.
Before My Changes:
* Do you use source control? No
* Can you make a build in one step? No
* Do you make daily builds? No
* Do you have a bug database? No
* Do you fix bugs before writing new code? No
*
*
*
*
*
*
*
Do
Do
Do
Do
Do
Do
Do
you have an up-to-date schedule? No
you have a spec? Almost
programmers have quiet working conditions? Kind of
you use the best tools money can buy? No
you have testers? No
new candidates write code during their interview? No
you do hallway usability testing? No
Around 1/12.
Now:
* Do you use source control? Yes
* Can you make a build in one step? Almost. This is a work in progress.
* Do you make daily builds? Almost. Another work in progress.
* Do you have a bug database? Yes
* Do you fix bugs before writing new code? Yes
* Do you have an up-to-date schedule? No
* Do you have a spec? Almost
* Do programmers have quiet working conditions? Kind of
* Do you use the best tools money can buy? No
* Do you have testers? No
* Do new candidates write code during their interview? No
* Do you do hallway usability testing? Yes
Around 5/12. Still poor, but improving.
3. Darren Oct 5, 08:25 AM
7, it was about 3 when we first read about it.
Sadly we’ve only ever been able to fix the ones that don’t need help from management. :-(
4. Jesse Oct 5, 12:40 PM
We scored pretty bad (5/12). At least I have the toys, err i mean tools part right ;) Good computers bring the
geeks to work on time. I just got my co-op student an iMac G5 to work on, how nice am I?
Being in a management position helps my ability to change things but the first task has been getting the
resources to have people actually focus on web projects. Uni work is so much fun.
As I sort out the team I can also incorporate some this. Just working on the subversion and trac installs, bring
us to 7, and a spec.. I wouldn’t consider scribbles on a white board spec. So that is an easy-ish to achieve 8.
Thanks Drew for this.. it was timely. I am sticking to my ‘this is the web business maturing’ line. Articles like
this help move things along.
5. Dewayne Mikkelson Oct 5, 04:41 PM
Great set of posts!!!
*
*
*
*
*
*
*
*
*
*
*
*
Do you use source control? Kinda Sorta, but nor really!
Can you make a build in one step? NO
Do you make daily builds? NO
Do you have a bug database? NO
Do you fix bugs before writing new code? NO
Do you have an up-to-date schedule? NO
Do you have a spec? NO
Do programmers have quiet working conditions? NO
Do you use the best tools money can buy? NO
Do you have testers? NO
Do new candidates write code during their interview? NO
Do you do hallway usability testing? NO
OUCH! I knew it was bad but that really settles the question. One half out of 12. That is terrible.
6. Brandon Dennis Oct 27, 05:26 PM
Great article, thanks so much for all your work. I wish I had reviewed this back when I was running my
company in 2001. We were so close on so many things, but still so far on others. However, I think one of the
important issues with some of this is the Clients as well. Certainly clients need different things and it’s not as
easy to adapt a diverse team into a compartmentalized platform for every project. Some things work, some
things break, and some things complain a lot :)
7. Jon Berg Dec 6, 01:31 AM
It’s good to have a list of things that will help the develop process go smoothly. One top I would rank excelent
people. It is all about the knowledge and work morale of the workers.
8. TomA Dec 13, 12:50 PM
Our small web shop scores 8/12, it would’ve been about 4/12 before I read the original Joel test (two weeks
ago). Now we have a svn repository, reorganized office and stuff. Getting better… build in one step is next.
Great article, thanks Drew!
9. Kevin Jump Feb 7, 07:47 PM
We are just a few weeks in, getting a quite small but politically crippled development team from at best a 1,
up to as high as we can go, at the moment, I think we have a 3, but we are trying to take a five year project
and put some organisation behind it.