UK Test Management Forum
Balls Brothers, London
30th October 2013
Software Developers In Test:
The Supply-Side Crisis Facing Agile Adopters
_____________
A reality check on where we are
Reflections on how we got here
A discussion on scaling supply
Facilitated By
Richard Neeve
www.richardneeve.net
Some Warnings
• The evidence herein is unashamedly anecdotal.
– Looking to work with Paul Gerrard and Graham Thomas on a
survey for review at the next Test Management Summit.
• If you’re opposed to agile or using SDITs, that’s fine but you may
not get much/anything from this session as I’m running this on
the premise that we need to adequately fulfil the market
demand regardless of its merits.
• There may not be any realistic answers.
– But any answers are probably in this room. I doubt anyone
beyond testing will solve this for us. So if ever there was a
topic that relied on the group’s participation and wider
community influence that is mentioned in the standard TMF
introduction, this is it!
– We won’t crack this today; I would like us periodically re-visit
this topic to gauge progress (hopefully in a more evidencebased fashion).
Definitions (1 of 2)
If you disagree, please indulge me as I don’t want to get bogged down here.
•
SDIT:
– “Software Developer In Test”.
– A (more?) common alternative label is SDET (Software Development
Engineer in Test). Others? Code-infected tester?
– Surprising paucity of succinct definitions offered online but we’re basically
talking about people who can:
• Write automation frameworks.
• Develop automated test cases.
• Use their coding skills to pair with a developer in TDD (for example).
• Independently develop non-trivial bespoke test tools as required.
• Conceivably produce production code if that’s deemed appropriate.
– Potential ‘Litmus Test’ for future debate: “Someone that could, in very
short order, become viable in a full commercial dev role if they were so
inclined”?
Definitions (2 of 2)
• Crisis:
– “A stage in a sequence of events at which the trend of
all future events, especially for better or for worse, is
determined; turning point”.
(Source: www.dictionary.com)
– Seems justified to me. May or may not represent an
existential threat to testing. I think it’s a top 3
challenge for testing (perhaps no.1). Large
opportunity cost already incurred? Could get a lot
better/worse depending on whether/how it’s
addressed.
Where are we?
The Broad Macro Picture
• Agile is now mainstream and calls for the blurring of test
and dev roles. So demand for SDITs is broad-based and
rapidly accelerating as agile continues to spread and
mature.
• Lots of warning (min 5yrs, arguably 10+yrs?) but we
haven’t/aren’t geared up to supply SDITs in volume so the
market has been caught flat-footed and appears to be in
serious under-supply.
• Existing limited supply is ‘tightly held’ hampering market
liquidity. SDITs are typically looked after well (financially
and otherwise). Clients increasingly realise backfilling will
be painful, uncertain and could easily take longer than the
incumbent’s notice clause (jeopardising knowledge
transfer).
Demand Under-pinned By Conviction
•
Where SDIT demand exists, it seems very ‘committed’. I’m sure that’s true of other
roles but it seems that:
– Once an environment philosophically commits to the SDIT concept, it really
wants them and they’re rarely seen as a ‘nice to have’ thereafter.
• A contrast with so many environments where the test manager has to take
what they can get through voluntary automation efforts post 6pm.
– SDITs are seen as key to achieving ‘sustainable agility’ through automation
enabling Continuous Integration and the avoidance of lapsing into ‘Wagile’ style
test cycles.
– SDITs are seen to speak directly to the “Shift Left” narrative of reducing
time/cost (and can enjoy strong senior sponsorship as a consequence).
• Observation: I’ve seen/heard ‘Shift Left’ used more in the last month than I
have in the last decade.
– SDIT demand is compounded by/with what appears to be similarly committed
(and growing) demand for Behaviour Driven Development (BDD) and Test Driven
Development (TDD). This ‘slip-streaming’ demand effect seems perfectly logical.
The Political/HR Dimension
•
•
•
•
•
•
•
In reality, implementing the agile evangelist’s vision in which everyone swaps between test and dev
activity as required, is very hard since:
– Many developers won’t do testing (beyond the unit testing mandated by the dev job market).
– Many testers don’t have comprehensive development skills.
I’ve seen devs threaten constructive dismissal claims over this as they see it as a diminution of their
role (and maybe it is).
I’ve seen testers resign (un-prompted) in fear of having to skill up in order to ‘go under the hood’.
These are very delicate matters, especially when you’re dealing with permanent staff.
Are agile evangelists/coaches living in a bubble to some degree? Are they disproportionately:
– Surrounded by self-selecting enthusiasts at the top end of the ability range as they move from
pilot to pilot?
– Exposed to highly innovative start-ups and small companies that are highly progressive in their
thinking and perhaps inherently have an “all hands to the pump” mentality?
– Shielded from the broader range of attitudes/concerns faced by those who have to scale the
agile adoption across the wider organisation once their agile coach’s involvement is winding
down or concluded?
If agile evangelists/coaches are setting undeliverable expectations, are they inadvertently causing
part of the problem and if so how might they be part of the solution?
Either way, these factors increase the appeal of those willing and able to bridge dev and test.
Tales Of Escape
You may have been active in SDIT recruitment but not impacted (yet) if:
• You’re acting on behalf of a tier 1 investment bank (or similarly priceinsensitive organisation) that can aggressively bid for resources.
• You’re acting on behalf of a firm like Google or Microsoft that can attract
people through money, high profile innovation that even my mum may
have heard of and an environment with all the trimmings (e.g. pool tables,
free food, consoles, etc).
• You’ve outperformed what you’re offering having lucked into one or more
appointments because the candidate(s) had particular reasons to join your
team (e.g. they live around the corner and have just had a new baby).
• You haven’t had to go to the external market having successfully tapped
an internal supply of willing or conscriptable resources.
• You’ve successfully gambled on a ‘wildcard’ appointment (e.g. a cheap
graduate or taking someone from a tangentially related function e.g. tech
support).
The Limits Of Those Tales
• That’s all fine but:
– On current trends even clients with a strong market position may soon
start to struggle.
– Everyone needs a bit of luck but it’s not a strategy in itself and is finite.
– Individual anecdotes about successfully using internal/wildcard
candidates are great on a micro level, but they’re not going to sate the
breadth of demand at the macro level.
• What if you’re hiring an SDIT for a relatively unknown and somewhat
unlucky SME that’s:
– On a trading estate in, say, Staines
– Close enough to the city to have to compete with Goldman Sachs et al
– Close enough to the M4 corridor to have to complete with MS et al
– Working to pay scales set by someone living in dreamland?
The Market Pricing Impact
•
(experienced committed demand) + (new potentially ill-informed demand) + (limited
illiquid supply) + (wide variations in price sensitivity) = impaired price discovery
…leading to extreme pricing and a dysfunctional market that’s struggling to clear
Or to put it in less abstract terms…
•
I keep seeing SDIT job adverts, sometimes side-by-side in my Linkedin feed, that are
comparable in specification, sector and location, paying basics of £35k and £75k.
– Former probably reflects ignorance about the SDIT market and prejudices
(perhaps baked into formal pay grades) about the value of functional testers.
– Latter probably reflects an experienced, committed and price-insensitive client.
But who knows? Contract rates for SDITs seem to exhibit similar disparities.
•
Audience: Any insights on the basic salaries/day rates at which most deals are being
done in London, both outside and inside of finance? How long are vacancies
typically open for? Any prospective SDIT hirers shocked at what they’re hearing?
Deeper ‘Trouble’ Ahead?
•
•
•
I have the Bloomberg mobile app with alert subscriptions (10 to 15 a day).
– I’ve had hundreds of positive alerts in recent months with literally just 3 or 4 negative
exceptions:
• Earnings Per Share for company X beats analysts’ estimates by Y.
• Confidence index X at expected level + Y.
• Unemployment in country X fell by Y more than expected.
• Etc.
– It has been notably relentless, for months.
ONS data for Q4 2012 showed cash held by corporates other than banks was £318bn.
FTSE100 cash reserves in 2013 are at £74bn, up from £12bn in 2008. Excessive cash reserves
often aren’t favoured by investors (e.g. the demise of GEC Marconi) especially in this low
interest rate environment.
Are board room execs also watching these positive metrics? Are they collectively about to
push a lot of green-coloured buttons?
– Good to recognise when problems are ‘nice to have’. But are we about to get punished
(at least in opportunity cost terms) for not skilling up while more projects were on ice?
– Homework: Read up on ‘elasticity of supply’ and consider its implications for this topic.
A Quick Re-Cap
• Broadening/accelerating/committed demand set against
limited/illiquid supply.
• So basically we’re all fishing in the same puddle for resources
considered to be pivotal. Can we at least turn the puddle into
a small stream?
• Indications of a dysfunctional market.
• Some unaware (through not having to go to market, or luck,
or having a strong market position).
• Some very aware through painful experience.
• Questionable preparedness for a large wave of capex by
corporates (if it happens).
• There are technical, financial and political factors in play.
• I think this is a huge challenge for the next economic cycle.
How did we get here?
Causal Factors: My Perspective (1 of 3)
My analysis is flawed as it only dates back to 1998 but for what it’s worth:
•
Historical Structural Separation Between Dev And Test
– The ‘test independence’ agenda was firmly in focus when I started; seemingly
less so now. Y2k an catalyst? Not saying independence is bad but the market
now seems to accept a risk of reduced independence as a fair price for closer
collaboration.
– Many strongly embraced the idea of test being an arms-length ‘formal
customer’ of dev; probably a by-product of the test independence agenda.
This suited lots of people including tool vendors.
– The ‘professionalisation’ of testing (manifested via ever-burgeoning standards,
certifications, tooling, community, vocabulary, etc) has perhaps encouraged
differentiation – and therefore an increased degree of separation – from dev,
even though there’s probably no intrinsic reason why that should be so.
– The ‘equal pay for testers’ lobby, which appears to have failed, seemed to ‘dig
in’ quite deep and this may ultimately have accentuated any separation.
Causal Factors: My Perspective (2 of 3)
•
•
Apparent long-term downward trend in the technical skills of candidates
– Reported by many with a long-standing involvement in technical recruitment (me included).
– I don’t want to get political here, but is this the downstream impact of ‘A’ Level Mathematics
and Computer Science being diluted, causing universities to react in sympathy to secure
income following cuts in central government funding?
Testing’s initial reaction to The Agile Manifesto seemed somewhat delayed/passive
– The Agile Manifesto isn’t the origin of agile but it seemed to be the trigger that made it
mainstream thinking among a whole generation of people like me.
– I’m not sure we came out of the blocks quickly enough on the whole agile testing piece. I
often felt there was a victim mentality among testers. We seemed unsure about
how/whether we fitted in which makes me doubt whether we fully exploited the opportunity
to provide thought leadership in this area.
– By the time the manifesto was published, many testing specialists were carrying some very
negative baggage regarding automation. I think many of us were unduly reticent about selling
automation again and using our automation skills/experience as a vehicle to engage in the
wider process.
– These factors perhaps created a sense that we weren’t leading the agile testing skills agenda
and by extension we weren’t collectively skilling up in line with the emerging market
requirements (even though some clever individuals were).
Causal Factors: My Perspective (3 of 3)
•
Unfavourable career path dynamics
– Devs don’t want to move into test as they see it as a diminution in their role and status.
– If you’re seriously technical, your career prospects are probably better in dev (since it seems more common
for senior managers seem to be drawn from dev rather than test).
– Quoting current SDIT rates often elicits a response along the lines of “we can get a dev for that” so you don’t
get far before hitting political constraints on SDIT pay, which impacts the demand signals being emitted in
the market.
– Scarcity of seriously technical test specialists has made succession planning difficult so ambitious people are
wary of getting stuck in their career.
– Many people tire of the constant treadmill of learning the latest tech and want to step off it.
– In most cases financial progression seems to necessitate going the hands-off management route - especially
for permanent staff. Again I don’t want to get political here, but over the last 10 years or so, property price
inflation has been a big issue for people in their 20s and 30s (i.e. those on the cusp of management).
– Recasting organisations around multi—disciplinary and product-centric teams has sometimes seen the
reduction/removal of the test management layer. That may or may not be the best thing, but where it has
happened it has often:
• Reduced the advocacy for the training and development of test specialists.
• Reduced career opportunities for testers who are aspiring test managers.
• Resulted in people like me being asked to return to hands-on work which generally isn’t attractive.
• Placed increased demands on the soft skills of testers and created circumstances where the tester at
the coal face is the most senior testing specialist in the organisation. Many testers have no appetite
for the politics from which they have previously been shielded and have acted accordingly.
How can we scale SDIT supply?
Or Is Doing Nothing An Option?
• Opportunity cost implications?
– Look enormous to me (w.r.t both influence & revenue)
• Do we have bigger fish to fry?
– With the possible exception of the offshoring
nightmare it’s hard to think of many (any?) compelling
candidates.
• Existential threat from eventual irrelevance? Is it
too late or is the problem simply intractable? Are
we so behind the curve that it’s a rear guard
action from here on in?
– Maybe but the situation still feels redeemable.
Tactical View: Where Are They Now?
• In the short term and on the margins, clients may benefit from
being more informed/proactive in knowing where the existing
supply can be accessed:
– Events that are disproportionately likely to attract the interest of
SDITs such as: LTG, Skills Matter seminars, eXtreme Tuesday
Club, Meetups for SDITs, etc.
– Focused Linkedin groups like this one.
– Specialist recruitment agents.
– Others?
– Bonus Poaching Tip: I’ve noticed media organisations seem to
do well in recruiting SDITs.
• On a selfish tactical level this may help some individual clients, but
it doesn’t create fresh supply in the market as a whole.
Strategic View: Some Systemic Options
•
•
•
•
•
•
Rolling back market norms?
– Paying to train contractors?
– Tackling specificity of job specs?
– Restructuring pay grades so people don’t have to go the management route to progress financially?
Taking more risks on graduates?
– More use of propositions like FDM and Novus?
– Deeper links with universities?
Changing contractual arrangements for recruitment?
– Tighter PSLs (carrot)?
– Looser PSLs (stick)?
– Can we do anything different commercially to encourage risk taking? Changing the risk/reward balance?
Exploring insights from recruiters?
– Any discernible themes in the reject pile? Are there risks in there that we could ultimately manage as we
do in other areas? How would we need to flex to turn the top, say, 10% of rejections into placements?
Any scope to skill-up non-techies or de-skill the SDIT role without impacting its effectiveness?
– Anything around tooling?
– Converting script readers to script writers? A matter of building confidence through coaching?
Can we cut demand too?
– Any scope to cannibalise the SDIT remit across other, more readily filled, roles?
– What might it take to make more (pure) developers want to do this kind of work?
Summary
•
•
•
•
•
•
•
•
•
•
The SDIT demand profile is broadening/accelerating/committed and set against
limited/illiquid supply, provoking symptoms of market dysfunction.
Recruitment consultants and direct hirers seem to be increasingly desperate.
Questionable preparedness for a large wave of capex by corporates (if it happens).
Status quo looks unsustainable to me.
A key challenge - both present and future – and not just for testing.
The dynamics involved are numerous, complex and inter-connected. They are financial,
technical and political; some having origins dating back many years.
Some clients are unaware (through not having to go to market, or luck, or having a
strong market position).
On the current trajectory, even those with a strong market position may soon start to
struggle.
On the margins, clients and their recruitment agents may be able to identify new
sources of existing candidates by being better informed and more proactive. But taking
a market-wide view, this is just ‘robbing Peter to pay Paul’.
Genuinely and sustainably freeing up supply in a way that will benefit the broader
market will require time, innovation and risk-taking (if it happens at all).
But In The Meantime…
• Thank you for listening and participating.
• I’ll create a blog entry on www.richardneeve.net
about this session so we can continue the
discussion in the comments section.
• The slides will also be on uktmf.com
• I‘m on Linkedin.
• Email: richard@richardneeve.net