Marti A. Hearst
Professor, iSchool, UC Berkeley
February 8, 2012
The views expressed in this talk are my own and do not necessarily represent those of any organization.
Intro

Goal: enabling a high-performance,
citizen centered, 21st century
government.
 Govt should be capitalizing on recent IT
advances (mobile, social, etc)
 $80 Billion Federal Govt IT portfolio
○ Over $500 billion in contracts and orders total
 Many IT projects fail; how to fix this
Talk Structure
Goal: IT to help enable a 21st Century
Government
 Many different parallel tracks:

 Citizen / customer focus
 Use the latest technology advances
 Reduce Waste and Costs
 Improve How Large IT Projects are Built
Improving Large IT Projects
The Cost of Failed IT
 Why Large IT Projects Fail Generally
 Why they Fail in Govt in Particular
 How To Help Govt IT Projects Succeed
 How YOU Can Help

Intro
Goal: enabling a high-performance,
citizen centered, 21st century
government.
 80 Billion IT portfolio
 Govt should be capitalizing on recent IT
advances (mobile, social, etc)
 Many IT projects fail; how to fix this

http://www.cio.gov/pages.cfm/page/Chapter-19-Information-Technology-Page-1
FY 2011 President's Budget, Analytical Perspectives, Special
Topics, Chapter 19
Costs of Failed IT
YEAR
COMPANY
SYSTEM
COST (US $)
2005
Hudson Bay Co. [CA]
Inventory Control
$33.3 million loss
2004-05
UK Inland Revenue
Tax Credit
$3.45 billion overpayment
2004
Avis Europe PLC [UK]
Enterprise Resource Planning
$54.5 million, cancelled
2004
Ford Motor Co.
Purchasing system
$400 million, abandoned
2004
J Sainsbury PLC [UK]
Supply-chain management
$527 million, abandoned
2004
Hewlett-Packard Co.
Enterprise Resource Planning
$160 million loss
2003-04
AT&T Wireless
Customer Relations Management
$100 million revenue loss
2002
McDonald’s Corp.
Purchasing System
$170 million, cancelled
Why Software Fails, R. Charette, IEEE Spectrum Sept 2005
Costs of Failed IT
The cost itself.
 The opportunity cost:

 FAA Air Traffic control system failure:
○ $2.6 Billion Spent (1984-1994)
○ Cumulative cost of delays estimated at $50 B.

Can jeopardize an organization’s future.
 FoxMeyer Drug Co., worth $5 billion,
plummeted into bankruptcy in 1996.
Why Software Fails, R. Charette, IEEE Spectrum Sept 2005
In search of the future of air traffic control, T.S. Perry, IEEE Spectrum 34(8), 1997.
Frequency of Failed IT


15-20% failure rate for projects with budgets > $10 million
(Charette, 2005)
From Standish Group CHAOS Report 2009 (via ProjectSmart):
Success rating achieved if the projects were completed on time, on budget and with
required features and functions (met user requirements).
Challenged rating if late, over budget and/or with less than the required features and
functions.
Why Software Fails, R. Charette, IEEE Spectrum Sept 2005
http://www.projectsmart.co.uk/the-curious-case-of-the-chaos-report-2009.html
“Adding [programmers] to a late
software project makes it later.”
 Frederick Brooks, The Mythical Man
Month, 1975

Some cost contributors to large-scale programs, Nanus & Farr, AFIPS 1964
(exponent of 1.5)
Why Large IT Projects Fail












Unrealistic or unarticulated project goals
Inaccurate estimates of needed resources
Badly defined system requirements
Poor reporting of the project's status
Unmanaged risks
Poor communication among developers, and users
Use of immature technology
Inability to handle the project's complexity
Sloppy development practices
Poor project management
Stakeholder politics
Commercial pressures
Why Software Fails, R. Charette, IEEE Spectrum Sept 2005
Why Did Your Project Fail? Cerpa & Verner, CACM 52(12), 2009
Large Govt IT Projects

According to the Government Accountability
Office (GAO):
 Planned federal IT spending > $81 billion for fiscal year
2012.
 Many are critical to the health, economy, and security
of the nation.


Unfortunately, these frequently fail or have
overruns while yielding little value.
Recent Examples:
 After $127M over 9 years on an outpatient scheduling
system, the VA had to start over.
 After $40M and 7 years, FEMA canceled insurance and
claims system as it did not meet user needs.
Why Do They Fail?






Oversight and funding models make modern
IT design methods nearly impossible.
Problems related to contracting.
Lack of IT expertise by IT project leaders.
Lack of focus from executives and
changeover in leadership.
Lack of a user-centered design tradition.
And serious legal hurdles.
Legal Hurdles to Adopting Off-theShelf Well-Designed Software







TOS
PII
OWACS
COOP
508
Cookies
NARA
Legal Hurdles to Adopting Off-theShelf Well-Designed Software







TOS
PII
OWACS
COOP
508
Cookies
NARA
(Terms of Service)
(Personally Identifiable Information)
(Security requirements)
(Disaster recovery requirements)
(Accessibility Standards)
(Persistent Cookie Restrictions)
(National Archives and Records)
Challenges for Large Govt IT Projects
Procurement Rules & Myths
 Waterfall Development

 Not User-Centered, Not Agile

Oversight
 Instead of Coaching / Advice

Personnel
 IT and Procurement

Legacy Systems and Data
UCD Legal Hurdles

Paperwork Reduction Act
 Goal: ensure that information collected is useful and
minimally burdensome for the public.

To ask a question of 10 or more people, must:
 Clear internal legal process, then
 60 days in the Federal Register
○ May need to iterate based on comments
 Submit documents to OMB
 Another 30 day federal register notice period
 ~60 days for OMB review
○ May need to iterate and even start again

Usually a 6 month process.
New PRA Fast Track
OMB clearance in 5 days!
 Applies to information collections that focus
on

 the awareness, understanding, attitudes, preferences, or
experiences of stakeholders
 relating to existing or future services, products, or communication
materials.

Specifically includes:
 Focus groups
 Remote usability testing
 Online surveys for customer feedback purposes

However, public distribution of results
cannot be intended.
Plain Writing Act of 2010

The purpose of this Act is to improve the effectiveness
and accountability of Federal agencies to the public by
promoting clear Government communication that the
public can understand and use.

The term ‘‘plain writing’’ means writing that is clear,
concise, well-organized, and follows other best practices
appropriate to the subject or field and intended audience.

Applies to any document that:
 Is necessary for obtaining any Federal Government benefit
 Provides information about any Federal Government benefit or
service
 Explains to the public how to comply with a requirement the
Federal Government administers or enforces;
Plain Writing Act of 2010

The purpose of this Act is to improve the effectiveness
and accountability of Federal agencies to the public by
promoting clear Government communication that the
public can understand and use.

The term ‘‘plain writing’’ means writing that is clear,
concise, well-organized, and follows other best practices
appropriate to the subject or field and intended audience.

Applies to any document that:
 Is necessary for obtaining any Federal Government benefit
 Provides information about any Federal Government benefit or
service
 Explains to the public how to comply with a requirement the
Federal Government administers or enforces;
Plain Writing Act of 2010
From www.plainlanguage.gov: BEFORE
Plain Writing Act of 2010
From www.plainlanguage.gov: AFTER
Executive Order on Streamlining Service
Delivery and Improving Customer Service
Follows on memos from 1993, 1995, and 1998.
 “With advances in technology and service delivery
systems in other sectors, the public's expectations of
the Government have continued to rise. The
Government must keep pace with and even exceed
those expectations.”
 Customer service plans to include:
 At least one major initiative using IT to improve the
customer experience;
 Mechanisms to solicit customer feedback on
Government services and use such feedback
regularly to make service improvements;
 Streamlining agency processes to reduce costs and
the need for customer inquiries.

President's Executive Order (EO) 13571, on "Streamlining Service Delivery and Improving Customer
Service,” April 2011.
Success Factors: A New Report from the
Government Accountability Office
GAO-12-7, Information Technology, Critical Factors Underlying Successful Major Acquisitions ,
October 2011, http://www.gao.gov/new.items/d127.pdf
Mapping Success Factors to
Current IT Reforms
1. Program officials were actively engaged with end users.
2. Program staff had the necessary knowledge and skills.
3. Senior department and agency executives supported the programs.
4. End users were involved in the development of requirements.
5. End users participated in testing of system functionality prior to formal
end user acceptance testing.
6. Government and contractor staff were stable and consistent.
7. Program staff prioritized requirements.
8. Program officials maintained regular communication with the prime
contractor.
9. Programs received sufficient funding.
User-centered Design/Agile
IT Staffing
Procurement
Governance/Accountability
GAO-12-7, Information Technology, Critical Factors Underlying Successful Major Acquisitions ,
October 2011, http://www.gao.gov/new.items/d127.pdf
Administration Initiated
IT-Related Reforms, 2009-2011

Reforming Major Government Operations







A Focus on IT




IT Reform
Procurement Reform
Hiring Reform
Regulation Reform
Waste Reduction
Open Data / FOIA
Emphasis on Leadership
25 Point Plan for Reforming IT
Promoting New Standards
An Emphasis on Users





Customer Service Executive Order
Plain Language Act
New PRA Guidelines
Federal Government Web Site Reform
Modern Communication Techniques
○ Leading with High-quality Web Site Design
○ Encouraging Information Visualization
○ Deploying Light-weight online discussion tools
“He’ll sit here and he’ll say ‘Do this! Do
that!’ And nothing will happen.”
 Harry S. Truman, On Presidential
Power, quoted in Neustadt, R.R.,
Presidential Power, Wiley 1960.

Administration Initiated ITRelated Reforms, 2009-2011

A Focus on IT





An Emphasis on Users






Emphasis on Leadership
25 Point Plan for Reforming IT
Enabling use of SaaS and Cloud Services
Encouraging Challenges and Prizes
Customer Service Executive Order
Plain Language Act and EO
New PRA Guidelines
Federal Government Web Site Reform
Modern Communication Techniques
○ Leading with High-quality Web Site Design
○ Encouraging Information Visualization
○ Light-weight online discussion tools
Reforming Major Government Operations




IT Reform
Procurement Reform
Hiring Reform
Regulation Reform
Strategy
Embrace modular development,
 Build on open standards, and
 Work with Congress to modernize IT
funding rules.

Steve VanRoekel, http://www.whitehouse.gov/sites/default/files/svr_parc_speech_final_0.pdf
Fixing Govt IT Development

First deliverables produced years after
work begins
 By then, program sponsors and end-user
needs have changed.
 Should have 18-24 month duration
25 Point Plan to Reform IT
IT Reforms

Emphasis on Leadership
 Elevating Federal CIO, naming CTO
25 Point Plan for Reforming Federal IT
 Enabling use of SaaS and Cloud
Services


http://www.whitehouse.gov/21stcenturygov/

http://www.whitehouse.gov/21stcenturygov/actions/reforming
-government-contracting#ethics-menu
IT Reform (key document)











http://www.whitehouse.gov/blog/2010/11/19/driving-it-reform-update
Tackling the information technology gap between the public and private sectors is one of most
effective ways we can make government work more effectively and efficiently for the American
people. IT has been at the center of the private sector’s productivity gains, but for too long Federal IT
projects have run over budget, behind schedule, or failed to deliver what on their promise. That’s why
fixing IT is a cornerstone of the President’s Accountable Government initiative.
Second, since far too many financial system modernization projects were running behind schedule
and over budget, we halted all new work on those projects pending review and approval by OMB.
Across the government, over 30 financial systems projects, with budgets totaling $20 billion, were
affected by this policy.
Our review of 20 agencies’ projects is now complete, and I am proud to report that we have taken
steps to save $1.6 billion on these projects.
Through our reviews, we determined that half the projects were basically on track. Of the half that
were not, we took the following actions:
At two agencies – the Department of Housing and Urban Development and the Environmental
Protection Agency -- we pulled forward meaningful functionality, resulting in almost $230 million in
budget reductions.
At two agencies – the Department of Veterans Affairs and the Small Business Administration – we
canceled their projects as a result of the review, resulting in over $500 million in budget reductions.
At three agencies – the Departments of Homeland Security, Justice, and Health and Human Services,
we are moving forward with plans to decrease the scope of and improve their financial system
projects, resulting in reduced costs and a greater focus on critical business needs. This revaluation of
these projects resulted in over $680 million in budget reductions.
An additional $200 million in budget reductions was identified in various agencies, with more to come.
(Zients, Nov 19, 2010)
Procurement Reform

http://www.whitehouse.gov/blog/2011/02/04/turning-tide-contract-spending






For the first time in 13 years, we have reduced spending on contracting and agencies
have stopped the costly upward spiral in contract growth. In FY 2010, agencies spent
nearly $80 billion less than they would have spent had contract spending continued to
grow at the same rate it had under the prior Administration.
In his State of the Union address, President Obama said that, “we can’t win the future
with the government of the past.”
instead, he said we must reform the way we do business in Washington and give the
American people a government that’s not only more affordable, but also more effective
and more efficient. This principle has been the cornerstone of our work on contracting
and across the Accountable Government Initiative. From reforming and cutting costly IT
systems, implementing unprecedented transparency and reporting efforts, buying in
bulk, establishing a government-wide Do Not Pay list, or moving toward electronic
government payments, we’re making real progress in changing the way government
does business.
Lew on February 04, 2011
http://www.whitehouse.gov/sites/default/files/contracting_reform.pdf
http://www.whitehouse.gov/omb/procurement_default/
IT Reform (key document)


CLOSING THE IT GAP AND MODERNIZING GOVERNMENT
Twenty years ago, when people came to work for the
government, they had access to the world’s best technology.
Today, government employees often have better technology
at home than at work. This gap between the public and
private sectors results in billions of dollars in waste, slow and
inadequate customer service, and a lack of transparency
about how dollars are spent. While a productivity boom has
transformed private sector performance over the past two
decades, the federal government has almost entirely missed
this transformation and now lags far behind on efficiency and
service quality. The Obama Administration has undertaken
an IT reform and government modernization effort so that the
government uses IT in a more efficient and cost- effective
way
 http://www.whitehouse.gov/sites/default/files/Background_on_Accountabl
e_Government_Initiative.pdf
25 Point Plan for Reforming IT

Issued Dec 9, 2010
 Vivek Kundra, Steven VanRoekel

Main Components:
 Apply “Light Technology” and Shared Solutions
 Strengthen Program Management
 Align the Acquisition and Budget Processes with
the Technology Cycle
 Streamline Governance and Improve
Accountability
 Increase Engagement with Industry
Procurement Improvements
Dan Gordon, Administrator for Federal
Procurement Policy
 MythBusting:

 http://www.whitehouse.gov/sites/default/files/om
b/procurement/memo/Myth-Busting.pdf


FAR: Federal Acquisition Regulation
More Information
Cio.gov
 http://www.whitehouse.gov/omb/procureme
nt_default/
 Whitehouse.gov/21stcentury
 Agency open government pages /open

Waterfall
vs Agile
Images from Effective UI, Anderson et al, O’Reilly Media, 2010.
Design happens During
Development


The later you make a decision in a project, the more likely it is to be the best
one by virtue of having been made from a position of greater experience.
Attempting to comprehensively define the functional requirements of a
product on day zero is absurd and futile;





that’s the day when you know the absolute least about the product, and any decisions you
make on that day are very likely to be incorrect
Development is where the majority of design happens, and design is the
activity that discovers unknown problems and their solutions, so development
should begin as soon as possible.
Functional requirements and specifications are written before development
begins, so they’re immediately handicapped by having been made from the
least informed perspective.
Putting together written requirements and specifications is not entirely without value, though, so long as it’s perceived in the right way. The goal should
not be to build a definitive description of the product, but rather to do a dry
run of the product design, to get the team to start thinking through the problems that lie ahead.
In preparing for battle, I have always found that plans are useless, but
planning is indispensable. -- Dwight Eisenhower
from Effective UI, Anderson et al, O’Reilly Media, 2010.
Project Scope

Any estimation of scope, having been
defined in the “mere twilight” of the project
kickoff or contract negotiation, cannot
possibly be accurate. And a comprehensive
description of the scope of a project is so
enormously complex that it simply can’t be
done; the only perfect description of a
product is the product itself.
from Effective UI, Anderson et al, O’Reilly Media, 2010.
Problems with Waterfall
Because each step is entirely separate, each group of contributors is
entirely siloed from the others. The people brainstorming and writing
requirements for the product never collaborate with the people
architecting and designing it. The software engineers never have the
opportunity to collaborate with the architects and designers, let alone
the business managers and stakeholders.
 This approach forces the engineering and quality assurance (QA)
stages to absorb almost all of the effects of the risks and unknowns
that arise during the project. Since the planning, architecture, and
design of the product are already ostensibly complete, there’s no
option for changes to them because the money for them has already
been spent and the resources have been allocated to other things.
This leaves it to the engineers to figure out how to account for the
inevitable unforeseen problems and unknowns—and to do it within
the budget and timeline they were allocated before the problems and
unknowns were identified.


from Effective UI, Anderson et al, O’Reilly Media, 2010.
Advantages of Agile

Favor:
 Individuals and interactions over processes and
tools
 Working software over comprehensive
documentation
 Customer collaboration over contract
negotiation
 Responding to change over following a plan

from Effective UI, Anderson et al, O’Reilly Media, 2010.
25 Point Plan for Reforming IT
Current status: cio.gov/modules/itreform
25 Point Plan for Reforming IT
25 Point Plan for Reforming IT
25 Point Plan for Reforming IT
Strengthen Program
Management
Design an IT management career path
 Enable mobility across govt and industry
 Launch a technology fellows program
 Require integrated program teams

Streamline Governance
Reform existing internal reviews
 Redefine role of agency CIOs
 TechStat reviews at bureau level

 Techstat: face-to-face evidence-based review of
an IT program with White House and agency
leadership.
Fixing Govt IT Development

First deliverables produced years after work begins
 By then, program sponsors and end-user needs have
changed.
 Should have 18-24 month duration

Solution: “modular” development
 Must deliver functionality to users at least every 12
months
 Must regularly capture and incorporate user feedback
through an iterative process that assesses user
satisfaction with each release, continually refining design.

Problem: government processes do not allow for this
kind of timeline
○ Planning, budgeting, procurement all require that agencies
do not do develop IT in this way.
From 25 Point Plan to Reform IT
25 Point Plan for Reforming IT
Procurement & Vendors
RULES
 Turn into MYTHS and HABITS

Other Angles




Reduce Waste Initiative
Prizes, Challenges
Supporting Standards Efforts
Entrepreneurs in Residence
 a time-limited recruitment of world-class
entrepreneurs and innovators to join highly-qualified
internal employees in the development of new
operational procedures in areas that impact
innovation; the goal is to deliver transformational
change by combining the best internal and external
talent in testing, validating and scaling what works

Lean Government Startup Team
How YOU Can Help
Form Coaching Councils / IT Clinics
 Study and Report on Vendor Activity
 Publish Very Short How-To’s
 Respond to Government RFCs
 Help with Future First Initiative
 Work in Government!

Provide Advice
Agencies IT projects have LOTS of
oversight, but NO coaching.
 Solution: Unbiased advisors / coaches.
 Outside groups have to initiate these
and offer the services to government.

Study Vendor Practices

Compile statistics on:
 Protests
 Interventions by Congresspersons
 Vendors’ Success Rates
Investigate Relevant Research
Problems

Study and publish on how to make large
IT projects succeed.
Sample Useful
How-To on
Social Media
Usage
(From EPA)
Participate in Government Dialogues
Example: The National Dialogue on Improving Federal Websites
Make How-To’s
Independent non-profits, bloggers, university groups
can have real impact.
 They do this work on their own initiative.
 Examples:
 Technology developed by OMB Watch for

fedspending.org used in relaunch of
USASpending.
 Sunlight Foundation compiled lists of strategies
for govt to address the OGD.
 UC Berkeley iSchool faculty posted guidelines
on how to improve the design of recovery.gov;
had a big influence.
 Open gov how-to workshops and website
Participate with Classwork
 Example Idea: Usability Clinic
 Professors teaching usability courses
○ Have their students critique a web site as a homework
exercise
○ Commit to a particular time period
 Organizations sign up for the clinic
○ Govt, non-profits, small businesses
 May turn into longer-term projects
Participate with Data Analysis
 Build tools that use govt data
 Expose inefficiencies
 Create new, useful functions
 Example:
 Analyze hiring latency on a per-agency basis
 Data isn’t there?
○ Comment on agencys opengov websites
○ Ask for time-to-hire data for each agency
○ Be persistent if necessary
Participate By answering Requests For
Comment

Example:
 OSTP Request for Comments
 Federal Register, May 21, 2009
 http://www.fas.org/sgp/news/2009/05/opengov.pdf
 Also on the OSTP blog

Sought advice on Open Government
topics:
 What alternative models exist to improve the
quality of decision making and increase
opportunities for citizen participation?
 What are the limitations to transparency?
 What strategies might be employed to adopt
A Recent
Federal Register
Request For
comment
Become a
Presidential Technology Fellow

A brand new program!
 Started Fall 2011
 2 year paid fellowship
 Rotate among agencies

Builds on the highly successful Presidential Management
Fellows program
 Trains leaders for Federal Government Service
 Usually a terrific cohort

Must apply in the Fall before you graduate with a masters
or PhD
 But very few opportunities for non-US citizens
Summary
It is still very difficult to successfully develop
IT in government.
 However, many reform efforts are greatly
improving the situation.






An IT-savvy administration
A focus on usability and citizen input.
The 25 point plan
Additional changes in procurement guidance.
Academics, researchers, and practitioners
can all help!