Lewis Shepherd
Chief Technology Officer
Microsoft Institute for Advanced Technology in Governments
KEY FACTORS IN
THE FUTURE OF
COMPUTING
in the Government Enterprise
An Exercise in Prediction, with the
Intelligence Community as an Example
“What technologies
will be required
over the next 10 years
to protect U.S. interests?”
What if we had asked that question,
10 years ago?
Some Surprises Post-9/11
“Asymmetric adversary” = an information challenge (“hard target”)
Seeming irrelevance of traditional methods for new targets
- Order of battle (counting military elements)
- State-to-state analysis
- “Kremlinological” approaches
Challenges of IT during wartime
- Stress on systems infrastructure of 2 wars
- Stress on software (link-analysis, SNA, “search”)
- Stress on collection capacity (sensor grids, Internet)
- Stress on analysts’ – and technologists’ – imagination
Limits of “Search” for Prediction
We don’t have a “Search”
capability to reach inside
enemy minds … yet
IT Challenge: Low-Observable Adversary
How does THIS …
… help perform analysis on THIS?
Our databases had no fields
for box-cutters, IM accounts
Case Study: Intelligence Community
 The IC’s post-9/11 challenge
 Some identified solutions:
1. Grid/Cloud computing
2. Secure SOA platform
3. Web 2.0 tools
(Intellipedia, A-Space)
 Implementation challenges
What Drives the Future of Enterprise Computing
The value for a new user of a service depends
on the number of existing users of the service…
“Critical mass” can lead to “Bandwagon effect”…
Side-Effects of Network Effect
 Exponential growth of networks, systems
Requires Scale
 Exposes networks to “edge audiences”
Requires Security
 Derives new wisdom from growing “crowd”
Makes Smart Systems
Scale
Scale: a Challenge for Large Commercial Enterprises
Remote Office IT
Scenarios
No Infrastructure
Characteristics
Microsoft Inc. as an Enterprise Example
141,000 end users
260,000 computers
550 Buildings in 98
countries
358,000 SharePoint sites
2,500 internal applications
2,500,000 internal E-mails
per day
18,000,000 incoming Emails per day (97% filter)
136,000 E-mail Server
accounts
1,000,000 remote
connections per month
29 billion E-mails sent
per day
280 billion page views per
day
435 million unique users
6 billion instant
messages (IMs) per day
Defense Intelligence Agency
as an Enterprise Example
 One of 16 agencies in the Intelligence Community
 9,000+ personnel
 DIA IT systems support the entire intelligence community
 100,000+ users of DIA’s Top Secret network, apps, data
 Global reach through IT support of all DoD Commands
 Pacific Command, European Command, etc.
 The only true “all-source” agency in the IC
 Collection (signals intell, human intell,
measurements & signatures, etc)
The Challenge: Stovepiped Analytic Capabilities
Security
The Security Side of “Enterprise 2.0”
Secret to a Walled Garden: Control
 Definition: On the Internet, a walled garden is
an environment that controls the user's access
to Web content and services. In effect, the
walled garden directs the user's navigation
within particular areas, to allow access to a
selection of material, or prevent access to other
material. [SearchSecurity.com]
Why Walled-Garden Content & Systems?
 Rationale on the Internet: Money
 Paid-Access Content Revenue
 Member-Fee Revenue
 Exclusive Ad Revenue (Segmented Eyeballs)
 Value of Intellectual Property
 “Enterprise” Rationale: Security
 Trade Secrets in Operational Data
 Competitive Advantages
 Regulatory Control over Data
Smart Systems
Need for Analytic Reform
 Traditional IC output: ~50,000 stand-alone reports/year




Many redundancies
Produced in agency/organization silos
Lack of collaborative capabilities across (and within) agencies
“Intelink” (the IC-wide shared domain) seen as a backlot
 Forcing Function: 9/11 Commission Report
 Key Recommendation: From Need-to-Know to Need-to-Share!
 Realization: “Something that’s 80 percent accurate, on-time,
and sharable, is better than something that is perfectly
formatted, but too much, too late, and over-classified.”
Chris Rasmussen, NGA
Birth Pangs of IC Web 2.0: 2004-2005
 Early Efforts were internal, agency-specific projects





CIA’s internal blogs, 2004
DIA’s internal “IntelliPedia” wiki, 2004
NGA’s internal blogs, early 2005
DIA’s AJAX mashups in “Lab X,” 2004-05
CIA’s del.ici.ous lookalike, Tag/Connect, 2005
 A “Wisdom of Crowds” Culture was forming by 2005
 Joint trips to outside conferences
 Cross-agency collaboration on metadata tagging
 Formation of “IC Enterprise Services” group, or ICES
 Tipping Points, sparked by ICES:
 August 2005 launch of “Intelink Blogs”
 April 2006 launch of IC-wide Intellipedia
One thing we learned wiki-wiki…
Key Distinctions, Intellipedia vs Wikipedia
Business Practices of intelligence analysis & reporting
demanded certain technical features:
 Not open to the public, only users with access to the IC’s
Top Secret network (JWICS), accounts created by ICES.
 No anonymity. All edits and additions are traceable.
 Intellipedia does not enforce a “neutral point of view”
 Actually intended to represent various points of view; viewpoints
are attributed to the agencies, offices, and individuals participating
 Consensus may or may not emerge!
300000
Apr-06
May-06
Jun-06
Jul-06
Aug-06
Sep-06
Oct-06
Nov-06
Dec-06
Jan-07
Feb-07
Mar-07
Apr-07
May-07
Jun-07
Jul-07
Aug-07
Sep-07
Oct-07
Nov-07
Dec-07
Jan-08
Feb-08
Page Views
50
35000
30000
30
20
10
10000
0
5000
Total Pages
Apr-06
May-06
Jun-06
Jul-06
Aug-06
Sep-06
Oct-06
Nov-06
Dec-06
Jan-07
Feb-07
Mar-07
Apr-07
May-07
Jun-07
Jul-07
Aug-07
Sep-07
Oct-07
Nov-07
Dec-07
Jan-08
Feb-08
60
Apr-06
May-06
Jun-06
Jul-06
Aug-06
Sep-06
Oct-06
Nov-06
Dec-06
Jan-07
Feb-07
Mar-07
Apr-07
May-07
Jun-07
Jul-07
Aug-07
Sep-07
Oct-07
Nov-07
Dec-07
Jan-08
Feb-08
Millions
Intellipedia’s Hockey-Stick Growth
Registered Users
40
25000
20000
15000
0
250000
200000
150000
100000
50000
0
The Top-Secret Wiki Gets Cloned
Summer 2007, ICES introduced 2 new Intellipedia versions:
•
One on the SECRET network “SIPRNET”
•
One on a “Sensitive But Unclassified” network “DNI(U)” (a
protected trunk apart from the regular Internet)
Rationale:
•
Many military intelligence analysts (and most soldiers) only
have access to SIPRNET
•
Many DHS personnel and Law Enforcement have no
clearances whatsoever for classified information
•
Many IC personnel like to work at home on research and
topical news items
Walled Gardens Within Walled Gardens:
Relative Value of Classified Information
Relative Number of Users,
Also Relative Volume of Data
DNI(U)
Relative Growth in Intellipedia Pages
300,000
250,000
SIPRNET
200,000
150,000
JWICS
SIPR
100,000
JWICS
(Top Secret)
50,000
0
DNI(U)
Anticipate a Network Effect for DNI(U)?
Expect increasing rates of growth for DNI(U)
usage and information sharing
 Improved realtime Internet data-mining
 Awareness of value of collaboration outside
traditional IC boundaries (DHS, LE, foreign
partners)
 Improved Web 2.0 tools deployed on DNI(U) to
mirror those on JWICS and the Internet
Intellipedia Totals on All Three Networks
300,000
250,000
200,000
64,782 users
2.3 million edits
150,000
JWICS
SIPR
100,000
50,000
0
DNI(U)
Bottom Line: Knowledge Work is Universal
New IC Focus:
“Analytic Transformation”
 Launched by ODNI, April 2007
 Both “analysis” side and “techie” side
 DDNI/A and DNI CIO are the two project owners
 Several key programs:
 Community-wide “IC Data Layer” to aggregate access

to “all” databases (no one knows the true number)
A-Space, a16-agency “collaborative environment for
analysis”
 DNI assigned job of ICDL and A-Space to DIA on
behalf of full IC - because of our SOA work
DIA’s Alien: All-Source Intelligence Environment
SOA Planning Begun 2005: Full web-services framework
 Alien is a framework, not a single tool
 Reliant on globally networked set of data centers
 New best-of-breed analytic software
Alien data services – tying data together
 Message traffic and other text sources
 Traditional single-INT databases
 Integrated security architecture for single sign-on
Alien allows tools to exploit semantically-enhanced data
METS: Metadata Extraction & Tagging Service
“Black-box Tagging Factory” combines 13 separate
best-of-breed entity-identifiers, natural-language
processors, disambiguators, tagging engines.
34
Key Desired Features of A-Space
Wikis, blogs, social networking,
personalized RSS feeds, collaborative
cloud-based word processing, mash-ups,
and content tagging…
… all built atop an underlying SOA.
A-Space: think “iGoogle,” “Live Spaces”
Metrics (a key post-9/11 recommendation)
37
A-Space Pilot Schedule: Bridge Too Far?










Pilot Awarded
Pilot Development and Integration
Pilot Development Freeze
Integration Testing and IPAT
Functional Testing (Approved Users)
Final Clean Up
C&A DIA*
C&A DNI*
Installation at DIA’s main Data Center
Prototype available to IC users
Sep 14, 2007
Sep 14-Nov 23, 2007
Nov
23, 2007
Nov 26-30, 2007
Dec 3-7, 2007
Dec 10-12, 2007
Dec 13-14, 2007
Dec 17-19, 2007
Dec 20-28, 2007
Dec
31, 2007
Lesson:
Many Enterprise IT Projects Fall Short of Expectations
Average IT Project Success
Time overruns
Budget overruns
Incomplete features
Incomplete functions
Cancelled prior to
completion
Abandoned
Source: CIO Executive Board research; Standish Group 2004 CHAOS Report
On time
On budget
Desired features
Desired functions
Other Government Examples: epa.wik.is
http://epa.wik.is/
epa.wik.is goes mashup bigtime
41
 Extensibility: Integration with Yahoo!,
Windows Live, Google, Flickr,
WidgetBox, YouTube, and much
more.
 “Data reuse in mashups will
revolutionize EPA data architecture,
data management, and data reuse
applications!”
EPA Architect Brand Niemann
Near-Future IT Enablers for the IC
Semantic Web - Global all-source system enabling rich
ontological information management
 autonomously and presumptively alerting analysts
 automatically populating knowledge bases
 cueing other military and IT systems
GIGINT - ability to mine and control the Global Information Grid
without human intervention, including the billions of sensor/
RFID/nano/autonomous devices communicating with the Grid.
 Gartner: By 2013, more than 200 billion processors will be in
daily use around the world
Virtual Worlds
 New methods of modeling,
simulation, and collaboration
are being created for analysts
and collectors
 “Knowledge Walls” and Crisis
Centers can be built more
cheaply in a Virtual World, still
using real-time feeds
43
Research Underway
for Future Enterprise Effectiveness
1.
2.
3.
4.
5.
6.
SOA environments driven entirely by business processes
Cross-Domain capabilities as embedded, intuitive services
Rapid increases in speed/volume of sensor and analytic feeds
Stateless devices (the ultimate thin client “computer”)
Wideband agile human interfaces, and true video tele-presence
The far edges of technological support for analysis:



Support to prediction;
Crisis uncertainty management;
Dynamic retasking of machines by machines...
Lesson: Joint Leadership Responsibility
Accountability
100
0
Business Leaders
CIO
IT Portfolio Lifecycle
Lewis Shepherd
Microsoft Institute for Advanced
Technology in Governments
www.ShepherdsPi.com