THINGS THAT THINK, SEE, FEEL AND DO…
Recording and being able to retrieve
and use everything I’ve ever
read (written),
and also heard (said), and
seen (participated in or presented)
Vanguard Group
11 November 1998
Gordon Bell
Microsoft Corporation
Recording Everything
Recording Everything
My project … putting my legacy
and future bits in Cyberspace

Personal paper documents
–


Encode video and audio lectures
Current electronic media
–
–

Books (encoded by CMU), papers, articles,
memos, notes, email, notebooks, drawings
email, files, papers, photos (now digital)
voice notes and records
Would like to capture photos and slides
Recording Everything
The other side… getting the
bits out of Cyberspace

Retrieval is the next challenge
–
–
–

Database(s)
Indexers (Magnifi, Virage, etc.)
Icons so we can help the computer
Being able to see and use electronic
bits as well as paper bits will have to
wait until display technology appears
Recording Everything
Storing all we’ve read, heard, & seen
Documents
page or fax
business card
snapshot
350 page book
image
100 K
5K
3M
25 M
Human data-types
read text, few pictures
speech text @120wpm
speech @1KBps
Video comp. 50KbPOTS
video comp. 200Kb VHS
video comp. 4.3Mb DVD
compressed
4K
500
25-100 K
1-2 M
/hr
200 K
43 K
3.6 M
22 M
90 M
1.8 G
/day/4Yr
2 -10 M/G
0.5 M/G
40 M/G
.25 G/T
1 G/T
20 G/T
#/GBy
10K;250K
200K;2M
10,000
40;750
/lifetime
60-300 G
15 G
1.2 T
25 T
100 T
1P
Recording Everything
Capture and encoding
Recording Everything
HP Digital Sender…
color document scanner
Recording Everything
Sony voice notes recorder…
Recording Everything
Evolving cameras to capture
images and voice notes
Recording Everything
A word about storage… CDs
hold 6 “near VHS quality”
videos (300 Kbps)
Recording Everything
Coexisting with Paper



Paper is the best GUI (screen dump) without peer
– portable: read anywhere, anyway, anytime
– contrast ratio
– resolution (3-30x screens)
– form factor, weight
– ubiquity, cost, infrastructure
– with good OCR, it is a loss-less medium
For many documents, paper is very expensive
– large and multiple documents are heavy
– hard to transmit
– difficult to store (only access is via people)
– computer cannot access them
Goal: use paper as aRecording
loss-less Everything
bit medium
Computer-Paper Co-existence


Goals for encoding paper for the computer
– Handling must be as easy as discarding it
– Input, Identify, OCR, Index, save, and
retrieve in every possible context
– Interface: unobtrusive and humane
Put all documents in Cyberspace
– Ubiquitous scanners are key!
– Totally eliminate the need for copiers
– Totally eliminate the need for file cabinets
Recording Everything
Paper to handle
Recording Everything
m
Fi
na
em n c
os ial
,
P etc
er
so .
nn
el
E Le
du g
ca al
tio
n
M al
e
H dic
ou a
se l
ho
ld
N
ot
e s Ot
, j he
ou r
rn
a
H ls
ob
b
C ies
a
M len
em d
a
or r
ab
3r
ili
a
d
P
pa h
rty oto
ar s
tic
le
s
M
is
c.
W
or
k
8. What type of information do
you file?
45
40
35
30
25
20
15
10
5
0
Recording Everything
Atoms vs electrons/flux/photons
for financial bits
Atoms (mass)
money
statements
bills / checks
coupons
Electrons, etc. (massless)
database, smart card,
credit card, debit card
web access
bill present / check free
cyber-coupons
stock
database, web
statements, reports
web access, email
+company infor, analyst reports, etc.
private placements
web access, email
trade confirmation
direct trades
Recording
Everything
mail voting
on
line voting
New or old money… it’s just bits
Credit
ATM /credit
Prepaid
Check
Cash
Recording Everything
Prepaid
Put those checks & statements in
Cyberspace or eliminate them!
Recording Everything
Paperless transactions: put them all
in Cyberspace
Recording Everything
Buying & selling stock: what a pain!
Faxes? (Electronic signatures are legal in Georgia.)
Recording Everything
Transfer document
Recording Everything
Electronic
bill
presentation
Recording Everything
Recording Everything
Clipped
articles
etc.
Personal files
2 weeks
mail
Family
memorabilia
Recording Everything
Lawsuit
Paper in
process
Supercomputers
vendor file
A
Lawsuit
Startups
to look at
Recording Everything
Recording Everything
Gordon’s
Output
Gordon’s
Output
Computer
support
stuff
4 files
Recording Everything
Storage I.e. disks and
databases …
Recording Everything
Storing … disks are inexpensive,
but what is the cost to find it?


Disks make storage very inexpensive
Which database?
–
–
–
–
–

Web pages will a page be there forever?
Personal file system with its folders
Outlook with its folders and contacts
Rolladex
Visioneer image database
All of these haveRecording
to be indexed!
Everything
Recording Everything
Recording Everything
Aids for Retrieval
Recording Everything
Recording Everything
Recording Everything
Magnifi’s SmartMedia
Context
Relevance
Image Recognition for Objects
Speech Recognition
Image Recognition, Lexical Cues
Transcripts
Close Caption
Lexical Analysis
Viewing
Color
Previews - Power Point, PDF,
B&W
Brightness Video, Sound, Artwork
Contrast
Volume
Speech-Music
Meta Information
Recording Everything
Survey of users in various
size corporations
Recording Everything
A survey of attitudes toward paper
(8/97) in various sized Co’s.





>3,000 users (admin. and profs.)
1/6 Response
Personal
47%
Work
52 %
More 10,000+ employee company
people choose to fill out survey for
work (63%)… response evenly split
among companies.
Time (hours/month) filing
12-20
Responses: (scaleRecording
1-7, 7 strongly
agree)
Everything
#1

I often wish I had an electronic (computer) copy
of my paper documents.
–
–
–
Average
5.06
Highest for companies of >1K people ~5.4
Lowest for companies of 10-99 people
4.67
Recording Everything
#2, 3

My filing system takes too much space.
–
–
–

Average
4.89
Admins care most at
5.32
followed by people in companies of 1,000+
employees
~5.16
My most recent documents tend to be my
most important.
–
–
Average
Admins highest
4.68
Recording Everything
5.08
#4, 5, 6

The effort I spend filing is worth it.
–

4.51
It is easy for me to find a document when I
need it.
–

Average
Average
4.38
I spend more time looking for a document
than I do filing a document.
–
Average
4.05
Recording Everything
#7,8

I am satisfied with my filing system.
–
–
–

Average
3.99
Admins
4.27
unemployed are most satisfied 4.71
My papers are organized in such a manner
that it represents actions I need to take, or
part of my to-do list.
–
Average
3.92
Recording Everything
9, 10

I clean out my filing system on a regular
basis.
–

Average
3.4
I frequently lose documents.
–
–
Average
3.05
Companies (1K-10K lose more) 3.43
Recording Everything
© and no one cares...

I often worry about copyright issues when I
file, photocopy or distribute documents.
–
Average
2.20
Recording Everything
18. I would file my paper
documents on a computer to …






make it easier to find documents
5.73
save space
5.49
better preserve my documents
5.46
make it easier to distribute documents 5.33
be able to have access to documents on a portable
computer or network
5.03
make it easier to track routing
4.91
Recording Everything
19. I would NOT file my paper documents on
a computer because…
•file formats might change
•I don’t have the equipment
3.81
3.50
•documents are less secure
3.27
•filing would take more time than it saved 3.11
•computer documents are harder to read 3.10
•I might lose a document
2.89
•documents take longer to access
2.78
•it is harder to control distribution
2.68
•applications are too hard to learn / use 2.63
•documents are harder toRecording
find
2.59
Everything
•documents are harder to share
2.42
20. I would not trust my important documents
or files on a computer because of…







possible network failure and/or reliability
possible disk failure (permanent loss)
4.63
4.07
a chance that someone might delete a file
a chance that I might delete a file
3.80
3.32
application reliability
3.29
general computer reliability
3.23
Recording
Everything
general distrust. (can't pinpoint
a reason)
2.66
21. Do you file documents on a
computer?


75% Yes
22. Do you file any documents on a
computer that used to be on paper? e.g,
scan the document into the computer… says
nothing about scanning though
–
–
–
26% Yes
Admins, 1,000 - 9,999, < 10
10,000+, 10-99
~33%
~20%
Recording Everything
Start now



Take the oath: put it in Cyberspace
Get rid of copiers
Get rid of 1/2 of the filing cabinets
Recording Everything
I would be interested in a computer
application that helps
me organize my paper documents
and files.


Average 5.49
No huge fluctuation, except for the
not employed at 4.79.
Recording Everything
The end
Recording Everything
Now how do you find or
use the rich information
Need the system to:
locate, retrieve, visualize, order,
up load the corporation’s IP assets
(text, proposals, images, videos,
presentations, etc.) …
with appropriate controls.
Recording Everything
Gains if 20, 40, & 60% / year for
processor & memory chips, and disks
1.E+21
Secondary Mem. ie. disks
60%=
Exaops
1.E+18
40%=
Petaops
1.E+15
20%=
Teraops =
terabytes
of prim.
Mem.
1.E+12
1.E +9
1.E+6
1995
2005
2015
2025
2035
2045
Recording Everything
Fujitsu 42” plasma panel
Recording Everything
The Cyber Admin or
the prosthetic memory…
When we can store
everything we’ve:
read/written,
heard/said,
seen/acted, plus
physical parameters.
Recording Everything
Administrivia, paper and
phone coexistence






With no administrative support, we are our own
administrative assistants, secretaries, and gofers.
Forms and travel arrangements
Calendar management
Paper handling and its database
– Identification, input, indexing, and interface
– Interface to message management database
Message management database
– email, voicemail, fax, contacts, calendar
– phone and videophone must be “on line”
Personal database(s) management, a major time sink
Recording Everything
Coexisting with the
telephone





Overall integration with the computer
Answering machine, recorder, and
transcriber
And database with transcription,
keywords, and voice
“Wildfire” it … finds me
ICQ capability for both computer and
phone
Recording Everything
What does Cyber Admin do?



Captures the creation of all
personal/professional information
Stores and organizes
Retrieval is the challenge
–
–


recalling readings, conversations,
presentations, images
helps the “guardian angel”
What are the apps?
when we can do this?
Recording Everything
Virage Video Cataloger
Recording Everything
Intrastructure
Recording Everything
SOHO (small office, home office)
network computing environment
POTS
(legacy services)
IP Dial tone (Internet, phone,
videophone) >1.5 Mbps
NT Server for:
comm/network, POTS/IP
gateway, file, print, compute
LAN
PC
Phone
PC ... NC*
*NC, NetPC,
Xterm, etc.
Everything
Phone Recording
Phone
...
The Guardian Angel … a case
of having bits about us and
how to help us plus being
able to communicate
Recording Everything
Your
husband
just died,
… here’s
his black
box
Recording Everything
Steve Mann
in
Cyberspace
Recording Everything
BAN, PAN or whatever for
people networks




Temperature monitoring with the flu
Saved me from a heart attack
Where is the person, exactly… a baby
sitter
Let’s interact… ala Barney
Recording Everything
Medtronics
Implanted
Cardioplastic
stimulator
cardiorecorder
Recording Everything
Audio, pix, T, P, ECG,
location, physiological
parameters…
1 GB
Recording Everything
Not shown:
ECG; GPS;
Compass;
altimeter
PCS; Pilot
Libretto,
.5mm
pencil
Recording
Libretto PS,
Ricoh Camera;
Swiss Army
Everything
Knife
Steve Mann
in
Cyberspace
Recording Everything
Time and Motion Study




Where does the time go?
Can anything be parallelized?
Is there an app to help?
– Will the gain be worth the pain?
What is likely to be a fruitless and/or
over- worked area e.g. telecollaboration?
Recording Everything
Time and motion of teleworker (me)
context & “doing”
Context
Doing
 Microsoft
 sleeping, feeding,traveling
 Families
 goofing-off, interacting,
– Wife
vacationing
– Friend
 supporting infrastructure
– Children &
 administriving
grandchildren  info- and cyber-grazing,
– TCM
learning
 GB Corp
 communicating
– Startup
 WORKING?*
– Consultant-at-large
* building, deciding, interacting,
presenting,
testing,
Recording
Everything
thinking, problem-solving, writing,
Group Structure of Interaction






solo
pair
small group of 3-6
group gropes & formal meetings 10-100
whole organization or organization-atlarge
world
Recording Everything