New Technology
Support for Government
and Social Research
Russian-U.S. Workshop
with Ilya Zaslavsky
San Diego Supercomputer Center
University of California, San Diego
ZASLAVSK@SDSC.EDU
Disclaimer
Besides my own slides and slides developed and
commonly shared within the DAKS group of
SDSC, this and other presentations given in
Moscow and Tomsk during the Russian-U.S.
workshop Feb 10-20, 2003, partly use fragments
of slide presentations available from multiple
sites on the Internet.
Copyright to these slides remains with their
respective owners.
The Moscow and Tomsk presentations are not
intended for Web publication.
“American Speakers”, Russian-US Workshop, Moscow-Tomsk, 10-20 Feb 2003
Overview of Day 2
• Distance Education: Personal
experiences
• Federal Statistics
• Social Statistics Online
• Sociology Workbench
– demo!!
“American Speakers”, Russian-US Workshop, Moscow-Tomsk, 10-20 Feb 2003
What is DE?
• Learning that happens when the
instructor and student are in different
physical locations
• First ‘distance education
technology’
“American Speakers”, Russian-US Workshop, Moscow-Tomsk, 10-20 Feb 2003
DISTANCE EDUCATION FOR
WHOM
Distance Education brings the
information to the student.
It is very useful for an individual
who cannot go to an ordinary class.
• WORKING PEOPLE
• HANDICAPPED PEOPLE
• INTERCONTINENTAL STUDENTS
• CONTINUING EDUCATION DURING
CARRIER
“American Speakers”, Russian-US Workshop, Moscow-Tomsk, 10-20 Feb 2003
Distance Education Tools
There are basically two
categories of distance
education delivery system.
SYNCHRONOUS
• Simultaneous participation
of both student and
instructor is required.
• Interaction is done in real
time.
Television, Audiographics,
Computer Conferencing,
IRC can be considered as
synchronous distance
education tools.
“American Speakers”, Russian-US Workshop, Moscow-Tomsk, 10-20 Feb 2003
ASYNCHRONOUS
• Doesn’t require the
simultaneous participation of
all students and instructor.
• Students may chose their own
instructional time frame
• More flexible than
synchronous.
E-mail, listservs, audiocasstte,
videotape and WWW can be
considered as asynchronous
distance education tools.
Distance Education Tools
• PRINT (1800s)
• POSTAL SERVICE (1850s)
• TELEPHONE (1875)
• RADIO (1895)
• AUDIO TAPE (1945)
“American Speakers”, Russian-US Workshop, Moscow-Tomsk, 10-20 Feb 2003
Distance Education Tools
• BROADCAST TELEVISION (1953)
• VIDEO TAPE (1960)
• AUDIO TELE-CONFERENCING (1960)
• CABLE TELEVISION (1965)
“American Speakers”, Russian-US Workshop, Moscow-Tomsk, 10-20 Feb 2003
Distance Education Tools
• COMPUTER ASSISTED INTRUCTION (1975)
• COMPUTER CONFERENCING (1980)
• SATELLITE DELIVERY (1980)
• AUDIOGRAPHIC TELE-CONFERENCING
(1980)
“American Speakers”, Russian-US Workshop, Moscow-Tomsk, 10-20 Feb 2003
Distance Education Tools
• FACSIMILE (1980)
• VIDEO CONFERENCING (1980)
• VIDEO DISC (1984)
•
•
•
•
CD-ROM COMPACT DISC (1985)
COMPRESSED VIDEO (1988)
MULTIMEDIA (1989)
WIRELESS WORKSTATIONS (1996)
“American Speakers”, Russian-US Workshop, Moscow-Tomsk, 10-20 Feb 2003
INTERNET tools in Distance
Education
• E-MAIL SOFTWARE: Free form text messages between lecturer and
individual students.
• MAILING LISTS: Delivering messages from instructor to students.
• TELNET, WEB PAGE: Delivery of learning materials and
assignments to student as needed.
• IRC, COMPUTER CONFERENCE: Text based commentary between
students about the learning resources.
• AUDIO CONFERENCE EQUIPMENT or INTERNET PHONE:
Synchronous audio communication students and lecturer
• ANSWERING MACHINE: Asynchronous audio communication
students and lecturer.
Ref: Distance Learning Glossary
“American Speakers”, Russian-US Workshop, Moscow-Tomsk, 10-20 Feb 2003
FedCon Alliance/NPACI Meeting
Synchronous Distance Teaching on a
Low Budget:
Ed Center’s Experiences
and Student Assessment
Ilya Zaslavsky, Ph.D.
zaslavsk@rohan.sdsu.edu
http://www.edcenter.sdsu.edu
http://www.edcenter.sdsu.edu/disted
voice: 619.594.0491
fax: 619.594.0433
The setting
Two classes: 567 “Geodata Handling” and 569 “GIS”,
both relatively small (8-14 students).
Additional hardware and software - not to exceed $200 per
machine, or $500 total
This is me (Ilya Zaslavsky) in
San Diego, teaching a class
This is the GIS lab (17 Pentium133, Windows 95, with
Windows NT server) at WMU, right after a lecture
Teaching
assistant
(Kathleen Baker)
at WMU
“American Speakers”, Russian-US Workshop, Moscow-Tomsk, 10-20 Feb 2003
Organization of the Distance Learning System
(SDSU - WMU, Geography 567 & 569, 1997-98)
Tools used:
hr
c
syn
us
ono
Web Server
at WMU
us
o
n
o
chr
A
Kalamazoo
n
Sy
Lab Demo
Computer
Web Server
at SDSU
- synchronous
NetMeeting
(sharing
applications,
whiteboard,
chat)
- asynchronous
San Diego
Instructor’s
Workstation
Lab Server
at WMU
class web pages
formatted in
DocReview
TA’s
Workstation
(web-based
discussions,
comments, Q&A)
E-Mail
“American Speakers”, Russian-US Workshop, Moscow-Tomsk, 10-20 Feb 2003
“American Speakers”, Russian-US Workshop, Moscow-Tomsk, 10-20 Feb 2003
the classroom, with camera
on tripod, Kathleen
in front of the class
Using Video
• Camera on the instructor’s side
–
–
pointing at instructor’s face, at active-matrix laptop
screen (not too clear), at scratch paper (possible,
but slow)
move as little as possible, for good overall signal
quality
• Camera in the lab:
–
pointing at students or TA, often turned off (it’s
important to watch that something is going on on the other side but not at the expense of audio!)
useful for student presentations
Video is still a “frosting”, so far most useful in one-toone interaction (with grad. students, etc.)
–
“American Speakers”, Russian-US Workshop, Moscow-Tomsk, 10-20 Feb 2003
Using Audio
• turned out to be the major problem
• full-duplex audio is a must, in half-duplex mode
microphone switches direction of transmission when
it picks up extraneous noises
• stay up-to-date with sound and network
drivers
• transmit signal in one direction (turn off one
microphone, one or both videos)
• don’t breathe, hold the mike in your teeth
• have a phone line as backup
“American Speakers”, Russian-US Workshop, Moscow-Tomsk, 10-20 Feb 2003
Sharing chat, whiteboard,
programs
• Chat window:
– good for questions from audience, repeating important
points, or when connection is poor
• Whiteboard:
– turned out very useful, though drawing with a mouse
requires some skill…
– having a set of pre-built images, and pasting them into
whiteboard was the best
• Sharing applications:
– very important; this is what makes it different from
standard videoconferencing
– SPSS, ArcView with extensions, Netscape, Stella, etc.
… but slow...
“American Speakers”, Russian-US Workshop, Moscow-Tomsk, 10-20 Feb 2003
“American Speakers”, Russian-US Workshop, Moscow-Tomsk, 10-20 Feb 2003
“American Speakers”, Russian-US Workshop, Moscow-Tomsk, 10-20 Feb 2003
“American Speakers”, Russian-US Workshop, Moscow-Tomsk, 10-20 Feb 2003
“American Speakers”, Russian-US Workshop, Moscow-Tomsk, 10-20 Feb 2003
The asynchronous part: web-
based discussions
• Just placing lectures on the Web is not enough
http://unix.cc.wmich.edu/zaslavsk/classes/567 and
/569
• E-mail discussion of lecture notes and
assignments - first 4-5 weeks
–
–
class e-mail lists;
have to forward personal messages to the list...
• DocReview, a system for Web-based
collaborative text editing/commenting
–
–
don’t have to switch to e-mail to do comments
boost in interaction in 567, more focused (contextual)
questions
“American Speakers”, Russian-US Workshop, Moscow-Tomsk, 10-20 Feb 2003
“American Speakers”, Russian-US Workshop, Moscow-Tomsk, 10-20 Feb 2003
The chronology (weeks 1-6)
new techniques
Week 3
Week 4
Week 5
overall
chat window, voice, video
unstable
decent at times
e-mail discussions, chat,
video of chalkboard
bad or absent, on
one or both sides
sporadic, in and
out
total disaster
lectures in DocReview
sporadic
50/50
TA gets a TA
unstable, but present 60/40
for entire classes
Week 1 of instructor, students
Week 2
connection
sharing whiteboard
sharing applic. (SPSS),
Week 6 whiteboard graphics kit; doing it from new
site, with a phone
569 students lead
backup
“American Speakers”, Russian-US
Workshop,
Moscow-Tomsk,
10-20
Feb
2003
live discussion
quite decent, not
a circus anymore
bad in the first
lecture, good in
the afternoon
The chronology (weeks 7-12)
new techniques
Week 7
connection
overall
** Ilya in K-zoo **
** midterms **
Week 8
** midterms **
Week 9
567 present their maps live,
video and audio on; sharing
ArcView
speakerphone for
569, good in 567
good
Week 10
students get folder passwords (files wiped out…)
speakerphone for
569, good in 567
good
finally… remote pointer
Week 11 on whiteboard...
both 567 and 569 go
over Internet fine
connection is bad, so
Week 12
students do lab as
planned;
“American Speakers”, Russian-US Workshop, Moscow-Tomsk, 10-20 Feb 2003 we try out Tango...
Ilya in San Jose at
Supercomputing’97
** midterms **
excellent!!
What do you do if you can’t do
anything ?
“… if a horde of guests suddenly appears at your doorstep, and you
have but absolutely nothing to treat them with, go to the
basement and pick up a mid-sized leg of lamb. This is what you
do with it…”
(the “Molokhovets book”, 1898, a famous Russian cookbook)
• try to connect to another network, even via a
modem - forget about transmitting any useful
signal, though
• type in the whole lecture using as many
interaction tricks as possible, have your TA do
standby commentary
• have a TA who can do it all without you
“American Speakers”, Russian-US Workshop, Moscow-Tomsk, 10-20 Feb 2003
Student Assessment
Standard assessment
• No significant difference in student performance compared to previous
years.
Student surveys
• 3 surveys: at the beginning, middle, and end of each semester
– no previous distance learning experience for all, standard Web tools are
fairly novel for many
• In the final survey (30 students, both factual and open-ended questions):
– all but three say that they are satisfied with what they have accomplished
in this course
– all joined virtual meetings, used class web pages
– 83% used whiteboard, 60% collaborated in shared applications (most
useful techniques); 30% used text chat
– lack of communication with instructor, missed eye contact, asked less
questions, though interacted more with fellow students
“American Speakers”, Russian-US Workshop, Moscow-Tomsk, 10-20 Feb 2003
Lessons Learned and Useful
Strategies
• Advanced preparation and careful lecture scenarios are
important, though some ad hoc freedom in using visual
tools is necessary. Have a pre-built set of images to share
in whiteboard.
• Involve students in presenting themselves (once they see
that they can do it, it works!)
• Re-create or maintain traditional channels of studentteacher interaction as much as possible.
• Maintain all possible interaction channels active (e-mail,
chat, whiteboard, DocReview, etc.)
• Archive discussions, whiteboards, chats.
• Meet with students face to face, at least couple times.
“American Speakers”, Russian-US Workshop, Moscow-Tomsk, 10-20 Feb 2003
Lessons Learned and Useful
Strategies (2)
• Encourage student interaction with peers.
• Have a set of back-up plans if network connection is bad,
including speakerphone.
• Have a reliable support on the "receiving side" !!!!
• Survey students and keep a class log, this will help in
detecting strategies that work.
• Use a second pair of computers to transmit different
components of the conference.
• Turn off video in one or both directions, to keep audio
clear.
• Accept that this approach is not necessarily good for every
student, and some won't be happy.
• Re-think your teaching style and habits!
“American Speakers”, Russian-US Workshop, Moscow-Tomsk, 10-20 Feb 2003
remote
synchronous
“Distance learning” in
Internet times
1. Lecture notes on the Web (everywhere, electronic
textbooks; Geographer’s craft - best example)
2. Lecture notes with some interactivity (tests, e-mail)
3. Notes + session recording and playback (WLS - best
example)
4. Web-based collaboration within the classroom
5. Synchronous chat, supplemented by occasional
audio/video with CU-SeeMe (NASA AMES to U. of North
Dakota, $170,000, others)
6. Collaborative environment, sharing Java applets:
Habanero (NCSA) - several sites
7. Synchronous audio/video, applications sharing +
interactive asynchronous mode: NPAC to Jackson State
U. (“Tango”) and WMU (“NetMeeting”)
“American Speakers”, Russian-US Workshop, Moscow-Tomsk, 10-20 Feb 2003
URLs
• This talk: http://unix.cc.wmich.edu/zaslavsk/classes/nov25/
• Michigan GIS Institute:
http://www.wmich.edu/geography/GISinstitute/
• Tango web collaboratory (NPAC, Syracuse):
http://trurl.npac.syr.edu/tango/
• Habanero collaboratory (NCSA):
http://www.ncsa.uiuc.edu/SDG/Software/Habanero/
• Web Lecture System (North Carolina St. U.):
http://renoir.csc.ncsu.edu/WLS/
• Ed. Center on Computational Science and
Engineering: http://www.edcenter.sdsu.edu/
• Geographer’s Craft:
http://www.utexas.edu/depts/grg/gcraft/contents.htm
“American Speakers”, Russian-US Workshop, Moscow-Tomsk, 10-20 Feb 2003
Distance Learning
Singapore-MIT Alliance (SMA) Program
• A very selective Master/Ph.D. joint program
• Involves National University of Singapore (NUS),
Nanyang Technological University (NTU) & MIT
• Launched in July 1999
• Students undergoes initial orientation in Singapore,
then 1 month in MIT followed by continuation of
program in Singapore.
• Students earn single degree with indigenous
registered institution
• 3 programs totaling about 67.5 conferencing
hours/week
• Expand to 5 programs totaling about 100 hours/week
in August 2001
“American Speakers”, Russian-US Workshop, Moscow-Tomsk, 10-20 Feb 2003
SMA DE - Configuration
Application Sharing via Internet 2
MIT
Auditorium
ISDN
Audio / Video Conferencing
Internet 2
Student
access
via web
for
revision
View video
Online
digitization
onto video
server
Video Server
“American Speakers”, Russian-US Workshop, Moscow-Tomsk, 10-20 Feb 2003
NUS Auditorium /
SMART Classroom
SMA DE Connectivity
NUS Classroom
OC-3
NUS
Gateway
SingAREN*
GigaPoP
14 Mbps ILC
MIT Classroom
MIT
Gateway
“American Speakers”, Russian-US Workshop, Moscow-Tomsk, 10-20 Feb 2003
Abilene
155 Mbps
LA
PoP
SMA DE – Distance Counseling
• Primary instructor works with remote assistants
• Multiple platforms:
–
–
–
–
Email
Discussion forum
Desktop conferencing
Chat rooms
• Lecturers equipped with Distance Education kits:
– Desktop conferencing
– Scanner
– Writing tablet mouse
• Students have access to desktop multimedia
conferencing stations
“American Speakers”, Russian-US Workshop, Moscow-Tomsk, 10-20 Feb 2003
SMA DE – Lessons Learnt
• Synchronous teaching/learning:
– A challenge to maintain Line of Presence in large
classroom
– Sustained audio quality a challenge (noise &
breakups)
– Single screen mode demands higher video
resolution
– Delay in camera tracking system poses problems
for Q&A sessions – improvements using push-totalk system
– Secondary communication channels for inter-team
session monitoring to ensure rapid problem
resolution
“American Speakers”, Russian-US Workshop, Moscow-Tomsk, 10-20 Feb 2003
SMA DE – Lessons Learnt
(cont)
• Pedagogical Issues:
–
–
–
–
Faculty needs to understand technical limitations
Adaptation required
Conducting an effective Q&A session is challenging
Backup connectivity activation disrupts flow of lessons
• Support Issues:
–
–
–
–
–
Requires well understood inter-team operational protocols
Inter-technical team building essential
Core technical staff available on mobile/beepers
1-man operation highly risky
Replicated services in local helpdesk a necessity for quality
service
“American Speakers”, Russian-US Workshop, Moscow-Tomsk, 10-20 Feb 2003
Distance Education Myths
• Distance education is about technology
• Distance education is about replacing
professors http://e-education.mtt.ca/
• Distance education is about saving/making
money
http://www.polycom.com/streams/streamstation
_overview/index.htm
• Distance education is about administration
http://129.128.5.72/main/index.jhtml
“American Speakers”, Russian-US Workshop, Moscow-Tomsk, 10-20 Feb 2003
Increasing Competition
UCLA’s OnlineLearning.net
• offers more than 1,000 online courses and
has enrolled over 12,000 students.
• offers 1,000 American Airlines frequent
flyer points when you enrol in a Spring
2000 online course.
“American Speakers”, Russian-US Workshop, Moscow-Tomsk, 10-20 Feb 2003
Federal Statistics
Social Statistics Online
“American Speakers”, Russian-US Workshop, Moscow-Tomsk, 10-20 Feb 2003
Federal statistics
• 70 agencies, Census and BLS are the largest, but
many more: SSA (program-based), EPA (for
regulations), BTS…
• Little coordination: Office of Management and Budget
+ Interagency Council on Stat Policy
• Population estimates, redistricting (Census, $180 bil
to local administration..)
• Consumer Price Index: wages, pensions, social
services
• Health, Crime, Unemployment, “Digital Divide”
• General Social Surveys
“American Speakers”, Russian-US Workshop, Moscow-Tomsk, 10-20 Feb 2003
U.S. Census
“American Speakers”, Russian-US Workshop, Moscow-Tomsk, 10-20 Feb 2003
U.S. Census
“American Speakers”, Russian-US Workshop, Moscow-Tomsk, 10-20 Feb 2003
General Social Survey
• The General Social Survey (GSS) is bi-annual personal
interview survey of U.S. households conducted by the National
Opinion Research Center (NORC). The first survey took place in 1972
and since then more than 35,000 respondents have answered over
2500 different questions. The mission of the GSS is to make timely,
high-quality, scientifically relevant data available to the social science
research community.
95. Do you think the use of marijuana
should be made legal or not?
Should
Should not
Don’t know
No answer
“American Speakers”, Russian-US Workshop, Moscow-Tomsk, 10-20 Feb 2003
Data Archives
• Inter-university Consortium for Political and
Social Research (ICPSR)
– international data collections
• Roper Center for Public Opinion Data
– Latin American Survey Data Bank and Japanese
Data Archive
• Data archives around the world - CESSDA
maps
• Differences in data access procedures
“American Speakers”, Russian-US Workshop, Moscow-Tomsk, 10-20 Feb 2003
Inter-University Consortium for
Political and Social Research (ICPSR)
http://www.icpsr.umich.edu
“American Speakers”, Russian-US Workshop, Moscow-Tomsk, 10-20 Feb 2003
Micro-level Data read into SPSS
World Values Survey (R. Inglehart); distributed by ICPSR
“American Speakers”, Russian-US Workshop, Moscow-Tomsk, 10-20 Feb 2003
Roper Center for Public Opinion
http://www.ropercenter.uconn.edu
“American Speakers”, Russian-US Workshop, Moscow-Tomsk, 10-20 Feb 2003
Surveys, and IT
• American Transportation Survey (CATI, 65
thous hholds), National Health Survey
(NCHS): 5000 respondents, interviews +
medical exams, National DNA sample
• All stages of survey development:
– Planning/design (What is general population;
sampling schemes…)
– Data collection (CATI, CAPI, CASI, PAPI)
– Processing (lack of standards…)
– Analysis
– Dissemination (for non-specialists, user-friendly,
single entry point: Fedstats, Ferret, etc.)
“American Speakers”, Russian-US Workshop, Moscow-Tomsk, 10-20 Feb 2003
What is FirstGov?
http://www.FirstGov.gov
“American Speakers”, Russian-US Workshop, Moscow-Tomsk, 10-20 Feb 2003
What is FirstGov?
• Point of access (“Web portal”) to
Government information and services
• Portal is a free service that indexes
Web-enabled government information
and transactions (links to “electronic
commerce” sites)
• FirstGov “Partners” program
“American Speakers”, Russian-US Workshop, Moscow-Tomsk, 10-20 Feb 2003
Information Discovery
Requirements of FirstGov
FirstGov is required to interface with
GILS-compliant servers,
finding information
Databases
in various
GILS
Bibliographic
formats,
Interface
Records
distributed
Locator
across the
Web
Records
Internet in
pages
catalogs, directories, databases
“American Speakers”, Russian-US Workshop, Moscow-Tomsk, 10-20 Feb 2003
The Sociology Workbench v2.0
“American Speakers”, Russian-US Workshop, Moscow-Tomsk, 10-20 Feb 2003
SWB and Digital Government
“American Speakers”, Russian-US Workshop, Moscow-Tomsk, 10-20 Feb 2003
SWB Architecture – XML related
Client
XML Survey
Document
Function
Query
XML
Output
XML Output with
Table Format
Stylesheet
SWB Interface
Table Querying
SQL statements
(via JDBC)
XML Survey
Document
XML Parser
XML Tree
SQL statements for table
creation and the insertion of all
question information (via
JDBC)
XML Processor
“American Speakers”, Russian-US Workshop, Moscow-Tomsk, 10-20 Feb 2003
Oracle
Table output
in XML
format
The DDI Initiative
• The Data Documentation Initiative
• A Project to Develop an XML Document Type
Definition for Data Documentation
• Maps to 15 elements of the Dublin Core
• 30 other recommended elements for social
science research & data management
• http://www.icpsr.umich.edu/DDI/codebook.ht
ml
“American Speakers”, Russian-US Workshop, Moscow-Tomsk, 10-20 Feb 2003
In a DDI DTD XML codebook you can
integrate meta-information about...
•
•
•
•
•
Intellectual content of a study
Its scope
Methodological details
Retrieval and dissemination policies
File location and format
“American Speakers”, Russian-US Workshop, Moscow-Tomsk, 10-20 Feb 2003
(+) References to accompanying
documents, e.g.
•
•
•
•
•
•
Reports on methodology,
Publications,
Classifications lists,
Questionnaires and similar,
Computer syntax files,
Tables of results,
etc.
“American Speakers”, Russian-US Workshop, Moscow-Tomsk, 10-20 Feb 2003
ddi in SWB
Challenges working with the ddi
– All tags in the Variable Description portion of the
DTD are optional.
– Flexibility of the ddi DTD allows for extensive
personal judgment in tag usage.
• Our Solutions
– Place restrictions on tags by making them
mandatory for our application. For a listing of SWB
mandatory tags and their usage:
http://edcenterdev.sdsu.edu/SOURCE/ddi.html
– Use XSL Stylesheets to preprocess XML documents
to adhere to our restrictions
“American Speakers”, Russian-US Workshop, Moscow-Tomsk, 10-20 Feb 2003
Example of a ddi compliant XML
document
<?xml version="1.0" ?> <codebook><docDscr>
<guide> Sample Data From SWB </guide>
</docDscr><stdyDscr><stdyInfo> <abstract>Example Survey </abstract> </stdyInfo>
</stdyDscr><dataDscr><var name="SPANKING" format="String" ID="q1"><valrng> <range min="0" max="9" /> </valrng>
<txt>Favor Spanking to Discipline Child</txt> <catgry> <catValu>1</catValu> <txt>Agree</txt> </catgry><catgry> <catValu>2</catValu> <txt>Neutral</txt> </catgry><catgry> <catValu>3</catValu> <txt>Disagree</txt> </catgry></var
</dataDscr>
</codebook>
“American Speakers”, Russian-US Workshop, Moscow-Tomsk, 10-20 Feb 2003
XML Output from Oracle
• API - oracle.xml.sql.query.OracleXMLQuery
• In order to produce XML output from Oracle the SWB
team had to create SQL functions to organize the
tables in a way in which we can easily convert from
Oracle tables to an XML document. This document
can either be displayed to the screen or stylized to
create a table.
• Output Tags- The tags are determined by the SQL
query, and vary with each type of table.
“American Speakers”, Russian-US Workshop, Moscow-Tomsk, 10-20 Feb 2003
Converting SWB to Web
Service
• Wrote SOAP wrappers for each SWB function that we
wanted to be a service
– Wrapper calls the servlet and parses the resulting XML.
Returns the root node of the XML document
<?xml version="1.0" ?>
- <ROWSET>
- <ROW num="1">
<DESCRIP>Missing Data</DESCRIP>
<AMOUNT>979</AMOUNT>
<AMOUNT_PERCENT>33.71</AMOUNT_PERCENT>
</ROW>…
</ROWSET>
“American Speakers”, Russian-US Workshop, Moscow-Tomsk, 10-20 Feb 2003
Survey Documentation and
Analysis (SDA) Program
• Written at UC Berkeley
• Used by ICPSR and others-- referred to
as DAS (Data Analysis System)
• Data files must be converted to SDA
format before use. ICPSR has
converted a number of data sets in their
topical archives into SDA format and are
converting more.
“American Speakers”, Russian-US Workshop, Moscow-Tomsk, 10-20 Feb 2003
Sources of Data at ICPSR
(http://www.icpsr.umich.edu)
• ICPSR topical archives
– National Archive of Computerized Data on Aging
(NACDA)
– National Archive of Criminal Justice Data (NACJD)
– International Archive of Education Data
– Substance Abuse and Mental Health Data Archive
(SAMHSA)
• General Social Survey
• National Election Study
“American Speakers”, Russian-US Workshop, Moscow-Tomsk, 10-20 Feb 2003
What Can You Do?
•
•
•
•
Browse codebook
Subset data
Download data and documentation
Run statistical procedures
“American Speakers”, Russian-US Workshop, Moscow-Tomsk, 10-20 Feb 2003
Statistical Procedures
•
•
•
•
Frequencies
Crosstabs
Comparison of means
Comparison of correlations
“American Speakers”, Russian-US Workshop, Moscow-Tomsk, 10-20 Feb 2003
What Else Can You Do?
•
•
•
•
Recode (temporarily)
Use control variables
Use filter variables
Use weight variable
“American Speakers”, Russian-US Workshop, Moscow-Tomsk, 10-20 Feb 2003
Frequencies Program -Specify Variables
• Row variable (required)
• Filter variables
• Weight variable
“American Speakers”, Russian-US Workshop, Moscow-Tomsk, 10-20 Feb 2003
Frequencies Program -- Select
Statistics
• Percents
• Central tendency -- mean, median,
mode
• Variability -- standard deviation,
variance
• Coefficient of Variation
• Standard error of the mean
“American Speakers”, Russian-US Workshop, Moscow-Tomsk, 10-20 Feb 2003
Crosstabs Program -- Select
Statistics
• Percents -- vertical (row), horizontal (column),
total
• Chi square (Pearson’s, Likelihood Ratio)
• Eta
• Gamma
• Tau-b and Tau-c
• Somer’s d
“American Speakers”, Russian-US Workshop, Moscow-Tomsk, 10-20 Feb 2003
Filter Variables
• Can also use filter variables to select
particular cases
• Variable name (____; ____; ___)
– Where _____ stands for a range of values or a
particular value
– E.g., sex (1)
– E.g., age (65-89)
• Using more than one filter variable
– E.g., sex (1), age (65-89) to select all those who
are 1 on sex and age 65 to 89
– Joins the two variables with an AND
“American Speakers”, Russian-US Workshop, Moscow-Tomsk, 10-20 Feb 2003
Subsetting Data Sets
• Select the files you want to construct
– Data file (ASCII)
– Codebook (ASCII)
– Data definitions for SPSS or STATA or SAS
• Select the cases to include (leave blank if you
want all the cases)
• Select the variables to include
“American Speakers”, Russian-US Workshop, Moscow-Tomsk, 10-20 Feb 2003
Choose “Groups of Variables”
Statistics on the Web
Chance News
• Newsletter on risk and uncertainty
http://www.darmouth.edu/~chance/chance_news/rec
ent_news/
“American Speakers”, Russian-US Workshop, Moscow-Tomsk, 10-20 Feb 2003
General
• Internat. Stat. Institute (ISI): http://www.cbs.nl/isi
• Statlib: http://lib.stat.cmu.edu/
• The Probability Web:
http://www.maths.uq.oz.au/~pkp/probweb/intro.html
– Links to WWW Virtual Library, etc.
•
•
Hyperstat online: http://www.ruf.rice.edu/~lane/hyperstat/contents.html
Info on statistical packages: MIDAS
http://midas.ac.uk/stats/
– Packages: SPSS, BMDP, SAS, STATA, S-PLUS, GAUSS, MLn,
NAG, TSP, LIMDEP, GLIM, etc.
– Includes data sets
“American Speakers”, Russian-US Workshop, Moscow-Tomsk, 10-20 Feb 2003
Probability distributions
• http://www.marketsys.com/pde.htm
– Introduction to distributions (def., etc.)
• http://www.astro.virginia.edu/~eww6n/math/
– graph, formula, and math. characteristics (e.g.
moments) for Bernoulli, Binomial, Geometric, Negative
Binomial, etc.
“American Speakers”, Russian-US Workshop, Moscow-Tomsk, 10-20 Feb 2003
Interactive statistical calculation:
http://members.aol.com/johnp71/javastat.html
Collection of programs:
http://www.stat.wisc.edu/statistics/software.html
“American Speakers”, Russian-US Workshop, Moscow-Tomsk, 10-20 Feb 2003
Textbooks
• Virtual Laboratories in Probability and
Statistics
http://www.math.uah.edu/~stat (univ. Alabama in
Huntsville)
– Basic probability and statistics (e.g. definitions,
problems, distributions)
– Relation exponential - Gamma - Poisson, and
Poisson - Binomial (cf. The Poisson Process)
– Simulation of experiments!
– Data sets
– Tables
– Resources on the Web
“American Speakers”, Russian-US Workshop, Moscow-Tomsk, 10-20 Feb 2003
Textbooks
• UCLA Statistics Textbook (Jan de Leeuw):
http://www.stat.ucla.edu/textbook/
– Includes Statistical Calculators for e.g. distributions (+
plots)
• Links to other electronic textbooks, electronic
journals, software, data and code archives:
http://www.stat.ruhr-uni-bochum.de/links.html (website maintained by
G. Rohwer and U. Poetter)
• List of short courses:
http://www.statistics.com/courses.html
“American Speakers”, Russian-US Workshop, Moscow-Tomsk, 10-20 Feb 2003