# 90-776 Manipulation of Large Data Sets Homework 4 Solutions 1) Program

90-776 Manipulation of Large Data Sets
Homework 4 Solutions
1) Program
/* u:\class\907776\class\HW4P1.SAS does the tasks in Homework 4, problem 1*/
/* In lab 4, I took the data set l:\academic\90776\data\text\country.txt, created an improved comma
delimited ASCII file, then brought it into SAS
and saved the file as u:\class\90776\data\lab4.sd2. This program cleans up
that data set */
/* Rob Greenbaum*/
/* 4/6/1999*/
options pageno=1 ps=150 ls=150;
/* create library reference for my SAS data sets */
libname mydat 'u:\class\90776\data\';
/* 1a. First, find the means (with proc means and univariat) of the numeric variables */
PROC MEANS DATA = MYDAT.LAB4;
/* I can see which observations are the outliers by including an ID subcommand
in my univariate procedure*/
PROC UNIVARIATE DATA=MYDAT.LAB4;
VAR YR_IND;
ID COUNTRY;
/*1b. Let’s set negative yr_ind observations to missing */
DATA FIXYR;
SET MYDAT.LAB4;
IF YR_IND &lt;0 THEN YR_IND=.;
/* Let’s see how the mean has changed */
PROC MEANS DATA=C4;
RUN;
/*1c. Change the missing values for the character variables to blanks */
DATA FIXCHAR;
SET FIXYR;
ARRAY ROB[*] _CHARACTER_;
DO I=1 TO DIM(ROB);
IF ROB[I]=&quot;?&quot; OR ROB[I]=&quot;-&quot; THEN ROB[I] = &quot; &quot;;
END;
DROP I;
RUN;
/* 1d. Print out the data set */
PROC PRINT DATA=FIXCHAR;
RUN;
/* 1e. Check religion */
PROC FREQ DATA=FIXCHAR;
TABLES RELIGION;
RUN;
/* For one of the observations, Lutheran was misspelled as Lutherian. That is easy to fix */
DATA FIXSPELL;
SET FIXCHAR;
IF RELIGION = “Lutherian” THEN RELIGION = “Lutheran”;
/* MAKE SURE WE FIXED THE PROBLEM */
PROC FREQ DATA=FIXSPELL;
TABLES RELIGION;
RUN;
/* 1f. let’s create an ID variable */
DATA NEWVAR;
SET FIXSPELL;
LENGTH CSMALL \$2; /* need to reset the length of the new variable*/
CSMALL = SUBSTR(COUNTRY,1,2); /* CSMALL is the first 2 chars of country name*/
ENDYR = MOD(YR_UN,100); *Get the last 2 digits of the year they entered the UN;
YR = PUT(ENDYR, 2.); /* convert ENDYR to CHARACTER*/
IF YR = “ .” THEN YR=”00”; /*convert missings to 00*/
ID = CSMALL||YR; /* put csmall and yr together*/
RUN;
/*1g. print out the data */
PROC PRINT DATA=NEWVAR ;
VAR COUNTRY ID;
RUN;
1) Output
The SAS System
16:05 Tuesday, April 6, 1999
1
16:05 Tuesday, April 6, 1999
2
Variable
N
Mean
Std Dev
Minimum
Maximum
-------------------------------------------------------------------TOTALPOP 93
55462086.65
151888397
27816.00
1149667000
CAPPOP
91
2196361.60
2379700.09
134393.00
10726000.00
YR_IND
92
1796.02
395.9593435
-1000.00
1980.00
YR_UN
89
1954.08
10.6602754
1945.00
1991.00
--------------------------------------------------------------------
1)a. There is at least one observation with a negative value for YR_IND.
The SAS System
Univariate Procedure
Variable=YR_IND
Moments
N
Mean
Std Dev
Skewness
USS
CV
T:Mean=0
Num ^= 0
M(Sign)
Sgn Rank
92
1796.022
395.9593
-4.69388
3.1103E8
22.04647
43.50658
92
45
2134
Quantiles(Def=5)
Sum Wgts
Sum
Variance
Kurtosis
CSS
Std Mean
Pr&gt;|T|
Num &gt; 0
Pr&gt;=|M|
Pr&gt;=|S|
92
165234
156783.8
27.93015
14267326
41.28162
0.0001
91
0.0001
0.0001
100%
75%
50%
25%
0%
Max
Q3
Med
Q1
Min
Range
Q3-Q1
Mode
1980
1960
1922.5
1821.5
-1000
99%
95%
90%
10%
5%
1%
Extremes
1980
1964
1962
1499
900
-1000
Lowest
ID
-1000(Ethiopia)
800(Denmark )
836(Sweden )
843(France )
900(USSR
)
Highest
ID
1964(Zambia )
1975(Angola )
1975(Mozambiq)
1980(Zimbabwe)
2980
138.5
1960
Missing Value
Count
% Count/Nobs
.
1
1.08
The SAS System
16:05 Tuesday, April 6, 1999
3
Variable
N
Mean
Std Dev
Minimum
Maximum
-------------------------------------------------------------------TOTALPOP 93
55462086.65
151888397
27816.00
1149667000
CAPPOP
91
2196361.60
2379700.09
134393.00
10726000.00
YR_IND
91
1826.75
265.9089064
800.0000000
1980.00
YR_UN
89
1954.08
10.6602754
1945.00
1991.00
--------------------------------------------------------------------
1)b. The mean YR_IND has now increased to 1827 – before it was 1796. Removing the one outlier made a large difference.
The SAS System
OBS
1
COUNTRY
TOTALPOP
CAPITAL
CAPPOP
REGION
Afghanistan
16922000
Kabul
1424400
Asia
YR_IND
YR_UN
1919
1946
16:05 Tuesday, April 6, 1999
RELIGION
LANG1
LANG2
Islam
Pashto
Persian
LANG3
4
2 Algeria
3 Angola
4 Argentina
5 Australia
6 Austria
8 Belgium
9 Bolivia
10 Brazil
11 Bulgaria
12 Burkina Faso
13 Burundi
14 Cambodia
15 Cameroon
18 Chile
19 China
20 Colombia
21 Cote d'Ivorie
22 Cuba
23 Czechoslovakia
24 Denmark
25 Dominican Repu
27 Egypt
29 Ethiopia
30 Finland
31 France
32 Germany
33 Ghana
34 Greece
35 Guatemala
36 Guinea
37 Haiti
38 Hong Kong
39 Hungary
40 India
41 Indonesia
42 Iran
43 Iraq
44 Italy
45 Japan
46 Kenya
The SAS System
25888000
10284000
32470000
17337000
7815000
108760000
9978000
7528000
153322000
9005000
9261000
27816
8781000
12239000
26941000
5823000
13385000
1149667000
33613000
12464000
10700000
15577000
5146000
48443
11079000
54609000
5392000
51617000
500100
56942000
79096000
15509000
10272000
9177000
7052000
6617000
5862000
10326000
871158000
181451000
57050000
18317000
57590000
123920000
25905000
Algiers
Luanda
Buenos A
Canberra
Vienna
Dhaka
Brussels
La Paz
Brasilia
Sofia
Bujumbar
Phnom Pe
Yaounde
Ottawa
H'Djamen
Santiago
Beijing
Santafe
Abidjan
Havana
Prague
Copenhag
Santo Do
Quito
Cairo
Helsinki
Paris
Berlin
Accra
Athens
Guatemal
Conakry
Port-auBudapest
New Delh
Jakarta
Tehran
Rome
Tokyo
Nairobi
1507241 Africa
1962
1134000 Africa
1975
2922829 Latin America
1816
310000 Oceania
1901
1487577 Europe
1918
5731000 Asia
1971
137966 Europe
1830
669400 Latin America
1825
1567709 Latin America
1822
1217024 Europe
1908
441514 Africa
1960
226628 Africa
1962
564000 Orient
1953
649000 Africa
1960
863900 America
1867
500000 Africa
1960
5236300 Latin America
1810
6800000 Orient
1523
4819696 Latin America
1810
1850000 Africa
1960
2077938 Latin America
1902
1212010 Europe
1960
1343916 Europe
800
1600000 Latin America
1844
1094318 Latin America
1822
6452000 Africa
1922
. Latin America
1841
1495266 Africa
.
492240 Europe
1917
2152423 Europe
843
3376800 Europe
1955
949100 Africa
1957
885737 Europe
1830
1095677 Latin America
1821
705280 Africa
1958
514438 Latin America
1804
. Orient
.
2016132 Europe
1918
7174755 Asia
1947
7829000 Oceania
1945
6042584 Asia
1906
5348117 Asia
1932
2803931 Europe
1861
8163127 Orient
1660
1504900 Africa
1963
16:05 Tuesday, April 6, 1999
1962
1976
1945
1945
1955
1974
1945
1945
1945
1955
1960
1962
1955
1960
1945
1960
1945
1945
1945
1960
1945
1960
1945
1945
1945
1945
.
1945
1955
1945
1973
1957
1945
1945
1958
1945
.
1955
1945
1950
1945
1945
1955
1956
1963
5
Islam
Catholicism
Islam
Catholicism
Buddhism
Lutherian
Islam
Greek Orthodox
Monotheism
Islam
Islam
Arabic
Portuguese
Spanish
English
German
Bengali
Dutch
Spanish
Portuguese
Bulgarian
French
Rundi
Khmer
French
English
Arabic
Spanish
Mandarin
Spanish
French
Spanish
Czech
Danish
Spanish
Spanish
Arabic
Spanish
Amharic
Finnish
French
German
English
Greek
Spanish
French
Haitian Creole
Chinese
Hungarian
Hindu
Bahasa Indonesian
Persian
Arabic
Italian
Japanese
Swahili
Cumulative Cumulative
RELIGION
Frequency
Percent
Frequency
Percent
ƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒ
Buddhism
2
8.7
2
8.7
Catholicism
3
13.0
5
21.7
Greek Orthodox
1
4.3
6
26.1
Hinduism
1
4.3
7
30.4
Islam
13
56.5
20
87.0
Lutheran
1
4.3
21
91.3
Lutherian
1
4.3
22
95.7
Monotheism
1
4.3
23
100.0
French
Aymara
French
English
French
French
Slovak
Swedish
French
English
English
English
German
Quechua
Frequency Missing = 70
The SAS System
16:05 Tuesday, April 6, 1999
6
16:05 Tuesday, April 6, 1999
7
Cumulative Cumulative
RELIGION
Frequency
Percent
Frequency
Percent
ƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒ
Buddhism
2
8.7
2
8.7
Catholicism
3
13.0
5
21.7
Greek Orthodox
1
4.3
6
26.1
Hinduism
1
4.3
7
30.4
Islam
13
56.5
20
87.0
Lutheran
2
8.7
22
95.7
Monotheism
1
4.3
23
100.0
Frequency Missing = 70
The SAS System
OBS
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
COUNTRY
Afghanistan
Algeria
Angola
Argentina
Australia
Austria
Belgium
Bolivia
Brazil
Bulgaria
Burkina Faso
Burundi
Cambodia
Cameroon
Chile
China
Colombia
Cote d'Ivorie
Cuba
Czechoslovakia
Denmark
Dominican Repu
Egypt
Ethiopia
Finland
France
Germany
Ghana
Greece
ID
Af46
Al62
An76
Ar45
Au45
Au55
Ba74
Be45
Bo45
Br45
Bu55
Bu60
Bu62
Ca55
Ca60
Ca45
Ch60
Ch45
Ch45
Co45
Co60
Cu45
Cz60
De45
Do45
Ec45
Eg45
El00
Et45
Fi55
Fr45
Ge73
Gh57
Gr45
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
Guatemala
Guinea
Haiti
Hong Kong
Hungary
India
Indonesia
Iran
Iraq
Italy
Japan
Kenya
North Korea
South Korea
Gu45
Gu58
Ha45
Ho00
Hu55
In45
In50
Ir45
Ir45
It55
Ja56
Ke63
No91
So91
Ma60