Introduction to R for Absolute Beginners: Part I Melinda Fricke Department of Linguistics University of California, Berkeley melindafricke@berkeley.edu D-Lab Workshop Series, Spring 2013 Why this workshop? "The questions that statistical analysis is designed to answer can often be stated simply. This may encourage the layperson to believe that the answers are similarly simple. Often, they are not… No-one should be embarrassed that they have difficulty with analyses that involve ideas that professional statisticians may take 7 or 8 years of professional training and experience to master.” (Maindonald and Braun, 2010. Data Analysis and Graphics Using R: An Example-Based Approach.) What we will cover today • Getting around in R – working directories, managing your workspace, creating and removing objects (types of variable assignment), inspecting objects, viewing functions, getting help • Types of data and basic manipulations – data types, object types, reading and writing data, modifying data and objects, basic functions (What we will cover next time) Downloading and installing external packages Common statistical tests correlation, simple linear regression, t-tests, ANOVA Graphing R makes really beautiful graphs, and is very flexible What we will not cover (ever) "The best any analysis can do is to highlight the information in the data. No amount of statistical or computing technology can be a substitute for good design of data collection, for understanding the context in which data are to be interpreted, or for skill in the use of statistical analysis methodology. Statistical software systems are one of several components of effective data analysis.” (Maindonald and Braun, 2010) Why use R? R is an incredibly flexible, high-level programming language that will allow you to conduct nearly any statistical analysis, and create any visualization you can think of. This video was created by Ben Schmidt using the ggplot2 package. http://sappingattention.blogspot.com/2012/10/data-narratives-and-structural.html A simpler example… Fricative and Vowel Spectra in Perseverative Context High Frequency Centroid fricative spectra at 20% fricative duration adults children 3750 adults, anticipatory ● ● children, anticipatory 5720 amplitude (dB) 10 3725 vowel context front round 0 5700 3700 ● ● 5680 −10 5660 3675 0 2000 4000 6000 8000 0 2000 4000 6000 5640 8000 Hz frequency (Hz) adults, perseverative vowel spectra at 20 ms before fricative onset adults children 5720 20 amplitude (dB) children, perseverative 3750 3725 10 vowel type front round 0 −10 −20 ● ● 3700 4000 6000 8000 0 frequency (Hz) 2000 4000 6000 ● 5660 −30 2000 ● 5680 3675 0 5700 5640 8000 beg end beg end (These are from my own work on the production of “s” sounds.) context ● front round But first, the basics… Creating objects Open R and type the following: x=1 y <- 2 3 -> z [enter] [enter] [enter] x y z [enter] [enter] [enter] There are 3 ways to assign variables in R. Creating objects Now try this: x+y+z x + y + z -> q q What’s the difference between the first line and the second? Creating objects These are vectors. A vector is just a bunch of values that have been concatenated together, in a sequence. When we type “q”, R tells us that the first element in the vector “q” is 6: [1] 6 (It’s also the only element, but that’s okay.) Creating objects We can create vectors that are longer than 1 element by concatenating multiple elements together: x = c(7, 8, 8, 7, 4, 1) x length(x) A little bit about “looping” R is a “high level” programming language. This means it takes care of a lot of things for us. x*2 x+y Most programming languages require you to write loops, but R takes care of a lot of “looping” on its own. Pop quiz! What will this code produce? length(x + y) A little bit about looping What will this code produce? length(x + y) [1] 6 A little bit about looping What will this code produce? length(x + y) [1] 6 The length of x is 6. (x + y) loops through the vector x, adding y to each number, yielding 6 new values (and therefore a vector of length 6). Remember: if you want to concatenate, use c(): length(c(x,y)) [1] 7 Pop quiz! What will this code produce? length(x + y) [1] 6 length(length(x + y)) Pop quiz! What will this code produce? length(x + y) [1] 6 length(length(x + y)) [1] 1 A little bit about looping What will this code produce? x = c(7, 8, 8, 7, 4, 1) y = c(1, 2) x+y A little bit about looping What will this code produce? x = c(7, 8, 8, 7, 4, 1) y = c(1, 2) x+y [1] 8 10 9 9 5 3 A little bit about looping What will this code produce? x = c(7, 8, 8, 7, 4, 1) y = c(1, 2) x+y [1] 8 10 9 9 5 3 “Loop through x and y simultaneously, adding 2 elements together.” For operations using 2 vectors of different lengths, the shorter one will be “recycled”. (Look at y + x.) Data types class(x) class(‘x’) Data types class(x) [1] “numeric” class(‘x’) [1] “character” Data types class(x) [1] “numeric” class(‘x’) [1] “character” There are different types of data in R. Putting quotes around something indicates you want R to treat it literally - as a character, not a variable. e.g. class(‘1’) Data types as.character(x) -> k k+1 Data types as.character(x) -> k k+1 Your first error message! as.numeric(k) -> k k+1 Data types as.factor(k) Data types as.factor(k) A factor is R-speak for a categorical variable: a type of data that can have one of several fixed levels. By default, factor levels are ordered alphabetically. levels(k) Data types as.factor(k) A factor is R-speak for a categorical variable: a type of data that can have one of several fixed levels. By default, factor levels are ordered alphabetically. levels(k) Oops! Why doesn’t this work? Data types as.factor(k) -> k A factor is R-speak for a categorical variable: a type of data that can have one of several fixed levels. By default, factor levels are ordered alphabetically. levels(k) [1] “1” “4” “7” “8” Data types c(10, 11, 10, 8, 6, 12, 11, 8, 10, 6, 10, 11) -> p Using what we’ve learned so far… c(10, 11, 10, 8, 6, 12, 11, 8, 10, 6, 10, 11) -> p How many items are in the vector ‘p’? How many unique values are in ‘p’? Using what we’ve learned so far… c(10, 11, 10, 8, 6, 12, 11, 8, 10, 6, 10, 11) -> p How many items are in the vector ‘p’? length(p) [1] 12 How many unique values are in ‘p’? Using what we’ve learned so far… c(10, 11, 10, 8, 6, 12, 11, 8, 10, 6, 10, 11) -> p How many items are in the vector ‘p’? length(p) [1] 12 How many unique values are in ‘p’? length(levels(as.factor(p))) Your first R weirdness as.factor(p) -> p as.numeric(p) What happened?? Your first R weirdness as.factor(p) -> p as.numeric(p) [1] 3 4 3 2 1 5 4 2 3 1 3 4 ( 10 11 10 8 6 12 11 8 10 6 10 11) When you try to change a factor directly into numeric mode, the factors are replaced by their “order”. How could we avoid this? Your first R weirdness as.numeric(as.character(p)) Your first R weirdness as.numeric(as.character(p)) What if we want to change the order of the levels? Your first R weirdness as.numeric(as.character(p)) What if we want to change the order of the levels? factor(p, levels=c(‘6’, ‘8’, ‘10’, ‘12’, ‘11’)) -> p levels(p) Ordered factors Factors may make more sense if we give our categories names other than numbers. Try this: mycolors = c(‘blue’, ‘yellow’, ‘green’, ‘purple’, ‘red’) class(mycolors) factor(mycolors, levels=c(‘red’, ‘yellow’, ‘green’, ‘blue’, ‘purple’)) -> mycolors class(mycolors) levels(mycolors) Taking stock Objects and operations values e.g. ‘1’ vectors c(‘1’, ‘4’, ‘a’, ‘word’) functions as.factor(x) variable assignment =, ->, <Data types (‘classes’) numeric 8 character ‘8’, ‘x’, ‘female’ factor ‘8’, ‘x’, ‘female’ the difference between strings of characters and factors is that factors have one of a set of fixed values e.g. ‘male’ vs. ‘female’ Some more useful functions Type these commands in to see what they do: ls() table(p) unique(p) sort(p) mean(p) median(p) sd(p) edit(p) ls Some more useful functions Type these commands in to see what they do: ls() table(p) unique(p) sort(p) mean(p) median(p) sd(p) edit(p) ls lists the objects currently in your workspace also useful: rm() (remove) Some more useful functions Type these commands in to see what they do: ls() table(p) unique(p) sort(p) mean(p) median(p) sd(p) edit(p) ls lists the objects currently in your workspace creates a table of counts Some more useful functions Type these commands in to see what they do: ls() table(p) unique(p) sort(p) mean(p) median(p) sd(p) edit(p) ls lists the objects currently in your workspace creates a table of counts lists all existing unique values Some more useful functions Type these commands in to see what they do: ls() table(p) unique(p) sort(p) mean(p) median(p) sd(p) edit(p) ls lists the objects currently in your workspace creates a table of counts lists all existing unique values sorts values from lowest to highest Some more useful functions Type these commands in to see what they do: ls() table(p) unique(p) sort(p) mean(p) median(p) sd(p) edit(p) ls lists the objects currently in your workspace creates a table of counts lists all existing unique values sorts values from lowest to highest mean of the values median (middle) of the values standard deviation Some more useful functions Type these commands in to see what they do: ls() table(p) unique(p) sort(p) mean(p) median(p) sd(p) lists the objects currently in your workspace creates a table of counts lists all existing unique values sorts values from lowest to highest mean of the values median (middle) of the values standard deviation edit(p) lets you interact directly with the data! “edit(p) -> p” to save your changes ls Some more useful functions Type these commands in to see what they do: ls() table(p) unique(p) sort(p) mean(p) median(p) sd(p) lists the objects currently in your workspace creates a table of counts lists all existing unique values sorts values from lowest to highest mean of the values median (middle) of the values standard deviation edit(p) lets you interact directly with the data! ls displays the internal workings of the function Getting help with functions ?sort help(sort) search current packages for a function (these two are equivalent) ??sort search all packages for a word Getting help with functions ?sort help(sort) search current packages for a function (these two are equivalent) ??sort search all packages for a word sort(p, decreasing = T) Data frames A really handy data structure! ind 1 2 3 4 5 dept ling ling anth hist econ year 1 4 2 5 2 prog R Excel Excel Stata SPSS Data frames are organized by rows and columns. Each column can contain a different type of data. Let’s try to create this data frame in R… Data frames ind 1 2 3 4 5 1) dept ling ling anth hist econ year 1 4 2 5 2 prog R Excel Excel Stata SPSS Create a vector for each column. Name the vector with the column header, e.g.: c(‘ling’, ‘ling’, ‘anth’, ‘hist’, ‘econ’) -> dept 2) Combine the vectors into a data frame: data.frame(ind, dept, year, prog) -> gradstats 3) Type ‘gradstats’ to display your whole data frame think about which ones should be factors! Data frames Did your data get entered properly? Check the data type for each column, and think about what it should be. e.g. class(gradstats$dept) Data frames Did your data get entered properly? Check the data type for each column, and think about what it should be. Factors: ind, dept, prog Numeric: year (probably…) Data frames Now try out these functions: head(gradstats, n = 3) tail(gradstats, n = 2) names(gradstats) summary(gradstats) dim(gradstats) length(gradstats) length(gradstats$ind) table(gradstats$dept) table(gradstats) Data frames Now try out these functions: head(gradstats, n = 3) tail(gradstats, n = 2) names(gradstats) summary(gradstats) dim(gradstats) length(gradstats) length(gradstats$ind) table(gradstats$dept) table(gradstats) displays the first n rows of the data frame last n rows gives the name of each column summarizes the whole data frame gives the dimensions, n rows x n columns Data frames Now try out these functions: head(gradstats, n = 3) tail(gradstats, n = 2) names(gradstats) summary(gradstats) dim(gradstats) displays the first n rows of the data frame last n rows gives the name of each column summarizes the whole data frame gives the dimensions, n rows x n columns length(gradstats) length(gradstats$ind) length of a data frame = # of columns length of a vector = # of values table(gradstats$dept) table(gradstats) Data frames Now try out these functions: head(gradstats, n = 3) tail(gradstats, n = 2) names(gradstats) summary(gradstats) dim(gradstats) displays the first n rows of the data frame last n rows gives the name of each column summarizes the whole data frame gives the dimensions, n rows x n columns length(gradstats) length(gradstats$ind) length of a data frame = # of columns length of a vector = # of values table(gradstats$dept) table(gradstats) table of counts for a single vector (column) table of counts for all vectors (crossed) Data frames Look at the help file for table(). Try to figure out how to make a contingency table for departments x stat programs. Data frames Look at the help file for table(). Try to figure out how to make a contingency table for departments x stat programs. table(gradstats$dept, gradstats$prog) Reading in data Download the data file located at http://linguistics.berkeley.edu/~mfricke/R_Work shop_files/salary.txt. This file contains data on professors’ salaries. (S. Weisberg (1985). Applied Linear Regression, Second Edition. New York: John Wiley and Sons. Page 194. Downloaded from http://data.princeton.edu/wws509/datasets/#salary on January 31st, 2013.) Reading in data: working directory Your working directory is where R looks for (and saves) files. Check to see what it is by typing: getwd() You can change it to the directory where you saved the data file with: setwd() setwd(“/Users/melindafricke/Desktop”) But there’s an easier way: On a Mac: command + d, then select your directory. In Windows: go to “File”, then “Change dir…”, and select your directory. Reading in data Open the help file for read.table() See if you can read in the data file we just downloaded and start inspecting it… Reading in data read.table(“salary.txt”, header=T) -> salary read.table() has several options, to deal with differently formatted files. file header sep quote dec row.names nrows skip the filename, in quotes (must be in working dir) does the first row contain column names? how are the fields separated? (e.g. ‘\t’, ‘,’) what character was used for quoting? (‘ ‘ ‘) what character is used as a decimal point? does one column contain row names? (if not, R will number the rows) how many rows to read in (default is all of them) how many rows to skip before reading data Using what we know already… How many rows does this data set contain? columns? What is the average salary (sl) for these professors? How many professors are male vs. female (sx)? For each rank (rk), how many professors have a doctorate vs. masters (dg)? Using what we know already… How many rows does this data set contain? columns? dim(salary) [1] 52 6 What is the average salary (sl) for these professors? How many professors are male vs. female (sx)? For each rank (rk), how many professors have a doctorate vs. masters (dg)? Using what we know already… How many rows does this data set contain? columns? dim(salary) [1] 52 6 What is the average salary (sl) for these professors? mean(salary$sl) [1] 23797.65 () How many professors are male vs. female (sx)? For each rank (rk), how many professors have a doctorate vs. masters (dg)? Using what we know already… How many rows does this data set contain? columns? dim(salary) [1] 52 6 What is the average salary (sl) for these professors? mean(salary$sl) [1] 23797.65 How many professors are male vs. female (sx)? table(salary$sx) female male 14 38 For each rank (rk), how many professors have a doctorate vs. masters (dg)? Using what we know already… How many rows does this data set contain? columns? dim(salary) [1] 52 6 What is the average salary (sl) for these professors? mean(salary$sl) [1] 23797.65 How many professors are male vs. female (sx)? table(salary$sx) female male 14 38 For each rank (rk), how many professors have a doctorate vs. masters (dg)? table(salary$rk, salary$dg) doctorate masters assistant 14 4 associate 5 9 full 15 5 Manipulating data frames Subscripting is a way to reference columns and rows in a data frame. The basic syntax is: salary[1,1] name of dataframe row #(s) column #(s) comma! N.B. You always need to include the comma when you use subscripting on a dataframe. Manipulating data frames You can combine this syntax with other conventions we’ve learned (and a few we haven’t!). Try these: salary[c(1,4), 1] salary[c(1,4), c(1,6)] salary[c(1:4), c(1,6)] salary[c(10:15), ] what does the colon do? what if you leave the column # out? salary[salary$sx==“female”,] salary[salary$sl>30000,] “display all the rows where sx is female” “display all the rows where sl is > 30,000” Using what you know now… What is the mean salary for a female professor? a male? What will this syntax tell us? length(salary[salary$yd>20, ]$sl) Using what you know now… What is the mean salary for a female professor? a male? mean(salary[salary$sx==“female”,]$sl) [1] 21357.14 mean(salary[salary$sx==“male”,]$sl) [1] 24696.79 What will this syntax tell us? length(salary[salary$yd>20, ]$sl) Using what you know now… What is the mean salary for a female professor? a male? mean(salary[salary$sx==“female”,]$sl) [1] 21357.14 mean(salary[salary$sx==“male”,]$sl) [1] 24696.79 What will this syntax tell us? length(salary[salary$yd>20, ]$sl) [1] 19 Using what you know now… What is the mean salary for a female professor? a male? mean(salary[salary$sx==“female”,]$sl) [1] 21357.14 mean(salary[salary$sx==“male”,]$sl) [1] 24696.79 What will this syntax tell us? length(salary[salary$yd>20, ]$sl) [1] 19 The number of professors that got their degree over 20 years ago. One more cool function aggregate() is really nice for creating data summaries. Try this: aggregate(salary$sl, list(salary$sx, salary$rk), mean) One more cool function aggregate() is really nice for creating data summaries. Try this: aggregate(salary$sl, list(salary$sx, salary$rk), mean) “aggregate salaries, contingent on both sex and rank, and take the mean” Group.1 female male female male female male Group.2 assistant assistant associate associate full full x 17580.00 17919.60 21570.00 23443.58 28805.00 29872.44 But… Aggregating What’s the average salary for people with doctorates vs. masters? On average, how many years ago did assistant vs. associate vs. full professors get their degrees? What’s the standard deviation for male vs. female salaries? Aggregating What’s the average salary for people with doctorates vs. masters? aggregate(salary$sl, list(salary$dg), mean) doctorate 23500.35 masters 24359.22 On average, how many years ago did assistant vs. associate vs. full professors get their degrees? What’s the standard deviation for male vs. female salaries? Aggregating What’s the average salary for people with doctorates vs. masters? aggregate(salary$sl, list(salary$dg), mean) doctorate 23500.35 masters 24359.22 On average, how many years ago did assistant vs. associate vs. full professors get their degrees? aggregate(salary$yd, list(salary$rk), mean) assistant 6.33 associate 18.93 full 22.95 What’s the standard deviation for male vs. female salaries? Aggregating What’s the average salary for people with doctorates vs. masters? aggregate(salary$sl, list(salary$dg), mean) doctorate 23500.35 masters 24359.22 On average, how many years ago did assistant vs. associate vs. full professors get their degrees? aggregate(salary$yd, list(salary$rk), mean) assistant 6.33 associate 18.93 full 22.95 What’s the standard deviation for male vs. female salaries? aggregate(salary$sl, list(salary$sx), sd) female 6151.873 male 5646.409 Writing data Let’s say you’ve produced some data you want to share with someone else, or have easy access to later. aggregate(salary$sl, list(salary$sx, salary$rk), mean) -> mfsalaries names(mfsalaries) names(mfsalaries) = c(“sex”, “rank”, “salary”) write.table(mfsalaries, “MFSalaries.txt”, sep=“\t”, row.names=F) object filename separator? include row names? Saving your workspace Look at all the objects we’ve created today! ls() If you want to save these, make sure you save your workspace before you exit. (“Workspace…”, “Save workspace file…”) R will create a file (in your working directory) that you can load for use later, which includes all of these objects. Saving your workspace Look at all the objects we’ve created today! ls() If you want to save these, make sure you save your workspace before you exit. (“Workspace…”, “Save workspace file…”) R will create a file (in your working directory) that you can load for use later, which includes all of these objects. If you want to save the text of your session, go to “File”, “Save”. Next up Downloading and installing external packages More sophisticated analyses correlation, simple linear regression, t-tests, ANOVA Graphing R makes really beautiful graphs, and is very flexible Which of these topics are highest priority to you? Thank you! melindafricke@berkeley.edu