Spring 2013 ArcGIS for Geoscientists GEOL 596 Lab 3: Importing XY data Making a point layer from Excel One of the most common things you will want to do in ArcGIS is bring in point data that you’ve collected, like sample locations, or downloaded, like earthquake locations. The steps that you will walk through today are useful for bringing any point data for which you have latitude and longitude coordinates into ArcGIS as a point layer. This will require some basic spreadsheet work, which we’ll do in Excel. As you’ve already seen, little mistakes can throw big wrenches into this process. Sometimes totally invisible mistakes throw big wrenches into the process. Try not to get frustrated – just be very systematic in your work. This lab has several parts: Formatting data you already have in Excel (p. 2) Downloading data and formatting it in Excel (p. 4) Adding XY data to ArcGIS (p. 8) Adding symbology and display options (p. 11) Properly displaying strike and dip data (p. 14) Adding lineations to strike and dip symbols (p. 18) This lab should give you the skills you need to get XY data into ArcMap from anywhere – really useful for a variety of purposes. 1 Spring 2013 ArcGIS for Geoscientists GEOL 596 Formatting data you already have in Excel Most research in geology involves some sort of spatially distributed point data that you’ve collected. Most geologic maps have some sort of structural information (strikes and dips on bedding, etc.), and most geologists also have sample locations. I’m going to use these two types of data as examples here – while the importing process is exactly the same, the techniques for setting up the data in Excel and applying symbology are somewhat different for each one. In order to import properly into ArcGIS, your spreadsheet must have: A single column for latitude in decimal degrees (xx.xxxxx), where N latitudes are positive and S latitudes are negative, and A single column for longitude in decimal degrees (xxx.xxxxx), where E longitudes are positive and W longitudes are negative. You must also know which datum the latitude and longitude data are referenced to. If you used a handheld GPS to collect the data yourself, you should be able to find that information on the GPS. If you estimated the lat-long by hand from a map, the paper map should have the datum printed on it. If you got the data from someone else, it’s possible they were responsible enough to annotate the spreadsheet with the datum. If not, you need to find out. Most commonly, the default datum for a handheld GPS unit is WGS 1984. (Remember, this is NOT a projection. It is just the reference for the geographic coordinates.) It is also possible to import point data that has projected coordinates, like meters from a UTM projection. The main idea is that you need to know about your own data – are they projected or not, what is the datum if not, what is the projection if so. You can then have any number of columns in your spreadsheet for whatever you want. You can also add columns once you bring it in to ArcGIS, but it is definitely easier to do so in Excel. Think about adding columns for information that you might use to display or sort the data. Depending on the state of your data, you may or may not need to go through all of these steps. 1. Open your spreadsheet in Excel. 2. Save a new copy by selecting “Save as…” and make a new folder within the folder that contains your geodatabase and the DEMs folder. Call the new folder XY_data, and save your spreadsheet as fieldarea_datatype.xlsx (where fieldarea and datatype are replaced by the appropriate words). 3. Look at your latitude and longitude columns. Are they in decimal degrees? There should be no symbols (like °) or letters (like N or S) in the column with the number. Do your data look different? Do not despair! There are lots of things you can do in Excel with formulas and text-to-columns to make this work. a. Is there a letter in the column with the number, separated by a space? Insert a column to the right. Select the column with the data in it. Go to Data Text to Columns… Select Delimited and use the space to delimit the column; the letter will now be separated into the new column and you can delete it. b. Is there a symbol in the column with the number without a space? Insert a column to the right. Select the column with the data in it. Go to Data Text to Columns… Select Delimited and use the symbol(s) to delimit the columns; the number might now be distributed among many columns (see below). 2 Spring 2013 ArcGIS for Geoscientists GEOL 596 c. Is the number in multiple columns? (e.g. degrees, minutes, seconds) Insert a column to the right of all three. Use a formula to calculate the decimal degrees from the degrees, minutes, seconds format. The formula is ((S/60) + M)/60) + D 4. For your spreadsheet to import correctly, the first row must be your column headers, with no other information above it, and no other information in rows below all of the data. Make sure that this is the case with your spreadsheet. 5. In addition, all of the column headers must be formatted according to strict rules: a. No spaces (these are often hiding at the end of words in Excel, and you should just check each header to make sure there are no hidden spaces). Underscores are allowed. b. No symbols, including ) ( < > , . : ; & % $ # * ^ etc. This rule means that you cannot have a header that is something like Elevation (m), which has both a space and two forbidden symbols. Elevation_m is OK. c. Column headers must start with a letter. d. Column headers must contain only letters, numbers, and underscores. e. Column headers must not exceed 64 characters. Check all of your headers to see that they follow these rules. In particular, make sure that the columns that have your latitude and longitude data are clearly labeled as such. 6. You also want to make sure that the data itself is formatted correctly, beyond the lat-long coordinates. In general, you want to keep a single value per column. For example, if you have a spreadsheet with geochronology data, you might have age and error data. Rather than having this in a single column, you want to put it into two columns. Bad! Age_error 12.31 +/- 1.21 Age_Ma Good! Error_Ma 12.31 1.21 Spend some time looking through all of your data and making sure you have it how you want it. You don’t have to have all numeric columns. 7. Your Excel spreadsheet can have multiple worksheets. Even if it only has a single worksheet, go ahead and give it a name – you will see why when we add the file to ArcGIS. Do this by double-clicking on the tab at the bottom of the spreadsheet, which has a default of Sheet1. The name will become highlighted, and you can type what you want. If you have multiple sheets, name them all. 3 Spring 2013 ArcGIS for Geoscientists GEOL 596 Downloading XY data and formatting it in Excel Here are some places where you might find point data that you want to download: The Global Earthquake Search (http://earthquake.usgs.gov/earthquakes/eqarchives/epic/) at the USGS is a great place to download earthquake locations, depths, and magnitudes. Note that this is in beta form right now. NAVDAT (http://www.navdat.org/), the Western North American Volcanic and Intrusive Rock Database, where you can download geochemical and geochronological data that has been submitted by others. Something paleo Something geothermal Paleo mag More examples Each of these has slightly different processes for downloading the data, but the process once you have the data is exactly the same. 1. Launch a browser and go to http://earthquake.usgs.gov/. 2. Click on Earthquakes in the upper left navigation. 3. Click on in the right-hand navigation. 4. Read the top paragraph on this page carefully. Imagine that you want to download all of the earthquakes in your field area, as far back in time as you can go. Fill in the search parameters accordingly. 4 Spring 2013 ArcGIS for Geoscientists GEOL 596 5. Once you are successful, you will get a return on the page that looks something like this: 5 Spring 2013 ArcGIS for Geoscientists GEOL 596 Click on CSV. This does not actually prompt a file download, but gives you something like this instead: Looks like a complete mess, right? Don’t worry. 6. Select everything on the page, and copy it (right-click and select Copy, or use the menu or keyboard shortcut). 7. Go to Excel and open a new workbook. 8. Right-click in the upper-left-most cell (A1) and select Paste. This pastes all of the data into different rows, but all in the same column. 9. With all of the data still selected (just the first column appears selected), go to Data Text to Columns… and you should see this window: If you are not already familiar with this process, you will be. It will become your friend. 10. The data you have copied and pasted is in CSV, or comma-separated values, format. That means it is “delimited” (data in different columns is separated by commas) rather than “fixed width” (which has an equal number of spaces per column). So keep the default selection of delimited and click Next. 11. Select the appropriate delimiter in the next screen (comma) and deselect the others. You should see the data separate into columns. Scroll through to see if it all looks OK. 6 Spring 2013 ArcGIS for Geoscientists GEOL 596 12. The final screen in the wizard offers you the opportunity to determine the format of each column. I tend to leave these all as “General” and just click right to Finish, here, because otherwise it can sometimes introduce errors into perfectly good data. So you can just click Finish. 13. Hooray! This is starting to look better, but there is still a lot of clean-up to do in order to bring it in to ArcGIS. Save your work in your XY_data folder, with a name something like fieldarea_EQs.xlsx 14. Name the worksheet Fieldarea_EQs_all_data, and add a second worksheet (you can either click on the plus sign in the tab next to the first worksheet or select Insert Sheet Blank Worksheet). Name the second worksheet Fieldarea_EQs_edited. 15. Take a look at all of the data. What all do you really want to have at your fingertips in ArcGIS? You know you want Latitude, Longitude, Depth, Magnitude. Do you care about anything else? Probably not. Copy and paste just the columns you want from EQs_all_data to EQs_edited. (Note: if you want to use the date and time information, you’ll need to clean it up to make it useful. You can do this by being clever with the textto-columns process described above, but I won’t go through it now.) 16. Save your work. 17. Now go through the process described above for formatting your own data correctly. Check the headers and scroll through to make sure things look OK. Now we can bring in both your data and the data you downloaded into ArcGIS. 7 Spring 2013 ArcGIS for Geoscientists GEOL 596 Adding XY data to ArcGIS There are a few different ways to bring your spreadsheet into ArcGIS. We’ll go through one of them here, and you can then find the other ways in the Help document if you need them. The principles are the same in all cases, so you should have no trouble. 1. Launch ArcMap. If you saved the map you made with the DEM and hillshade, open that. If not, add them and make the colors and shading look the way you want them. Go ahead and save this if you haven’t already. 2. The process for bringing in new data from an Excel file is a little different from adding a raster image or a feature class. Go to File Add Data Add XY data… 3. This will open a new dialogue box for you to fill in. Click on the file folder and navigate to your Excel spreadsheet. You will see that the spreadsheet has a symbol like this: 8 . Spring 2013 ArcGIS for Geoscientists GEOL 596 This is NOT what you want to add. You want to double-click on this, and you will be able to see the individual worksheets within the workbook, which will have symbols like this: . This is why you want to rename your worksheets with meaningful names. Select the sheet you want to add, and click Add. 4. Most likely, some of the drop-down boxes will auto-populate, perhaps even with the right information! But you might need to modify them. The fields correspond with the column headers you cleaned up. When you click on the drop-down boxes, you should see all of your column headers. NOTE: If you don’t see all of your headers, there is probably something wrong with your formatting, and you should go back and edit the Excel file, checking for extra spaces and random symbols. In the X field, you should select your Longitude column; in the Y field, select Latitude. If you have elevation data in a column, you can select that in the Z field, but it’s not necessary. 5. You also need to check the coordinate system. The program is probably just guessing, if there is anything at all showing up in the description box. Read what is in the box, and then click the Edit… button to find the right one. 6. This brings you to the familiar spatial reference window, where you select the correct coordinate system. IMPORTANT: Here you are telling ArcGIS the coordinate system that the data are currently in, NOT the one you want them to be in. So if your GPS data are geographic, lat-long coordinates referenced to the WGS 84 datum, that is what you would select here. 7. Once you’ve selected the coordinate system, it will show up in the description window, and you can click OK. 9 Spring 2013 ArcGIS for Geoscientists GEOL 596 8. Most likely, the following warning will appear: Read it. It’s important. Because we are going to export to a feature class so that you can do the things you want to do, and because it makes your data more transferable. But for now, you can note that you have been warned, and click OK. 9. Once you click OK, your points should show up on your map. If they are not where you expected them to be, you probably chose the wrong coordinate system. If you can’t even find them, you can right click on the layer and select Zoom to layer to see them – this is probably another indication that your coordinate system is off. 10. In the Table of Contents, you will notice that your spreadsheet has the word “Events” attached to it, so it looks like this: This is related to that warning you got. Before you do any messing around with the symbology and displays, you want to export this to a feature class so that the data live within your geodatabase and can be manipulated and edited the same as all of the other feature classes. So right-click on the layer and select Data Export Data… 10 Spring 2013 ArcGIS for Geoscientists GEOL 596 11. You want to export your data to the feature dataset where all of the rest of your data lives: The radio button may already be selected as the default – that’s fine. Then click on the file folder to navigate to the feature dataset. You should get to a window where you can see the continents file that we imported, along with the feature class you created. Change the default name (Export_Output) to something more meaningful (e.g., fieldarea_samples). Click Save. 12. Click OK. You will be asked: The answer is yes! You do. 13. The new layer will show up in the Table of contents. You can now delete the Events layer, and work entirely with the new feature class you’ve created. 14. Save your work. Congratulations! You’ve now successfully brought point data in to both your feature dataset and your map. 11 Spring 2013 ArcGIS for Geoscientists GEOL 596 Adding symbology and display options There are infinite ways to display and label point data using symbology options in ArcMap. This will go through a few of the most common things you would do in a geologic map or figure. 1. To get started, add your earthquake XY data to the map you’ve created, following the same steps as above. 2. Right-click on the layer and select Properties. 3. Click on the Symbology tab. The default symbology when you add a layer is typically a simple symbol for every feature, so you are likely to see something like this for your symbology window: You can, of course, edit that symbol however you’d like by clicking on it, but the result is that every feature in that feature class will be displayed using the same symbol. That’s not terribly informative in the case of earthquake data, where you have not only location, but depth and magnitude, which you can display. For example, let’s say you want to color the symbols by depth and scale them by magnitude. You can do that! Let’s start with coloring the earthquakes by depth. 4. Select Quantities in the left menu of the symbology tab. 12 Spring 2013 ArcGIS for Geoscientists GEOL 596 5. Select Graduated colors underneath Quantities. In other words, you want to base the color of the symbol on a quantity in one of the fields, in this case, depth. 6. In the Value field, select Depth. This will automatically create some number of categories, often five, and color them according to whatever color bar you have selected. But you don’t need to go with the default – and you rarely want to. 7. Click on Classify, so that you can determine exactly how the breaks occur. The default is to classify by natural breaks in the data, but you probably want to be a bit more specific. You probably want to choose breaks manually according to more meaningful depths in the earth. What you choose is going to depend on your data, so the best way to figure this out is just to play around with it. You can change the number of them and where they occur. When you are happy, click OK. 8. Back in the Symbology tab, select the color ramp you want to use to display the quantities. You will want to choose something that is a continuous gradient and creates different enough colors to be able to see them. 9. Now we want to make the symbols different sizes based on the magnitude of the earthquake. Click on Advanced and select Size… 10. Here you should be able to select Magnitude as the factor that determines the size of the symbol. Click OK, and OK again in the main window. NOTE: I often find this result somewhat unsatisfying, and I do other things to make this work better. One is to separate the earthquake data into two separate feature classes, one that is earthquakes with a magnitude greater than 5.5, which is the lowest end considered “significant”, and one for magnitudes less than 5.5. Then you can color by depth and make the symbol size distinction more prominent. 13 Spring 2013 ArcGIS for Geoscientists GEOL 596 Properly displaying strike and dip data If you are doing any geologic mapping, it is likely that you will have points that have associated structural data, including strikes and dips. If you already have that data, here’s how to display the symbols appropriately in ArcGIS. The final section of this lab will deal with best practices in the field for collecting strike and dip data so that it is easily imported into ArcGIS. 1. If you already have a spreadsheet with strike and dip data, you want to follow procedures as outlined above to make sure that you have latitude and longitude coordinate data that is properly formatted to import this data into ArcGIS. You need to have a column for strike that is in azimuth degrees, and a column for dip that uses the right-hand rule. (Azimuth degrees means that N = 0, E = 90, S = 180, W = 270. It is possible to convert, but best practice is to collect in azimuth to begin with.) 2. You also want to add a column, which I typically name Feature, in which you describe in a word or two what you are measuring. The purpose of this column is to take advantage of the symbology features in ArcGIS. You want to limit the number of options that you fill in here to the number of different symbols you would use. Typically, here are the options I would use: Bedding Horizontal bedding Vertical bedding Foliation Fault Dike Slicks Lineation You might have some other feature, so add anything you want to use a different symbol for. 3. Import the XY data, and export the data to make a new feature class. 4. Open the Properties dialogue box and navigate to the Symbology tab. 5. Select Categories Unique Values in the left menu. 14 Spring 2013 ArcGIS for Geoscientists GEOL 596 6. In the Value Field, select the field you created called Feature. 7. Click Add All Values. This should populate the window with all of the possibilities you typed into the spreadsheet, and a count of how many of those kinds of symbols there are. Hopefully, you didn’t make too many typos, but it’s not a big deal if you did – you can either correct it in the attribute table or just give the typos the same symbols. 8. Right-click on bedding and select Properties for selected symbols… to get the symbol selector window. All of the appropriate geological symbols are already in ArcGIS, but you might have to add them to what you are viewing. Scroll down initially to see what is there – if you don’t see geology symbols, then click on Style References… 9. Scroll down and check the box next to Geology 24K (and anything else you want to be able to use symbols for, but this is what you need right now). Click OK. 10. Back in the Symbol Selector window, scroll down until you see the geology symbols. 15 Spring 2013 ArcGIS for Geoscientists GEOL 596 Select the strike and dip symbol, which is called Inclined bedding – showing strike and direction of dip. Change the size of the symbol to 30, though you might need to mess around a bit to get them to show at the right size. 11. Now click on Edit Symbol… Unfortunately, Arc has this very annoying habit of not plotting the struke and dip symbols directly on the points, so you need to fix this. Beneath the Preview window, change the view to 400% so you can actually see this problem. Now use the offset X and Y boxes to move the symbol so that the place where the dip tick intersects the strike line is exactly on the center of the crosshairs. Click OK when you are done. 12. Click OK in the Symbol Selector dialogue. When you get back to the Symbology tab, click Apply to apply your changes without closing the window. 16 Spring 2013 ArcGIS for Geoscientists GEOL 596 13. Now you want to rotate the symbols according to their strike. So with Bedding still selected in the symbol list window, click on Advanced and select Rotation… 14. You want to rotate your points by the value in the Strike field, and use Geographic coordinates (where N = 0). Click OK, and Apply in the Symbology tab. 15. Now you want to label the symbols with the dip. Click on the Labels tab in the Layer Properties window. 16. Select the check box to Label features in this layer. Click on the drop-down menu in order to Define classes of features and label each class differently. 17. Click on Get symbol classes. This should populate the drop-down list with the same list that you are selecting symbols for. Now you can select each item in the dropdown list and check/uncheck the checkbox indicating whether or not they will be labeled. You will not, for example, label horizontal or vertical bedding with the dip, since that is implied in the symbol you will use. 17 Spring 2013 ArcGIS for Geoscientists GEOL 596 18. In the Label Field, select Dip. 19. You will need to mess around with the size of the text and the placement properties in order to get the labels to look the way that you want. This is most effectively done through trial and error, to be honest. You can keep the window open and apply the changes you make as many times as you want, however. 20. When you are happy with the labels so far, go back to the Symbology tab, right-click on the other features, and select the appropriate symbol for horizontal bedding, vertical bedding, fault planes, etc. You will go through the same process for all of these symbols. 21. Be sure to save your work. 18 Spring 2013 ArcGIS for Geoscientists GEOL 596 Adding lineations to strike and dip symbols If you have locations where you collected structural data that includes both planar and linear features, there are some additional steps to getting those to display correctly. Once again, you want to make sure your spreadsheet is formatted correctly to begin with. If you collected a strike and dip of a metamorphic foliation and lineation at the same location, those two things need two separate lines in your spreadsheet, even though the coordinates will be exactly the same (in practice, this means I actually take two waypoints in the field for the two different measurements). Assuming your spreadsheet is formatted correctly and you already have gone through the section about properly displaying strike and dip symbols, here are the next steps. 1. In the Symbology tab, right-click on your lineation feature and select Properties for selected symbols… 2. Choose the appropriate symbol for your lineation (probably the one called Lineation – showing bearing and plunge). 3. Click on Edit symbol. In the ultra tiny preview window, notice the location and orientation of the arrow. Where the cross-hairs meet, that’s the location of your point. That preview is showing you how the symbol will appear relative to the location of your point. Because this is a lineation, you want the butt end of the arrow to be at your point, not the middle of the arrow. You also want the default rotation of the arrow to be pointed straight up, or north, so that you can rotate it by the trend. 4. Change the Angle so that the arrow points up. 5. Change the X and Y offset so that the butt end of the arrow is directly on the crosshairs. Click OK. 6. Then you want to use the same process to rotate the lineation symbol according to the trend and label with the plunge. 19