Geoprocessing Using ModelBuilder Transcript Copyright © 2005 ESRI All rights reserved. The information contained in this document is the exclusive property of ESRI. This work is protected under United States copyright law and other international copyright treaties and conventions. No part of this work may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying and recording, or by any information storage or retrieval system, except as expressly permitted in writing by ESRI. All requests should be sent to Attention: Contracts and Legal Services Manager, ESRI, 380 New York Street, Redlands, CA 92373-8100, USA. The information contained in this document is subject to change without notice. @esri.com, 3D Analyst, ADF, AML, ARC/INFO, ArcAtlas, ArcCAD, ArcCatalog, ArcCOGO, ArcData, ArcDoc, ArcEdit, ArcEditor, ArcEurope, ArcExplorer, ArcExpress, ArcFM, ArcGIS, ArcGlobe, ArcGrid, ArcIMS, ArcInfo Librarian, ArcInfo, ArcInfoProfessional GIS, ArcInfo-The World's GIS, ArcLocation, ArcLogistics, ArcMap, ArcNetwork, ArcNews, ArcObjects, ArcOpen, ArcPad, ArcPlot, ArcPress, ArcQuest, ArcReader, ArcScan, ArcScene, ArcSchool, ArcSDE, ArcSdl, ArcStorm, ArcSurvey, ArcTIN, ArcToolbox, ArcTools, ArcUSA, ArcUser, ArcView, ArcVoyager, ArcWatch, ArcWeb, ArcWorld, Atlas GIS, AtlasWare, Avenue, BusinessMAP, Database Integrator, DBI Kit, ESRI, ESRI-Team GIS, ESRI-The GIS Company, ESRI-The GIS People, FormEdit, Geographic Design System, Geography Matters, Geography Network, GIS by ESRI, GIS Day, GIS for Everyone, GISData Server, InsiteMAP, JTX, MapBeans, MapCafé, MapObjects, ModelBuilder, MOLE, NetEngine, PC ARC/INFO, PC ARCPLOT, PC ARCSHELL, PC DATA CONVERSION, PC STARTER KIT, PC TABLES, PC ARCEDIT, PC NETWORK, PC OVERLAY, PLTS, Rent-a-Tech, RouteMAP, SDE, SML, Spatial Database Engine, StreetEditor, StreetMap, TABLES, the ARC/INFO logo, the ArcCAD logo, the ArcCAD WorkBench logo, the ArcCOGO logo, the ArcData logo, the ArcData Online logo, the ArcEdit logo, the ArcExplorer logo, the ArcExpress logo, the ArcFM logo, the ArcFM Viewer logo, the ArcGIS logo, the ArcGrid logo, the ArcIMS logo, the ArcInfo logo, the ArcLogistics Route logo, the ArcNetwork logo, the ArcPad logo, the ArcPlot logo, the ArcPress for ArcView logo, the ArcPress logo, the ArcScan logo, the ArcScene logo, the ArcSDE CAD Client logo, the ArcSDE logo, the ArcStorm logo, the ArcTIN logo, the ArcTools logo, the ArcView 3D Analyst logo, the ArcView Business Analyst logo, the ArcView Data Publisher logo, the ArcView GIS logo, the ArcView Image Analysis logo, the ArcView Internet Map Server logo, the ArcView logo, the ArcView Network Analyst logo, the ArcView Spatial Analyst logo, the ArcView StreetMap 2000 logo, the ArcView StreetMap logo, the ArcView Tracking Analyst logo, the Atlas GIS logo, the Avenue logo, the BusinessMAP logo, the Data Automation Kit logo, the ESRI ArcAtlas Data logo, the ESRI ArcEurope Data logo, the ESRI ArcScene Data logo, the ESRI ArcUSA Data logo, the ESRI ArcWorld Data logo, the ESRI Digital Chart of the World Data logo, the ESRI globe logo, the ESRI Press logo, the Geography Network logo, the MapCafé logo, the MapObjects Internet Map Server logo, the MapObjects logo, the MOLE logo, the NetEngine logo, the PC ARC/INFO logo, the Production Line Tool Set logo, the RouteMAP IMS logo, the RouteMAP logo, the SDE logo, The World's Leading Desktop GIS, Water Writes, www.esri.com, www.geographynetwork.com, www.gisday.com, and Your Personal Geographic Information System are trademarks, registered trademarks, or service marks of ESRI in the United States, the European Community, or certain other jurisdictions. Other companies and products mentioned herein are trademarks or registered trademarks of their respective trademark owners. Copyright © 2005 ESRI. All rights reserved. 1 Geoprocessing Using ModelBuilder Presenter: Jeff Bigos ESRI Redlands, California Copyright © 2004 ESRI. All rights reserved. Good afternoon. My name is Jeff Bigos and I’m an instructor for the Educational Services team here in Redlands. Joining me today is Catherine Jones, who is a product specialist for ModelBuilder. I want to welcome you all to today’s live training seminar. Copyright © 2005 ESRI. All rights reserved. 2 Seminar overview ¾ Topics Creating models Editing models Documenting and distributing models ¾ Format Each topic followed by a software demonstration, review, and Q & A session. Copyright © 2004 ESRI. All rights reserved. In today’s seminar, we are going to cover some of the basics of geoprocessing with ModelBuilder. In this seminar we will cover creating models of our GIS workflow, editing model elements and repairing broken models, and documenting and distributing models. After each topic we will have a software demo, followed by a review and question and answer session. Copyright © 2005 ESRI. All rights reserved. 3 Creating models Copyright © 2004 ESRI. All rights reserved. First off, we are going to touch upon the basics of geoprocessing and creating models. Copyright © 2005 ESRI. All rights reserved. 4 Creating a model Geoprocessing basics What is a model? Why build models? Model elements Creating a model Model states Running a model Copyright © 2004 ESRI. All rights reserved. In topic one, we are going to cover the following issues: • Geoprocessing basics • What is a model? • Why build models? • Model elements • Creating a model • Model states, and • Running a model Copyright © 2005 ESRI. All rights reserved. 5 What is geoprocessing? Tools for management, conversion, and analysis Accessible through ArcToolbox Can access models, tools, and scripts Tools available with ArcGIS extensions and all ArcGIS Desktop applications Copyright © 2004 ESRI. All rights reserved. To get started, we have to look at what geoprocessing is and how it applies to model building. Defined, geoprocessing is a GIS operation used to manipulate data stored in a GIS workspace. Geoprocessing is one of the most important aspects of a GIS because it provides us with the tools necessary to perform data analysis, conversion, and management. At its core, geoprocessing is as simple as converting data from shapefiles to geodatabase feature classes, or performing a buffer analysis on your input data. Many ArcGIS Desktop extensions have a toolbox associated with them, such as Spatial Analyst, Network Analyst, Military Analyst, and Data Interoperability. Geoprocessing tools work with all ArcGIS-supported data formats. They are also available with ArcMap, ArcCatalog, ArcScene, and ArcGlobe. Copyright © 2005 ESRI. All rights reserved. 6 Using geoprocessing tools Copyright © 2004 ESRI. All rights reserved. There are four different ways to interact with the geoprocessing framework, all of which are initially taken from ArcToolbox. They can be accessed and run through their tool dialogs from ArcToolbox, interactively typed in the command line window, used in a geoprocessing script environment like Python, and finally, we can work with the tools in ModelBuilder to build GIS task-based workflows. Let’s investigate using models with geoprocessing. Copyright © 2005 ESRI. All rights reserved. 7 What is a model? String of processes composed of data and tools Copyright © 2004 ESRI. All rights reserved. In ArcGIS, a model is displayed as a model diagram like what you see here. You can automate your workflow by stringing data and tools together in the model diagram that will run in sequence. Copyright © 2005 ESRI. All rights reserved. 8 Why build models? Automate a geoprocessing workflow Ability to share geoprocessing work Create custom tools Easy to run and rerun Visual representation of geoprocessing work Copyright © 2004 ESRI. All rights reserved. So why would you want to build a model? Building a model helps you to manage and automate your geoprocessing workflow. It can help with simple multistep tasks, such as adding new fields to a feature class and then calculating values for those fields. Models are custom tools that you create. They can be run multiple times, and you can apply different datasets each time you run them. Also, they give users of your models a clear and concise representation of the geoprocessing work that will take place. Copyright © 2005 ESRI. All rights reserved. 9 What is ModelBuilder? Interactive window used to create models inside of ArcGIS Copyright © 2004 ESRI. All rights reserved. Here is the ModelBuilder interactive window. This is the window where your model building is going to occur. When open, you will drag and drop in geoprocessing tools and project data and connect them into processes which are then connected to create a workflow. Let’s take a look at the different kinds of elements you will use when building your model. Copyright © 2005 ESRI. All rights reserved. 10 Model elements Connector Process Copyright © 2004 ESRI. All rights reserved. First off, you will be working with project data. Project data is your input data that exists before the model runs. Most of you work with this data on a day-to-day basis. This is the input data to your model, the data that will go through the model and generate your result. Tools represent the geoprocessing operation that will be performed on the project or input data. This might be a buffer, clip, or union. Tools fall into two categories: system and custom. System tools are provided to you by ESRI through ArcToolbox. Custom tools are models created by you in ModelBuilder, script tools built with Python (or any scripting language), or tools you’ve written using ArcObjects code. Derived data is the result of tools run in the model. It is the data that is generated as the outcome of a tool in your model process. This generated result can also be used as input into another tool in the model chain. It will also be the end result of your model. ModelBuilder provides that kind of functionality to chain tools and results together. At its simplest, the line connector is a line that shows the sequence of processing. However, it is much more than that. The line connector will allow you to connect your project or input data into Copyright © 2005 ESRI. All rights reserved. 11 a geoprocessing tool. Using this element will populate some of the necessary input parameters to the tool. Together, these elements create a process within our model, and a model is composed of one or more processes. Copyright © 2005 ESRI. All rights reserved. 12 Creating the model Create a custom toolbox or have an existing custom toolbox Opens the ModelBuilder interactive window Copyright © 2004 ESRI. All rights reserved. To begin, you have to create a model. To do that, you need only to right-click on a custom toolbox and then select New > Model. This will bring up the ModelBuilder interactive window that we talked about earlier. Before we apply our new knowledge and create and run a model, we need to look at model states and running the model. Copyright © 2005 ESRI. All rights reserved. 13 Model states Not ready to run Ready to run Has been run Copyright © 2004 ESRI. All rights reserved. As you create and edit models, you will begin to notice a change in the color of model elements. These colors give us an indication of the state of a model. When you open a model you will see one of three different states: • Not ready to run: In this case there may be parameters not set, referenced data no longer exists or has been moved, or the parameter value is invalid. • Ready to run: All the parameters are valid and executing the process should be successful. • Has been run: A shadowbox around elements tells us that they have already been run successfully. Copyright © 2005 ESRI. All rights reserved. 14 Running a model There are three ways to run model processes: Run a single process Run ready-to-run processes only Run all processes Status messages are returned with each run Copyright © 2004 ESRI. All rights reserved. Now that we have looked at the basic building blocks and how to create the model, let’s run a model. Once you are ready to execute your model, you can run a single process of the model, like the select or clip tool. We can run all processes that have not yet been run. And finally, we can run the entire model, including rerunning any previously run processes. Keep in mind that the more processes being run, the longer the processing time, hence the reason you are able to run just ready-to-run processes. When each tool is run, there are status messages generated. Status messages will include parameter values, the start time and end time of execution, status of execution, and any error messages, to name a few. We will look at these again in topic three when we look at model reports. Copyright © 2005 ESRI. All rights reserved. 15 Software demonstration Copyright © 2004 ESRI. All rights reserved. For now, let’s take a look at a demonstration on how we are going to go about creating a model. To start, we are here in ArcMap and I’ve got a custom toolbox loaded in ArcToolbox called Model_LTS. Inside of Model_LTS I’ve got some system tools, a Python script, and some models. Now to start, I have to right-click on my custom toolbox and go to New > Model. Now that brings up the ModelBuilder interactive window. Now, for our analysis we want to create a model that will help us determine the total surface area of water for a selected country. I’ve got two layers loaded in my ArcMap table of contents. I’ve got one that represents countries of the world, and I’ve also got another one, called World_lakes, which represents some of the major and minor lakes around the world. So first off, what I want to do is drag and drop the country layer into my model. What this is going to do is the countries will serve as the input data or the start of my model. Copyright © 2005 ESRI. All rights reserved. 16 Okay, now the first tool I want to use in my analysis is the Select tool. The Select tool is in the Analysis Tools toolbox, in the Extract toolset. So here is the Select tool. To use it, I’m just going to drag and drop it into my model, and there it is. Now what I need to do is notice that the tool is in a not-ready-to-run state. I need to supply some parameters to it. To do that, I’m going to use the line connector tool, click on the countries layer, and then click on the Select tool. Now my countries layer is an input parameter to my Select tool. Now I need to supply some additional parameters to the Select tool to get it to run. For me to generate surface area waters for a country like Iceland, I have to supply an expression. So I’m going to build one here in my Query Builder. I’m going to set out my country name equal to Iceland. Click OK to each of these dialogs. Now the Select tool is ready for me to run. The next tool I want to use in my analysis is the Clip tool. I’m going to go ahead and drag and drop that into the ModelBuilder interactive window. Now instead of using the line connector, I’m going to supply the parameters a different way. I’m going to go ahead and double-click on the Clip tool and supply the parameters through this dialog here. So for my input features, I’m going to go ahead and click on the World_lakes. Now, this World_lakes is coming from my ArcMap table of contents. Go ahead and select the lakes. Now, for the Clip Features, I want to use the output from the Select tool, and that is noted here: World_CNTRY92_Select. That is the output to the Select tool. Now I’ve got the parameters supplied to the Clip tool. One thing I want you to take note of is the Clip tool—it is white, so it is not ready to run, there are no line connectors, and we haven’t added the World_lakes feature class yet. So now when I click OK, all of that has been added. The appropriate parameters have been added and World_lakes have been added to the model. Let me resize that so we can start to see our model more. The last tool I want to use in my analysis is the Summary Statistics tool. It is located here in the Statistics toolset. Let’s go ahead and drag and drop that in and resize our model. I’m just going to use the line connector to provide that World_lakes_Clip as the input parameter to the Summary Statistics tool. Now I also need to apply some additional parameters to this tool. Copyright © 2005 ESRI. All rights reserved. 17 I’m going to go ahead and double-click to open that up. In my World_lakes feature class, I’ve got a SQKM (square kilometers) attribute. Now I’ve clipped my lakes to Iceland, and now I want to sum up the SQKM field to tell me how many square kilometers of surface area water is available in Iceland. So for Statistic Type, select SUM. Now my model is ready to run. My input data is here and my tools are in a ready-to-run state. Now one more thing I want to point out is that, as each tool is running, it will turn red, indicating that they are currently processing the data. To run this model, I’m going to click the Run button. There goes the Clip tool, the Summary Statistics, and any messages that the tools generate are being written to this Model progress dialog. The clip took about a second and the statistics ran through quickly as well. So, three of three processes were executed and our model is complete. Let’s click Close. Now, the neat thing is that we have generated a result—Summary Statistics generates a table. Inside of the ModelBuilder interactive window, if I want to see that table, I just right-click on the result element and click on Add To Display. Now to see the table in ArcMap, I have to set the table of contents to the Source tab; that is how you see tables in ArcMap. Here is the result of the Summary Statistics tool. When I open it, it tells me that there are over 1600 km² of surface area waters available in Iceland. So let me save and close this model, as we will need it for the second demonstration. So in this demonstration, we looked at how to create and run a model, and we looked at the different model states and elements. Copyright © 2005 ESRI. All rights reserved. 18 Review and Q & A ¾ Geoprocessing basics ¾ What is a model? ¾ Why build models? ¾ Model elements ¾ Creating a model ¾ Model states ¾ Running a model Copyright © 2004 ESRI. All rights reserved. To review, in the first topic, we covered: Geoprocessing basics; What is a model?; Why build models?; Model elements; Creating a model; Model states; and finally, Running a model. Now, let's turn it over to Catherine for some of your questions. Thanks, Jeff. The first question that we'll look at here is from Neal in Albuquerque, and he asks, "Can AMLs be imported into ModelBuilder?" Well, Neal, you can actually create a custom script tool that references your AML and then, because a script tool is just like any other tool, you can drag and drop that into ModelBuilder and chain it together to the other processes that you have there as well. So, yes. Our second question comes from Emma in Wellington, and she asks, "Can you use ModelBuilder to iterate through a set of individual features, performing processing on each one?" So, she's asking if we can do batch processing in ModelBuilder. And the answer, again, is yes. What you will need to do is to create a script tool again, a custom script tool, and then you can include that in your model, and then that will do the batch processing for you in that way. In fact, there is a technical article about that on the Knowledge Base that will tell you exactly how to go about doing that. Copyright © 2005 ESRI. All rights reserved. 19 There is also a question here from Brandon in Chicago, and he says, "How can we schedule to run a model at midnight, that kind of thing?” Well, you can do that; it's no problem. You can schedule your task to run at a different time. What you need to do is tap into the Windows Scheduled Tasks, and there is actually again a Knowledge Base doc about how to do that for you, and the actual Knowledge Base doc number is 27553. We have another question here from Charlie in Dallas that says, "Can models be shared with other GIS users?" Again, the answer is yes. Your custom models or script tools are saved inside of your custom toolboxes, and those toolboxes are saved on your system as .tbx files. Those are quite small binary files that you can very easily attach to an e-mail. I think we'll pass it back to Jeff now. Thanks. Copyright © 2005 ESRI. All rights reserved. 20 Editing models Copyright © 2004 ESRI. All rights reserved. In topic two, we are going to cover editing our models. Copyright © 2005 ESRI. All rights reserved. 21 Editing models Editing a model Exposing model parameters Creating model variables Working with intermediate data Validating your model Repairing a model Copyright © 2004 ESRI. All rights reserved. Your organization is dynamic: data comes in, data goes out, databases and their paths change daily or weekly, but your geoprocessing tasks may not. Perhaps we want to apply one model to multiple datasets or databases. Our data might have moved from the C to the D drive. We want our models to accommodate a dynamic work environment In topic two, we are going to cover some of the basics of model editing, ways to validate and repair models, and ways to make our models more dynamic with parameters and variables. Copyright © 2005 ESRI. All rights reserved. 22 Editing a model Choose Edit on the model’s context menu to open the ModelBuilder interactive window Copyright © 2004 ESRI. All rights reserved. A model is not static, so often times we have to take a preexisting model and add or remove data or additional tools to it, or perhaps change parameter values, or we may need to generate a different analysis or to gather additional information about our data. To do any of these things will have to edit the model. To do so, right-click the model tool in the ArcToolbox window and choose Edit. This will bring up the ModelBuilder interactive window. Copyright © 2005 ESRI. All rights reserved. 23 Editing a model Ability to add and remove data and tools Adding functionality to an existing model Apply environment settings Alter parameter values Copyright © 2004 ESRI. All rights reserved. With the ModelBuilder interactive window open, we can begin to add additional tools and data to our model. Perhaps in our analysis of determining how much surface area of water there is in Iceland, we want to determine the number of water features in Iceland. With our model already created, we can drag in the Get Count tool from the Data Management toolbox. This tool will helps us determine the answer to that question. In addition, we might have written and imported a Python script that helps determine additional characteristics of the water bodies that we need in our analysis. You can incorporate system tools, script tools, or other model tools into your model. Perhaps we want to apply the same surface area analysis to another country, like Poland. How can we make our model accommodate this kind of change? Copyright © 2005 ESRI. All rights reserved. 24 Model parameters Allow for user input Used when running a model as a dialog Create a data-independent tool Environment settings Copyright © 2004 ESRI. All rights reserved. To accommodate using another country for the same surface area analysis, we are going to expose some model parameters. In geoprocessing, a parameter is a required or optional value supplied to a tool in order for the tool to run. For instance, the buffer tool requires a distance value to generate buffers. For our analysis, we need to be able to select any country to generate information about the surface waters. Exposing a model parameter allows us to accomplish this task because it allows for user input. All exposed model parameters will appear on the model tool dialog. Copyright © 2005 ESRI. All rights reserved. 25 Creating parameter variables Share values between processes Expose as variables Expose for user input Copyright © 2004 ESRI. All rights reserved. Creating a parameter as a variable creates a list of all the tool’s parameters. This is a great way to create an empty parameter that you could expose on a dialog, which would allow you to enter the value at a later date or allow for user input. Creating a parameter variable also allows you to use the same value for different tools. For example, if your model included more than one buffer tool, you could create a variable out of the buffer distance and connect it to both buffer processes. It’s as easy as right-clicking a tool inside our model and making a variable from a parameter. We will look at this in more depth in the upcoming demo. Copyright © 2005 ESRI. All rights reserved. 26 Intermediate data Derived data is flagged as intermediate Uncheck Intermediate from your final result Copyright © 2004 ESRI. All rights reserved. At this point, we have added additional tools and set up parameters for our model. Since we are still editing our model, we need to take into consideration intermediate data. Intermediate data is temporary data that is created by tools within your model. In our analysis of surface waters, the first tool to run is the Select tool. The output of that tool is used to clip our water features. That is the extent to which we are going to use the result from the Select tool. It is not the final result of our model, but it just data needed to continue the analysis. There are a few points to think about with intermediate data: models run from a dialog box or the command line delete intermediate data automatically; and, models run within the ModelBuilder window maintain intermediate data on disk. One thing to remember is that your end result will be flagged as intermediate data. Also, if you run the model through its dialog or at the command line, your final result will be deleted. Copyright © 2005 ESRI. All rights reserved. 27 Model validation Ensures that parameters are valid Model state may be affected Copyright © 2004 ESRI. All rights reserved. In topic one, we mentioned the three different states of a model: not ready to run, ready to run, and has been run. The state of a model when you first open it will reflect its status when you last ran it. Any time you open a model, it is good practice to validate the entire model. This validation will verify all parameter values to make sure they are correct, for example, whether input data is in the same location or has been moved, or whether custom tools are still available or valid. To validate your model, in the ModelBuilder interactive window, select Model > Validate Entire Model. We will look at this further in the next demo. Copyright © 2005 ESRI. All rights reserved. 28 Repairing a model Associated data or tools have been moved or renamed Parameter values are no longer valid Model elements change back to not-ready-to-run state Copyright © 2004 ESRI. All rights reserved. Like most environments, the work environment is very fluid. At times, we might move our data to different network drives, or tools might get moved from project folder to project folder. Actions like this might change elements within our models. When this happens, your model might need to be repaired. If a change that affects the model occurs, then ArcToolbox will change the icon of your model to reflect this. When you see a red X through the model icon, you know you need to repair something in your model. Also, when you validate your model and elements return to the notready-to-run state, you will need to repair your model. When we edit the model it might look like the graphic here. Notice the change in color to the Countries oval. It is no longer blue, and in addition, there is a red X through the Summary Statistics tool. There are properties about those two elements that need to be fixed before we can execute the model, and we would do this by finding lost data, checking parameter values, and fixing any broken custom tools. Copyright © 2005 ESRI. All rights reserved. 29 So let’s take a look at how we will go about editing, repairing, and making our model more dynamic Copyright © 2005 ESRI. All rights reserved. 30 Software demonstration Copyright © 2004 ESRI. All rights reserved. Okay, first off I want to look at the model we created in the first demonstration. Now what I want to show is the Model dialog. If I double-click on the model from the first demo, I see this text here which says, “This tool has no parameters.” If I were to click OK, the model would run and perform the same analysis for the country of Iceland. What I want to do is expose a model parameter to allow people to specify their own country for this analysis. So let’s look at how to do that. To do that, first we have to edit our model. Right-click on the model and select Edit. Now what I need to do is expose a model parameter. I need to let people select a country that they want to perform an analysis on, perhaps Poland. To do that, I need to expose a parameter of the Select tool. To do that, right-click on the Select tool, select Make Variable > From Parameter, and select Expression. Notice that now I get a new element; it’s this one here named Expression. If I double-click on it, notice it is just the same tool that allowed me to build a SQL expression. I’m going to close those dialogs. Now, to make this a model parameter, I have to right-click on this element and select Model Parameter. Now if I take a closer look at this after selecting this element as a model parameter, you’ll notice this P. This P Copyright © 2005 ESRI. All rights reserved. 31 indicates that this is a parameter of the model. So let’s look and see what this has done to our Model dialog. Save the model and close it. Now, double-click on the model in the custom toolbox in ArcToolbox. Notice now that the same expression that we just looked at is here on the Model dialog. If I wanted to reset the analysis for Poland, I could just type in Poland, click OK, and my analysis for the surface area would run for Poland. I need to add some additional tools to the model, so I’m not going to run it from there. We are actually going to run it from the ModelBuilder interactive window. So let’s go back in and edit our model. Before we add in some additional tools, let’s go ahead and validate our model to make sure our data and tool references have been maintained. So go to Model > Validate Entire Model. The states haven’t changed, so if I wanted to, I could rerun the analysis. Before that, I want to generate my surface analysis, but I also want to know how many water features I had in this analysis. So I think that a while back somebody told me about a Get Count tool. I only know about it, but I don’t know what toolbox it is in. So what I can do is click on the Search tab here, type in count, and what we have here is some results. So what the Search tab did is look through ArcToolbox searching for every instance of “count” and it found it in these three tools here. The first result just happens to be that Get Count tool, which is in the Data Management toolbox. The neat thing about the Search tab is that, not only does it tell me what toolbox it is in, but if I click on the Get Count tool, I can go ahead and drag and drop it into my model. Okay, now I need to supply the parameters to it, so I’m going to use the line connector tool to go from my clipped lakes into the Get Count tool, so now that is ready to run. So the last thing I need to do to reset the analysis for Poland is to reopen this Expression element, and let’s go ahead and type in Poland. All my parameters are set and my model is ready to run. Let’s go ahead and regenerate the analysis. Okay, so our model ran through successfully; four of four processes were executed. Right here on the Model progress dialog, notice it says, “Row Count = 17.” This is the output from the Get Count tool. It tells me that there are 17 features that were clipped for the country of Poland. Copyright © 2005 ESRI. All rights reserved. 32 I’m going to go ahead and save the model, as we will need it for the third demonstration, and I’ll close it down. In our ArcMap table of contents, I have the Summary Statistics result here, so if I right-click and open it up, we can see that there is over 3000 km² of surface area water available in Poland. So that’s editing a model. Now let’s look at how we would go about repairing a model. So, in my custom toolbox, Model_LTS, I have this model here that has a red X icon through it. That indicates to me that there is something broken inside the model. Let’s take a look and see what it is. Right-click on the model and select Edit. I just need to resize the model so that we can all see it. Okay, now we have it in full view. Okay, the input data for Country and Lakes has lost its reference, and also the Summary Statistics tool. So let’s go ahead and repair these. To repair the Country layer, I’ve already got my countries loaded in ArcMap, along with my lakes. So if I wanted to repair Country, I just go ahead and double-click on Country, use the pulldown, set it to the World_CNTRY92 layer that I have loaded in ArcMap, and click OK. Let’s go ahead and do the same thing for Lakes. We’ll set this to World_lakes, and now our Select and Clip tools are in a ready-to-run state. Now let’s look at the Summary Statistics tool. Now, I might have had this tool in a custom toolbox that got deleted, or the Summary Statistics tool got deleted from my toolbox, so let’s go ahead and repair that. To do that, double-click on the Summary Statistics tool. Now, in my custom toolbox, Model_LTS, I already put in my Summary Statistics tool, so that is where I’m going to point the reference to. Click Add, and now I have successfully repaired my model. So now I can regenerate the analysis of surface waters for a given country. In this demonstration we looked at how to edit and repair an existing model. Copyright © 2005 ESRI. All rights reserved. 33 Review question Locate the Review question link in the upper-left corner of your browser window. Click this link now. Copyright © 2004 ESRI. All rights reserved. So in this demonstration, we looked at how to edit and repair an existing model. Now we have a question for you. Locate the Review question link in the upper-left corner of your browser window. Click this link now. Copyright © 2005 ESRI. All rights reserved. 34 Review question Which type of data element in a model is considered virtual? A) Project data B) Tool C) Derived data D) Environment To submit your answer, click A, B, C, or D then click the Send button. Copyright © 2004 ESRI. All rights reserved. Okay, the question is: Which type of data element in a model is considered virtual: project data, tool, derived data, or environment? To submit your answer, click A, B, C, or D, and then click the Send button. I'll leave this in view for a few seconds so you can determine the answer. Copyright © 2005 ESRI. All rights reserved. 35 Review and Q & A ¾ Editing a model ¾ Exposing model parameters ¾ Creating model variables ¾ Working with intermediate data ¾ Validating your model ¾ Repairing a model Copyright © 2004 ESRI. All rights reserved. To review, in this last topic, we looked at editing a model, exposing model parameters, creating model variables, working with intermediate data, validating your model, and finally, repairing a model. Now, let's turn it over to Catherine for some of your questions. Thanks, Jeff. Let's take a look at the first question that we have here. This is from Dorothy, and she asks, "Can I use all data types in ModelBuilder (coverages, grids, shapefiles, and feature classes), and can the data type be mixed or do I need to convert to a common data type?” Well, Dorothy, you can use any one of the valid ArcGIS data types as input and output for your tools, and then those tools are in ModelBuilder, so the answer to your first question is yes. The second part of that I think I also answered—yes, you can mix them around. For one of your tools, the input could be a shapefile, then for the next process, one of the inputs could be a geodatabase feature class, and so on. Copyright © 2005 ESRI. All rights reserved. 36 So let's see. The next question we have then is from Ricardo, and he's saying, “Is there a limit on the number of processes and elements that you can have in a model?” The only limitation, Ricardo, is your imagination. So whatever you need to put into your model diagram, you can have in there. Keep in mind though, and I'll make a suggestion to you that I've made to other people, that, just like you would do if you were creating a file structure in Windows Explorer on your computer, you would have a sort of a plan in mind where you would have things broken down into folders and then subfolders and so on. I would recommend that you do a similar sort of thing with your models as well. Break them down into workable units. It will make your life a lot easier when it comes to trying to make changes or if you need to do any kind of editing, or if you need to fix things in that, and then you've got each one of those smaller models being called in another model. So, sort of break it down into workable units. Our next question is from Terry, and he says, “Can you demonstrate how to create a custom script tool that runs in AML?” Well due to time constraints, Terry, we can't demonstrate that for you, but I won't leave you empty-handed. I'm going to tell you this: Go to the Desktop Help System, open that up, and there is a book under there called Writing geoprocessing scripts. There is another book underneath that called Messaging and script tools, which contains a page called Using AMLs with script tools. This will explain everything to you and actually give you a step-by-step process on how to get it done. So, there’s plenty of information there for you for that. Then we have a user in West Lafayette who says, “Can I use ModelBuilder and a script together in tandem?” I want to make it very clear to everybody—because this is something that a lot of newer people have trouble with—that there is no difference between a system tool that we provide you or a custom tool, such as a model and a script that you use. So yes, they can be used together. You can drag and drop script tools into your ModelBuilder interactive window and use those to build your workflow—no problem. And then inside of your script, you can also call a model tool, just like you would call one of our system tools, so there’s no difference there. So there's the answer to that question as well. I think we'll pass it back to Jeff now. Copyright © 2005 ESRI. All rights reserved. 37 Review question results To view the results, click the Review question link in the upper-left corner of your browser window. Which type of data element in a model is considered virtual? A) Project data B) Tool C) Derived data D) Environment Copyright © 2004 ESRI. All rights reserved. Okay, thanks, Catherine. Let's take a look at the answer to the review question. So the question was: Which type of data element in the model is considered virtual? The answer is C, derived data. It looks like about 60 percent of you got that right. Now the reason that it is derived data is that derived data has not been created until the model is run. Copyright © 2005 ESRI. All rights reserved. 38 Documenting and distributing models Copyright © 2004 ESRI. All rights reserved. At this point, we have a working knowledge of how to create and edit a model. We now want to take advantage of the portability of our geoprocessing tools. Before we think about distributing our tools we must first document our model. This way our users have some descriptive information about the process they are going to run. Copyright © 2005 ESRI. All rights reserved. 39 Documenting and distributing models Labeling Documenting model processes ArcToolbox Documentation Editor Model reports Model properties Distributing models Copyright © 2004 ESRI. All rights reserved. To do that we will consider the following: labeling; how we can document processes within the ModelBuilder interactive environment; using the Documentation Editor to create help for our model; how we can use model reports; we’ll take a brief look at model properties; and, finally, things to consider before distributing our work Copyright © 2005 ESRI. All rights reserved. 40 Labeling Two different types: Stand-alone labels Element labels Can be connected to tools or line connectors, or free-floating Copyright © 2004 ESRI. All rights reserved. Labels are a quick and easy way to provide useful information to your user. You might use labels to briefly describe a geoprocessing tool or some input data, describe a process, or your model as a whole. There are two different kinds of labels you can use in ModelBuilder. There are stand-alone labels—these are not connected to process elements. They might provide an overview of a model. Then, there are element labels—these are labels that are connected to a geoprocessing tool, data element, or line connector. These labels will stay with their associated element if they are moved in the model’s interactive window, and they will be deleted if the element is deleted. Copyright © 2005 ESRI. All rights reserved. 41 Documenting a process Additional information about each process Copyright © 2004 ESRI. All rights reserved. As you go through and construct the model, you are going to want to provide help to users about the model you created. There are several ways to accomplish this. As you go through and build your model, you can write help about the tools that you have incorporated into your model. You can create documentation about a process while editing the model. Just right-click on a tool and select Edit Documentation. This will launch the process Documentation Editor, where you can provide information about the tool within your process, perhaps the kind of parameter values the tool is expecting, or what the process will generate. The information that you provide about the individual tools gets integrated into the model’s help file. You would do this when you want to provide users with information about parameter values that have been set, what the process will achieve, or links to additional documentation. Copyright © 2005 ESRI. All rights reserved. 42 Documenting a model Three types of documentation: Help page Metadata Dialog side panel Create help the same way as system tools Copyright © 2004 ESRI. All rights reserved. Documentation and metadata can be created for all geoprocessing tools that you create. This help is created by using the Documentation Editor, accessible in ArcToolbox through the model’s context menu. The Documentation Editor is easy to use and gives you a lot of functionality for documenting your models. With the Editor, you can provide summaries of the model’s processing capabilities, examples of how to use the model, describe the different parameters required, usage tips, and so on. Provided that documentation has been created for your model, you will be able to access it three different ways: through the Help page, the Metadata tab in ArcCatalog, and the model’s dialog side panel. Copyright © 2005 ESRI. All rights reserved. 43 Documentation: Help page Help page allows you to view: Parameters Model-associated tools Example syntax to run in different geoprocessing environments Copyright © 2004 ESRI. All rights reserved. Here is an example of the kind of output you can expect when looking at help for the model from ArcToolbox. The output created from the model is a professional-looking HTML Help file that has been integrated within your tool. The Help page will open in your default browser and will show information such as a description of the model, usage tips that will help you run the model successfully, syntax necessary to execute the model in a different geoprocessing environment (such as the command line or in a script), examples, and details about each process in your model, to name a few. You can access this Help page by right-clicking on the model in ArcToolbox and selecting Help on the context menu. Copyright © 2005 ESRI. All rights reserved. 44 Documentation: ArcCatalog metadata Keywords integrated into ArcToolbox and ArcCatalog will help you search authoring information Copyright © 2004 ESRI. All rights reserved. In ArcCatalog, you can click on the Metadata tab to return simple metadata about your model, such as a brief description of the tool, who designed it, and how to contact them. It should also include keywords that will allow you to utilize the search capabilities of the application. Copyright © 2005 ESRI. All rights reserved. 45 Documentation: dialog side panel Brief description of tool and parameters Linked to full Help page Copyright © 2004 ESRI. All rights reserved. Finally, you can create help that will appear on the model tool’s dialog side panel. A brief description of the tool and its parameters will appear. Plus, by clicking the Help button on the dialog you can open the full Help page. Copyright © 2005 ESRI. All rights reserved. 46 Model reports Provide information about the model Show information regarding variables and processes Display status messages Copyright © 2004 ESRI. All rights reserved. A model report is a good way to track properties of your model. The report will provide information about the last time it was ran, such as which datasets were used in the model, the different tools used, model variables, status messages for each process, and processing times. Copyright © 2005 ESRI. All rights reserved. 47 Distributing models Setting model properties Storage options Things to remember before you distribute your model Copyright © 2004 ESRI. All rights reserved. So let’s move from documenting your model to learning how to distribute your model. To do that we’ll look at setting model properties, storage options, and things you need to remember before distributing your models. Copyright © 2005 ESRI. All rights reserved. 48 Distributing models: properties Set model name, label, description, and dialog style sheet Can set relative paths to toolbox location Preview exposed model parameters and environments Copyright © 2004 ESRI. All rights reserved. The Model Properties dialog allows you to set a name and label for your model, write a description for your tool, and set a custom style sheet for the dialog. Included on the General tab is an option that pertains to distributing your geoprocessing work. On the Model Properties dialog we have the ability to set relative paths, meaning that pathnames will be maintained relative to the toolbox or workspace that contains the model. Copyright © 2005 ESRI. All rights reserved. 49 Distributing models: storage options Model is saved as a tool in a toolbox Toolboxes are saved as *.tbx files or in a geodatabase Default location for *.tbx files is the user’s profile: \\Documents and Settings\<user>\Application Data\ESRI\ArcToolbox\My Toolboxes Copyright © 2004 ESRI. All rights reserved. There are two different storage options for your toolboxes. They may be stored as stand-alone .tbx files or stored inside a geodatabase. Stand-alone .tbx files are shareable. For example, they can easily be attached to an e-mail. This type of toolbox is located in your Documents and Settings folder by default, but you can change the save location through the Options dialog found in the application. Copyright © 2005 ESRI. All rights reserved. 50 Distributing models: things to consider Set the toolbox to read/write Share: All data Scripts or additional models used All documentation Additional files (graphics and style sheets) Set relative paths If data moves, the model won’t break Copyright © 2004 ESRI. All rights reserved. Automating your own workflows and automating other people’s workflows is ultimately the point of ModelBuilder. Automating your own workflows is simple. You know where your data is located and, if it moves, where your toolbox is located and so on. Automating workflows for other people is just as easy as long as you keep several things in mind. When you share toolboxes or tools (in this case model tools) with other users, you have to make sure you: set the toolbox to read/write, share all data used in the tool, share any scripts or additional models used, share all documentation, share additional files such as graphics and style sheets, and set relative paths to data so that, if it moves, the model won’t break Copyright © 2005 ESRI. All rights reserved. 51 Software demonstration Copyright © 2004 ESRI. All rights reserved. So let’s take a look at how we can set up documenting and distributing our models. We are back in ArcMap and we’re in our custom toolbox. This model (named Model) is the one we created in the first and second demonstrations. Now, the first thing I want to point out to you is model properties. To get there, right-click on your model and select Properties. This brings up the Model Properties dialog box. We are going to take a look at the General tab. The first thing that we want to enter is a name. The name is used to reference our model in the command line and scripting environment. An important thing to remember is that we cannot have any spaces in the name. So we are going to call this one “SQKMSurfaceArea.” There are no spaces in the name. Now in Label, what we type here is going to be displayed in the custom toolbox. So we are going to add the same text, but we are going to make use of the ability to add spaces. Now, I want a short description to show up on our model’s dialog side panel, so here is where I enter in that description. So we’ll just enter in a short description here: This model will determine the surface area of lakes. If I wanted to apply a custom style sheet, I could apply that here. Copyright © 2005 ESRI. All rights reserved. 52 Most important with the General tab and distributing our work is this option here to store relative pathnames. This is the property that will make all pathnames within our tool relative to the toolbox location. Let’s go ahead and check that off. So that is the General tab. On the Parameters tab, in the second demonstration we exposed a parameter of the Select tool and created a model parameter from it, and that is what we see here. So we have the name as Expression and the data type is SQL Expression. You’ll see all of your model’s parameters here in this tab. We can also set up some environment settings. If we had a .chm file, or compiled help, then we could apply that here. So let’s go ahead and click OK. Notice that the label of our model has changed from Model to SQKM Surface Area. Now let’s take a look at what it did to our dialog side panel. When we double-click on the model, we have a brief description on the side: This model will determine the surface area of lakes. So now I want to document some of the processes within the model. To do that, I have to be editing the model. Now, I briefly want to describe the Select tool. To do that, right-click on the Select tool and select Edit Documentation. This brings up the Select Documentation editor. I can provide a paragraph of text, some bulleted items, provide a link to a file or an illustration (maybe I had a picture of what some of the lakes looked like). So I can provide that here. For now, I’m just going to provide a simple paragraph. So let’s put in: Will select the country provided in the expression. So now we have basically documented the Select tool. Click OK. Now let’s document something about the Clip tool. Right-click on the Clip tool and select Edit Documentation. Now instead of doing a paragraph, I’m going to add some bulleted items. So here I’m going to say: Uses the country from the Select tool. So there is one bullet, and let’s add another: Clips the lakes with the selected country. So now we have two bullets that are going to briefly describe tools within our model. So that has now been applied to the Help associated with our model. So let’s go ahead and save our model and close it out. Now we are going to look at the Documentation Editor of the model. Up till now, we have just described two processes within our model. So let’s take a look at what that has done for us. If I right-click and go to the Help, and If I click on Model, here is the Select and here is the Clip tool. Copyright © 2005 ESRI. All rights reserved. 53 Here are the two bits of information I provided about the tools: the paragraph and the two bulleted items. So those are the processes that I just documented inside the model. Now let’s go into the Documentation Editor. So right-click on your model and go to Edit Documentation. That brings up this Documentation Editor. Now there are two sections to look at here. There is General Information and there is Help. The General section is going to apply to the metadata of your model. So we can provide an abstract, some keywords, author information, and perhaps constraints to running the model. So I’m going to provide an abstract to the model. So let me get that here, and I’m just going to paste that in, and fix up the text here so that it will display properly. This is just an abstract about what the model is going to do. Now, an important thing to consider is that what I place here will replace the description that I have in my model’s side panel. So, before I just entered in a very simple line of text when we were looking at the Model Properties dialog. This abstract is going to replace that description. For Keywords, I want to be able to use a search to find my model. So I’m going to type in Iceland and let’s also type in Surface Area. So if I wanted to locate my model, I could just type in Surface Area and it would go and find my model. If I had information about the author, I could provide that here and how to contact them, and finally, constraints for running the model. Now we are going to jump into the Help section. So when I right-click on my model tool and select Help and see the Help information, this is where I enter that text. So for Summary, I’m just going to add a paragraph, and I’m going to make it the same text as the abstract. Again, let me just fix this up. So my summary is ready to go. I can provide an illustration, or provide some usage tips on how to run it. For Command Example, if I wanted to provide additional information about how to run this model from the command line, I could provide that here. Same with the script example—if you wanted to run the model from a geoprocessing script environment like Python, you could provide that example here. What I want to do is enter in some information about the parameters. Notice here we exposed a parameter in the second demonstration called Expression. Here I have two things that I can document, Command Reference and Dialog Reference. Command Reference applies to the model Copyright © 2005 ESRI. All rights reserved. 54 Help. So I’m going to just type in a quick sentence: Use a SQL expression to select a country. I’m going to use the same text for the Dialog Reference. Now the Command Reference shows up on the model Help, the Dialog Reference shows up on the Model dialog, and I’ll show you what I mean by that in just a second. Go ahead and click on the paragraph, and let’s just paste in that same text. Now I’m going to click Finish. Now if I right-click on my model and go to Help, let’s look at the scripting syntax, and here is what I entered: Use a SQL expression to select a country. Here is the summary of my model here, and of course the processes that I documented in my model before. Now if I double-click and open up my model, here is the abstract of the model. Now if I click on the expression, notice that the text has changed to “Use a SQL expression to select a country.” So in this demonstration, I showed you how to document individual processes within a model, and also how to use the Documentation Editor. Copyright © 2005 ESRI. All rights reserved. 55 Review and Q & A ¾ Labeling ¾ Documenting model processes ¾ ArcToolbox Documentation Editor ¾ Model reports ¾ Model properties ¾ Distributing models Copyright © 2004 ESRI. All rights reserved. To review, in the last topic we looked at labeling, documenting model processes, the Documentation Editor, model reports, model properties, and distributing models. And now let’s turn it over to Catherine for some of your questions. Thanks, Jeff. I think we have a couple minutes to answer just a few brief questions. We have a user here from Kalamazoo that wants to know the Knowledge Base article number for batching models, and the number you're looking for is 26892. There's also a Help page titled About batch processing with geoprocessing scripts. If you go to the Desktop Help system and search for that, you'll find that as well. We have another user in Champaign that says, "Do all tools in a model share the same model environments, and can each tool have its own environment setting?" It's a great question. Environment settings throw off a lot of people. There is a hierarchy to environment settings. The lowest level is if you set it in the application or if you set it in the ArcToolbox window, those environments will apply to all of your custom tools and all of the system tools as well, so every tool. If you open up the ModelBuilder interactive window and you set environments inside the Copyright © 2005 ESRI. All rights reserved. 56 window, that will apply to all of the tools and/or processes within just that one model. And that will override what you've set in the application. You can also right-click on each one of the tools and set specific environments just for that tool, and that will override all of the other settings. Hopefully that will clear it up. There are Help pages that will help you with that as well. We have another question here. It says, "Can you clarify the role of intermediate data? When is it temporary and when is it saved to disk?" Intermediate data is that data which is within your model. As you step through each one of your processes, there's an input and an output, and that output then becomes the input to the next process. This is what we classify as intermediate data. More than likely, we're not interested in each one of those individual steps. What we want is the end result of our model. Those individual steps along the way, those results are classed as intermediate. All intermediate data is flagged as temporary unless it's data from an exposed model parameter or the output of the process updates the input of the process, so like an Add Field tool, for example, where the output is just an updated version of the input. It's saved to disk if you uncheck the Intermediate option or if you run your model from the interactive window. Then, all of your intermediate data is always saved that way. It can maintain the model state for you the next time that you look at your model. However, if you run your tool from the ArcToolbox window or at the command line, your intermediate data will be deleted at the end of execution. One other brief thing to question here is from Brian in Jackson, and he says, "Are there any rules about which process will run first in my model?" This is something for you to keep in mind. There is no guarantee as to the order and how the processes will run. The only way you can guarantee that is if you set something called preconditions, and there's quite a bit of write-up about that within the Help system to help you get through that. Let me sort of briefly explain that. If you had a process or a string of processes, and it was a single string (so you have three processes, and they all chained one from the other), it would be acceptable for you to assume, and it wouldn't be wrong, that it would start at the project data and step through the three processes until it reached the end. For the majority of us, when we build models, it's going to have more than one chain involved, and you'll find then that the chains will converge and diverge as you work through your model. In those cases, the only way to guarantee what will get run first and what will have to be run before other things is to set something called preconditions. That's all the time we have for questions right now, so I'm going to pass it back to Jeff. Copyright © 2005 ESRI. All rights reserved. 57 For more information Virtual Campus course Geoprocessing with ArcGIS Desktop Instructor–led training Advanced Analysis with ArcGIS Product book or tutorial Geoprocessing in ArcGIS Copyright © 2004 ESRI. All rights reserved. For more information about ModelBuilder and geoprocessing, you can visit the resources listed here. Copyright © 2005 ESRI. All rights reserved. 58 Thanks for participating Copyright © 2004 ESRI. All rights reserved. That concludes today’s presentation. Thank you for listening. Copyright © 2005 ESRI. All rights reserved. 59