PECAS Model Development
MapIt Installation and Configuration
Tech Memo 2 – WORKING DRAFT
Calgary, Alberta
October 2012
PECAS Model Development: MapIt - Installation and Configuration
System Documentation Technical Note
File: MapIt documentation_PostGIS2.0.docx
10.18.2012/ARM/GFM/JEA/JDPM
1. What is MapIt?
MapIt is a tool developed by HBA Specto that provides support to a group of tasks involved in every
day PECAS modelling work. MapIt allows users to visualize PECAS model results in the form of maps
at the LUZ (land use zones) and TAZ (transportation analysis zones). MapIt runs on a webserver and
interacts with a database, so that multiple users can simultaneously use it to investigate model run
results.
Users define their own query in terms of: scenarios (one or two), which file (ActivityLocations,
ExchangeResults, others), which attributes (prices, quantities, utilities, etc.), start year and end year.
MapIt then shows a list of options depending on the file selected. For example, MapIt shows activity
groups or commodity groups based on the file previously chosen.
Two features make MapIt essential in PECAS modelling work:
 It compares the results from 2 different scenarios (for instance W00 and C01).
 It combines values from different years.
Although MapIt generates rudimentary maps, its main purpose is to combine PECAS outputs into
layers in a geodatabase and in shape files. Users can then use the GIS and visualization tools they
are comfortable with to view PECAS output.
Figure 1. MapIt interface
MapIt is used by HBA staff to build numeric comparisons between scenarios and years and store the
instructions for the comparison in the database. Users can download these comparisons as GIS
shape files, or view them from the GIS database. These comparisons are used in calibration and
development (plan an implement future changes or improvements in the models) and in policy
evaluation and analysis.
2
PECAS Model Development: MapIt - Installation and Configuration
System Documentation Technical Note
File: MapIt documentation_PostGIS2.0.docx
10.18.2012/ARM/GFM/JEA/JDPM
The purpose of this brief document is to describe the MapIt installation and configuration on our
client’s computer. The description includes how to get the source code, what are the prerequisite
software packagess and their corresponding installation, how to load shape files into the database,
how to configure MapIt setting files, how to create the map files for MapServer, how to configure
Apache and how to run MapIt. Figure 1 shows MapIt interface and some user options.
2. Getting the source code:
1. Install the Tortoise SVN software
2. Create a folder where you want to install the program i.e. “C:\djangoProjects”
3. With the right bottom click on the folder “djangoProjects” and check out the code from
SVN using
Mac Users: in a terminal windows, change to your development directory and type
svn co http://svn.hbaspecto.com/svn/pecas/PECASSoftware/trunk/web_output
You will need to write for the URL of repository:
http://svn.hbaspecto.com/svn/pecas/PECASSoftware/trunk/web_output
And for the checkout directory:
C:\djangoProjects\web_output
Figure 2. SVN Code Checkout
3
PECAS Model Development: MapIt - Installation and Configuration
System Documentation Technical Note
File: MapIt documentation_PostGIS2.0.docx
10.18.2012/ARM/GFM/JEA/JDPM
You will see the Tortoise copying the code to the specified folder like in this window:
Figure 3. SVN windows copying files into specified folder
And you will end up with the files and programs that you need in the following path:
C:/djangoProjects/web_output which from now will be called the “PROJECT PATH”
Figure 4. final content of the specified folder
3. Prerequisites Software:
1.
2.
3.
4.
5.
6.
PostgreSQL 8.4 (pgAdmin III) or higher with PostGIS enabled.
Python-2.66 32bit (Python 2.7 and later versions work, but Python 3.x is not compatible)
psycopg2 (e.g. for Windows: Psycopg2-2.0.14.win32-py2.6-pg8.4.3 )
Django 1.2.1
MapServer 2.3 (e.g. for Windows ms4w_2.3.1)
Google Visualization Python (gviz_api)
4
PECAS Model Development: MapIt - Installation and Configuration
System Documentation Technical Note
File: MapIt documentation_PostGIS2.0.docx
10.18.2012/ARM/GFM/JEA/JDPM
Note: You can get the all the software you need (except PostgreSQL) from:
C:\djangoProject\Web_output\SoftwareToInstall-essentials.rar
Mac Users: You can use MacPorts to install everything. Install the prerequisites for mac ports as
instructed on http://www.macports.org/ (XCode and its command line utilities.) With macports
installed you can install postgresql, except the port for postgresql does not include the required
tablefunc module. To fix this, you should first type:
sudo port edit --editor "open -e" postgresql91
And find the line that begins with “set contribs”. In that line there is a list of modules, add the
tablefunc module to the list. Save the file, then you can type
sudo port install postgresql91
to install postgres, and
sudo port install postgresql91-server
sudo port load postgresql91-server
to start it running.
In this guide, we will assume that PostgreSQL is already installed, then going down the list the next
program to install is python.
3.1
Python & Psycopg
Installing python and psycopg is straightforward. You only need to double click on the installer (e.g
python-2.7.3.msi and psycopg2-2.4.5.win32-py2.7-pg9.1.3-release.exe) in the
SoftwareToInstall-essentials folder.
Mac Users: with macports installed, type:
sudo port install python26
and
sudo port install py26-psycopg2
Note there are python 27 versions of these, but as of this writing there is no python 27
integration with apache and mod_wsgi, so we are installing the python 26 versions.
3.2
Django 1.2
From this point many programs are going to be installed using the CMD (command prompt) and it is
recommended to open CMD as an Administrator. One way to do this in windows vista or in windows 7 is
going to the search programs and files bar, type cmd and instead of pressing Enter, press
CTR+SHIFT+ENTER. That will launch the user account control option. Click yes and that will take you to
the CMD windows as an Administrator.
Once in the CMD, to install Django, change to
C:\djangoProjects\web_output\SoftwareToInstall-essentials\Django-1.2.1
Run the following command:
C:\djangoProjects\web_output\SoftwareToInstall-essentials\Django1.2.1> C:\Python27\python setup.py install
5
PECAS Model Development: MapIt - Installation and Configuration
System Documentation Technical Note
File: MapIt documentation_PostGIS2.0.docx
10.18.2012/ARM/GFM/JEA/JDPM
Mac Users: sudo port install py26-django
As you can see it is needed to specify the path of the program we want to install, which is django.
But, also the path of python and the command “python setup.py install”.
WARNING: Make sure that the version of python used is the same as you installed earlier in 2.1. For
more help on Django installation, please see:
http://docs.djangoproject.com/en/1.2/intro/tutorial01/#intro-tutorial01
Mac Users: to check your version of python, type
python --version
3.3
MapServer
To install MapServer, run the ms4w setup program in
C:\djangoProject\web_output\SoftwareToInstall-essentials\
(e.g. ms4w-3.0.4-setup.exe) and install ms4w to the C:\ drive. Then, run the following command:
C:\ms4w> apache-install.bat
Mac Users: sudo port install mapserver +proj +postgis
For more help, please see: http://maptools.org/ms4w/index.phtml?page=README_INSTALL.html#bextracting-ms4w-for-the-first-time
WARNING: if there is any trouble please check that the comments (signed as #) inside of the file
httpd_weboutput.conf in the path C:\ms4w\httpd.d start at the beginning of the line
3.4
Gvis_api
This program is used to generate the bar charts for an individual zone. To install it, run the following
command:
C:\djangoProject\web_output\google-visualization-python> C:\Python32\
python setup.py install
Mac Users: cd google-visualization-python; sudo python setup.py install
3.5
OWSlib
This program is used to generate the legend of the maps in Mapit. To install it, run the following
command:
C:\djangoProjects\web_output\SoftwareToInstall-essentials\OWSLib>
C:\Python27\ python setup.py install
This requires installing setuptools for the python version that you have installed in your computer. The
file to install is called “ez_setup.py” and you can take it from the SoftwareToInstall-essentials folder
located in C:\djangoProject\web_output. Open the program in python and run it. For more complete
instructions go to : https://pypi.python.org/pypi/setuptools/1.1.5#windows
6
PECAS Model Development: MapIt - Installation and Configuration
System Documentation Technical Note
File: MapIt documentation_PostGIS2.0.docx
10.18.2012/ARM/GFM/JEA/JDPM
4. Setting up the database
You may already have a database setup, which you used for the PECAS Space Development (SD) module.
Mapit uses two schemas in the database, the “output” schema and the “analysis” schema, which can
coexist in the same database as the SD schemas. However the SD database may not have some of the
functionality (PostGIS, crosstab functions, etc.) so it is better to create a new database.
To set up a new database for MapIt, you need to follow the steps below:
1- Open the program pgAdmin III. It will probably ask for the user password, which usually is:
usrPostgres
2- In the Object browser pane on the left, right click on the Databases icon (under the
PECAS_Demo server), and choose “New Database …” option.
3- Fill in the New Database form as in the figures below. Click OK.
Properties Tab:
7
PECAS Model Development: MapIt - Installation and Configuration
System Documentation Technical Note
File: MapIt documentation_PostGIS2.0.docx
10.18.2012/ARM/GFM/JEA/JDPM
Definition Tab:
Figure 5. New Database forms
4- Now that you have created (or chosen) your database for MapIt, you need to right click on the
newly created Database (in this case MapIt), and choose “Restore…”
8
PECAS Model Development: MapIt - Installation and Configuration
System Documentation Technical Note
File: MapIt documentation_PostGIS2.0.docx
10.18.2012/ARM/GFM/JEA/JDPM
Above to restore only structure
5- On the ”File Options” tab in the Restore Database form, choose
C:\djangoProjects\web_output\SoftwareToInstallessentials\MapIt_DB_template.backup
9
PECAS Model Development: MapIt - Installation and Configuration
System Documentation Technical Note
File: MapIt documentation_PostGIS2.0.docx
10.18.2012/ARM/GFM/JEA/JDPM
6- On the “Restore Options #1” tab, make sure no options are checked.
7- On the “Restore Options #2” tab, make sure the “Verbose messages” box is checked.
Figure 6. Restore in mapit database
10
PECAS Model Development: MapIt - Installation and Configuration
System Documentation Technical Note
File: MapIt documentation_PostGIS2.0.docx
10.18.2012/ARM/GFM/JEA/JDPM
Note needed on restore errors. There may be many duplicate functions, keys, etc. if restoring over an
existing database.
When the Restore finishes, and after Refreshing (View>>Refresh), you should be able to see three (3)
schemas, namely:



output: used to store the models’ run
analysis: used to store MapIt views and crosstabbed data
public: used to store the zone shapefiles (TAZ and LUZ) and some system tables
Note: if you previously installed the “PECASDemo” database, there will be 2 additional schemas shown:
topology and w00.
NOTE: addgeometrycolumn function is changed in PostGIS 2.0, so addgeometrycolumn functions
without “boolean” arguments must be dropped (by dropping the postgis2.0 extension if needed using
the “Drop cascaded...” command on the postgis extension in the restored database) so shapefiles can be
imported (in the next step) if upgrading from a postgreSQL version <9.0 and PostGIS version <2.0.
Refer to http://www.bostongis.com/blog/index.php?/archives/187-How-to-upgrade-your-database-toPostGIS-2.0-let-me-count-the-ways.html
Running on PostGIS 2.0.0 PostgreSQL 9.0 or lower and moving to 9.1 with extensions
Yes, This also applys to folks who were running 32-bit PostgreSQL on Windows and decided what the hey lets see what this
PostGIS on 64-bit PostgreSQL windows is all about.
1.
2.
3.
4.
Backup your old database
Make sure the PostGIS 2.0.0 binaries are installed on your new PostgreSQL service
Restore your old db on new server service
run:
postgis_upgrade_20_minor.sql
-- this one only if you had installed with raster support before
rtpostgis_upgrade_20_minor.sql
-- otherwise you'll need to install
rtpostgis.sql
-- run this 00
topology_upgrade_20_minor.sql (if you had topology support this may be
in a topology sub folder)
5.
Run sql statement
CREATE EXTENSION postgis FROM unpackaged VERSION '2.0.0';
-- only if you have topology support
CREATE EXTENSION postgis_topology FROM unpackaged VERSION '2.0.0';
-- (if you don't have topology and want it)
CREATE EXTENSION postgis_topology;
11
PECAS Model Development: MapIt - Installation and Configuration
System Documentation Technical Note
File: MapIt documentation_PostGIS2.0.docx
10.18.2012/ARM/GFM/JEA/JDPM
The query files (*.sql) are located in:
C:\Program Files\PostgreSQL\9.2\share\contrib\postgis-2.0
You will end up with a view as shown in Figure 7:
12
PECAS Model Development: MapIt - Installation and Configuration
System Documentation Technical Note
File: MapIt documentation_PostGIS2.0.docx
10.18.2012/ARM/GFM/JEA/JDPM
Figure 7. Schemas in mapit database
5. Load shapefiles to database
1- Make sure you have the shapefiles for TAZs and LUZs. They must be in WGS84 spatial reference
which is in PostGIS SRID=4326. (If not, you can convert them after, see below.)
2- Use the PostGIS Shapefile Import/Export Manager plugin (located under “Plugins” menu in
pgAdmin III) to load the shapefiles to the public schema.
13
PECAS Model Development: MapIt - Installation and Configuration
System Documentation Technical Note
File: MapIt documentation_PostGIS2.0.docx
10.18.2012/ARM/GFM/JEA/JDPM
14
PECAS Model Development: MapIt - Installation and Configuration
System Documentation Technical Note
File: MapIt documentation_PostGIS2.0.docx
10.18.2012/ARM/GFM/JEA/JDPM
Figure 8. Importing shape files to PostGIS
WARNING: Depending on the version, you need to specify which file to add, specify the path and the
name of the geometry column as “the_geom” as indicated on the window, and the SRID number of
your shapefile, then press Import. You need to find the projection of your shapefile, and the
corresponding SRID. http://www.bostongis.com/PrinterFriendly.aspx?content_name=postgis_tut01 has
some information on finding an SRID. If your shapefile is not in SRID 4326, do not call the column
the_geom, rather call it the_geom_mysrid, then execute commands similar to the following (TEST THIS)
to update the projection:
select AddGeometryColumn(‘tazs’, ‘the_geom’, 4326, ‘MULTIPOLYGON, 2);
update tazs set the_geom = ST_Transform(the_geom_mysrid, 4326);
15
PECAS Model Development: MapIt - Installation and Configuration
System Documentation Technical Note
File: MapIt documentation_PostGIS2.0.docx
10.18.2012/ARM/GFM/JEA/JDPM
WARNING: The name of the columns with the geometry in your luzs and tazs tables should be the same
as indicated in Figure 8, “the_geom”. If they are not the same, changes should be done clicking with the
right button on the column name and going to properties to change it to “the_geom”. This is shown in
Figure 9. And the final column names are shown in Figure 10.
Figure 9. Changing the name of the column with the geometry in the luzs and tazs tables
Figure 10. Column names “the geom” for luzs and tazs
3- Apply the Land Area script: From pgAdmin 3: open and run the following SQL script:
C:\djangoProjects\web_output\SQL Changes\addLand_area to SpatialLayers.sql If land area
already exists in the table, then this script will return an error.
16
PECAS Model Development: MapIt - Installation and Configuration
System Documentation Technical Note
File: MapIt documentation_PostGIS2.0.docx
10.18.2012/ARM/GFM/JEA/JDPM
4- Make note of the extent: similar to the previous step, run the query in
C:\djangoProjects\web_output\SQL Changes\calculateLayerExtent.sql
Right the resulted numbers down as you will need to put them in settings.py file
5- Each polygon in the TAZ and LUZ has to have a point associated with it for drawing charts on the
polygon. Open the query C:\djangoProjects\web_output\SQL Changes\addGraphPointGeom.sql
and run it. (If there is an error about topology, you can change the text “ST_PointOnSurface” to
“ST_Centroid”, and the graphs will appear at the centroid of the zone, which in some cases may
not be on the zone.)
[NEED TO SET COMMODITY NUMBERS, COMMODITY TYPES, ACTIVITY NUMBERS, ACTIVITY
TYPES. COMMODITYNUMBERS.CSV AND ACTIVITYNUMBERS.CSV CAN BE IMPORTED FROM
MODEL OUTPUTS, TYPES NEED TO BE DEFINED BY THE USER.]
6. Configure MapIt settings file
1- In Web_output folder, make a copy of “default_settings.py” file, and call it: mapit_settings.py
2- Edit the mapit_settings.py, and make changes as appropriate (follow the CHANGE ME
comments)
3- Save and close
WARNING: The name of the database is mapit, and by default postgres is configured to not accept
connections by IP address, so if the database is on the same machine as mapit use 'localhost' as a IP
address
WARNING: in the “default_settings.py” there is a "fieldNameforJoin" that should be specified, and it
is the luz or taz number column name. This field should be integer instead of numeric. If the only
pre-existing one is numeric should be changed to integer inside of Postgres.
7. Create the Map files for MapServer
1- Edit the “mapFilesCreator.py” file in Web_output folder
2- Make sure that the script is importing mapit_setting file that was created in Section 6. In other
words, the mapFilesCreator.py must have the following line in it:
import mapit_settings as settings
3- In command prompt, run the script:
C:\djangoProject\web_output> C:\Python32\python mapFilesCreator.py
When mapFilesCreator runs, it will create Mapfiles for luz and taz shapefiles in specific directories.
There will be a contextlayers.map file which is empty. You can edit this file to define background layers
of situational context in mapserver layer format. Such context could include water bodies, major transit
lines and freeways. If you want to use font symbols in this layer, edit the file
C:/djangoProjects/web_output/fontset.txt to point to the truetype fonts on your system (e.g. in the
windows system directory). Layers that you want to show up in the dot density maps should be listed in
the mapit_settings.py file as a list of comma separated layers CONTEXT_LAYERS. Keep a copy of this file
17
PECAS Model Development: MapIt - Installation and Configuration
System Documentation Technical Note
File: MapIt documentation_PostGIS2.0.docx
10.18.2012/ARM/GFM/JEA/JDPM
as well, as it will be overwritten if you rerun the mapFilesCreator (e.g. after upgrading your mapit
installation).
8. Configure Apache
1- In C:\djangoProjects\web_output\apache folder, make a copy of “default_django.wsgi” file, and
call it: “mapit_django.wsgi”
2- Edit mapit_django.wsgi file, and make changes that reflects your new path and mapit_settings
file.
3- Save and close mapit_django.wsgi file.
4- Then, copy the httpd_weboutput.conf file from
c:/djangoProjects/web_output/apache/httpd.conf to C:\ms4w\httpd.d\httpd_weboutput.conf
and edit it.
WARNING: Comments must be at the beginning of the line in this type of file
5- Make sure that you point correctly to mapit_django.wsgi file in the following line:
WSGIScriptAlias / C:/djangoProjects/web_output/apache/django.wsgi
6- Make sure that all the paths in httpd_weboutput.conf file are correct.
7- Save and close httpd_weboutput.conf file.
8- Install mod_wsgi. Copy and rename the appropriate mod_wsgi.sofile (e.g mod_wsgi-win32ap22py27-3.3.so for Windows with Python 2.7)
from C:/djangoProjects/SoftwareToInstall-Essentials
to c:\ms4w\Apache\modules\mod_wsgi.so .
Be sure to rename the file to mod_wsgi.so after copying it. If neither of these work (e.g. you
have Python 2.6 installed on Windows) download the appropriate mod_wsgi.so file from
http://code.google.com/p/modwsgi/
WARNING: You will need to restart Apache from Windows Services after any changes made to
mapit_settings.py. If apache fails to restart from windows services, try to restart it by opening a
command prompt and running the program c:\ms4w\apache-restart.bat. If you double click on this
.bat file you will not see any error messages, so you need to start the command prompt separately,
and run the .bat from the command line.
Mac Users: Installing mapserver installs apache but does not start it, and it also does not install
mod_wsgi.
To install mod_wsgi:
sudo port install mod_wsgi
18
PECAS Model Development: MapIt - Installation and Configuration
System Documentation Technical Note
File: MapIt documentation_PostGIS2.0.docx
10.18.2012/ARM/GFM/JEA/JDPM
Note the following message
########################################################
# To enable mod_wsgi add
# LoadModule wsgi_module modules/mod_wsgi.so
# to your apache2 config file:
# /opt/local/apache2/conf/httpd.conf
########################################################
Edit the file /opt/local/apache2/conf/httpd.conf to make the changes suggested above, and also
add a line at the bottom
include “/<your installation directory>/httpd_weboutput.conf"
Copy the file httpd_weboutput.conf from inside the windows ms4w directory to your installation
directory, and make the necessary changes to it. Once this is done you can delete the windows
ms4w directory.
To start the apache that was installed with Mapserver, do
cd /opt/local/apache2/bin; sudo ./apachectl start
to have apache start at every boot, type
sudo launchctl load -w /Library/LaunchDaemons/org.macports.apache2.plist
If you need to restart apache (because you’ve changed the configuration files) type
sudo /opt/local/apache2/bin/apachectl restart
You may have permissions problems, do
sudo chmod -R a+rwx mapfiles/
9. Restart Apache Service
1234-
Right click on My Computer, and then choose Manage.
Choose Services and Applications >> Services >> ApacheMS4WWebServer
Right click on ApacheMS4WWebServer, and choose restart
Make sure that the new status of ApacheMS4WWebServer says “Started”. If it does not start,
run the program c:\msrw\apache-restart.bat from a command prompt to see the error
message.
Mac Users: To restart apache (because you’ve changed the configuration files) type
sudo /opt/local/apache2/bin/apachectl restart
19
PECAS Model Development: MapIt - Installation and Configuration
System Documentation Technical Note
File: MapIt documentation_PostGIS2.0.docx
10.
10.18.2012/ARM/GFM/JEA/JDPM
Run MapIt
If you followed this manual you should be able to access and run MapIt by typing this URL in a web
browser: http://<IP address of MapServer>/mapit.
For example, if MapServer is installed on SantaMonica (IP: 192.168.90.105). Then, MapIt URL is:
http://192.168.90.105/mapit or if MapServer is installed in your machine, then MapIt URL is
http://localhost/mapit
20