Salvador Bayarri
Consultant, World Bank
sbayarri@gmail.com


GeoServer components
Direct access to GeoServer
◦ Styling with SLD
◦ Adding data from external databases

Direct access to the Geonode spatial
database
◦ Examples of DB analysis processes
◦ Projecting data in DB

Access from desktop GIS
◦ Connecting and displaying map and feature
services
◦ Editing the DB
◦ Projecting data in GIS

Using GIS to prepare data
◦ Importing X,Y & KML data
◦ Georeferencing images
◦ Uploading mobile data
GeoNetwork
Web interface
GeoServer
Web interface
Application layer
Service layer
Data layer
GeoServer is the component that provides WMS, WFS and
WCS services
 It can be used to upload and style data using its own Web
front-end
 However, your GeoNode user won’t be recognized by
GeoServer, you need to use a GeoServer user, normally with
administration privileges
Connect with your Web browser:
http://geonodeURL/geoserver
Login with your workshop user (you will need administration
privileges)



It is easy to work with SLD styles directly within GeoServer
Click on the ‘Styles’ link in the left-side menu and select
the style you want to edit, or add a new one
In the style editor, you
can




Change the name of the
style
Copy the SLD from an
existing style
Change the code of the
SLD
Upload a saved SLD file


If you create a new style, you need to associate it
with a layer
To do so, select the desired layer in the Layers
page

Select the ‘Publishing’ tab in the layer properties
In the WMS Settings
section you can choose the
default style and add more
style choices
Don´t forget to ‘save’ at the page bottom!

The styling changes will show up in GeoNode. Go
to the ‘manage styles’ and make them available

Exercise:
◦ Modify and add new styles for some of your vector
layers
◦ Modify the style for the elevation layers
◦ See the changes in GeoNode
For questions about the SLD format:
http://docs.geoserver.org/stable/en/user/styling/sldreference/index.html


GeoNode only supports file-based data sources,
shapefiles and geoTIFF, but GeoServer offers more
options
Data sources are referred in GeoServer as stores

The uploaded shape files are inserted in the geonode PostGIS
database, so they are all in the same store called “datastore”
We can access any data from the
PostGIS DB ‘geonode’ using the
same store

The uploaded geoTIFF files are copied in the geonode data
folder. Each raster file has its own store.

We can add other data sources for GeoNode by creating
GeoServer stores
More raster formats

GeoServer has extensions to create stores from other
database vendors:
More information in:
http://docs.geoserver.org/latest/en/user/webadmin/data/store
s.html


In this version of Geonode,
to remove a layer correctly:
◦ Remove the layer in GeoServer
(and the data store if it’s a
raster)
◦ Remove the layer styles in
GeoServer
◦ Run ‘geonode updatelayers’
◦ In Geonode, use ‘Edit
layer/Remove layer’ to
eliminate the layer
IMPORTANT NOTE:
If you add external data directly into GeoServer, the GeoNode
superuser must run the following shell command to synchronize
these new layers to be visible in GeoNode.
geonode updatelayers
• GeoNode uses a PostGIS database management
system to store uploaded vector data as well as
the metadata
• To see the contents of this DB, open the pgAdmin
Postgres DB administration tool (this has been
installed for the workshop)
• Read the /etc/geonode/local_settings.py file to
see the DB connection parameters (esp the password)
• Use the ‘Add Server’ command to connect to the
geonode DB
Use the user
and
password
from the
config file
here
Under
Databases/geonode/Schemas/
public/Tables we will see the
geonode tables, and among
them those corresponding to
our vector data
(double-click on the tree
elements, to expand it)
By right-clicking on a table,
we have access to some
interesting tools

We can create reports, like a data dictionary report

The reports are HTML (Web) files that will
automatically open in the default Web browser

We can view the data records and filter them

We can execute SQL queries to select, insert or
delete records
We can apply operators
to perform calculations,
like average, count,
standard deviation,
summation, etc.

Exercise:
◦ Use simple SQL queries to extract columns from some
layers:
SELECT some_columns
FROM some_data_source
WHERE some_condition;
Remember column names in UPPERCASE must use QUOTES:
“LIKETHIS”
◦ Use Sum() and Count() operators with SELECT to
aggregate results
For instance, find out how many communes make each
department and what are their total areas
There are also spatial operators that perform calculations on
geometries:













ST_GeometryType(geometry) returns the type of the geometry
ST_SRID(geometry) returns the spatial reference identifier number of the
geometry
ST_X(point) returns the X ordinate
ST_Y(point) returns the Y ordinate
ST_Length(linestring) returns the length of the linestring
ST_StartPoint(geometry) returns the first coordinate as a point
ST_EndPoint(geometry) returns the last coordinate as a point
ST_NPoints(geometry) returns the number of coordinates in the linestring
ST_Area(geometry) returns the area of the polygons
ST_NRings(geometry) returns the number of rings (usually 1, more if there
are holes)
ST_ExteriorRing(polygon) returns the outer ring as a linestring
ST_InteriorRingN(polygon, integer) returns a specified interior ring as a
linestring
ST_Perimeter(geometry) returns the length of all the rings
And operators that generate a text description of the geometries
in different formats, or vice versa:











ST_AsText(geometry) returns WKT text
ST_AsEWKT(geometry) returns EWKT text
ST_GeomFromWKB(bytea) returns geometry
ST_AsBinary(geometry) returns WKB bytea
ST_AsEWKB(geometry) returns EWKB bytea
ST_GeomFromGML(text) returns geometry
ST_AsGML(geometry) returns GML text
ST_GeomFromKML(text) returns geometry
ST_AsKML(geometry) returns KML text
ST_AsGeoJSON(geometry) returns JSON text
ST_AsSVG(geometry) returns SVG text
We can use the DB to perform more complex spatial analysis
operations
Many ‘GIS analysis’ tasks are based on a combination of
geometric operations between features (intersection, distance)
and aggregation
PostGIS provides the following geometric operators:










ST_Contains(geometry A, geometry B)
ST_Crosses(geometry A, geometry B)
ST_Disjoint(geometry A , geometry B)
ST_Distance(geometry A, geometry B)
ST_DWithin(geometry A, geometry B, radius)
ST_Equals(geometry A, geometry B)
ST_Intersects(geometry A, geometry B)
ST_Overlaps(geometry A, geometry B)
ST_Touches(geometry A, geometry B)
ST_Within(geometry A, geometry B)
The SQL JOIN operator allows us to add fields to a table by
performing spatial tests
For instance, if the ‘landslide points’ table contains locations of
landslides, and the ‘hydrogeology’ table contains types of soil,
we can find out which is the type of soil in each landslide area
We can join spatially the table by using the ‘ST_Contains’ test, to
find out the type of soil for each landslide point
SELECT
slides.fid, slides."PLACE",
hydrogeo."TYPE"
FROM hti_hazardgeology_landslidesdetail_point_022010 as slides
JOIN hti_geology_hydrogeology_bme_polygon_2010 as hydrogeo
ON ST_Contains(hydrogeo.the_geom, slides.the_geom);
We can also create new attributes by aggregating the
information of all the features in the joined dataset
Use JOIN + GROUP BY, and an aggregation operator (Avg, Sum…)
For instance, we want to calculate the total flood area crossed by
each river:
SELECT rivers."NOM",
Sum(ST_Area(floodzone.the_geom)) as flood_area
FROM hti_inlandwaters_rivers_cnigs_line_062006 as rivers
JOIN hti_hazardhydrology_propitiousfloodzone_polygon_052010 as floodzone
ON ST_Intersects(rivers.the_geom, floodzone.the_geom)
GROUP BY rivers."NOM";

Exercise:
◦ Use spatial joins to add attributes to point layers (for
instance, in which commune or department is located each
CNIGS centre)
◦ Find out which communes are crossed by a certain road or
river
◦ You can get more ideas from:
http://workshops.opengeo.org/postgisintro/joins_exercises.html

We can export the result of a SQL Query to a text
file, which can be loaded into a spreadsheet
Saving your results

We can also use the results of a query to create a
new table
CREATE
TABLE table_name AS
SELECT … FROM … WHERE … ;
ALTER TABLE table_name ADD PRIMARY KEY (fid);

Or create a view, which gets updated when the
original table changes. This is how we can easily
add more columns with calculated values
CREATE VIEW view_name AS
SELECT … FROM … WHERE … ;
Saving your results

If we want to create a SPATIAL table that we can
reuse in GeoNode, we also need to register the
table as spatial
SELECT
Populate_Geometry_Columns(‘table_name’'::regclass);
Saving your results

Exercise:
◦ Create a spatial table by copying an existing one and
adding some columns
CREATE TABLE communes AS
SELECT fid, the_geom, ST_Area(the_geom)
FROM hti_boundaries_communes_adm2_cnigs_polygon;
-- Define a primary key
ALTER TABLE communes ADD PRIMARY KEY (fid);
-- Spatially enable it
SELECT Populate_Geometry_Columns('communes'::regclass);
◦ Create the new layer in GeoNode from the ‘datastore’ store
◦ Run the ‘updatelayers’ command
◦ Display the new layer in a map and use the new column to
style it

If we upload a .PRJ file with our data, Geonode will
assign a SRID (Spatial Reference ID) in the PostGIS
database. Take a look at the geometry_columns
table.
This ID is also called the EPSG code, a standard way to refer
to Spatial References. See http://spatialreference.org/


Another internal PostGIS table (spatial_ref_sys) describes in
text format each SRID
We can see there two standard text representations of each
Spatial Reference (srtext is the same description used in .prj
files)



Using different projections for different layers in a
map should not be a problem, since:
◦ Web servers project the data to the SRID the client requests
◦ GIS clients can project data to the view SRID on-the-fly
However, sometimes it is convenient for efficiency to project
data to a different SRID, or we must do it because the server
or GIS does not perform correctly the conversion (for
instance, when doing some operation between layers of
different SRID)
For example, imagine we want to change from a UTM zone
(18N for Haiti) on NAD27 datum to use a different datum like
WGS84 (differences in datum often account for annoying
displacements between layers in maps)

Transforming geometries to a different spatial
reference in PostGIS SQL is as simple as:
SELECT ST_Transform(the_geom, DestinationSRID)
FROM original_table

We’ll see we can do the same as easily in a GIS

With a query we can find all SRIDs based on UTM
zone 16N:


We can now transform data from UTM16N-NAD27
(26716) to UTM16N-WGS84 (32616)
In this case differences are about 2 meters

To learn more about working with PostGIS:
http://workshops.opengeo.org/postgis-intro/
http://postgis.refractions.net/documentation/
manual-1.4/ch04.html


Desktop or Web GIS tools include powerful
display and analysis functionality that we
can use with our GeoNode
From a GIS, we can connect to GeoNode’s
◦ PostGIS DB
◦ Map Service (WMS) published by GeoServer
◦ Feature Service (WFS) and Coverage Service (WCS),
as published by GeoServer
◦ Catalog Service (CSW), but in practice this is not
well solved yet
GeoNode
File upload
Web front end
pyCSW/GeoNetwork
GeoServer
File download
CSW
WFS, WFS, WCS
GIS
SLD
WFS-T(Editing)
PostGIS
DB
(Editing, geoprocessing)

For the workshop we will use Quantum GIS,
an open-source GIS available in Linux and
Windows

To connect to the GeoNode WMS, we use the
URL http://mygeonodeURL/geoserver/wms. Click on
‘New’ to create a new connection





Click on ‘Connect’
The layers published in the GeoNode map service will be displayed
(even layers that are not added to a map)
Also the maps that you published as WMS
Select one or more layers (expand to select from available styles)
Click on ‘Add’ and then ‘Close’

The selected layer(s) will be displayed using images
from the map server

We can identify features in the map (internally uses
GetFeatureInfo request)

It is straightforward to add a WMS server to ArcGIS
by using the ‘Add Data’ tool


After adding the WMS service to the table of contents in
ArcMap, we can select specific layers to draw
However, the Identify feature does not seem to work well




By accessing WFS services we can access data with full
geometry and attribute information, as if it were on a local
file or database
Because data might come in a different spatial reference, in
some GIS we need to make sure the on-the-fly projection is
enabled
In qGIS, we use the File/Project Properties dialog for the
current project…
And, for future projects, the Settings/Options dialog

To connect to the GeoNode WFS, we use the
URL http://mygeonodeURL/geoserver/wfs




Click on ‘Connect’
The layers published
in the GeoNode
feature service will be
displayed
Select one layer
Click on ‘Apply’ and
then ‘Close’
Note that we can use a
Coordinate Reference
System for the request
that
is different from the
current qGIS project view


The selected layer will be displayed using features
downloaded from the server
We can style them with the qGIS style properties to perform
visual GIS analysis



We can also edit the WFS feature layer
Use the ‘Layer/Toggle Editing’ command to enable editing
After doing so, the edit tools become enabled in the qGIS Edit
menu

Use edit tools (try Node Tool or Reshape in qGIS)
and the table of attributes to change the geometry
and field values of a selected feature

We can also add
or remove
columns in the
table view
• If the GIS or Web editor supports WFS-T, like qGIS,
you can save changes directly to the server
• Just click again the ‘Toggle editing’ tool and accept
saving
• If everything goes well, you should be able to see the
changes in the Geonode maps
• However, in the current pre-release version there is a
Geonode bug that prevents this editing transactions
to succeed
• You can save the editing results to a file
• Then you can use ‘Replace layer’ in Geonode
• Exercise:
Create a map in qGIS by combining several WMS
and WFS layers from GeoServer (you can style the
WFS, but not the WMS)
Save the project
• It is necessary to have the Data Interoperability
extension installed (not a free add-on)
• From ArcCatalog or the Catalog tab in ArcMap,
add a new Interoperability Connection, select
the WFS format and enter the GeoNode WFS URL
• Click the ‘Parameters’ button and open the
Feature Types selector
• A list of the layers from the WFS server should
appear (*), and you can select to import one or
more
(*) Does not seem to work with the Geoserver WFS, try the URL showed her
• Once the interoperability connection to the WFS
server is created, you can add data from the
connection to ArcMap
• You can also use it in geoprocessing (just drag
the layer from the legend to the input field for a
feature class



Desktop GIS systems can also connect directly to the spatial
DB and use it as a data source
We create a connection just like we did from pgAdmin
We need to use the user & password in
/etc/geonode/local_settings.py



Once the connection is added, click ‘Connect’ to get the
list of available layers
Select one layer and click ‘Add’. You can repeat this for
other layers and click ‘Close’ when you are done
We can also specify a definition query for each layer
with ‘Set Filter’ (see next slide). This can also be done
for WFS layers.

With the query, we select only features complying
with some condition

With the query, we limit the features to be
displayed, edited, etc.


We can toggle editing and use the same tools as with
WFS or other feature layers
This time, we will be able to Save Edits in the same DB
table, or create a new one. All services based on this
layer will be affected

Exercise:
Upload in qGIS some feature layers from the PostGIS
DB
Edit them
Save the results to the DB
Go to GeoNode and see that the layers and maps
reflect the changes



ArcGIS support PostGIS spatial databases via
ArcSDE
It is necessary to install the ArcSDE PostgreSQL
AddOn
In theory, this will allow us to access the GeoNode
spatial data from ArcGIS

Data can be processed:
◦ Before uploading to GeoNode, by using a GIS on
shapefiles or the original DB, or SQL on the DB
◦ After uploading to GeoNode, by connecting to the
PostGIS DB or WFS from a GIS, processing the DB with
SQL, or by downloading the data and updating the layer
in GeoNode

GIS is often used to prepare data for SDI publication:
◦ Importing data point with X-Y values in a table
◦ Importing from KML
◦ Importing GPS or mobile data
◦ Georeferencing images

In qGIS, use the “Add Delimited Text Layer”
command

In ArcGIS, use the Add XY Data command
http://support.esri.com/es/knowledgebase/techarticles/detail/27589

Google Maps and Google Earth, as well as mobile
and desktop apps have tools to digitize geographic
data as KML (see
http://maps.cga.harvard.edu/qgis/wkshop/import
_kml.php for Google Maps)

KML files can be imported in qGIS with the ‘Add
vector layer’ command
Imported layers can then
be saved to a shapefile and
uploaded to Geonode

In ArcGIS, a geoprocessing tool can be used to read
KML and create vector layers
Exercise:
Create a map in Google Maps and digitize some
points, lines, polygons
Export them as KML
Import in qGIS and save the shapefiles
Upload in Geonode and display the layers in a
map



Another common data input comes from rasters
which have been scanned or photographs with no
spatial reference
We need to assign them coordinates which allow us
to superimpose the raster on a map and maybe use
it for analysis
• In qGIS, use the
Plugins/Manage dialog
to enable the
Georeferencer plugin,
based on the GDAL
library for raster data
• Then select the
Georeferencer tool in the
raster menu

First, load a reference layer in qGIS, like the country
boundaries


Open a raster to the Georeferencer, which displays
it in its own window
Then we need to create at least 4 control points to
define the correspondence between the raster and
the reference data
Open the
“HaitiImage.png”
image, assigning the
default project CRS
Then click on a
distinctive point, like a
coast feature (you can
use the Georeferencer
navigation tools to get
closer and pan
around)
When you click on the image, a dialog will appear to enter
the reference coordinates for that point
Click on the ‘From map canvas’ button to use the reference
map by clicking on it. The coordinate input fields will get
values and you can click ‘Ok’
Enter at least other 3 reference points in the same way,
better if they are spread in different directions
You can remove points from the GPS table later on

Once you have at least 4 points, click on “Start
geoferencing” (the Play icon) and choose the
transformation options


A new GeoTIFF file with the CRS and coordinate information
will be generated. This image can then be uploaded in
Geonode
You can check the georeferencing correctness in the map
view and upload the GeoTIFF in GeoNode if the result is
correct
More details in
http://courses.umass.edu/nrc297s/PDFs/Lab_Georeferencing_with_QGIS.pdf
http://glaikit.org/2011/03/27/image-georeferencing-with-qgis/

Exercise:
Capture a view in Google Maps or any other
map source
Save the capture to an image file
Georeference the image and upload it in
GeoNode




GPS-enabled devices are becoming a quick
and easy way to input data in GIS and SDIs
The GPS eXchange Format (GPX) is an open
format that can be downloaded or uploaded
from/o GPS-enabled devices
A GPX file contains points (waypoints),
tracks (lines) and routes (future reference
points and turns) using latitude/longitude
coordinates on the WGS84 datum
It is widely supported in many systems and
software, including the Open Street Map
project


qGIS provides a convenient plugin to import GPX
files. It can also be used to download or upload the
files to a device
First, enable the plugin in the Plugins/Manage
dialog, then go to the ‘Vector/GPS/GPS Tools’
command to pop up the import dialog
Exercise:
Import some of the sample GPX files in the workshop data folder,
which may contain only points or tracks
(or search in

http://es.wikiloc.com/wikiloc/find.do?t=&d=&lfr=&lto=&src=&act=&q=haiti)
Do not forget to enable the ‘on the fly projection’ project option
Use the OpenLayers plugin to set a nice background map (you may
need to reset the project CRS after adding this layer)


Once imported, you can process GPX data as any
other feature, and save it to a shape file and upload
in GeoNode
As we will see, mobile data can be an important
source for ‘crowdsourced’ data acquisition, as in the
OpenStreetMap project
OpenGeo education center:
http://workshops.opengeo.org/

Introduction to GIS using qGIS:
http://www.baruch.cuny.edu/geoportal/practicum/

Harvard qGIS tutorials
http://maps.cga.harvard.edu/qgis/

Quantum GIS documentation
http://www.qgis.org/en/documentation.html

PostGIS documentation from its makers, Refractions Research:
http://postgis.refractions.net/documentation/

Using Geonode from ArcGIS, qGIS, Google Earth
http://docs.geonode.org/en/latest/tutorials/users/integration/index.html

GeoServer documentation (User Manual)
http://docs.geoserver.org/stable/en/user/
