Spatial Analysis in Government
Lecture 2- Data & Information
By Guy Thigpen, M.Phil., MUSA
Purpose of Lecture
Review the Data and Information contained in
GEODB2
Explore an example (ULRS) of the information
available in the data warehouse
Examine the data structure of a parcel layer
Frameworks of Understanding
Systems
Thinking
Cartography
Geography
Spatial
Analysis
Philadelphia’s Data Warehouse
Data and Information
282 feature files (tables, points, etc)
20 departments provide information
26 geodata folders with additional
feature files
GEODATABASE
A database
that is in some way referenced
to location on the earth.
VECTOR DATA
ATTRIBUTE DATA
spatial data represented. as
points, lines and polygons
additional information about
each spatial feature housed
in tabular format.
METADATA
Documentation of GIS datasets.
Contains information such as,
•the coordinate system,
•when the data was created,
•contact of who created it
•definitions for any of the
code attribute data.
RASTER DATA
cell-based data such as aerial
imagery and digital elevation
models
ArcGIS Services
ESRI Arc GIS Server 10.1
Some of the services that we provide include, City Aerial Photography, City Hybrid Map,
Parcel Layer, Building Footprint Layer, Septa Lines etc.
All these services can be overlaid upon each other according to the user's choice in any
web application or desktop around the city.
•
Caching
One of the main characteristics that makes the server very capable of publishing maps over web is Caching.
Additionally, we have more control over how the cache gets built and updated. Caching particular areas on
the map and On Demand caching are the other main features.
•
Security
Management of services with ArcGIS Server's role based security has been a major advantage. Also, the
product supports token based security for web services and applications.
•
Imagery
ArcGIS Server 10.1's Imagery capabilities enable us to handle large amount of GIS data and provides access to
it enterprise wide. It also allows processing and analysis of images on the client side.
Pictometry
Pictometry is the leading provider in digital, geo-referenced, oblique aerial
imagery and innovative geospatial technologies.
ORTHO
OBLIQUE
Shot straight down and oriented North-up.
shot at a 40~45 degree angle.
Individual image is available from North, South, East or West.
SPECIFICATION:
RESOLUTION: Pictometry offers two levels of resolution, “community” level and “neighborhood” level. The
average resolution of oblique imagery at community level (shot from around 4,500 ft) is 1-foot per pixel, and at
neighborhood level (shot from around 3,500 ft) is 4-inch per pixel.
DATE/ TIME STAMP : Each image is date and time stamped with the time of capture as reported by the
time information transmitted from the Global Positioning System.
CAPTURE AREA:
The oblique imagery is captured for the City
of Philadelphia, including
•
center city area,
•
the entire Philadelphia International
Airport, which is located in Delaware
County, and
•
the Wyndmoor portion of Montgomery
County.
With Pictometry Electronic Field Study (EFS) software, licensed users are allowed to view and work with the
aerial images contained in the Pictometry Image Library
How to make a connection to GEODB2?
1)
In ArcCatalog:
The easiest way to create a connection to the central data warehouse, is by using ArcCatalog and connect to the ArcSDE
application server (aka. Geodb2). Geodb2 will broker all your operations to the database.
2)
Connecting with Oracle Client:
The central warehouse is in essence an Oracle database, and for power users out there who perform more complex operations
and queries, you can connect to the warehouse directly using any Oracle Client .
GIS Spatial Data Sharing and Distribution
•
In general the staff and departments who have been in the forefront of the development of the City’s spatial data support
the sharing of this data within the City and to the public where appropriate and where the City’s costs to provide this data
access can be reimbursed.
•
The interns at the Philadelphia redevelopment Authority are also provided access to the database.
Publically Available Data from GEODB2
@http://maps11.phila.gov/EnterpriseGIS/Downloads.aspx
Philly GIS Forum
•
•
•
Bring together the GIS users in all the departments of City of Philadelphia.
The topics discussed here are expected to get attention from the concerned users
who may make posts pertaining to the discussion.
Threads can serve as future reference for various issues should the need arise.
Group structure with restrictions:
• Threads in the forum pertaining to certain specific topics will be having access
limitations; consequently, the forum users will be divided into groups according to
the topics that are discussed and these groups will be having access those threads.
One user can be present in multiple groups depending on the topics he is engaged
in. Another one will be the read only restriction for certain threads and for certain
users.
•
•
Solutions can be found soon without numerous email tags and phone call
exchanges as the problems are viewed by the community.
Betterment in speed, resource utilization and problem-solving of city-wide GIS
analysis and development.
Data and Information Process
Challenge
Regulation
Compliance
Standard
Resources
Committed
Sponsor
Spatial
Analysis
Resources
Distribution
Medium
Team
Data
Processing
Primary
Data
Capture
Data and Information
The foundation of an analysis include DATA and INFORMATION.
The City of Philadelphia has solid foundation of data and information.
DATA
Information
Knowledge
Understanding WISDOM
W = Wisdom
U = Understanding
K = Knowledge
I = Information (URLS parcel layer and batch geocoder)
D = Data (various shapefiles geospatially located)
S
I
P
O
C
Suppliers
Inputs
Processes
Outputs
Customers
COMPLIANCE
STANDARD:
DATA:
DATA
PROCESSING:
Data is
uploaded,
stored and
collated to be
available as
data or
information
Programs/projects
are required or
encouraged by
federal, state, and
local sources to
collect data
carried out by
contractors and
city agencies
Programs and
projects
produce data
PLATFORM:
Public
Open Data
Philly
PASDA
Clients
phila.gov/map
City
Agencies
Data and Information Relates to
Analysis
Wisdom
Understanding
Knowledge
Basic
Analysis
Information
Data
Parcel Polygons
ULRS: Information in GEODB2
A simple task – displaying an address spatially
What do we know about an address?
- What are the sources for that address?
- (process of standardization)
- What parcel from the Department of Records does this address belongs too?
- (process of geocoding)
Address standardization
Incoming addresses are scrubbed to one or more ADDRESS_IDs
An ADDRESS_ID is any unique combination of
STREET_NUMBER
STREET_NUMBER_SUFFIX
STREET_PREFIX
STREET_NAME
STREET_TYPE
STREET_SUFFIX
(~ similar to LOC field in REAL_ESTATE table)
STREET_CENTERLINE + any other street known by
OPA informs the “valid” streets
!!! ADDRESS_ID is an artifact of the ULRS !!!
Parcel geocoding
For each ADDRESS_ID, assigned to active DOR parcel
2 methods
- by direct reference to DOR: if the address is in the parcel layer
- by estimation: if the address is not in the parcel layer, ULRS
will perform “educated guess”
!! Not every ADDRESS_ID can be associated with a parcel !!
Relations in ULRS
Process of geocoding
Process of address standardization
BRT Tax Accounts
023150808
453137110
PARCEL
145N140168
026S050256
ADDRESSES
883333239
1234 MARKET ST
...etc...
2700 SPRING GARDEN ST
068N220032
070N090134
...etc...
1206 NAUDAIN ST
LNI Hanssen Address Keys
...etc….
141144
141145
141147
141148
..etc...
PWD WaterRev
757315026001
5614004537001
8088001300002
...etc...
SPATIAL
NON-SPATIAL
Unified Land Records System
Online
Zoning Map
Crime
Map
CityMaps
RESA
Police PHA
interface
BUILD
Address
Explorer
DHS
Facts 2
Streets
Workorder
311
PWD
Plumming
Conveyance
Fraud Tracker
Map
Plugin
PWD
workorder
311
ParcelExplorer
Authentication
PhiCAM
S
Batch
Geocoder
Service area
lookups
Legistar
Zoning
Archive
Pothole
Reporter
BRT
Web
Streets
Snowcategory
CRS
Online
LNI
FPC
Workorder
Treemanag
311
ement OHCD Prop
lookup
Revenue
Online
PWD Doc
managemernt
Generate map
thumbs
Published Data
(~reports~)
Geocoding
GIS web services
Land Record
Extraction
ULRS system
lookups
Address
standardization &
verification
Reverse
Geocoding
BRT
LNI
Hanssen
Water
Revenue
(Weekly)
ULRS
Other spatial data
DB
(e.g. police district boundaries, trash
(“address switchboard”)
pickup, …)
(monthly-ish)
MASTER_ADDRESS
MASTER_ADDRESS_ALL
PARCEL_ULRS
BRTREALESTATE_DORPARCEL
MASTER_ADDRESS_BRTPICT
….
(upon request)
….
ULRS access?
Exposed through various means
» Webservices + their clients
» Ready-made applications: AddressExtractor,
AddressExplorer
» bulk data reports in the “GIS_ULRS_DATA” schema
Web Services
• = developer technology
• Two flavors
– CityMaps Webservices
• “generic GIS” functionality:
• relies on ULRS for only two pieces: the address cleanup logic &
parcel-based geocoding
– ULRS WebServices
• Main window in the ULRS.
– Allows you to extract any information that is stored in the ULRS
• Provides simple read access to other db systems.
Web Services
• Who uses the webservices?
–
–
–
–
Around 25 clients from various departments: lni, police, revenue, dot, brt, ohcd, dhs, water, sherrif’s
office,…
2,000,000+ lookups against the web services per month
Supports web sites + batch processing
Majority of ULRS clients do not rely on “geocoding logic”
• mainly to resolve address input
• switchboard to retrieve account information based on address
(!!! after migration to ArcGIS Server, some functionality was lost!!! )
Existing Applications
Address Explorer
Replaces the ULRS Explorer – which was discontinued after June 30, 2009.
Why the rename?
- address is central – not the parcel, not the BRT#,…
- also, underline the fact that ULRS does not store any other information other than
relationships between addresses & parcels on the one hand, and addresses &
accounts on the other hand. (nothing more, nothing less)
Mimic three basic ways of retrieving information from the web services
- by address
- by parcel #
- by account # (topic key)
~corresponds to 3 ways of organizing the data
Existing Applications
Batch Geocoder/Address Extractor
-Allows you to process address based data
- Standardize; clean up an address(= most basic operation)
- Validate: clean up an address & check if imported by ULRS
- Geocode: clean up an address + find location
- Extract land records (~~~ topic pane from Address Explorer)
- Extract DOR parcel info: retrieve parcel identifiers by address
Data Reports
- bulk delivery of ulrs data under GIS_ULRS_DATA on geodb2 (on monthly
basis)
- Replaces ulrs publisher: low usage + data not always meaningful
GOALS
-organize data in meaningful way
-secure core applications
-flexible schema
- data added upon request: make sure that every report actually has a
user
- not recommended for application use – but not enforced either.
-SO FAR:
-ALIAS_ADDRESS
-MASTER_ADDRESS
-MASTER_ADDRESS_ALL
-MASTER_ADDRESS_BRT_PICT
-PARCEL_ULRS
-BRTREALESTATE_DORPARCEL
Exploring DATA: how it is generated
This Square
Legal Description and Survey Plan
Consistency - some legal descriptions do not indicate
streets and just use bearings where others do indicate the streets with
the bearings.
Understanding Bearings
Example- South 03 degrees 42 minutes 30 seconds East
S03º42’30”E
•
1: First it gives the rotation of the line with respect to the
geographical north in degrees, minutes and seconds. It is in the
middle part between the directions.
•
2: Second it gives the direction of the line from the starting point
to the end point in one of the four predefined ways - South to
East, South to West or North to East, North to West. These
directions are at the start & end.
Reading Bearings From a Legal Description
Bearing Definition
NE
NW
15º
55º
60º
20º
SW
SE
How to Define Bearings
N
LO = N 55º43’59” W
OL = S 55º43’59” E
bL =?
Lb =?
b
O
55º43’59”
15º21’45”
gL =?
Lg =?
L
W
g
E
20º46’12”
60º10’25”
W
S
How to Check Bearing of Parcel
Each Segment should have bearing
Angles in Bearings are never greater than 90º
Bearings are referenced from North or South
Four predefined ways - South to East,
South to West
North to East,
North to West
A Parcel Should have Bearings either in a clockwise or
Anticlockwise Direction
Minutes & Seconds should not be greater than 60º
bL = S 15º21’45” W
Lb = N 15º21’45” E
gL = N 60º10’25” W
Lg = S 60º10’25” E
Example: Parcel 61B
Missing Bearing
Either – 74.417’
Or – 74’5”
Two Bearing Segments Missing from Legal Description
Missing Segments Assumed At Right Angle
Bearing To
Be Verified
2nd Deed
2nd Shape in identified parcel
Final Remainder
This Square
Remainder Legal Description
•
ALL THAT CERTAIN lot or piece of ground Situate in the 6th Ward of the City of Philadelphia and described
as follows:
•
Beginning at a point of intersection of the Southeasterly side of 46th Street ( ) and the Southeasterly side
of Fairmount Avenue (60 feet wide);
•
Thence extending N86°17'30"E, along the Southeasterly side of Fairmount Avenue a distance of 243.417
feet;
•
Thence extending S03°42'30"E, along a line at right angles to said Fairmount Avenue a distance of
249.000 feet;
•
Thence extending S86°17'30"W, along a line parallel with said Fairmount Avenue, a distance of 243.417
feet to a point on the Southeasterly side of said 46th Street;
•
Thence extending N03°42'30"W, along the said Southeasterly side of said 46th Street, a distance of
249.000 feet to the point of beginning.
•
Containing 60,610.83 square feet or 1.3914 acres, more or less.
•
BEING KNOWN AS:?
The resulting shape in GIS layer
This Square
Next week’s Topic: The Front Door
We will discuss the challenges
and analysis that led to the
implementation of the ‘Front
Door’