Authoring an ontology of place
semantics using volunteered
geographic information
Alistair Edwardes and Ross Purves
Department of Geography
University of Zurich
1
Overview
• Motivation for considering place
• Why this is useful in the context of image
retrieval
• Where can we find place descriptions
• How might we build semantic resources
from these
2
Motivation
• GI bias towards spatial representations of
Geography
– BUT
• Not all geographic information is spatial
3
Motivation
4
Motivation
• GI bias towards spatial representations of
Geography
– BUT
• Not all geographic information is spatial
• Doesn’t reflect how people experience, remember
and talk about geography
• What else is there?
5
Place
• Space-Place Continuum
– Objective-Subjective
– Universal-Personal
– Machine-Human
6
Where is place important?
•
Location-Based Services
– Movement towards technologies closer to everyday, direct
experience, activity based.
“The historical demarcation in psychological and behavioural geography between
direct and indirect experience blurs when handheld devices are used as an adjunct
to reality in the field.” (Longley, 2004)
•
Web 2.0
– Social interaction, user generated information, personal memories
7
Where is place important?
[Baostar] documents place in a way that embodies neogeography,
where human perspective and social interaction supercede
latitude and longitude.
8
Where is place important?
•
Location-Based Services
– Movement towards technologies closer to everyday, direct
experience, activity based.
“The historical demarcation in psychological and behavioural geography between
direct and indirect experience blurs when handheld devices are used as an adjunct
to reality in the field.” (Longley, 2004)
•
Web 2.0
– Social interaction, user generated information, personal memories
•
Geographic Information Retrieval
– Vernacular geography, organising activities
•
Photographs
“GI theory articulates the idea of absolute Euclidean spaces quite well, but the
socially-produced and continuously changing notion of place has to date proved
elusive to digital description except, perhaps, through photography and film.”
(Fisher and Unwin, 2006)
9
Place and photographs
• Observer/Viewpoint
– Different from universal perspective of maps
• Information is perceptual
– Closer to direct experience
– Pre-cognitive
– Many ways to interpret
• Highly ephemeral
– moment in time
10
Problems
•
Many such moments in space and time
– How do we sort through them?
– Image Retrieval
1. How do we access a description of the contents
of an image?
2. What do we describe about an image?
11
Image Retrieval Approaches
(CBIR)
• Content Based
Image Retrieval
• “Natural” for format
– Use primitive
features like colour,
shape and texture
Smeulders et al, 2000
12
The Semantic Gap
•
Concept based image retrieval
– Define high-level semantic concepts
•
Defined in loosely structured word lists (LSCOM)
“The
gapusing
is the lack
of coincidence
between vectors
the information that
– semantic
Detect
low-level
feature
one can extract from the visual data and the interpretation that the same
data have for a user in a given situation.” (Smeulders et al, 2000)
13
Image Retrieval Approaches (TBIR)
•
Text-based Image Retrieval
– Describe contents in text
1. How do you access this description?
2. What should be described?
14
How do you access a description?
• Manual annotate
– Expensive and time consuming
• Definitively won’t scale
• Need to automate
– Inconsistency amongst annotators (Markey, 1984)
• Inter-annotator agreement (e.g. Ahn and Dabbish, 2004)
• Controlled Vocabulary
– Getty Images 12,000 keywords with 45,000 synonyms
(Bjarnestam, 1998)
– Specialist knowledge
15
Tripod Approach
• Describe location instead
– Spatial data
– Geographic knowledge
– Web resources
16
Obligatory Project Slide
•
•
•
•
European Commission Sixth Framework Programme Project
3 years (started January 2007)
€3,150,000
Partners
–
–
–
–
–
–
–
–
–
–
•
•
University of Sheffield, United Kingdom
University of Zurich, Switzerland
Dublin City University, Ireland
Otto-Friedrich-Universität, Bamberg, Germany
Cardiff University, United Kingdom
Ordnance Survey, United Kingdom
Centrica, Italy
Geodan, The Netherlands
Fratelli Alinari Istituto Edizioni Artistiche, Italy
Tilde, Latvia
Focus on image retrieval by users of professional stock photo libraries
Focus on particularly geographic images
– e.g. Natural landscapes
17
What should be described?
Modes
Facets
Specific Of
Generic Of
About
Individually named
Kinds of persons,
persons, animals, things animals, things
Mythical beings,
abstraction manifested
or symbolised by
objects or beings
What?
Individually named
events
Actions, conditions
Emotions, Abstractions
manifested by actions
Where?
Individually named
geographic locations
Kind of place
geographic or
architectural
Places symbolised,
abstractions manifest
by locale
When?
Linear time; dates or
periods
Cyclical time; seasons,
time of day
Emotions or abstraction
symbolised by or
manifest by
Who?
18
Panofsky-Shatford facet matrix – Shatford (1986)
What should be described?
GenericOf: Engineering
About: Innovation, technical brilliance, complexity
SpecificOf: [Da Vinci Chambord staircase]
19
What should be described?
• Advertising
– Mystery
– Isolation
– Chocolate
20
Dimensions of Place
• Theoretical Dimensions of Place
– Physical setting, activities, meanings Relph (1976)
– location, material form, investment in meaning
Gieryn (2000)
– location (spatial distribution activities), locale
(the setting), sense of place Agnew (1987)
• Similar to Shatford
– SpecificOf – Location/Identity
– GenericOf – Setting
– About – Sense of place, meanings, activities
21
Organisation in Tripod
• Concept Ontology
– GenericOf
• Scene types
• Elements
– About
• Sense of Place
– Affective, Cognitive, Conative
• Qualities and Activities
• Toponym ontology
– SpecificOf
• Identity, location
22
How can we elicit place
descriptions?
• Inductively
– Ask people
• Adjective Check List
• Category Norms / Basic Levels
– e.g. Mountains, Parks, Beaches, Cities
– Attributes, Activities, Parts
– Pick terms from a dictionary and validate
– Code unstructured domain knowledge
– Data mine web resources
• Deductively
– Look at structured semantic resources
• Use a combination of these approaches
23
Empirical Elicitation
Free description
• Online interactive experiments
• Database of 150 landscape
photographs from Switzerland,
Germany, Holland, Italy, Portugal
and the UK.
Controlled vocabulary
24
Sort and describe
Overall ∩ Elements
landscape village valley countryside wall field top
sunlight hills cliffs castle peak fortress narrow birch
town plain land hillside lawn farm creek orchard
harbour fall rocks fields flowers buildings ridge
bushes woods track stone ruins meadows
undergrowth dunes deciduous cross coast brush
wooden weather walls villa monument lane city
waterfall vineyard valleys tundra trail terraced
temple sunrise station ship shelter rolling riverbank
pylons peninsula pathway park outcrop munros
loch lightly historical hilltop ground gardens foot
flood dock cycle croft covered catwalk cargo
canopy canal burned brook boulder bay arm angler
altitude acre
Overall ∩ Qualities
mediterranean summer quiet winter hilly calm
isolated rural steep open mediteranian cold hot
blurry windy free spring remote lonely clean
dangerous cool tranquil arid well rugged rough
picturesque dark vista urban snowy sandy
rotting northern medieval intervention home
hike fertile end dusk back
Overall
wooded country surrounded scene mountainous low dense coastline
young world wonderland wintry waterside variation uniform two twilight tuscany tide
three sunlit suburban structure streaming snowland sicily sedimented seaside
seacoast savanna ruined route romantic rollign roadside riverside riverbed rising
rigi region reach ranch prospect populated pond plateau photographed peeking
pastures passage overgrown outback november mountaintops moor montains mist
mediterran lonley lodge leafy lakeside junction heaven hanging gurgling greek
france foreground following floor featureless falling dive development dawn
countour copse cliif babbling area alice
Overall ∩ Activities
running rowing touring biking looking enjoying bycicle parking live estate
25
Subjectivity
27
How can we elicit place
descriptions?
• Inductively
– Ask people
• Adjective Check List
• Category Norms / Basic Levels
– e.g. Mountains, Parks, Beaches, Cities
– Attributes, Activities, Parts
– Pick terms from a dictionary and validate
– Code unstructured domain knowledge
– Data mine web resources
• Deductively
– Look at structured semantic resources
• Use a combination of these approaches
28
Volunteered Resources
Gatliff Trust Hostel on Berneray: Picture taken from the
beach on Berneray of the historic Gatliff Trust Hostel. Visited in
the 1990s, shortly before the causeway linking Berneray to
North Uist was built.
29
Validation of terms
www.geograph.org.uk
Examine frequency and cooccurrence of scene types
and terms with respect to a
database of image captions
Word lists
(WordNet)
Scene types
30
Scene Type Descriptions
Activities
Beach
Surfing
Bathing
Defence
Swimming
Tourism
Wading
Protection
Sport
Shipping
Golf
Elements
Shingle
Sand
Cliff
Headland
Bay
Sea
Rock
Coast
Shore
Island
Sandy
Deserted
Eroded
Soft
Rocky
Warm
Glacial
Low
Beautiful
Lovely
Conservation
Reading
Fishing
Playground
Defence
Bowling
Tourism
Football
Entertainment
Sitting
Elements
Pub
Shop
Inn
Church
Housing
Edge
Cottage
Main Road
Village green
Stone
n=16232 Mountain
Hill
Climbing
Skiing
Holidays
Observation
Sitting
Walking
Running
Cycling
Preservation
Escape
Adjectives
Activities
n=2824 Village
Fort
Top
Summit
Horizon
Ridge
Sheep
Valley
Side
Trees
Track
Steep
Distant
Wooded
Black
Rough
Grassy
Round
Big
White
Broad
Biking
Kayaking
Outing
Mountaineering
Escape
Walks
Fun
Racing
Climbing
Cycling
Adjectives
n=12707
Deserted
Pretty
Green
Quiet
Lovely
Pleasant
Beautiful
Remote
Unusual
Large
n=1256
Peak
Summit
Ridge
Moorland
Quarry
Stream
Sheep
Forest
Top
Path
Distant
Black
Remote
Rocky
Grassy
Steep
Natural
Dark
Broad
Running
31
How can we elicit place
descriptions?
• Inductively
– Ask people
• Adjective Check List
• Category Norms / Basic Levels
– e.g. Mountains, Parks, Beaches, Cities
– Attributes, Activities, Parts
– Pick terms from a dictionary and validate
– Code unstructured domain knowledge
– Data mine web resources
• Deductively
– Look at structured semantic resources
• Use a combination of these approaches
32
Data Mining
• Analysed Nouns (those used >100 in captions)
– Aim
• Identify similar element concepts (equivalence relationships)
– Analyse noun co-occurrence with the landscape adjectives of Craik
• Identify related element concepts (associative relationships)
– Analyse noun-noun co-occurrence
– Methodology (vector space analysis)
•
•
•
•
•
Identify a list of nouns (inter-annotator agreement)
Form co-occurrence vectors for each noun with nouns or adjectives
Remove insignificant occurrences (tested with chi-squared p>0.01)
Filter out vectors with few occurrences (<3)
Analyse (cosine) similarity between idf-weighted co-occurrence
vectors
• Visualise using hierarchical clustering and multi-dimensional
scaling
– Throw out largest cluster (noise)
33
Element Similarities
Landforms
Headland, outcrop, coastline,
knoll, shoreline, promontory,
outcrops, foreshore
hill, bank, slopes, hillside,
slope, cliffs, banks, crag,
crags, incline, descent, fall,
ascent, coombe, gradient
Valley(s),
ravine, gorge
Land cover
Gully, channel(s), cutting
ditch(es), holes, pool, ford
shaft
land, forest, fields,
farmland, moor, moorland,
countryside, heathland,
grassland, downland
Areas, block, granite,
blocks, shed, boulder(s),
expanse, slab(s), pieces
34
Water
Related Elements
Built Environment
Church, tower, clock,
nave, porch, font, aisle
House, hall, stone, wall, home,
manor, brick, grounds, roof, walls,
door, structure, floor, parts,
window(s), period, glass, mansion,
lady, storey, gable(s), architect,
doorway, moat, material, façade,
wing, doors, bricks, materials
rubble, columns, foundations, keys,
wings, village, entrance, castle,
pub, cottages, inn, avenue
Bridge, river(s), water(s), bank, burn, stream(s),
brook, footbridge, dam, ford, pool, flood, banks,
drain, plain, weir, tributary, fish, source, waterfall,
bed, meadow(s), levels, gorge, fen, aqueduct,
sewage confluence, riverside, reservoir(s), pipe,
sluice, salmon, pools, meanders, trout, floods,
waterfalls, springs, anglers, channels, table,
fishery, outflow, watercourse, wharfe, otter, dike,
floodplain, watershed,
Transport
Nature
Trees, edge, wood, forest,
woodland, plantation, forestry,
oak, beech, inclosure, birch,
pine, heathland, ponies,
conifers, pines, plantations,
holly, oaks, lawn, conifer,
pony, spruce, sitka, larch,
commoners
railway(s), line, station,
view, farm, lane, hill, footpath,
train(s), branch, viaduct,
valley, farmland, bridleway,
cutting, embankment, rail(s),
hedge, heath, hillside, horse,
stations, trackbed, goods,
stile, walkers, copse, leaves,
passenger(s), terminus,
chalk, picnic, warren, cyclists,
mainline, gauge, platform(s),
spinney, trails
locomotive, overbridge, Road, way, track, junction(s), route(s), section, access,
sidings, freight, diesel
crossing, course, traffic, direction, mile, pass, roundabout,
35
motorway, camera, links, yards, bypass, carriageway,
network, lights, pedestrian(s), loop, barrier, flyover
Flickr
Work with Christian Matyas
of Bamberg University
Weather
Sun, bluesky, cloud, clouds, dusk,
horizon, sky, sunshine, himmel
insel, meer, sonne, strand
sonnenuntergang, wasser, wolken
Beaches
atlantic, beach, coast, ocean,
pacific, pier, plage, playa, sand,
sea, seagull, seaside, shore,
surf, wave(s), water
Aviation
aeroplane , aircraft, airline, airplane,
airport, aviation, boeing, flughafen,
flugzeug, plane, air, apache, flight,
helicopter
Street Art
cityart, graffiti, graffitiart, graffitti,
grafitti, graphiti, stencil, streetart,
urbanart, paint, spray, stickers,
street, wall(s), mural
Nature
barn, bench, countryside, environment, farm, fence,
field, flood, fog, grass, meadow, mist, moss, mud,
parks, path, pine, rain, rural, storm, weather, wiese,
wood, baum, fall, forest, leaf, leaves, tree(s), woodland,
woods
butterfly, insect(s), insectes,
landscape, landschaft, natur, nature,
scenery, bloom, blossom, cherry, flora,
flower, flowers, garden, gardens,
orchid, plant, plants, rose, wildflowers
Built environment
countycourthouse, county
courthouse, courthouses,
cityhall, capitolbuilding,
texascountycourthouses,
texascourthouses, court,
building(s), architecture
alley, billboard, brick(s),
castle, cathedral, centre,
centreville, chimney,
church, clock, county,
door, entrance, façade,
glass, houses, interior,
metro, roof schloss, shop,
sign(s), stairs, steps, store,
city, cityscape, downtown, streets, suburb, subway,
court, skyline, skyscraper, tower(s), town,
urban, capitalcity, innercity underground, ville,
36wall,
innerlondon, capitol
window(s), wires
Develop Taxonomies
Actual
Nouns
Concept
categories
Concept
s
Land cover
Agricultural land
crops
farmland
agriculture
Arable land
field
fields
wheat
…
Forest
Landforms
Topographic eminences
plantation
wood
woods
…
Mountains
beinn
mountain
sgurr
....
Hills
hill
down
cnoc
....
Taxonomy groups
37
Conclusions
• Place
– Facets
– Importance to geographical semantics
• Eliciting place
• Volunteer sources
– Usefulness
– Potential biases
• Left open
– Infrastructure
– Concept detection
38
Acknowledgements
• We would like to gratefully acknowledge
contributors to Geograph British Isles, see
http://www.geograph.org.uk/credits/2007-02-24,
whose work is made available under the
following Creative Commons AttributionShareAlike 2.5 Licence
(http://creativecommons.org/licenses/by-sa/2.5/).
• This research reported in this paper is part of the
project TRIPOD supported by the European
Commission under contract 045335.
39