PX-Web 2013
Petros Likidis, Statistics Sweden
Mikael Nordberg, Statistics Sweden
New features and improvements
•
•
•
•
•
•
User friendly urls
Better breadcrumbs
Clickable cells
Home page per database
New keywords
API
• Can be activated on individual databases
New keywords (CNMM 2.3)
• FIRST-PUBLISHED
• A date that indicates the date that the table was first
published
• DATANOTE
• A character that is show together with the table title,
variable name or the value to indicate that there is a
footnote connected to this item.
• META-ID
• A reference to a external metadata system. This
keyword could be set on the entire table a variable
or a value.
Scraping
Design of the API
• Build with the PC-Axis components and uses the
same “workflow”
• Browsing the database.
• Selecting table, variables and values.
• Retrieving a data cube.
• Data cube format.
•
•
•
•
“Standard” formats PX, CSV, Excel
PNG (experimental)
New JSON format (oData DSPL)
JSON-STAT
Addressing the API
Version Language
http://api.scb.se/OV0104/v1/doris/en/ssd/
Base address
Tribute database
Default is set to /api/v1
Browsing
[{"id":"BE","type":"l","text":"Population"},{"id":"FM","type":"l","text":"Financial markets"},{"id":"HE","type":"l","text":"Household
finances"},{"id":"LE","type":"l","text":"Living
conditions"},{"id":"MI","type":"l","text":"Environment"},{"id":"NV","type":"l","text":"Business
activities"},{"id":"PR","type":"l","text":"Prices and Consumption"},{"id":"AM","type":"l","text":"Labour
market"},{"id":"BO","type":"l","text":"Housing, construction and building"},{"id":"HA","type":"l","text":"Trade in goods and
services"},{"id":"JO","type":"l","text":"Agriculture, forestry and
fishery"},{"id":"ME","type":"l","text":"Democracy"},{"id":"NR","type":"l","text":"National accounts"},{"id":"OE","type":"l","text":"Public
finances"},{"id":"UF","type":"l","text":"Education and research"}]
{
"id": "BE",
"type": "l",
"text": "Population"
}
http://api.scb.se/OV0104/v1/doris/en/ssd/BE
Browsing
http://api.scb.se/OV0104/v1/doris/en/ssd/BE/BE0101/BE0101A
[{"id":"BefolkningNy","type":"t","text":"Population by region, marital status, age and sex. Year 1968 2012"},{"id":"FolkmangdNov","type":"t","text":"Population 1 November by region, age and sex. Year 2002 2012"},{"id":"FolkmangdSmaort","type":"t","text":"Population by smaller localities (places with 50-199 inhabitants). Every fifth year 1995 2010"},{"id":"FolkmangdTatort","type":"t","text":"Population by localities. Every fifth year 1990 2010"},{"id":"FolkmangdTatortH","type":"t","text":"Population by localities with older/changed names. Every fifth year 1960 - 1980"}]
{
"id": "BefolkningNy",
"type": "t",
"text": "Population by region, marital status, age and sex. Year 1968 - 2012"
}
Browsing
http://api.scb.se/OV0104/v1/doris/en/ssd/BE/BE0101/BE0101A/BefolkningNy
{"title":"Population by region, marital status, age, sex, observations and
period","variables":[{"code":"Region","text":"region","values":["00","01","0114","0115","0117","0120","0123","0125","0126","0127","0128","0136","0138","0139","0140","0160","0162","0163","0180","0181","0182","0183","0184","
0186","0187","0188","0191","0192","03","0305","0319","0330","0331","0360","0380","0381","0382","04","0428","0461","0480","0481","0482","0483","0484","0486","0488","05","0509","0512","0513","0560","0561","0562","0563","
0580","0581","0582","0583","0584","0586","06","0604","0617","0642","0643","0662","0665","0680","0682","0683","0684","0685","0686","0687","07","0760","0761","0763","0764","0765","0767","0780","0781","08","0821","0834","
0840","0860","0861","0862","0880","0881","0882","0883","0884","0885","09","0980","10","1060","1080","1081","1082","1083","12","1214","1230","1231","1233","1256","1257","1260","1261","1262","1263","1264","1265","1266","
1267","1270","1272","1273","1275","1276","1277","1278","1280","1281","1282","1283","1284","1285","1286","1287","1290","1291","1292","1293","13","1315","1380","1381","1382","1383","1384","14","1401","1402","1407","1415
","1419","1421","1427","1430","1435","1438","1439","1440","1441","1442","1443","1444","1445","1446","1447","1452","1460","1461","1462","1463","1465","1466","1470","1471","1472","1473","1480","1481","1482","1484","1485
","1486","1487","1488","1489","1490","1491","1492","1493","1494","1495","1496","1497","1498","1499","17","1715","1730","1737","1760","1761","1762","1763","1764","1765","1766","1780","1781","1782","1783","1784","1785","
18","1814","1860","1861","1862","1863","1864","1880","1881","1882","1883","1884","1885","19","1904","1907","1960","1961","1962","1980","1981","1982","1983","1984","20","2021","2023","2026","2029","2031","2034","2039","
2061","2062","2080","2081","2082","2083","2084","2085","21","2101","2104","2121","2132","2161","2180","2181","2182","2183","2184","22","2260","2262","2280","2281","2282","2283","2284","23","2303","2305","2309","2313","
2321","2326","2361","2380","24","2401","2403","2404","2409","2417","2418","2421","2422","2425","2460","2462","2463","2480","2481","2482","25","2505","2506","2510","2513","2514","2518","2521","2523","2560","2580","2581
","2582","2583","2584"],"valueTexts":["Sweden","Stockholm county","Upplands Väsby","Vallentuna","Österåker","Värmdö","Järfälla","Ekerö","Huddinge","Botkyrka","Salem","Haninge","Tyresö","UpplandsBro","Nykvarn","Täby","Danderyd","Sollentuna","Stockholm","Södertälje","Nacka","Sundbyberg","Solna","Lidingö","Vaxholm","Norrtälje","Sigtuna","Nynäshamn","Uppsala
county","Håbo","Älvkarleby","Knivsta","Heby","Tierp","Uppsala","Enköping","Östhammar","Södermanland county","Vingåker","Gnesta","Nyköping","Oxelösund","Flen","Katrineholm","Eskilstuna","Strängnäs","Trosa","Östergötland
county","Ödeshög","Ydre","Kinda","Boxholm","Åtvidaberg","Finspång","Valdemarsvik","Linköping","Norrköping","Söderköping","Motala","Vadstena","Mjölby","Jönköping
county","Aneby","Gnosjö","Mullsjö","Habo","Gislaved","Vaggeryd","Jönköping","Nässjö","Värnamo","Sävsjö","Vetlanda","Eksjö","Tranås","Kronoberg
county","Uppvidinge","Lessebo","Tingsryd","Alvesta","Älmhult","Markaryd","Växjö","Ljungby","Kalmar
county","Högsby","Torsås","Mörbylånga","Hultsfred","Mönsterås","Emmaboda","Kalmar","Nybro","Oskarshamn","Västervik","Vimmerby","Borgholm","Gotland county","Gotland","Blekinge
county","Olofström","Karlskrona","Ronneby","Karlshamn","Sölvesborg","Skåne county","Svalöv","Staffanstorp","Burlöv","Vellinge","Östra
Göinge","Örkelljunga","Bjuv","Kävlinge","Lomma","Svedala","Skurup","Sjöbo","Hörby","Höör","Tomelilla","Bromölla","Osby","Perstorp","Klippan","Åstorp","Båstad","Malmö","Lund","Landskrona","Helsingborg","Höganäs","Eslöv","Y
stad","Trelleborg","Kristianstad","Simrishamn","Ängelholm","Hässleholm","Halland county","Hylte","Halmstad","Laholm","Falkenberg","Varberg","Kungsbacka","Västra Götaland
county","Härryda","Partille","Öckerö","Stenungsund","Tjörn","Orust","Sotenäs","Munkedal","Tanum","DalsEd","Färgelanda","Ale","Lerum","Vårgårda","Bollebygd","Grästorp","Essunga","Karlsborg","Gullspång","Tranemo","Bengtsfors","Mellerud","Lilla
Edet","Mark","Svenljunga","Herrljunga","Vara","Götene","Tibro","Töreboda","Göteborg","Mölndal","Kungälv","Lysekil","Uddevalla","Strömstad","Vänersborg","Trollhättan","Alingsås","Borås","Ulricehamn","Åmål","Mariestad","Lidk
öping","Skara","Skövde","Hjo","Tidaholm","Falköping","Värmland
county","Kil","Eda","Torsby","Storfors","Hammarö","Munkfors","Forshaga","Grums","Årjäng","Sunne","Karlstad","Kristinehamn","Filipstad","Hagfors","Arvika","Säffle","Örebro
county","Lekeberg","Laxå","Hallsberg","Degerfors","Hällefors","Ljusnarsberg","Örebro","Kumla","Askersund","Karlskoga","Nora","Lindesberg","Västmanland
county","Skinnskatteberg","Surahammar","Kungsör","Hallstahammar","Norberg","Västerås","Sala","Fagersta","Köping","Arboga","Dalarna county","Vansbro","MalungSälen","Gagnef","Leksand","Rättvik","Orsa","Älvdalen","Smedjebacken","Mora","Falun","Borlänge","Säter","Hedemora","Avesta","Ludvika","Gävleborg
county","Ockelbo","Hofors","Ovanåker","Nordanstig","Ljusdal","Gävle","Sandviken","Söderhamn","Bollnäs","Hudiksvall","Västernorrland
county","Ånge","Timrå","Härnösand","Sundsvall","Kramfors","Sollefteå","Örnsköldsvik","Jämtland county","Ragunda","Bräcke","Krokom","Strömsund","Åre","Berg","Härjedalen","Östersund","Västerbotten
county","Nordmaling","Bjurholm","Vindeln","Robertsfors","Norsjö","Malå","Storuman","Sorsele","Dorotea","Vännäs","Vilhelmina","Åsele","Umeå","Lycksele","Skellefteå","Norrbotten
county","Arvidsjaur","Arjeplog","Jokkmokk","Överkalix","Kalix","Övertorneå","Pajala","Gällivare","Älvsbyn","Luleå","Piteå","Boden","Haparanda","Kiruna"]},{"code":"Civilstand","text":"marital
status","values":["OG","G","ÄNKL","SK"],"valueTexts":["single","married","widowers/widows","divorced"],"elimination":true},{"code":"Alder","text":"age","values":["0","1","2","3","4","5","6","7","8","9","10","11","12","13","14","15"
,"16","17","18","19","20","21","22","23","24","25","26","27","28","29","30","31","32","33","34","35","36","37","38","39","40","41","42","43","44","45","46","47","48","49","50","51","52","53","54","55","56","57","58","59","60","61",
"62","63","64","65","66","67","68","69","70","71","72","73","74","75","76","77","78","79","80","81","82","83","84","85","86","87","88","89","90","91","92","93","94","95","96","97","98","99","100+","tot"],"valueTexts":["0 years","1
year","2 years","3 years","4 years","5 years","6 years","7 years","8 years","9 years","10 years","11 years","12 years","13 years","14 years","15 years","16 years","17 years","18 years","19 years","20 years","21 years","22 years","23
years","24 years","25 years","26 years","27 years","28 years","29 years","30 years","31 years","32 years","33 years","34 years","35 years","36 years","37 years","38 years","39 years","40 years","41 years","42 years","43 years","44
years","45 years","46 years","47 years","48 years","49 years","50 years","51 years","52 years","53 years","54 years","55 years","56 years","57 years","58 years","59 years","60 years","61 years","62 years","63 years","64 years","65
years","66 years","67 years","68 years","69 years","70 years","71 years","72 years","73 years","74 years","75 years","76 years","77 years","78 years","79 years","80 years","81 years","82 years","83 years","84 years","85 years","86
years","87 years","88 years","89 years","90 years","91 years","92 years","93 years","94 years","95 years","96 years","97 years","98 years","99 years","100+
years","total"],"elimination":true},{"code":"Kon","text":"sex","values":["1","2"],"valueTexts":["men","women"],"elimination":true},{"code":"ContentsCode","text":"observations","values":["BE0101N1","BE0101N2"],"valueTexts":["Po
pulation","Population
growth"]},{"code":"Tid","text":"period","values":["1968","1969","1970","1971","1972","1973","1974","1975","1976","1977","1978","1979","1980","1981","1982","1983","1984","1985","1986","1987","1988","1989","1990","1991","1
992","1993","1994","1995","1996","1997","1998","1999","2000","2001","2002","2003","2004","2005","2006","2007","2008","2009","2010","2011","2012"],"valueTexts":["1968","1969","1970","1971","1972","1973","1974","1975","1
976","1977","1978","1979","1980","1981","1982","1983","1984","1985","1986","1987","1988","1989","1990","1991","1992","1993","1994","1995","1996","1997","1998","1999","2000","2001","2002","2003","2004","2005","2006","2
007","2008","2009","2010","2011","2012"],"time":true}]}
Selection
{
"query": [
{
"code": "ContentsCode",
"selection": {
"filter": "item",
"values": [
"BE0101N1"
]
}
},
{
"code": "Tid",
"selection": {
"filter": "top",
"values": [
"5",
]
}
}
],
"response": {
"format": "json",
}
}
Fetching the data table
• Do a HTTP POST request to the table URL
•
http://api.scb.se/OV0104/v1/doris/en/ssd/BE/BE0101/BE0101A/BefolkningNy
• Add the query as the content of the request
Limitations
• Number of cells that can be fetched => 10000
• Number of calls => 10 calls/10 second base on IP
address
Experience
at Statistics Sweden
Experience
• Slow adaptation
•
•
•
•
•
Still just a Beta version.
Lunched only through a Facebook group.
Some blog post.
One project on GitHub (Ruby Client).
Not that many support request.
Experience
• We really don’t know our users since you don’t
have to register use the API. But we log the users
IP address.
• Private persons in Sweden.
• Server hosting companies around the world.
• Companies in the financial sector.
20130620
20130624
20130624
20130627
20130629
20130704
20130706
20130708
20130712
20130718
20130723
20130727
20130731
20130807
20130809
20130811
20130813
20130815
20130817
20130819
20130821
20130823
20130825
20130827
20130829
20130831
20130902
20130904
20130906
20130908
20130910
20130912
API usage
100000
10000
1000
Metadata
100
Data
10
1
Usage