Innovations in data collection, data dissemination, data access and data analytics “Modernisation: Evolution or revolution” Pádraig Dalton, John Dunne & Donal Kelly Global Conference on a Transformative Agenda for Official Statistics New York 15-16 January 2015 What am I going to talk about? Introductions Context for the paper Environmental issues Modernisation challenges Highlight questions What am I hoping for? Open discussion Amongst colleagues Inherited our organisations and systems Trying to wear a global hat! Provocative and perhaps challenging Lots of questions – fewer answers? Consider the power of “One Voice” Modernisation – Broad perspective Expectation may be technology, Big Data, CSPA Context and audience led to broadening of definition of modernisation and relativity Asking questions to tap into experience at conference and develop understanding Environmental context Only constant - change Pace of change Increased user demand Emergence of global user Digital age – technology, sources, strategic alliances? Skills: Re-profiling, outsource? Availability of skills? Environmental context - continued Politicisation – crisis, targets and indicators Resources – are we too passive, a consistent message? Urgency some areas – revolution? Other areas will take more time – evolution? How do we decide – links to collaboration? Modernisation Maturity Model? Modernisation Maturity Model (MMM) Just an idea – has it a role? Strategy Position Govern • Understand national & international directions & factors • Determine organizational vision & values • Determine organizational value proposition • Determine organizational goals • Communicate values & expectations Influence & collaborate • Develop strategies for achieving organizational goals • Prioritize statistical portfolio • Prioritize capability portfolio • Allocate portfolio & programme budgets • Build & maintain internal statistical & professional excellence • Build & maintain strategic relations, nationally & internationally • Build & maintain external statistical excellence • Advance inter-agency & international collaborations • Secure support for statistical & capability portfolio Capability Plan capability improvements • Identify 'disruptive' & other capability improvements • Propose capability improvement projects, including shared infrastructure • Manage capability improvement programmes Develop capability improvements • Undertake background research • Develop detailed capability requirements • Design capability solution • Build & release capability solution, including shared infrastructure • Manage capability development project Manage capabilities Corporate support Support capability implementation • Maintain capabilities, including • Support design shared infrastructure • Support operations • Promote capabilities • Support use externally • Evaluate capabilities • Manage data, metadata and process data • Manage quality Manage business & performance • Manage business performance • Manage change and risk • Manage legislation & compliance • Manage physical assets, including building facilities Manage finances • Maintain accounts (including assets & liabilities) • Manage procurement & contracts Manage human resources Manage IT • Manage employee • Manage IT services performance • Manage IT & • Manage & develop information security skills • Manage talent • Manage recruitment • Ensure succession planning Manage information & knowledge Manage consumers & suppliers • Manage documents & records • Manage knowledge • Manage information standards & rights • Manage communications and media relations • Manage stakeholder consulations • Educate users and improve statistical literacy • Manage cross-product user support Modernisation Challenges Institutional setting Without appropriate institutional settings for NSIs, despite having numbers, do we have robust, trusted, reliable and internationally comparable data? Statistical Framework Pillars Data Protection Freedom of Information (FOI) Statistical legislation Statistical legislation the starting point? Long-standing problem outside power of NSIs – but can we influence? Without Data Protection and FOI can we have an effective statistical framework? Secondary data sources Variety of data sources Hot topic is Big Data but……. Administrative data - Administrative data also evolving - Access remains an issue – Why? - Value proven where in use - Have we a communications problem to address? Have we a role in educating custodians on why and value? Big Data Must continue our work in this area Are considerable opportunities and challenges Mobile phone data, smart metering, traffic loop data Need to move from potential to actual delivery Are challenges/questions - Conflating volume with insight - Sustainability of data sources over time - Availability of skill sets - Technology challenges - Privacy and data security concerns exist Big Data Not necessarily better than traditional sources but there is value Is it over-hyped at this point and are we managing expectations? Those that can should push ahead but …. Coherent output(s) to maximise value of investment Create a road-map for others to follow Privacy & Data Protection Are a range of challenges - Linking secondary and primary data sources - Implication of strategic alliances - Privacy and efficiency debate Privacy & Data Protection How do we retain trust in such an environment? Do we have a role in surfacing the issues and bringing a balance to the debate? Beyond legal issues – cultural also? Communications challenge? One Voice Technology Proliferation of mobile technology Web services Web scraping Visualisation Pace of change is incredible NSIs must share experiences (and tools) – collaboration Technology With technology and “systems development tools” changing so quickly in what skills etc. should NSIs invest? Public-private partnerships – if so in what areas? What is the appropriate model for these partnerships? Should we “club” together to increase influence/negotiating power? Could or should we pool resources? Standards based modernisation A lot of work ongoing UNECE HLG MOS – GSBPM, GSIM, GAMSO, CSPA DDI, SDMX Stability/maturity of standards - Is there development for development sake in some areas? Standards based modernisation Is there development for development sake in some areas? Does it mitigate against implementation? Proliferation of players (ownership and maintenance)– is it a challenge? How can we collaborate and influence the development and maintenance of modernisation relevant standards? Communications Already referenced - resources and privacy Quality-timeliness debate Race to the bottom? Must improve timeliness but trade-off limit? Are we clear about where we want to position official statistics – our brand? Are we all agreed on our competitive advantage(s) and unique selling point? Communications continued Each of us strike our own balance or One Voice? How can we progress these communications challenges? What will be the most effective approach? Conclusions Couldn’t address all of the modernisation issues Focussed on what we thought were the big strategic issues Lots of questions – fewer answers!! Broader definition of modernisation required Is an urgency to modernise in many areas Reality - are also modernisation challenges that will take time WHO, WHAT, WHERE and WHEN? MMM to help? Conclusions Collaboration is key – easy to say, difficult to do effectively Are a lot of modernisation programmes underway at both national and international level What are the most effective and efficient mechanisms, and divisions of labour, to support the transformative agenda?