Registration and
Data Pilot
Progress Update
June 2014
Document control
Title
Registration and Data Pilot Progress Update
Author
Paul Fry / Tim Wagstaff
Date
13/06/2014
Version
1.0
Purpose of this paper
The purpose of this report is to provide an update to the OWML Board on the progress
and interim findings of the Registration and Data Pilot following the recent completion
of Iteration 1 of Stage 1.
This report is for information only.
2
Contents
1. Executive Summary
4
2. Introduction
5
3. Pilot Approach
5
4. Pilot Participants
6
5. Progress Against Plan
7
6. Headlines from the pilot
8
7. Pilot Company Experiences and Learning
9
Data Observations
9
Process observations
9
Resource and Effort
Appendix A
3
10
Error! Bookmark not defined.
1. Executive Summary
In March 2014 a prototype market dataset was created using two national dataset
(Ordnance Survey AddressBase Premium and the Valuation Office Business Rates).
This resulted in 1,652,147 potential contestable non-household premises being
identified. This market dataset was then divided into sub-sets for each pilot company
which together totaled 1,196,017 premises, representing 72% of the overall market
dataset.
Pilot companies received their datasets on 17th March 2014 with the aim to begin
matching the premises in the dataset to the premises on their own corporate systems.
The purpose of the matching process is to identify the 1Unique Property Reference
Number (UPRN) for every premise in the market and thereby creating a link between
the central systems and those of the companies.
On the 16th May data was returned to Open Water from pilot companies. Of the overall
data returned 43% of premises had been matched. This varied between companies
from 15% to 82% matched. It was recognised by Scottish market participants currently
working on the 3rd party references working group that this was to be expected and
consistent with their own experience.
A number of key learning points have been identified. The time required to understand
the data provided, how it links to the company’s own data and then beginning the
matching process has taken longer than water companies expected. Again, this is
consistent with the lessons learned from the implementation of competitive markets in
Energy (GB and Ireland) and Water in Scotland that activities associated with data
preparation and cleansing ahead of market opening took much longer and were more
involved than had been anticipated; this was one of the programmes key
considerations when seeking to establish the pilot at an early stage.
The opportunity to discuss data and systems with executive teams has also initiated
further discussion within companies about the future market and what their approach
will be for systems and data.
The market dataset created has identified a wider range of premises than companies
expected. A learning and exploration process of the companies own internal billing
definitions for premises of domestic and commercial usage rather than household and
non-household has made a number of companies clarify their understanding of the
market dataset.
The method used to identify non-households and produce the market dataset is
different to that of the price review methodology. This was identified by OFWAT in their
Cost Allocation Guidance in March 2014. By using the national datasets there will be a
variation between the volume of non-households in the market dataset and those in
company’s business plans. This difference in overall numbers and the variance by
company is not yet known at this stage of the pilot, but will become clearer through the
later stages.
UPRN is the reference used by Ordinance Survey to uniquely identify premises
across all of its addressing products for GB.
1
4
It is also recognised that the pilot process may enable companies to identify nonhousehold premises currently being served but which are not registered on their
systems. This may result in premises being registered and customers subsequently
billed for water and wastewater service they receive. The numbers of such premises
and the variance between companies is not yet known but will become clearly as the
pilot progresses.
The current stage of the pilot, which is informing the design around addressing and
eligibility, is in continuation with a further interim report being expected in July before a
final report in September.
2. Introduction
In July 2013 the High Level Group (HLG) endorsed the Open Water Programme’s
recommendation that a central register of the contestable market should be developed
to facilitate the future retail market. This register will be hosted by the market’s market
operator and will be made available to all market participants. The register will play a
key role in ensuring all market participants have a common understanding of what
makes up the contestable market and in enabling an efficient and transparent way for
customers to exercise their right to switch retailer.
The HLG agreed that to inform the development of this central register, the Open
Water Programme would work with the industry to develop and pilot a prototype
register. The pilot will enable the testing of the Programme’s proposal for developing
the central register, allowing for it to be refined and for the companies participating in
the pilot to share their lessons learnt for the benefit of all the industry as we move
towards full development and roll-out of a central register in advance of market opening.
This is the first of the interim findings reports following the successful completion of the
first iteration of stage one
3. Pilot Approach
Work to develop and pilot a prototype register will be staged, separating out particular
elements in order to deal with them in ‘bite size’ pieces. This will enable us to explore,
test and address issues in a managed way, ensuring we have a robust solution at each
stage before we move on to the next challenge. This approach will also better enable
the planning of resources for the companies working with us during the pilot and help
us to capture lessons learnt as early as possible for when we come to working will all
companies to develop the actual register that will be used for the new contestable
market.
As such the pilot is staged as follows:


5
Stage 1 will focus on establishing the register of eligible premises
Stage 2 will focus on mapping service point and metering data against the
identified eligible premises

Stage 3 will focus on determining how the register will process churn in the
data it comprises
Each of the stages of the pilot will comprise three phases:



Set-up – where the Programme will develop the data tools we are looking to
test and the questions we are looking for companies to explore as they work
with these tool.
Execution – where companies will take away the data tools and questions
provided by the Programme and test them against their own systems and
their understanding of the contestable market and network in their area of
appointment.
Findings – where the Programme and the companies involved in the pilot
will reconvene to discuss their findings and lessons learnt from the
execution stage, in particular drawing out specific challenges, replicable
quick-wins and an understanding of the type and quantity of resource
required for that element of the register’s development.
Stage 1 started by taking two national, independent datasets: Ordnance Survey’s
Address Base Premium and the Valuation Office Agency’s non-domestic rates dataset.
These two datasets were then combined to produce a prototype market dataset. The
market dataset was then split by water company area.
The activity for water companies in Stage 1 is to match the premises identified in the
market dataset to the premises held within their own systems. The purpose of the
matching process is to identify the 2Unique Property Reference Number (UPRN) for
every premise in the market enabling cross-referencing to data from other
organisations whilst maintaining market data integrity
The findings at each stage of the pilot will be reported to the OWML Board and
published on the Open Water Programme’s website, such that other companies that
are not directly involved in the pilot can see the progress being made and feed the
lessons learnt into their own future resource planning for the roll-out of the full central
register.
4. Pilot Participants
The list below identifies the companies that are participating in the Pilot:



Albion Water (NAV)
Anglian Water (WaSC)
Thames Water (WaSC)
UPRN is the reference used by Ordinance Survey to uniquely identify premises
across all of its addressing products for GB.
2
6




Northumbrian Water (WaSC
Severn Trent (WaSC)
Essex and Suffolk Water (WOC)
South Staffs Water (WOC)
The physical locations of the pilot companies will allow us to take account of and better
understand issues associated with:






Premises served by more than one incumbent
Incumbents covering two distinct geographies.
Operational area bordering with Scotland
Operational area bordering and entering into Wales.
WoC operational areas being enclosed by a WaSC or bordering with
multiple WaSCs.
Challenges posed by large urban areas in terms of multi-occupancy
premises.
5. Progress Against Plan
The Stage 1 Programme Plan is shown below.
Stage 1 of the Pilot is broken down into three iterations. Iteration 1 commenced in
March 2014 when each of the Pilot Companies were issued with a cut of the market
dataset for their area of supply along with file specifications and a questionnaire
designed to capture their feedback, experiences and learning from the execution of this
part the pilot. The focus for iteration 1 was for companies to understand the data,
begin the matching process and then report back on the findings and share there with
the pilot group.
The Questionnaire responses and additional data files were returned on the 16th May.
Analysis of the data was then completed between the 19th and 30th May.
7
A workshop was then held on the 5th June in London to gather together and share the
outcomes and the experiences gained from Iteration One. The workshop was a
success with details of methodologies used, issues arising and solutions found shared
openly with the group. It was attended by all of the Pilot Companies, Ordnance Survey,
OFWAT, Scottish Water, CMA Scotland, Business Stream, C&C Group (database
developers) and PwC (consultancy support).
The attendance from Scottish market participants provided additional support and
confirmation of shared experience together with lessons learnt from a marketplace
which is more mature and is still facing challenges associated with addressing and
market data. The opportunity to hear the Scottish experience also highlighted points of
focus and discussion for future iterations. The Scottish market participants also offered
further support at a technical level, sharing experiences between markets.
6. Headlines from the pilot
In iteration 1 1,196,017 premises were identified as being potential non-household
premises across the pilot companies areas of appointment. The pilot companies have
successfully mapped 509,706 (43%) of these so far and supplied the matching UPRN
and own premise references back to the Programme. These results represent a
positive start to establishing a register of eligible premises. The results achieved here
are similar to the experience in Scotland where a similar exercise is currently taking
place to map the Scottish Market Operators address data with the Scottish Assessors
Association (SAA) own records. The SAA is the Scottish equivalent of the VOA.
Iteration 1 has taken longer than initially planned as companies found that the time
needed to fully understand the structure and quality of their own data and how
relationships between the national data and their data would be formed took much
longer than anticipated. The additional time investment at this stage has enabled the
water companies to build facilities to achieve automated matching and deepen their
understanding of the requirements of the matching process.
All the water companies involved with the Pilot are engaging positively with the process
and working collaboratively together to share learning and improve matching solutions.
Learning has also been shared from the Scottish market participants.
The definition of eligibility is emerging as a key discussion point which will require
further focus and exploration in the subsequent two iterations.
Prior to the start of the pilot a number of companies thought the process of matching
addresses would be much easier and less time consuming than has proven to be the
case.
Participation in the Pilot has required some companies to discuss the data and system
requirements with their own executive teams. This is initiating discussions about the
implications of market opening from a broader perspective and is encouraging Senior
Management in each of the companies to give early consideration to plans for the
future set up and shape of their businesses.
8
Pilot Company Experiences and Learning
Data Observations
After the completion of Iteration 1 the total volume of premises matched to the pilot
dataset in this iteration is 509,706 which is 43% of the total.
Percentage matches range from 15% to 82% though in consultation with Scottish
colleagues these results can be viewed as typical for this stage of the matching
process.
The first cut of the Pilot data identified a greater number of premises than water
companies expected. This was due to the inclusion of buffering around border areas,
unserved premises in area and types of premises that historically may not have been
considered by companies to be non-household.
For every premise matched to the market dataset a flag for billed or unbilled was added.
The purpose behind this was to begin to identify unbilled premises from companies
systems. For 3 of the water companies the total unbilled was 588 representing 1.54%
of the matched premises. Companies identified a number of specific categories within
the unbilled definition. The first is where a premise is on a non-billed contract so the
company is aware of the premise. There are also arrangements in place with billing
completed on behalf of another water company. This additional level of detail accounts
for the larger volumes of unbilled premises returned by water companies. Any premise
served but unbilled outside of these cases will be explored further in Iteration Two.
The Pilot has enabled Companies to further explore eligibility definitions. A number of
premises classifications have been discussed including Beach Huts, Advertising
Hoardings and Automatic Teller Machines (ATMs). This discussion will continue in
Iteration Two as the matching process matures and the focus switches from addressing
to eligibility.
The data pilot uses Households and Non-households as identified in OFWATs
document “Water Supply licensing – guidance on eligibility”. This has generated
discussion and a deeper understanding of eligibility issues, how guidance may be
interpreted differently across companies and how the use of a single data source is
helping to bring a common understanding.
A key challenge for companies that is emerging is that the difference between a
Domestic or Commercial customer is not the same as the difference between a
Household and Non-Household customer. Whilst companies have appeared to
maintain records or the former (typically associated with Tariff data) the latter
classifications are not always known within companies own systems.
Process observations
A range of methodologies has been used by the companies. One company decided to
start with a specific part of their supply area, in order to understand the data provided
and to aid development of an automated matching system, then apply this to the whole
supply area. Other companies focused on a sub-set of their data which included all
commercial billed premises to narrow the initial matching query whilst others went for
an approach of matching it all from the outset.
9
A wide range of data fields were used to aid matching with variation across all the
companies, reflecting to some degree the different data structures within their own data
sets.
For this first iteration, companies have exclusively used automated methods for data
matching. Semi-automatic or manual processes have not yet been utilised to any
degree but will be explored in future iterations.
A number of water companies (quite appropriately) bill for a service on behalf of
another. Up to date data is therefore not always held by the original water company.
Data requests/negotiations are therefore required between water companies to enable
a full dataset of premises to be re-established. Further iterations will provide greater
understanding in this area but pilot companies are working closely together to resolve
these issues.
Resource and Effort
Resourcing of the Pilot has been successful for the majority of the companies within
Iteration 1 though a small number of the Pilot Companies have experienced some
difficulties obtaining resources when they were needed.
Access to data has proved difficult for one of the companies due to two factors. Data
Management is outsourced and the data set itself was of a significant volume. (Several
million records).
Most of the Companies believed that the matching process was going to be easy prior
to the Pilot but they have all revised that view. Further time was required for exploring
the sub-register, their data sets and how the two related. The additional time invested
has proved valuable as it has ensured that each company has a deeper understanding
of the data sets and is better placed to engage with the pilot during the later stages.
5. Conclusions from Iteration
One
The Pilot Companies have engaged enthusiastically with Iteration One and have
developed a range of approaches for the execution of the pilot. Common themes for
discussion have emerged such as the definition of eligibility, the approach to defining
gap sites and the challenges posed by legacy non-industry standard classifications of
data (especially where mergers have taken place).
The Pilot Companies have all now built systems and methodologies for conducting the
data matching and these will now be finessed further and re-used for the next iteration.
The initial effort and investment in building the data matching facilities for the pilot will
now be re-usable for the remainder of the pilot.
Iteration one should be regarded as a solid “first pass” where the degree of successful
matches has been moderate but nevertheless not out of alignment with success rates
in similar exercises undertaken in Scotland.
10
A key objective of the Pilot was to expose potential issues around the borders between
different regional and national geographies and this has been highlighted in the returns.
A spirit of collaboration and willingness to share learning was clearly in evidence at a
recent workshop involving all of the companies. Common themes and issues emerged
which are now helping each organisation to finesse their approaches in preparation for
Iteration Two and there is an expectation that with a combination of approaches which
are more refined as a result of Iteration One, together with the exploration and use of
semi-automatic and manual processes, there will be a further increase in the level of
successful matching achieved.
Participants are aware that there are differences between the premises dataset being
used for the Pilot (to match premises for the purposes of eligibility) and the customer
data used for the PR14 business plan and price control-setting. The Pilot datamatching process is also throwing up likely instances of unbilled/ improperly billed
premises. These differences ultimately flow through to issues around customer
numbers, cost allocation and tariffing, and the potential for incidence effects. These
issues have been discussed at previous Pilot company workshops, which have been
attended by Ofwat, but are outside the remit of the Pilot itself.
6. Iteration Two and next
steps
Iteration 2 which will run from 5th June to the 4th July will continue to focus on
establishing a prototype register of eligible premises. The activity in Iteration 2 will
therefore be to continue matching premises in the sub-register provided to pilot
companies by Open Water to their corporate systems.
The objectives for iteration 2 are to:
•
Clarify the accuracy of matches completed through the automated processes
used
•
Increase the level of matching through refining the automation process and
beginning to explore semi-automatic and manual matching processes.
•
Continue the conversation on eligibility through the development of scenarios
with specific examples.
•
Continue to share learning and findings between pilot companies and the wider
industry.
11