Drainage Data Formats Introduction This guidance note should be read in conjunction with Interim Advice Note (IAN) 147/12.1 It describes how to produce drainage data suitable for inclusion on HADDMS. It fully supersedes all previous advice on drainage data formats, including that given in the “draft drainage IAN on HADDMS data population”. It is generally backwards compatible, such that data currently being prepared to this previous guidance does not necessarily need to be changed. This note is written in terms of Highways Agency practice in England, and with application to the HADDMS; however, the formats and procedures generally apply for application in Wales for the WADDMS. Feedback or questions on this guidance note are welcomed at support@haddms.com. This guidance may be periodically updated; the most recent version will be available from the Downloads page on the Help menus of HADDMS and WADDMS. Version history Issue 0 – November 2011 • Preliminary issue to Service Providers to coincide with implementation of data format changes on HADDMS. Appendix E was not included in this issue of the document. Issue 1 – August 2012 • Issued to coincide with publication of IAN 147/12. No further changes have been made to HADDMS or are planned, as these were launched in November 2011. • Only change to format/codes/etc: Video image storage code (Table B-21) for media files corrected from “MPG” to “HDD”. • Minor clarifications to text particularly regarding CCTV data submission procedures. • Addition of Appendix E for submission of drawing sets. This is an update of Appendix C of the “draft drainage IAN on HADDMS data population” relating to preparation and submission of located and georeferenced drawings. Drainage schemes All drainage asset data on HADDMS is grouped within a “drainage scheme”, which is a selfcontained set of connected drainage assets. Historically, a drainage scheme has been related to data originating from a set of as-built drawings or a survey; however, drainage schemes should be defined as required. Where possible drainage schemes should contain all drainage assets within their extents, and overlap and duplication of assets between schemes should be avoided. All physically connected drainage assets within a catchment should be included in the same scheme. 1 IAN 147/12, “Drainage surveys and data”, August 2012 Guidance note on drainage data formats, Issue 1. August 2012 1 of 93 Index tools and example datasets, as referred to in the respective appendices, are available for download from HADDMS via the “Help > Downloads” menu item. Drainage inventory and condition asset data There are three different formats of drainage asset data acceptable for HADDMS as defined below. Each is described in detail in the appropriate appendix of this document. • Digital inventory data, in shapefile format. This may be derived from as built drawings or new field surveys. The required file format, checking criteria and procedure for upload to HADDMS is defined in Appendix A. Codes to be used in “Code” fields are listed in Appendix B. This data is the format resulting from validation surveys2 and soakaway/outfall surveys.3 • Digital inventory and condition data, in shapefile format. This format is an optional extension of the above and is defined in Appendix A. Additional codes covering defect-level condition are in Appendix C. This data is the format resulting from connectivity surveys.4 • Digital inventory and condition data, from CCTV surveys in DraingeXML format. Pipework inventory and condition data surveyed by CCTV in accordance with SD15/03 can be submitted for upload to HADDMS, provided it is in the WinCan format specified in Appendix D. This document does not describe the XML format in detail, but covers how it is to be produced and submitted or uploaded. Drainage drawings Scanned or other electronic drainage drawings are grouped within a “drawing set” on HADDMS. This would normally comprise all drainage-related drawings from a single construction or maintenance scheme. In some cases, a drawing set may correspond with a drainage scheme. • Located files are typically scanned whole drawings in TIFF or PDF format, linked from the HADDMS map by a rectangle defining their approximate location in relation to the HA network. • Georeferenced drawings are solely the map elements of the drawings (also provided in their entirety as located files) provided in GeoTIFF format, and are displayed aligned with the other background mapping layers on HADDMS. This is described in full in Appendix E. List of Contents Drainage Data Formats ......................................................................................................... 1 Appendix A. HADDMS Digital Inventory and Condition Data Specification ......................... 5 Appendix B. Inventory Code Tables...................................................................................30 Appendix C. Defect-level assessment of drainage asset condition.....................................44 2 Guidance Note on Drainage Validation Surveys 3 Guidance Note on Drainage Outfall and Soakaway Surveys 4 Guidance Note on Drainage Connectivity Surveys Guidance note on drainage data formats, Issue 1. August 2012 2 of 93 Appendix D. HADDMS Piped Drainage CCTV Survey Records.........................................79 Appendix E. HADDMS Drawing Records Specification......................................................84 List of Tables Table A-1: Point item (point.dbf) data format ......................................................................... 7 Table A-2: Continuous item (continuous.dbf) data format .....................................................13 Table A-3: Region item (region.dbf) data format ...................................................................17 Table A-4: scheme.dbf data format ......................................................................................20 Table A-5: linkdata.dbf data format.......................................................................................21 Table A-6: Observation (observation.dbf) data format ..........................................................22 Table B-1: Data source codes (source of original asset record)............................................30 Table B-2: Item type codes for point inventory items ............................................................31 Table B-3: Item type codes for continuous inventory items...................................................32 Table B-4: Item type codes for region inventory items ..........................................................33 Table B-5: Shape codes for continuous inventory items .......................................................33 Table B-6: Material and lining material codes .......................................................................34 Table B-7: Surface material codes for continuous inventory items........................................35 Table B-8: System type codes..............................................................................................36 Table B-9: Owner codes.......................................................................................................36 Table B-10: Cover shape and chamber shape codes for point inventory items.....................36 Table B-11: Cover duty codes for point inventory items........................................................37 Table B-12: Manhole construction type codes for point inventory items................................37 Table B-13: Lining type codes ..............................................................................................38 Table B-14: Location type codes for point and continuous inventory items...........................39 Table B-15: Strategic drain codes for point and continuous inventory items .........................40 Table B-16: Purpose of inspection codes .............................................................................40 Table B-17: Flow control measure codes .............................................................................41 Table B-18: Precipitation codes............................................................................................41 Table B-19: Temperature codes ...........................................................................................41 Table B-20: Method of inspection codes...............................................................................42 Table B-21: Video image storage media codes ....................................................................42 Table B-22: Video image timing reference system codes .....................................................42 Table B-23: Photograph storage system codes ....................................................................43 Table B-24: Field validation status codes for point inventory items.......................................43 Table C-1: Inspection related observation codes..................................................................47 Table C-2: Defect-level structural condition assessment of pipes .........................................49 Guidance note on drainage data formats, Issue 1. August 2012 3 of 93 Table C-3: Defect-level service condition assessment of pipes ............................................52 Table C-4: Inventory observations for pipe surveys ..............................................................55 Table C-5: Defect-level structural condition assessment of ditches ......................................57 Table C-6: Defect-level service condition assessment of ditches..........................................58 Table C-7: Defect-level structural condition assessment of enclosed channels ....................59 Table C-8: Defect-level service condition assessment of enclosed channels........................59 Table C-9: Defect-level structural condition assessment of open channels...........................61 Table C-10: Defect-level service condition assessment of open channels............................61 Table C-11: Defect-level structural condition assessment of grassed channels....................62 Table C-12: Defect-level service condition assessment of grassed channels .......................62 Table C-13: Defect-level structural condition assessment of filter drains ..............................64 Table C-14: Defect-level service condition assessment of filter drains..................................64 Table C-15: Defect-level structural condition assessment of chambers ................................66 Table C-16: Defect-level service condition assessment of chambers ...................................69 Table C-17: Defect-level structural condition assessment of gullies .....................................71 Table C-18: Defect-level service condition assessment of gullies.........................................72 Table C-19: Defect-level structural condition assessment of inlets/outlets............................73 Table C-20: Defect-level service condition assessment of inlets/outlets ...............................74 Table C-21: Defect-level structural condition assessment of miscellaneous assets ..............75 Table C-22: Defect-level service condition assessment of miscellaneous assets .................75 Table C-23: Defect-level structural condition assessment of region items ............................77 Table C-24: Defect-level service condition assessment of region items................................78 List of Figures Figure E-1: Display of Located Files on HADDMS................................................................85 Figure E-2: Display of georeferenced drawings on HADDMS ...............................................85 Figure E-3: A Located File with one mapping segment.........................................................91 Figure E-4: A Located File with two mapping segments .......................................................92 Guidance note on drainage data formats, Issue 1. August 2012 4 of 93 Appendix A. HADDMS Digital Inventory and Condition Data Specification A1. Introduction A1.1 This Specification defines a file format for uploading drainage inventory and condition data to HADDMS in general accordance with HD43 and SD15. A2. File format A2.1 The drainage inventory model can originate from the manual digitising of as built drawings (either scanned paper or CAD drawings), or from field CCTV and/or validation surveys. A2.2 Regardless of the origin of the data, drainage inventory data must be supplied in shapefile format as specified by ESRI (White Paper of July 1998 – http://www.esri.com/library/whitepapers/pdfs/shapefile.pdf). A shapefile comprises three mandatory files: the geometry (.SHP file), database (.DBF file) and index (.SHX file). A2.3 For each scheme, the point, continuous and region items are provided in separate shapefiles of the following formats: • point items are represented as GIS point features • continuous items are represented as GIS polyline features, snapped to an “upstream” point item at one end, and a “downstream” point item at the other end • region items are represented as GIS polygon features, with a point item at their centroid A2.4 A shapefile for the point items is mandatory. Shapefiles for continuous and region items are only required if these types of items have been surveyed or exist within the scheme. Note that a scheme with no continuous items (or with many unconnected point items) may be of limited use, as unconnected point items cannot be used for flow modelling. A2.5 Shapefiles must be provided to the current Ordnance Survey grid definition, including PAI (Positional Accuracy Improvement) corrections. A2.5.1 Shapefile geometry must only include ‘X’ and ‘Y’ coordinate values; ‘Z’ (elevation) and ‘M’ (user-defined) coordinate values are not permitted. All height/level data is included as attributes, as defined in the relevant tables in this Appendix. A2.6 This Specification gives details of the database attribute files, as defined in HD43, SD15 and this document. Permitted entries for certain fields are listed in the appropriate table in Appendix B. A2.7 Sample shapefiles with the appropriate attribute fields already set up can be downloaded from HADDMS. It is recommended that these are used as the basis of all shapefiles to be submitted to HADDMS. Shapefiles of existing schemes downloaded from HADDMS will be provided in the correct format and will contain all fields, even if these are blank. A2.8 One set of shapefiles must contain the inventory for one drainage scheme only. Guidance note on drainage data formats, Issue 1. August 2012 5 of 93 A2.9 A2.10 A2.11 The following field types are indicated in the shapefile definition tables. The sample shapefiles available on HADDMS are already defined with the correct field types. • Integer – a whole number. May be formatted as text to prevent software defaulting blank values to 0; however, only numeric characters are permitted. • Number – a number with or without decimal places (expected units for each field are indicated). May be formatted as text to prevent software defaulting blank values to 0; however, only numeric characters with or without a decimal point are permitted. • Date – date in DD/MM/YYYY or YYYY-MM-DD format. May be formatted as text to prevent software defaulting blank values to 0; however, the text must be in either “20/06/2010” or “2010-06-20” format. • Y/N – text field containing “Y” if present, “N” if not present, blank if not specified or not known. • Code – text field containing a code or other pre-defined text given in the referenced table in Appendix B (e.g. “PW”). • Text – text field, up to 254 characters in length allowing any standard ASCII character. More extensive text would need to be placed in an attached document (see section A8). The following status values are shown in the shapefile definition tables: • SYS – HADDMS system assigned, must be left blank for new inventory data, and must not be modified in data downloaded from HADDMS. • M – Mandatory, must always be completed in order for the data to be used in HADDMS. • (M) – Mandatory under certain circumstances, otherwise must be blank. For example, if a point item has a cover shape of “OT” (“other”), then the description of the other cover shape is mandatory – otherwise, it is not relevant and should be left blank. • D – Desirable field for good quality data, which makes the data useful. Likely to be required (where relevant to the item type) for new surveys under HD43 but may not be available when digitising old records. • O – Optional field, may not be required or may not be relevant for particular item types. • A – Not part of database. To be uploaded as an attached document using the HADDMS interface, or as part of the ZIP file as described in section A8. The revision status is shown where appropriate in the shapefile definition tables as: • N – New – not previously present in either HD43 and SD15 • M – Modified – definition changed since HD43 or SD15 (change explained in table) • D – Deleted – not to be used. (These items have been removed because they duplicate information provided elsewhere.) Guidance note on drainage data formats, Issue 1. August 2012 6 of 93 A2.12 The source of each item is shown in the shapefile definition tables as the relevant HA standard from which the field is derived, e.g. HD43, SD15. A3. Point item attribute data fields A3.1 The following attributes are to be recorded in the “point.dbf” file, for each point item mapped in the “point.shp” file. A3.2 For region node point items located at the centroid of a region item, only the fields marked as “Mandatory” in the Status column below are normally recorded. HADDMS will populate those marked as “System”. Attributes that are available for region items (see Table A-3) should be populated against the region item itself, rather than the region node. If any other information needs to be recorded related to the region asset, it may be populated against the region node. Table A-1: Point item (point.dbf) data format Field name Description Status Type SYS Units Revision Source Text M HD43 Asset referencing and connectivity ASSET_REF Unique HADDMS item reference in HD43 format (modified: assigned by HADDMS at time of first upload; not changed if asset location is corrected and falls outside original 10m square) SUPP_REF Supplier's item reference (must not be in HD43 format) M Text N SUPP_SCH Supplier's scheme reference M Text N ORIG_SCH Original supplier’s scheme reference (assigned by HADDMS as a permanent record if the scheme is merged with another on the system) SYS Text N ASSET_CONN HADDMS connectivity item reference in HD43 format (assigned by HADDMS) SYS Text N SUPP_CONN The continuous item SUPP_REF that this point item connects with. This point item should not be an upstream node. (mandatory for connector nodes - ITEM_TY_CO = “CN”) (M) Text N (IDENTIFIER) Suffix letter at end of HD43 item reference (deleted: included in ASSET_REF by system) – – D EASTING OS Easting coordinate of the point item M Number m HD43 NORTHING OS Northing coordinate of the point item M Number m HD43 ITEM_TY_CO The item type code (codes given in Table B-2) (modified: additional codes) M Code M STATUS Validation status of the asset (codes given in Table B-24) M Code N SYSTEM_TYP The type of drainage system (codes given in Table B-8) (modified: codes merged and rationalised; additional codes) D Code M HD43 Core details Guidance note on drainage data formats, Issue 1. August 2012 7 of 93 HD43 HD43/ SD15 F17 Field name Description Status Type LOC_TY_CO Location type code (codes given in Table B-14) (modified: now applicable to point items) D STRATEGIC Strategic drain code based on strategic importance of drain (codes given in Table B-15) (modified: now applicable to point items) OWNER Units Revision Source Code M SD15 F12 D Code M SD15 F28 The owner of the item (codes given in Table B-9) (modified: additional codes) D Code M HD43 CONSTR_YR Year of construction (YYYY format only, e.g. 1970) (modified: range of years not permitted, now applicable to point items) D Number M SD15 F35 MH_CONTUC Manhole construction type (codes given in Table B-12) D Code HD43 DOWNSTREAM The name of the downstream watercourse O Text HD43 COVER_LEVE The level of the cover of the item D Number mAOD DEPTH The depth of the lowest point of the item below cover level (e.g. the depth of the sump below cover level) D Number m N INVERT_LEV The level of the lowest point of the item (e.g. the level of the sump) D Number mAOD N COVER_WIDT The width or diameter of the cover O Number m HD43 COVER_LENG The length of the cover O Number m HD43 WIDTH The width or diameter of the shaft O Number m HD43 LENGTH The length of the shaft O Number m HD43 CHAM_WIDT The width or diameter of the chamber O Number m HD43 CHAM_LENGT The length of the chamber O Number m HD43 COVER_SHAP The shape code of the cover (codes given in Table B-10) (modified: additional codes) O Code OTHER_SHAP Description of other cover shape (mandatory if COVER_SHAP = "OT") (M) Text CHAM_SHAP The shape code of the chamber cross-section (codes given in Table B-10) O Code Dimensions and shape HD43 M HD43 HD43 N Features and components GRATING Indicates the presence of a grating D Y/N REM_BARS Indicates the presence of a cover comprising removable bars or beams D Y/N N COVER_DUTY The duty of the cover (codes given in Table B-11) (modified: additional codes) O Code M MANFACTER The manufacturer of the cover O Text Guidance note on drainage data formats, Issue 1. August 2012 8 of 93 HD43 HD43 HD43 Field name Description Status Type COVER_HING Units Revision Source Indicates whether the cover (or grating) is hinged O Y/N N HEADWALL Indicates the presence of a headwall, wingwall and/or apron (modified: description clarified) D Y/N M LADDER Indicates the presence of a ladder in the item D Y/N HD43 STEPIRONS Indicates the presence of step irons in the item D Y/N HD43 SIDEENTRY Indicates the presence of a side-entry in the item D Y/N HD43 NO_OF_LAND The number of landings present D Integer HD43 THROTTLE Indicates the presence of a throttle pipe D Y/N N SIGN Indicates the presence of a sign specifying the location of the item (description in "remarks") O Y/N N MATERIAL The material of the main structure of the item (not the material of components such as the cover) (codes given in Table B-6) (modified: now applicable to point items) O Code M LINED Whether the point item is lined (modified: now applicable to point items) O Y/N M HD43 LINING_MA Material of lining (codes given in Table B-6) (modified: now applicable to point items) O Code M HD43/ SD15 F24 LINING_TY Type or method of lining (codes given in Table B-13) (modified: now applicable to point items) O Code M SD15 F25 GUARDRAIL Indicates the presence of a guardrail O Y/N N SAFETY_CHA Indicates the presence of a safety chain O Y/N N SAFETY_BAR Indicates the presence of a safety bar O Y/N N SUMP Indicates the presence of a sump O Y/N N TRAPPED Indicates the outlet of a gully is trapped (for gullies only) O Y/N N VORTEX Indicates the presence of a vortex (e.g. used for silt separation) O Y/N N VENT_PIPE Indicates the presence of a ventilation pipe O Y/N N BENCHING Indicates the presence of benching within a chamber O Y/N N CHANNEL Indicates the presence of a channel within a chamber O Y/N N HD43 HD33 Features and components (flow control) FLAP_VALVE Indicates the presence of a flap valve D Y/N HD43 ORIFICE Indicates the presence of an orifice plate D Y/N HD43 SLUICE_GAT Indicates the presence of a sluice gate D Y/N HD43 WEIR Indicates the presence of a weir D Y/N HD43 SCREENS Indicates the presence of a screen or grill D Y/N HD43 PENSTOCK Indicates the presence of a penstock D Y/N HD43 Guidance note on drainage data formats, Issue 1. August 2012 9 of 93 Field name Description Status Type FLUME Units Revision Source Indicates the presence of a flume D Y/N HYDRO_FLOW Indicates the presence of a hydro-dynamic flow control device (modified: not previously shown on inspection sheets) D Y/N M HD43 BAFFLE Indicates the presence of a baffle D Y/N N HD33 HANG_WALL Indicates the presence of a hanging wall D Y/N N HD33 HANDSTOP Indicates the presence of a handstop D Y/N N HD33 DATE_OF_SU The date the item was most recently inspected (or digitised if from as-built records) (modified: description clarified) M Date M HD43 ORIGIN_DAT The origin of the initial record of this item (codes given in Table B-1) (modified: description clarified; additional codes) D Code M HD43 OTHER_SOU Name of other source of origin of data (mandatory if ORIGIN_DAT = "OT") (M) Text INSPECTOR Name of inspector and survey contractor’s company (modified: now applicable to point items) O Text M SD15 F2 TIME The time the item was inspected in HH:MM format (modified: now applicable to point items) O Text M SD15 F9 PRECLEANED Whether the asset was cleaned prior to inspection (modified: now applicable to point items) O Y/N M SD15 F26 PURPOSE Purpose of the inspection (codes given in Table B-16) (modified: now applicable to point items) O Code M SD15 F29 FLOW_CTRL Temporary measures taken to deal with the flow during inspection (codes given in Table B-17) (modified: now applicable to point items) O Code M SD15 F30 PRECIPITAT Type of precipitation at time of inspection (codes given in Table B-18) (modified: now applicable to point items) O Code M SD15 F31 TEMPERATUR Temperature at time of inspection above/below freezing as a code (codes given in Table B-19) (modified: now applicable to point items; numerical values no longer permitted) O Code M SD15 F32 METHOD Method of inspection (codes given in Table B-20) (modified: now applicable to point items) O Code M SD15 F36 STANDARD Version of the standard used to record data, e.g. “EN 135082:2001” (modified: now applicable to point items) O Text M SD15 F37 VIDEO_STOR Type of media used for storing video (codes given in Table B-21) (modified: now applicable to point items; additional codes permitted) O Code M SD15 F38 HD43 Inspection related Guidance note on drainage data formats, Issue 1. August 2012 10 of 93 HD43 Field name Description Status Type VIDEO_TIME Method of recoding timing code (codes given in Table B-22) (modified: now applicable to point items) O VIDEO_VOL Filing reference of volume containing video data (modified: now applicable to point items) PHOTO_STOR Units Revision Source Code M SD15 F39 O Text M SD15 F40 Type of media used for storing photos (codes given in Table B-23) (modified: now applicable to point items) O Code M SD15 F41 PHOTO_VOL Reference number of film or CD containing photo (modified: now applicable to point items) O Text M SD15 F42 CONSTRAINT Any constraints to inspection/maintenance O Text N DATE_MAINT Date asset was last maintained D Date N Condition SERV_GRADE Service condition grade (1–5, 9, 0 or blank) (calculated by HADDMS if observations are present, otherwise can be manually specified) D/ SYS Integer N SERV_PEAK Peak service score (calculated by HADDMS from service observations) SYS Integer N SERV_MEAN Mean service score (calculated by HADDMS from service observations) SYS Integer N SERV_TOTAL Total service score (calculated by HADDMS from service observations) SYS Integer N STRU_GRADE Structural condition grade (1–5, 9, 0 or blank) (calculated by HADDMS if observations are present, otherwise can be manually specified) D/ SYS Integer N STRU_PEAK Peak structural score (calculated by HADDMS from structural observations) SYS Integer N STRU_MEAN Mean structural score (calculated by HADDMS from structural observations) SYS Integer N STRU_TOTAL Total structural score (calculated by HADDMS from structural observations) SYS Integer N Supplementary details REMARKS General remarks O Text N DISTRICT District or division (modified: now applicable to point items) O Text M SD15 F6 LOC_ST Street name or place name at location of asset (modified: now applicable to point items) O Text M SD15 F10 LOC_TOWN Town or village at location of asset (modified: now applicable to point items) O Text M SD15 F11 ATT_DOCS Comma separated list of files to be attached to the asset (refer to section A8) O Text N Guidance note on drainage data formats, Issue 1. August 2012 11 of 93 Field name Description Status Type – Location: Scanned image or other file showing location of item (modified: provide as attached document, refer to section A8) A – Details: Scanned image or other file showing details of item (modified: provide as attached document, refer to section A8) A Revision Source – M HD43 – M HD43 Guidance note on drainage data formats, Issue 1. August 2012 12 of 93 Units A4. Continuous item attribute data fields A4.1 The following attributes are to be recorded in the “continuous.dbf” file, for each continuous item mapped in the “continuous.shp” file. Table A-2: Continuous item (continuous.dbf) data format Field name Description Status Type SYS Units Revision Source Text M HD43 M Text N SD15 F7 Asset referencing and connectivity PIPE_REFER Unique HADDMS item reference in HD43 format (modified: assigned by HADDMS at time of first upload; does not change if flow direction is corrected or the recorded upstream asset otherwise changes; incorrect example given in HD43 inspection sheets) SUPP_REF Supplier's item reference / pipe length reference (PLR) (must not be in HD43 format) UPSTR_REF HADDMS reference of upstream point item (modified: assigned by HADDMS) SYS Text M HD43 DOWNSTR_RE HADDMS reference of downstream point item (modified: assigned by HADDMS) SYS Text M HD43 SUPP_REFUP Supplier's reference for upstream point item M Text N SUPP_REFDN Supplier's reference for downstream point item M Text N SUPP_SCH Supplier's scheme reference M Text N ORIG_SCH Original supplier’s scheme reference (assigned by HADDMS as a permanent record if the scheme is merged with another on the system) SYS Text N CERT_CONN Certainty that this item connects the specified up- and downstream items M Y/N N FLOW_DIR Certainty of flow direction as defined by the specified up- and downstream items M Y/N (CONDUIT_RE) Suffix number at end of HD43 item reference (deleted: included in PIPE_REFER by system) – – D HD43 ITEM_TY_CO The item type code (codes given in Table B-3) (modified: codes merged and rationalised; additional codes) M Code M HD43/ SD15 F18 SYSTEM_TYP The type of drainage system (codes given in Table B-8) (modified: codes merged and rationalised; additional codes) D Code M HD43/ SD15 F17 LOC_TY_CO Location Type Code (codes given in Table B-14) (modified: additional codes) D Code M SD15 F12 STRATEGIC Strategic drain code based on strategic importance of drain (codes given in Table B-15) (modified: additional codes) D Code M SD15 F28 HD43 Core details Guidance note on drainage data formats, Issue 1. August 2012 13 of 93 Field name Description Status Type OWNER The owner of the item (codes given in Table B-9) (modified: additional codes) D CONSTR_YR Year of construction (YYYY format only, e.g. 1970) (modified: range of years not permitted) DOWNSTREAM The name of the downstream watercourse Units Revision Source Code M HD43 D Number M SD15 F35 O Text HD43 Dimensions and shape HEIGHT The height of the item (diameter of circular shaped items) (modified: not previously required by HD43/04 for circular items; codes merged and rationalised) D Integer mm M HD43/ SD15 F20 WIDTH The width of the item (modification to HD43/04: not to be used for diameter of circular items) D Integer mm M HD43/ SD15 F21 PIPE_DIA The diameter of a circular cross-section (SHAPE_1 = “CI”) piped item. Must be left blank (or 0) for all other assets. (D) Integer mm N SHAPE_1 The cross sectional shape of the item (codes given in Table B-5) (modified: codes merged and rationalised) D Code UPSTR_DEPT The depth of the upstream invert below cover level D Number m HD43 UPSTR_INVE The upstream invert level of the item D Number mAOD HD43 DOWNSTR_DE The depth of the downstream invert below cover level D Number m HD43 DOWNSTR_IN The downstream invert level of the item D Number mAOD HD43 UPSTR_BACK The upstream backdrop level of the item O Number mAOD HD43 DOWNSTR_BA The downstream backdrop level of the item O Number mAOD HD43 LENGTH The length of the continuous item D Number m HD43/ SD15 F34 PIPE_U_LEN Pipe unit length of individual pipe units that comprise the continuous item O Number mm SD15 F33 MATERIAL The material of the item (codes given in Table B-6) (modified: codes merged and rationalised) D Code M HD43/ SD15 F23 SURFACE_MA The surface material of the item (codes given in Table B-7) (modified: additional codes) D Code M HD43 POROUS Indicates whether the pipe wall is porous (including pipes that are part of a Combined Surface and Sub-Surface Filter Drain) D Y/N N PERFORATED Indicates whether the pipe wall is perforated (including pipes that are part of a Combined Surface and Sub-Surface Filter Drain) D Y/N N LINED Whether the continuous item is lined O Y/N LINING_MA Material of lining (codes given in Table B-6) (modified: codes merged and rationalised; additional codes) O Code M HD43/ SD15 F24 LINING_TY Type or method of lining (codes given in Table B-13) (modified: additional codes) O Code M SD15 F25 M HD43/ SD15 F22 Features Guidance note on drainage data formats, Issue 1. August 2012 14 of 93 HD43 Field name Description Status Type Units Revision Source Inspection related DATE_OF_SU The date the item was most recently surveyed (or digitised if from as-built records). (modified: description clarified) M Date M HD43 ORIGIN_DAT The origin of the initial record of this item (codes given in Table B-1) (modified: additional codes) D Code M HD43 OTHER_SOU Name of other source of origin of data (if ORIGIN_DAT = "OT") (M) Text HD43 INSPECTOR Name of inspector and survey contractor’s company O Text SD15 F2 TIME The time the item was surveyed in HH:MM format O Text SD15 F9 PRECLEANED Whether the continuous item was cleaned prior to inspection O Y/N SD15 F26 PURPOSE Purpose of the inspection (codes given in Table B-16) (modified: additional codes) O Code M SD15 F29 FLOW_CTRL Measures taken to deal with the flow during inspection (codes given in Table B-17) (modified: additional codes) O Code M SD15 F30 PRECIPITAT Type of precipitation at time of inspection (codes given in Table B-18) O Code TEMPERATUR Temperature at time of inspection above/below freezing as a code (codes given in Table B-19) (modified: numerical values no longer permitted) O Code M SD15 F32 METHOD Method of inspection (codes given in Table B-20) (modified: additional codes) O Code M SD15 F36 STANDARD Version of the standard used to record data, e.g. “EN 135082:2001” O Text VIDEO_STOR Type of media used for storing video (codes given in Table B-21) (modified: additional codes) O Code M SD15 F38 VIDEO_TIME Method of recoding timing code (codes given in Table B-22) (modified: additional codes) O Code M SD15 F39 VIDEO_VOL Filing reference of volume containing video data O Text PHOTO_STOR Type of media used for storing photos (codes given in Table B23) (modified: additional codes) O Code PHOTO_VOL Reference number of film or CD containing photo O Text CONSTRAINT Any constraints to inspection/maintenance O Text N DATE_MAINT Date asset was last maintained D Date N D / SYS Integer N SD15 F31 SD15 F37 SD15 F40 M SD15 F42 Condition SERV_GRADE Service condition grade (1–5, 9, 0 or blank) (calculated by HADDMS if observations are present, otherwise can be manually specified) Guidance note on drainage data formats, Issue 1. August 2012 15 of 93 SD15 F41 Field name Description Status Type SERV_PEAK Units Revision Peak service score (calculated by HADDMS from service observations) SYS Integer N SERV_MEAN Mean service score (calculated by HADDMS from service observations) SYS Integer N SERV_TOTAL Total service score (calculated by HADDMS from service observations) SYS Integer N STRU_GRADE Structural condition grade (1–5, 9, 0 or blank) (calculated by HADDMS if observations are present, otherwise can be manually specified) D / SYS Integer N STRU_PEAK Peak structural score (calculated by HADDMS from structural observations) SYS Integer N STRU_MEAN Mean structural score (calculated by HADDMS from structural observations) SYS Integer N STRU_TOTAL Total structural score (calculated by HADDMS from structural observations) SYS Integer N Source Supplementary details REMARKS General remarks O Text SD15 DISTRICT District or division O Text SD15 F6 LOC_ST Street name or place name at location of upstream manhole O Text SD15 F10 LOC_TOWN Town or village at location of drain O Text SD15 F11 SMIS_ID The ID of the item as recorded in SMIS (especially a culvert) O Integer N ATT_DOCS Comma separated list of files to be attached to the asset (refer to section A8) O Text N – Location: Scanned image or other file showing location of item (modified: provide as attached document, refer to section A8) A – M HD43 – Details: Scanned image or other file showing details of item (modified: provide as attached document, refer to section A8) A – M HD43 Guidance note on drainage data formats, Issue 1. August 2012 16 of 93 A5. Region item attribute data fields A5.1 The following attributes are to be recorded in the “region.dbf” file, for each region item mapped in the “region.shp” file. A5.2 Each region item must have an associated region node point item at its centroid. Attribute data other than that listed below is stored against the region node point item. Table A-3: Region item (region.dbf) data format Field name Description Status Type Units Revision Source HD43 Asset referencing and connectivity ASSET_REF Unique HADDMS item reference of the region node point item at the centroid (modified: assigned by HADDMS) SYS Text M SUPP_REF Supplier's item reference of the region node point item at the centroid M Text N SUPP_SCH Supplier's scheme reference M Text N ORIG_SCH Original supplier’s scheme reference (assigned by HADDMS as a permanent record if the scheme is merged with another on the system) SYS Text N (IDENTIFIER) Suffix letter at end of HD43 item reference (deleted: included in ASSET_REF by system) – – D ITEM_TY_CO The item type code (codes given in Table B-4) (modified: additional codes) M Code SYSTEM_TYP The type of drainage system (codes given in Table B-8) (modified: codes merged and rationalised; additional codes) D Code M HD43/ SD15 OWNER The owner of the item (codes given in Table B-9) (modified: additional codes) D Code M HD43 CONSTR_YR Year of construction (YYYY format only, e.g. 1970) (modified: range of years not permitted) D Number M SD15 CAPACITY Maximum designed capacity D Integer RAISED_BA Indicates presence of any raised banks D Y/N N INUND_PLAN Indicates whether a Flood Inundation Plan is in place D Y/N N DOWNSTREAM The name of the downstream watercourse O Text (EASTING) Six-digit OS Easting coordinate of the centroid of the asset (deleted: specified in EASTING of Region Node point item) – – D HD43 (NORTHING) Six-digit OS Northing coordinate of the centroid of the asset (deleted: included in NORTHING of Region Node point item) – – D HD43 Key details Guidance note on drainage data formats, Issue 1. August 2012 17 of 93 HD43 m³ N HD43 Field name Description Status Type Units Revision Source Features LINED Whether the region item is lined (modified: now applicable to region items) O Y/N M LINING_MA Material of lining (codes given in Table B-6) (modified: now applicable to region items) O Code M LINING_TY Type or method of lining (codes given in Table B-13) (modified: now applicable to region items) O Code M DATE_OF_SU The date the item was most recently surveyed (or digitised if from as-built records). (modified: description clarified) M Date M HD43 ORIGIN_DAT The origin of the initial record of this item (codes given in Table B-1) (modified: additional codes) D Code M HD43 OTHER_SOU Name of other source of origin of data (mandatory if ORIGIN_DAT = "OT") (M) Text CONSTRAINT Any constraints to inspection/maintenance O Text N D / SYS Integer N Inspection related HD43 Condition SERV_GRADE Service condition grade (1–5, 9, 0 or blank) (calculated by HADDMS if observations are present, otherwise can be manually specified) SERV_PEAK Peak service score (calculated by HADDMS from service observations) SYS Integer N SERV_MEAN Mean service score (calculated by HADDMS from service observations) SYS Integer N SERV_TOTAL Total service score (calculated by HADDMS from service observations) SYS Integer N STRU_GRADE Structural condition grade (1–5, 9, 0 or blank) (calculated by HADDMS if observations are present, otherwise can be manually specified) D / SYS Integer N STRU_PEAK Peak structural score (calculated by HADDMS from structural observations) SYS Integer N STRU_MEAN Mean structural score (calculated by HADDMS from structural observations) SYS Integer N STRU_TOTAL Total structural score (calculated by HADDMS from structural observations) SYS Integer N Supplementary details REMARKS General remarks O Text N ATT_DOCS Comma separated list of files to be attached to the asset (refer to section A8) O Text N – Location: Scanned image or other file showing location of item (modified: provide as attached document, refer to section A8) A – M Guidance note on drainage data formats, Issue 1. August 2012 18 of 93 HD43 Field name Description – Details: Scanned image or other file showing details of item (modified: provide as attached document, refer to section A8) Status Type A – Guidance note on drainage data formats, Issue 1. August 2012 19 of 93 Units Revision Source M HD43 A6. Scheme metadata file formats A6.1 When a scheme of inventory and condition data is downloaded from HADDMS, two additional DBF4 format files will be included in the ZIP file. The content of these files is listed in Table A-4 and Table A-5, respectively. It is not expected that users attempt to create or edit these files; section A6 of this document is for reference purposes only. A6.1.1 scheme.dbf – contains information regarding the scheme data, and the version of the data being downloaded. A6.1.2 linkdata.dbf – contains a list of assets in the scheme that have either attached documents or an entry in the priority asset register. A6.2 The files are intended to provide: • assistance with offline data management of schemes that have been downloaded from the system, by fully identifying the data • an offline indication of the potential “usefulness” of an online scheme, in terms of the number of documents that are attached to a scheme or its assets, and the number of assessed priority assets A6.3 Much of the information can be changed directly or indirectly by editing the scheme or its inventory on HADDMS, or uploading a new version of the scheme; hence, it is a correct snapshot at the time of download only. A6.4 Some fields in the scheme.dbf file have an “e-GMS equivalent” element, compliant with the minimum standards of the UK Cabinet Office’s “e-Government Metadata Standard” (v3.1, August 2006). A6.4.1 Note that the “SCHEME_ID and “V_VERNUM” fields in the scheme.dbf file must be read in combination to derive the e-GMS “Identifier” element. A6.5 Users should not attempt to create or edit these files themselves. The files are not required to be included when data is uploaded, but if provided will be ignored by the system, such that they do not have any effect on data on the system. Table A-4: scheme.dbf data format e-GMS equivalent element Type Publisher Text either “HADDMS” or “WADDMS” Source Text SUBJECT “Drainage asset inventory and condition data” Subject Text SCHEME_ID Unique, permanent system scheme ID (assigned by system) Identifier Integer SUPP_SCH Supplier's scheme reference AREA Area or DBFO number of scheme ROADS Comma separated list of roads assigned to the scheme, e.g. “M4,A34,A329M” TITLE Scheme title Field name Description PUBLISHER either “Highways Agency” or “Welsh Assembly” SOURCE Text Integer Text Title Guidance note on drainage data formats, Issue 1. August 2012 20 of 93 Text e-GMS equivalent element Field name Description Type LOC_FROM Local name at start of scheme Text LOC_TO Local name at end of scheme Text DATATYPE Type of data scheme is based on: design, as-built or field survey Text V_VERNUM Version number of exported scheme V_ORIGDATE Date this version of the scheme was imported to the system V_USER Name of HADDMS user that imported this version of the scheme to the system Text V_ACTIVITY Type of survey activity for this version of the scheme (e.g. CCTV Survey, etc) Text V_ORG Name of company or organisation that produced this version V_COMMENT Free-form comment entered during import of this version V_CRACKSCR Completeness Rating Assessment of Catchment Knowledge (CRACK) overall score in % for this version of the scheme. EXPORTDATE Date data was exported from the system Date.issued Date EXPORTUSER Name of HADDMS user that exported data from the system Addressee Text EXPORTLOCK “Y” if the user that exported the data locked the scheme N_SCHDOC Number of documents attached to scheme (this does not include documents attached to assets or observations) Integer N_ASTDOC Number of assets with documents attached at asset or observation level (including those with documents attached to historic observations) Integer N_PRIASSET Number of assets within the scheme that are of a priority asset type (soakaways, outfalls, culverts) – includes “Not Determined” Integer N_PRISTAT Number of assets within the scheme that have an assessed priority asset status (i.e. that is not “Not Determined”). Integer Identifier Integer Date.acquired Date Creator Text Text Integer Y/N Table A-5: linkdata.dbf data format Field name Description Type ASSET_REF Unique, permanent HADDMS Asset Reference (as assigned by HADDMS) Text SUPP_REF Supplier's Asset Reference Text ASSET_TYPE Type of asset (point, continuous or region) Text ITEM_TY_CO Item type code Text PRIREGSTAT Priority register overall status (for the version of the asset downloaded – will be from the register indicated by the item type code) – including those with status of “Not Determined”. Text NUM_DOCS Total number of documents attached to the asset or its observations Guidance note on drainage data formats, Issue 1. August 2012 21 of 93 Integer A7. Observations data file format A7.1 This section describes the data format for recording observations. It should be read in conjunction with Appendix C, which describes in full what information is to be recorded. A7.2 Observation data for assets in a scheme must be included in an “observation.dbf” DBF 4 format file. This single file per scheme includes observations attached to point, continuous and region assets. The format of the file is described in Table A-6. A7.3 This file will only be included when data is downloaded from HADDMS if at least one asset in the scheme has observations. A7.4 Location data for observations can be recorded as absolute OS coordinates or relative to the asset itself (e.g. chainage). Hence, all location data is stored within the observation.dbf file and it is not in a full SHP file format. A7.5 Observations in all asset types can occur either at a point or as a “continuous observation” along an extent of the asset, as defined in section C1.9. Where an observation is only present at a point, only the “START” or “FROM” location fields are populated. For observations covering a longer extent of the asset the “END” or “TO” location fields must also be populated. Table A-6: Observation (observation.dbf) data format Field name Description Type Status SUPP_REF Supplier's item reference / pipe length reference of the item the observation occurs on (must match the SUPP_REF of a point, continuous or region item in the same scheme) Text M ASSET_REF Unique HADDMS item reference (assigned by HADDMS – must be left blank for new assets) Text SYS OBS_REF Unique numeric observation reference assigned by HADDMS (for observations downloaded from HADDMS only) Integer SYS START_EAST For observations in continuous and region assets only: OS easting of a point observation, or the start of a continuous observation as defined in section C1.9 Number – START_NOR For observations in continuous and region assets only: OS northing of a point observation, or the start of a continuous observation as defined in section C1.9 Number – START_CH For observations in continuous assets only: Distance in metres from the upstream end of the continuous asset of a point observation, or the start of a continuous observation as defined in section C1.9 Number – END_EAST For observations in continuous and region assets only: OS easting of the end of a continuous observation as defined in section C1.9. Left blank for point observations. Number – END_NOR For observations in continuous and region assets only: OS northing of the end of a continuous observation as defined in section C1.9. Left blank for point observations Number – END_CH For observations in continuous assets only: Distance in metres from the upstream end of the continuous asset of the end of a continuous observation as defined in section C1.9. Left blank for point observations. Number – DEPTH_FROM For observations in point assets only: the depth in metres of the top of the observation below the cover level Number – Guidance note on drainage data formats, Issue 1. August 2012 22 of 93 Field name Description Type Status DEPTH_TO For observations in point assets only: the depth in metres of the bottom of a continuous observation as defined in section C1.9. Left blank for point observations. Number – OBS_CODE The HADDMS code of the observation as given in Appendix C Code M CLOCK_FROM For observations in point and continuous assets only: Where appropriate, the clockface position of the circumferential extent of the observation between 1 and 12, as defined in section C1.12. Integer – CLOCK_TO For observations in point and continuous assets only: Where appropriate, the clockface position of the circumferential extent of the observation between 1 and 12, as defined in section C1.12. Must be left blank for observations affecting a single point within the circumference. Integer – OBS_VALUE1 Where defined in Appendix C, the first non-percentage quantification for an observation. The units of this numeric value must be as defined in Appendix C and must not be included in the data. Number – OBS_VALUE2 Where defined in Appendix C, the second non-percentage quantification for an observation. The units of this numeric value must be as defined in Appendix C and must not be included in the data. Number – OBS_PERC Where defined in Appendix C, a percentage quantification for an observation, specified as a whole number between 0 and 100. The “%” symbol must not be included in the data. Integer – VIDEO_REF The filename of any video file showing the observation Text – VIDEO_TIME The timecode within the file defined in VIDEO_REF where the observation is shown, in the format HH:MM:SS (e.g. “00:01:23”) Time – REMARKS Free-form description of observation, limited to 254 characters Text – PHOTO_REFS Comma separated list of files to be attached to the observation (refer to section A8) Text – SCORE Numeric observation score assigned by HADDMS. This field is not intended to be populated by users and will be recalculated by HADDMS when data is uploaded. Integer SYS Guidance note on drainage data formats, Issue 1. August 2012 23 of 93 A8. Referencing of attached documents A8.1 HADDMS has an online interface to allow upload and attachment of individual documents to one asset or observation at a time. Where significant numbers of documents are to be uploaded, it is preferable to reference them in the data and include them when the scheme is uploaded. A8.2 The following file types and filename extensions are permitted. Additional file types may be added in future. A8.3 • images in JPG, PNG or TIF format • documents in DOC, DOCX or PDF format • drawings in DGN, DWG, DWF or DXF format (drawings are preferred in a self-contained, view-only format such as DWF or PDF) • files in TXT, XLS, XLSX, PPT and PPTX format are also permitted The following file naming rules must be observed: • Filenames must be meaningful, as the document’s title is inferred from them. • Each file must have a unique filename. • Filenames must not contain commas or spaces, or any generally nonpermitted characters such as asterisks and backslashes. It is recommended that the underscore “_” symbol is used instead of a space, or “CamelCase”. A8.4 Attached documents should be kept small such that the total size of a scheme and its attached documents when zipped is generally less than 100MB. HADDMS may impose limits on schemes significantly above this size, which will require users to contact the Support Desk. Many attached documents will be photographs in JPG format, which should be optimised so they are of an appropriate resolution and quality for their intended purpose, such that file sizes are minimised. Other documents containing embedded photographs should be similarly optimised. A8.5 The “ATT_DOCS” attribute of the asset or “PHOTO_REFS” attribute of the observation to which the file(s) are attached must specify the filename of the file. If more than one file is to be attached to a single item, the filenames are listed in the attribute, separated by commas, e.g. “photo1.jpg,photo2.jpg”. The attribute may not exceed 254 characters so filenames should be kept short if multiple files are attached to an asset. The system is not intended for attaching excessive numbers of documents (i.e. more than about 10) to single assets or observations. A8.6 A single file can be attached to more than one asset or observation by specifying its filename in the relevant attribute for each of the items. If a single file is to be attached to a large number of assets, it may be appropriate to attach it to the scheme on HADDMS instead. Files attached to observations for an asset should not also be attached to the asset itself. A8.7 When downloading drainage asset and observation data from HADDMS, the attached document related attributes will be blank, allowing additional documents to be attached. Existing attached documents will be retained by the system provided the HD43 asset reference is not changed and the data is uploaded to the same scheme. If a new document is provided which has the same filename as an existing document, this will not affect the existing document. Guidance note on drainage data formats, Issue 1. August 2012 24 of 93 A9. Procedure for uploading to HADDMS A9.1 The shapefiles must be named “point”, “continuous” and “region”, with the appropriate file extensions; i.e. point.shp, point.dbf, point.shx, continuous.shp, continuous.dbf, continuous.shx, region.shp, region.dbf and region.shx depending on the inventory types being submitted. A9.2 If observation data is to be provided in accordance with section A7, this file must be named observation.dbf. A9.3 Any files to be attached to assets or observations (in accordance with section A8) are to be included in the ZIP file. All files must be in the top-level folder of the ZIP file; no subfolders are permitted. A9.4 All of the files for one scheme are to be uploaded to HADDMS as a single ZIP file, which may have any name with a “.zip” file extension. A9.5 The user who uploads data to HADDMS will need to have appropriate HADDMS access rights, which must be requested by the Service Provider’s Drainage Liaison Engineer (DLE). Such access is only granted to Service Provider’s own staff, although DLEs may request checking access for their data supply contractors. A9.6 Inventory can be uploaded to either: A9.7 • a new drainage scheme on HADDMS • a new version of an existing scheme on HADDMS, superseding the current version. For new schemes, the following information will need to be entered on HADDMS as a minimum, prior to uploading the zipped shapefiles: • Area (the user will be presented with a list of the Area/DBFO(s) that they have access to) • Scheme Reference (must be unique within the Area/DBFO) • Road(s) • Scheme Title • Year that the original records were produced • Date of first submission of scheme to HADDMS Other details are available and should be completed as far as possible. A9.8 The zipped shapefiles will then be validated against the criteria defined in Appendix A10, and any errors encountered will be listed or confirmation given that the data can be proceed to be imported into HADDMS. A9.9 For existing schemes, if digital inventory data already exists, then it will be replaced by the uploaded data; hence, the supplied shapefiles must contain the full inventory for the scheme. Any located files or georeferenced drawings will not be affected. Guidance note on drainage data formats, Issue 1. August 2012 25 of 93 A10. Acceptance criteria for digital inventory data A10.1 Various automated checks are performed on the inventory and condition data files when they are uploaded to HADDMS, as described below. The system will only carry out a limited set of checks if the data structure (e.g. field names, provided files) is incorrect. A10.2 Drainage contractors and other data suppliers may be given access to HADDMS for the purposes of checking data prior to submission to Service Providers. They are not permitted to amend the HADDMS database or import their data into the database. Data to be imported into the database must be finally uploaded and checked by the Service Provider. A10.3 If any of the checks fail, a report of the problems will be provided. Failures can result in either a “critical error” or a “warning”, as indicated below. A10.3.1 The presence of any “critical errors” indicates an illogical or ambiguous model, and means that the whole scheme must be rejected. Such errors must be fixed and the data re-uploaded before the data can be added to the system. A10.3.2 The presence of any “warnings” means that the scheme can be added to the system, provided there are no critical errors. Warnings may indicate incorrect data, or can arise due to unusual drainage layouts or incomplete records (e.g. due to a limited scope survey). Any warnings should be reviewed before confirming that the scheme can be added to the system. A10.4 The scope of checking is to ensure that a logical and unambiguous model has been defined in the supplied data and that it is provided in the correct format; checking of data quality and completeness is otherwise not checked at import stage by HADDMS, and is the responsibility of the data supplier. A10.5 If data is uploaded as a new version of a scheme, additional checks are carried out to detect potential loss of existing data, for example the removal of a large number of assessed priority assets from a scheme. If significant data loss is detected, users may need to discuss this with the HADDMS Support Team before they can complete the data import. A10.6 Additional checks resulting in “warnings” may be added to HADDMS periodically to help promote improvements in data quality. Basic checks A10.7 The following checks confirm that all files are present, hold appropriate types of data in the correct format, and that all mandatory fields have been completed. Checks A10.7.1 and A10.7.2 relate to the fundamental structure of the data; failure of these will generally prevent the system from carrying out any further checks. A10.7.1 Critical check: The shapefiles submitted in the .zip file must be correctly named and contain the appropriate type of geometry (i.e. points, lines or polygons for “points”, “continuous” and “region” respectively). A10.7.2 Critical check: All fields in the data file specification must be present and contain the appropriate type of data (e.g. text, number, date, etc.). To avoid some software defaulting blank number fields to 0, “Integer” and “Number” fields may be formatted as text, provided these only contain a number or are left blank. Guidance note on drainage data formats, Issue 1. August 2012 26 of 93 A10.7.3 Critical check: All system-assigned fields (“SYS”) for new inventory items must be blank. When re-uploading a scheme that has been downloaded from HADDMS, the values in these fields must be left unchanged. If the values are deleted, then those inventory items will be treated as new items and assigned a new HD43 reference, and the old items will be removed from HADDMS. A10.7.4 Critical check: All mandatory fields (“M”) must be completed. A10.7.5 Critical check: The scheme reference (“SUPP_SCH”) must be the same for all entries in each of the submitted shapefiles for a scheme. This is to confirm that all inventory items are part of the same scheme. A10.7.6 Critical check: All fields identified as requiring a code (e.g. item type, system type, material, shape, etc) will be checked for valid code entries, as defined in Appendix B. Referencing checks A10.8 The following checks confirm that there is connectivity between the point, continuous and region items, based on their supplier’s item references. A10.8.1 Critical check: As per check A10.7.3, all HD43 asset reference fields are system assigned, and must be left blank for newly submitted inventory items. A10.8.2 Critical check: All supplier’s reference fields (“SUPP_REF”) will be checked to ensure there are no duplicates within the submitted scheme. (Duplication with items in other schemes is permitted.) A10.8.3 Critical check: The supplier’s reference of any point asset must not match the supplier’s reference of any continuous asset in the same scheme. A10.8.4 Critical check: All upstream and downstream references of continuous items (“SUPP_REFUP” and “SUPP_REFDN” respectively) must exist as point references within the same scheme. A10.8.5 Critical check: All region items must have a corresponding region node point item (item type code = “RN”) with the same supplier’s reference within the same scheme. A10.8.6 Warning check: All supplier’s reference fields (“SUPP_REF”) will be checked to ensure that they are not in an HD43 format. Connectivity referencing checks A10.9 The following checks relate to checking the referencing used in connectivity attributes and connectivity nodes. A10.9.1 Critical check: If the connectivity attribute (“SUPP_CONN”) of a point item is specified, it must exist as the reference of a continuous item within the same scheme. A10.9.2 Warning check: If the connectivity attribute of a point item is specified, the item should not be used as the upstream node of any continuous item. A10.9.3 Warning check: The connectivity attribute should not be specified for outfalls, soakaways, soakaway boreholes, ghost nodes, region nodes or phantom nodes. A10.9.4 Critical check: The connectivity attribute must be specified for connectivity nodes (item type = “CN”). Guidance note on drainage data formats, Issue 1. August 2012 27 of 93 Upstream and downstream item checks A10.10 The following checks ensure that appropriate point items have been used as upstream and downstream nodes of continuous items. A10.10.1 Warning check: Point items that are gullies should not be used as the downstream point item for a continuous item. (In some cases this is required; however, in the majority it is not appropriate so it is flagged as a warning.) A10.10.2 Warning check: Outfalls should not be used as the upstream point item for a continuous item. A10.10.3 Warning check: Ghost nodes and phantom nodes should be specified as either the upstream or the downstream node of at least one continuous item. The only exception is a ghost node at the centroid of an isolated region item. Connectivity nodes (ITEM_TY_CO = “CN”) should be specified as the downstream node of at least one continuous item. All other point items may exist in isolation. A10.10.4 Warning check: Point items that are only referenced as downstream nodes of continuous items should be outfalls, soakaways, soakaway boreholes, connectivity nodes, phantom nodes, region nodes or ghost nodes. A10.10.5 Warning check: The region node at the centroid of a region item should only be the up- or downstream item of region connectors (ITEM_TY_CO = “RC”). Conversely, region connectors should have either their upstream or downstream node assigned as a region node. Location checks A10.11 The following checks ensure that spatial geometry is appropriate, and that point items are located correctly relative to any continuous item to which they are connected. A10.11.1 Critical check: All locations given in the geometry and attribute data must be to OS Grid (for display on OS MasterMap, post Positional Accuracy Improvements) within the appropriate range of Eastings and Northings. A10.11.2 Critical check: All point items must have “EASTING” and “NORTHING” attributes that match the easting and northing defined in the SHP geometry. A10.11.3 Critical check: The geometry Eastings and Northings of the ends of the line representing each continuous item must match the Eastings and Northings of its upstream and downstream point items. Observation related checks A10.12 The following checks ensure that observations are referenced to a valid asset. If observations are referenced to an inappropriate asset (e.g. a pipework observation is related to a gully), then it will be accepted but not scored. Any failures of the checks in this section would result in a critical error. A10.12.1 Critical check: All observations must have a “SUPP_REF” that matches a point, continuous or region item in the same scheme. A10.12.2 Critical check: If an “ASSET_REF” is provided in the observation data, it must match the “ASSET_REF” or “PIPE_REFER” of the same asset as its “SUPP_REF”. Guidance note on drainage data formats, Issue 1. August 2012 28 of 93 A10.12.3 Critical check: All observations must have an “OBS_CODE” entry. A10.12.4 Warning check: The “OBS_CODE” of an observation should be associated with an appropriate asset item type, otherwise it will not be scored. Attached documents checks A10.13 The following checks ensure that any documents which should be attached to the assets or observations are present and referenced in the data. A10.13.1 Critical check: Documents in the ZIP file must be of one of the acceptable types, as listed in section A8. A10.13.2 Warning check: Any document that is named in an “ATT_DOCS” or “PHOTO_REFS” attribute should be present in the root folder of the ZIP file. This warning could indicate a document has been omitted, incorrectly named or referenced, or placed in a sub-folder in the ZIP file. A10.13.3 Warning check: Any document that is present in the ZIP file should be referenced in at least one “ATT_DOCS” or “PHOTO_REFS” attribute. A10.13.4 Warning check: Referencing a single document to large numbers of assets or observations is discouraged, so a warning is generated for any document referenced more than once. Guidance note on drainage data formats, Issue 1. August 2012 29 of 93 Appendix B. Inventory Code Tables B1. Introduction B1.1 The following tables supersede those given in HD43/04 and SD15/03, as specified below. Other tables in HD43/04 and SD15/03 are unchanged. B1.2 It is permitted to use any of the codes listed in the “HADDMS acceptable codes” columns; for example, item type codes of “RM” or “B” are acceptable for a Rising Main, as shown in Table B-3. In future, HADDMS may automatically convert such codes to a single “preferred” code. B1.3 The source of each acceptable code is shown in the tables as: • HD43 – HD43/04 (original Table number where appropriate) • SD15 – SD15/03 (original Field or Part number where appropriate) • New – Code not included in HD43/04 or SD15/03. Where these are derived from other HA or industry standards, the description column states the source. B1.4 “Equivalent EN codes” are given in those tables where they are available. EN codes are provided for cross-referencing only and are not accepted by HADDMS. B2. Tables Table B-1: Data source codes (source of original asset record) HADDMS acceptable codes Description HD43 Table B1 SD15 New Internal Asset Survey (carried out by Service Provider) IS – – External Asset Survey (not carried out by Service Provider) ES – – Digital As Built Drawing DP – – Hardcopy As Built Drawing HP – – RMMS Database RS – – Other (description to be entered in “OTHER_SOU”) OT – Z Guidance note on drainage data formats, Issue 1. August 2012 30 of 93 Table B-2: Item type codes for point inventory items HADDMS acceptable codes Description Equivalent EN code for information only HD43 Table B2 SD15 Part 5 New Not to be used Bifurcation or Storm Overflow BI – – – Catchpit CP CP – – Connector Node – – CN – Flow Control Device – – FC – GN – – – – – GI – Gully GU GY – – Inlet IT – – – Inspection Chamber – IC – B Interceptor IN – – – Lamphole – LH – D Manhole MH MH – A Oil Separator – OS – – Other special chamber – OC – Z Outfall OU OF – E Outlet – – OL – Phantom Node – – PN – Piped Grip (superseded by Grip Inlet) PG – – – Pumping Station PS – – – Region Node – – RN – Rodding Eye RE RE – C Soakaway Chamber (or unknown type of soakaway) SO SK – – – – SB – Ghost Node (was “Ghost Manhole”) Grip Inlet Soakaway Borehole (or ‘Well’) Note: Codes denoting “Start Node Type” and “Finish Node Type” observations in CCTV surveys are not changed by the above. These codes are listed in Table C-4. Guidance note on drainage data formats, Issue 1. August 2012 31 of 93 Table B-3: Item type codes for continuous inventory items HADDMS acceptable codes Description Equivalent EN code for information only HD43 Table B4 SD15 Field 18 New PW – – – Gravity Drain – A – A Rising Main RM B – B Culvert CU – – – Land Drainage LD – – – Syphon (inverted) SY – – – Surface Water Channel CH – – – Edge Channel EC – – – Drainage Channel Block DB – – – Grip GP – – – Ditch DI – – – Grassed Surface Water Channel or Swale SC – – – Linear Drainage Channel LI – – – Combined Surface and Sub-Surface Filter Drain CF – – – Fin Drain FN – – – Narrow Filter Drain ND – – – Filter Drain (medium) – when not containing a pipe FD – – – Counterfort Drain CD – – – Informal Drainage ‘Over the Edge’ OE – – – Combined Kerb and Drainage Channel CK – – – Combined Pipe and Channel Drain CS – – – Region Connector RC – – – Phantom Connector – – PL – Linear Cellular Storage System (not used as soakaway) – – LS – Exfiltration Ditch (superseded by Soakaway Trench) – – – – Soakaway Trench – – ST – Pipework Guidance note on drainage data formats, Issue 1. August 2012 32 of 93 Not to be used Table B-4: Item type codes for region inventory items HADDMS acceptable codes Description HD43 Table B3 SD15 New Detention Basin DP – – Retention Pond RP – – Sediment Pond SP – – Infiltration Basin IB – – Pollution Containment Pond/Tank PC – – Wetlands WL – – Reed Bed Treatment System – – RB Reservoir Pavement – – RV Pond (undifferentiated) – – PU Table B-5: Shape codes for continuous inventory items HADDMS acceptable codes Description Equivalent EN code for information only HD43 Table B5/B6 SD15 Field 22 New Not to be used Circular CI C – A Rectangular RE R – B Egg shaped EG E – C Oval OV O – F Arch shaped AC A – E Triangular TG – – Z Trapezoidal TP T – XC U-shaped US U – D Barrel – B – XA Horseshoe – H – XB Kerb block – K – XD OT Z – Z Other (description to be included in “REMARKS”) Note: Codes to be used as appropriate for below and above ground continuous items Guidance note on drainage data formats, Issue 1. August 2012 33 of 93 Table B-6: Material and lining material codes HADDMS acceptable codes Description Equivalent EN code for information only HD43 Table B7 SD15 Field 23/24 New Not to be used Vitrified clay VC VC – AE Puddled clay – – PC Z Concrete CO CO – AG MDPE (medium density polyethylene) MD MD – Z PVC / PVC-U PV PVC – AX HPPE (high performance polyethylene) HE – – Z Cast iron CI CI – AM Asbestos cement AC AC – AA Bitumen – BL – AB Asphalt – – AS Z Pitch fibre PF PF – AC Brickwork – BR – AD Cement mortar – CL – AF Reinforced concrete – RC – AH Sprayed concrete – SPC – AI Cast in-situ concrete – – ISC Z Pre-cast concrete segments or units (SD15 code corrected) – CS – AJ Fibre cement (SD15 code corrected) – FC – AK Fibre reinforced plastics (SD15 code corrected) – FRP – AL Grey cast iron – GI – AN Ductile cast iron – DI – AO Steel – ST – AP Unidentified type of iron or steel – XI – AQ Masonry (coursed) – MAC – AR Masonry (uncoursed) – MAR – AS Epoxy – EP – AT Polyester – PS – AU Polyethylene (change: if not MDPE or HPPE) – PE – AV Polypropylene – PP – AW Guidance note on drainage data formats, Issue 1. August 2012 34 of 93 HADDMS acceptable codes Description Equivalent EN code for information only HD43 Table B7 SD15 Field 23/24 New Not to be used Butile compounds – – BU Z Unidentified or other type of plastics – XP – AY Granular material – – GR Z Geo-synthetic (lining material) – – GS Z Gabions – – GA Z Stone pitch – – STP Z Concrete bagwork – – CB Z Unidentified material – X – AZ Other (description to be included in “REMARKS”) – Z OT Z Table B-7: Surface material codes for continuous inventory items HADDMS acceptable codes Description HD43 Table B8 SD15 New Granular (undifferentiated) GR – – Granular (crushed stone) – – GRC Granular (shingle) – – GRS Turf TU – – Pre-coated Chips CC – – Tar Spray TS – – Bitumen Bonded Shredded Tyres BB – – Removable Bars RB – – Continuous Gratings (note: legacy code) CG – – Continuous Gratings (integral) – – CGI Continuous Gratings (removable) – – CGR Earth – – EA Vegetation other than turf – – VE Other (description to be included in “REMARKS”) – – OT Other (description to be included in “REMARKS”) – – Z Guidance note on drainage data formats, Issue 1. August 2012 35 of 93 Table B-8: System type codes HADDMS acceptable codes Description Equivalent EN code for information only HD43 Table B9 SD15 Field 17 New Not to be used Surface SW S – B Foul FW F – A Combined (foul and surface water) CO C – C Trade effluent drain or drain – T – D Culverted watercourse – W – E Other (description to be included in “REMARKS”) – Z OT Z Table B-9 to Table B-12 list codes that were previously indicated only on the HD43 inspection sheets, but are now provided in tables for completeness. Therefore, no HD43 Table reference is shown. Table B-9: Owner codes HADDMS acceptable codes Description HD43 SD15 New Highway authority (was: Highways Agency) HA – – Public PU – – Private PR – – Other (description to be included in “REMARKS”) – – OT Other (description to be included in “REMARKS”) – – Z Table B-10: Cover shape and chamber shape codes for point inventory items HADDMS acceptable codes Description Equivalent EN code for information only HD43 SD15 New Not to be used Square SQ – – A Circular CI – – B Triangular TR – – C – – RE A OT – Z Z Rectangular Other (description to be entered in “OTHER_SHAP”) Guidance note on drainage data formats, Issue 1. August 2012 36 of 93 Table B-11: Cover duty codes for point inventory items HADDMS acceptable codes Description HD43 SD15 New Heavy (BS:EN 124 grade D, E or F) H – – Medium (BS:EN 124 grade C) M – – Low (BS:EN 124 grade A or B) L – – Other (description to be included in ‘REMARKS’) – – OT Other (description to be included in “REMARKS”) – – Z Table B-12: Manhole construction type codes for point inventory items HADDMS acceptable codes Description HD43 SD15 New Brick BR – – Pre-cast concrete PC – – In-situ concrete IC – – Bolted concrete segments BS – – Plastic – – PL Steel – – ST Other (description to be included in “REMARKS”) – – OT Other (description to be included in “REMARKS”) – – Z Guidance note on drainage data formats, Issue 1. August 2012 37 of 93 The codes in Table B-13 to Table B-23 were previously provided in SD15/03 without HD43/04 equivalents, but are now available for use in all surveys. Table B-13: Lining type codes HADDMS acceptable codes Description Equivalent EN code for information only HD43 SD15 Field 25 New Not to be used Lining inserted during manufacture – M – A Sprayed lining – SP – B Cured in place lining – CIP – C Cast in-situ lining – – CIS Z Segmental linings – SEG – D Lining with discrete pipes – DP – E Lining with continuous pipes – CP – F Close fit lining – CF – G Spirally wound lining – SW – H Other (description to be included in “REMARKS”) – Z OT Z Guidance note on drainage data formats, Issue 1. August 2012 38 of 93 Table B-14: Location type codes for point and continuous inventory items Description HADDMS acceptable codes Equivalent EN code for information only HD43 SD15 Field 12 New Not used – XS1 – XHA Left Footway Not used – XS2 – XHB Left Verge Verge – XS3 – XHC Lane 1 Hard shoulder – XS4 – XHD Lane 2 Lane 1 (SD15: “Left lane”) – XS5 – XHE Lane 3 Lane 2 (SD15: “Middle lane”) – XS6 – XHF Lane 4 Lane 3 (SD15: “Right lane”) – XS7 – XHG Right Verge Central reserve / Right verge – XS8 – XHH Right Footway Not used – XS9 – XHI Right outside Verge Not used – XS0 – XHJ Earthwork slope Earthwork slope – – XSA XHP Between earthwork and HA ownership boundary Between earthwork and HA ownership boundary – – XSB XHP Outside HA ownership boundary Outside HA ownership boundary – – XSC XHP Not used Lane 4 – – XSD XHP Right turning lane Lane 5 – XSE – XHK Slip road Slip road – XSQ – XHL Bus lane Lane 6 – XSR – XHM Crawler lane Crawler lane – XST – XHN Left turning lane Not used – XSW – XHO – XSX XSP XSY – XHP Roads other than motorways Motorways Left Outside Verge (adjacent to footway) Crosses / straddles lanes (continuous items only, all road types) Any other option Any other option – Guidance note on drainage data formats, Issue 1. August 2012 39 of 93 Not to be used Table B-15: Strategic drain codes for point and continuous inventory items HADDMS acceptable codes Description HD43 SD15 New Drains within the Central Reserve are of greater strategic importance due to the consequences of flooding – A1 – Transverse drains have a high strategic importance due to the problems associated with access for remedial works – A2 – Carrier pipes and associated manholes adjacent to running lanes of very heavily trafficked roads where flooding or remedial work will seriously affect traffic movement (modified: description changed) – B1 – Toe drains to earthworks (modified: description changed) – B2 – Other drainage systems – C1 – Sewer Rehabilitation Manual Category A * – – A Sewer Rehabilitation Manual Category B * – – B Sewer Rehabilitation Manual Category C * – – C Other (description to be included in “REMARKS”) – – OT Other (description to be included in “REMARKS”) – – Z *Note: SRM codes are permitted to support historic CCTV data only. New surveys must supply codes from the “SD15” column. Table B-16: Purpose of inspection codes HADDMS acceptable codes Description Equivalent EN code for information only HD43 SD15 Field 29 New Not to be used Final control of new construction – E – A End of warranty period – J – B Routine inspection of condition – G – C Suspected structural problem – A – D Suspected operational problem – H – E Suspected infiltration problem – B – F Final control of renovation or repair – C – G Transfer of ownership – D – H Investment planning – I – I Sample survey – F – J Other (description to be included in “REMARKS”) – X Z Z Guidance note on drainage data formats, Issue 1. August 2012 40 of 93 Table B-17: Flow control measure codes HADDMS acceptable codes Description Equivalent EN code for information only HD43 SD15 Field 30 New Not to be used No flow control – N – A Flows blocked upstream – BL – B Flows partially blocked upstream – PB – C Other (description to be included in “REMARKS”) – X Z Z Table B-18: Precipitation codes HADDMS acceptable codes Description Equivalent EN code for information only HD43 SD15 Field 31 New Not to be used No precipitation – N – A Rain – R – B Melting snow or ice – S – C Table B-19: Temperature codes HADDMS acceptable codes Description Equivalent EN code for information only HD43 SD15 Field 32 New Not to be used Temperature is above freezing – A – A Temperature is below freezing – B – B Guidance note on drainage data formats, Issue 1. August 2012 41 of 93 Table B-20: Method of inspection codes HADDMS acceptable codes Description Equivalent EN code for information only HD43 SD15 Field 36 New Not to be used Direct inspection (man-entry) – A – A CCTV – B – B Inspection from manhole or inspection chamber only – C – C Non-visual (e.g. sonar, radar, etc) – – N – Other (description to be included in ‘REMARKS’) – – OT – Other (description to be included in “REMARKS”) – – Z – Table B-21: Video image storage media codes HADDMS acceptable codes Description Equivalent EN code for information only HD43 SD15 Field 38 New Not to be used VHS video cassette tape – VHS – A Video CD – CD – B Video DVD – --- DVD Z Media file (e.g. .mpg) on computer disk – --- HDD Z Other (description to be included in “REMARKS”) – X Z Z Table B-22: Video image timing reference system codes HADDMS acceptable codes Description Equivalent EN code for information only HD43 SD15 Field 39 New Not to be used The recording time in hours and minutes since the start of the tape / CD / DVD / media file – A – A A machine dependent numeric counter – B – B Other (description to be included in “REMARKS”) – X Z Z Guidance note on drainage data formats, Issue 1. August 2012 42 of 93 Table B-23: Photograph storage system codes HADDMS acceptable codes Description Equivalent EN code for information only HD43 SD15 Field 41 New Not to be used Printed or other non-electronic still photographs – A – A Electronic still photographs held on computer, in JPG format – B – B Other (description to be included in “REMARKS”) – X Z Z Table B-24: Field validation status codes for point inventory items HADDMS acceptable codes Description HD43 SD15 New Asset is present as per the recorded item within 5m. This status applies to all items recorded in a new field survey. – – Confirmed Asset is present within 5m but is of a different item type to that of the original records – – Modified Asset is present on site but was not in the original records – – New Asset as recorded has not been located on site. This status applies to validation surveys only, where such assets are not deleted. – – Unproven Item has not been subject to a field survey. Default assumed value if no status specified. – – Unvalidated Note: For full details of field validation statuses, refer to the Guidance Note on Validation Surveys. Guidance note on drainage data formats, Issue 1. August 2012 43 of 93 Appendix C. Defect-level assessment of drainage asset condition C1. Introduction C1.1 The tables below list all available observation codes for recording a detailed level of information regarding the structural condition and service condition of drainage assets. C1.2 Where EN codes are available, these are shown for comparison purposes only. All observations provided to HADDMS must use the “Accepted HADDMS Code”. C1.3 HADDMS observation codes are fully inclusive of any previous “characterisation” or joint codes and must not contain spaces. C1.4 HADDMS calculates all scoring from the HADDMS observation code with any required quantifications. Where more than one score is listed for a single observation code, the score depends on the quantification, which is then mandatory. Further information on quantification of observations is in section C1.13. C1.5 Users should not manually score observations, as this will always be carried out by HADDMS, which will then calculate the asset’s structural and service grades. C1.6 Where an asset has been assessed at the observation level, this takes precedence over any manually assigned grades, which are then always calculated from the observations. C1.7 Additional “inspection” observations are also available in Table C-1 or recording general remarks, etc. These observations apply to all asset types and are not scored. C1.8 Where field names such as “CLOCK_FROM”, and “OBS_PERC” are referred to, these relate to the data format in Table A-6 or the equivalent in software used to collect survey data. Recording the location of an observation C1.9 C1.10 The HADDMS inventory data locates the position of the asset; it is also necessary to locate the position of an observation within the asset as follows: • in point items, the depth of the observation below cover level • in continuous items, either the chainage measured from the upstream end of the asset, or the OS easting/northing of the observation • in region items, the OS easting/northing of the observation Observations may occur either at a point, or as “continuous observations” along an extent of an asset. A point observation requires only one location; a continuous observation requires a location to be recorded at both ends. An observation is “continuous” in the following cases: • in a pipe, where the observation continues for a length greater than 1 metre • in other continuous items, where the observation continues for a length greater than 5 metres • in region items, more than 5 metres of a linear feature (e.g. banks) affected Guidance note on drainage data formats, Issue 1. August 2012 44 of 93 • C1.11 C1.12 C1.12.1 in point items such as chambers, where the observation continues for a depth greater than 1 metre It is not necessary to provide an observation location in the following scenarios: • the observation type’s description means it can only exist in that specific part of a point asset, such as a cracked gully grating • the observation by its nature cannot be located, such as a lack of access route to a pond In addition to the above, observations in point and continuous assets may be located on the circumference of the asset using “clockface” position attributes. Examples include the position of an incoming connection, or circumferential extent of a crack. “From” and “To” parameters are available to indicate extent, with just “From” used to indicate the observation relates to a point on the circumference. “To” is always measured in a clockwise direction from the “From” parameter such that if “From” is 8 and “To” is 4, the extent will include 8–12–4. The clockface is oriented as follows: • For continuous assets such as pipes, 12 is at the top of the asset and the clock-face is viewed looking downstream. (Clock-face positions provided in DrainageXML format from CCTV surveys, as described in Appendix D, will be converted by HADDMS based on the survey direction and flow direction of the pipe.) • For point assets such as chambers, 12 is at the position of the lowest outgoing pipe. If there is more than one outgoing pipe at this level, 12 is at the position of the largest of these. Some examples of using the clockface parameters include: • An incoming connection on the left-hand side of a pipe (when viewed downstream or in direction of survey): “CLOCK_FROM” = 9, “CLOCK_TO” left blank. • Cracks covering the top half of a pipe: “CLOCK_FROM” = 9, “CLOCK_TO” = 3. • Cracks covering the bottom half of a pipe: “CLOCK_FROM” = 3, “CLOCK_TO” = 9. • A crack in a manhole wall, opposite the position of the lowest outgoing pipe: “CLOCK_FROM” = 6; “CLOCK_TO” left blank. • As 0 may be confused with a blank value, it is essential that 12 is used to represent the “top” of the clock. Quantifying a defect C1.13 Observations in the tables in this appendix having several scores for the same code require a quantification to be entered. In such cases it is mandatory, and if not provided the observation will not be scored. It is not mandatory to use these parameters for other observations. Many observations for non-pipework assets have separate observation codes to identify differing severity of defects, and scoring of these does not depend on any quantification. C1.14 Mandatory quantifications take either of two formats: Guidance note on drainage data formats, Issue 1. August 2012 45 of 93 C1.15 • A radial extent using the “clockface” parameters, as described above. “¼ radial extent” would indicate three-twelfths of the clock, e.g. 3–6. The scoring will therefore depend on the values of “CLOCK_FROM” and “CLOCK_TO”. • A percentage such as % cross-sectional area filled with sediment. The scoring will depend on the value of “OBS_PERC”. “OBS_VALUE1” and “OBS_VALUE2” parameters are provided to allow numeric values to be included in accordance with SD15 and are not used for quantification purposes when scoring defects. However, due to the variety of these for pipework observations, it is recommended that such additional information is included in a descriptive remark. Information related to inventory data (e.g. the diameter of an incoming pipe) should be recorded in the inventory data of the appropriate asset itself. Recording that an asset has no defects C1.16 If an asset has been fully inspected and has no structural or service defects, and no observations are recorded, the structural and service grades for that asset may both be recorded as 1. C1.17 Alternatively, an observation indicating the asset has been fully inspected and does not contain one or both types of defect may be recorded as follows: • If a drainage asset has no structural defects, an observation with the code “SOUND” should be assigned to confirm that the asset has been assessed. HADDMS will assign a structural observation score of 1 and, provided no other structural observations are assigned to the asset, calculate an assetlevel structural grade of 1. • If a drainage asset has no service defects, an observation with the code “CLEAN” should be assigned to confirm that the asset has been assessed. HADDMS will assign a service observation score of 0.5 and, provided no other service observations are assigned to the asset, calculate an asset-level service grade of 1. Recording that part of a continuous asset has not been inspected C1.18 Sections of a continuous asset that have not been assessed can be identified by specifying an observation code of “BLOS” (beyond limit of survey) over the appropriate extent of the asset. C1.18.1 HADDMS will automatically calculate the location of any non-surveyed extent of pipework assets from CCTV survey data provided in the format described in Appendix D. For such surveys, it is recommended that this observation is not manually specified. C1.18.2 Continuous assets that have not been inspected at all should not have any observations recorded, but should be graded as described in the Guidance Note on Drainage Connectivity Surveys, depending on the reason for non-inspection. C1.18.3 This observation is not applicable to point or region items. Guidance note on drainage data formats, Issue 1. August 2012 46 of 93 C2. Inspection related observation codes The codes in the table below apply to all asset types, unless indicated. With the exception of those to indicate there are no defects, they do not result in any observation score and do not affect the asset-level grade. Table C-1: Inspection related observation codes Accepted HADDMS Code Equivalent EN Code (not to be used) Observation type Description No defects Asset fully inspected and no structural defects present (structural score = 1) SOUND – Asset fully inspected and no service defects present (service score = 0.5) CLEAN – BLOS – GP BDA General remark (describe in REMARKS) REM BDB Unspecified reason (preferred codes below) SA BDC Due to obstruction SAOB BDCA Due to high water level SAWL BDCB Due to equipment failure EQ BDCC Due to other reasons (describe in REMARKS) Z BDCZ WL BDD Not inspected Extent of continuous asset not inspected (e.g. beyond limit of survey) Additional details General photograph (filename of photo in PHOTO_REFS) Inspection abandoned Water level Water level (specify % height) Clear water level WLC BDDA Turbid water level WLT BDDB Flow from incoming asset FW BDE FWC BDEA FWCS BDEAB FWT BDEB Wrong foul flow from incoming surface water asset FWTF BDEBB Flow from incoming asset not visible FWYY BDEYY Flow from incoming asset (specify radial position of incoming asset using clockface parameters) (connection observation, e.g. CN, also required) Clear flow from incoming asset Wrong surface water flow from incoming foul asset Turbid flow from incoming asset Guidance note on drainage data formats, Issue 1. August 2012 47 of 93 Accepted HADDMS Code Equivalent EN Code (not to be used) Observation type Description Atmosphere within asset Oxygen deficiency OD BDFA Hydrogen sulphide HS BDFB Methane ME BDFC Other GZ BDFZ Presence of pollutants (describe in REMARKS) PZ – Camera under water CUW BDGA Silt CUD BDGB Steam CUS BDGC Other CUZ BDGZ (pipes or chambers only) Pollutants (does not apply to oil separators or assets where specific observations for pollutants are available) Loss of vision Guidance note on drainage data formats, Issue 1. August 2012 48 of 93 C3. Highway Drainage Pipes – Defect-level assessment Inventory item type(s): Pipework (PW), Gravity Drain (A), Rising Main (RM, B), Combined Surface and Sub-Surface Filter Drain (pipe) (CF), Land Drain (LD), Culvert (CU), Syphon (SY) Scoring of some observations depends on the pipe’s material type, as defined in Table B7, and grouped as follows: • “Flexible” – all plastic materials, steel and glass fibre. • “Rigid” – all other materials listed in Table B-6. For those HADDMS observation codes where there is an optional suffix, this is shown in brackets. This must be appended to the main code without the inclusion of a space or any other character. For example: • a circumferential surface crack at a joint has the HADDMS observation code “HFCJ” • a defective intruding connection has the HADDMS observation code “CXI” Where more than one observation code is shown, either may be used for compatibility. For example, visible aggregate can be recorded by using the code “SAV”, “SSM” or “SWM”. For a combined surface and sub-surface filter drain where a pipe is present (item type code CF), the pipe is assessed as below, and the filter media as per the tables in Appendix C8. The overall grades for the combined asset are based on the highest scoring structural and service defects. Table C-2: Defect-level structural condition assessment of pipes Score Accepted HADDMS Code Equivalent EN Code (not to be used) D BAA Defect type Description / quantification Deformation Flexible material: 6–10% 20 Flexible material: 11–20% 40 Flexible material: 21–33% 80 Flexible material: >33% 165 Rigid material: 1–5% 40 Rigid material: 6–10% 80 Rigid material: >10% 165 Surface crack Cracked Longitudinal 2 HFL(J) BABAA Circumferential 2 HFC(J) BABAB Multiple / Complex 2 HFM(J) BABAC Spiral / Helical 2 HFS(J) BABAD Longitudinal 10 CL(J) BABBA Circumferential 10 CC(J) BABBB Guidance note on drainage data formats, Issue 1. August 2012 49 of 93 Score Accepted HADDMS Code Equivalent EN Code (not to be used) Multiple / Complex 40 CM(J) BABBC Spiral / Helical 40 CS(J) BABBD Puncture (Internal wall – flexible materials only) 40 CPU(J) – Longitudinal 40 FL(J) BABCA Circumferential 40 FC(J) BABCB Multiple / Complex 80 FM(J) BABCC Spiral / Helical 80 FS(J) BABCD Split (Internal wall – flexible materials only) 80 FSP(J) – Broken Broken 80 B(J) BACA Hole Radial extent: <¼ 80 H(J) BACB Radial extent: ¼+ 165 X/ XP(J) BACC Defect type Fractured Description / quantification Soil or void visible through hole – see SV or VV Collapsed Surface damage Collapse 165 Increased roughness due to wear 10 SW / SWS BAFA Increased roughness due to spalling 10 SS / SSS BAFB Visible aggregate 40 SAV / SSM / SWM BAFC Aggregate projecting from surface 40 SAP BAFD Missing aggregate 120 SAM / SSL / SWL BAFE Visible reinforcement 120 SRV BAFF Reinforcement projecting from surface 165 SRP BAFG Corroded reinforcement 165 SRC BAFH Porous due to manufacturing defect 40 PP BAN Guidance note on drainage data formats, Issue 1. August 2012 50 of 93 Equivalent EN Code (not to be used) Defect type Description / quantification Defective connection / junction Defective (intruding) 165 CX(I) / JX BAH Defective (intruding), position incorrect 165 CXP(I) / JXP BAHA Defective (intruding), gap 165 CXG(I) BAHB Defective (intruding), partial gap 165 CXH(I) BAHC Defective (intruding), connecting pipe is damaged 165 CXD(I) / JXD BAHD Sealing ring intruding 40 SR BAIA Visible, not intruding 10 SRN BAIAA Hanging loop above centre 80 SRHA BAIAB Hanging loop below centre 120 SRHB BAIAC Broken 80 SRB BAIAD Other sealant intruding 40 SO BAIZ Medium: 1–1.5x wall thickness 10 OJM BAJA Large: >1.5x wall thickness 40 OJL Medium: 1–1.5x wall thickness 10 JDM Large: >1.5x wall thickness 40 JDL (append “I” to indicate intrusion) Intruding Sealing Material Displaced/Open Joint (does not apply to pipes in filter drains) Radially displaced joint Score Accepted HADDMS Code Open (Soil visible) – see SV BAJB >10% diameter & visible soil – see SV Angular displaced joint >1.5x wall thickness 10 AJ BAJC 165 LXD BAKA Discolouration of lining 1 LXC BAKB Defective end of lining 120 LXE BACK Longitudinal wrinkled lining 40 LXWL BAKDA Circumferential wrinkled lining 40 LXWC BAKDB Spiral wrinkled lining 40 LXWS BAKDC Blistered lining 80 LXB BAKE >10% diameter & visible soil – see SV Lining defect Lining detached Guidance note on drainage data formats, Issue 1. August 2012 51 of 93 Defect type Description / quantification Other minor lining defect (describe in REMARKS) Defective repair Score Accepted HADDMS Code Equivalent EN Code (not to be used) 40 LXZ BAKZ RX BAL Radial extent: <¼ of radius 80 ¼+ of radius 165 Weld failure Longitudinal 40 WXL BAMA (Plastic pipes only) Circumferential 40 WXC BAMC Spiral / Helical 80 WXS BAMB Soil visible Soil outside the pipe is visible through a defect 165 SV BAO Void visible Void outside the pipe is visible through a defect 165 VV BAP Point repair Pipe replaced 20 RPR BCBA (If defective, record as RX) Localised lining 20 RPL BCBB Injected mortar 20 RPI BCBC Other injected sealing material 20 RPS BCBD Hole repaired 20 RPH BCBE Other trenchless method 20 RPZ BCBZ Any continuous lining material inserted as remedial measure. (Lining material to be specified in inventory/header data.) 20 RCL Lining (If defective other than discolouration, record as an LX code) Table C-3: Defect-level service condition assessment of pipes Score Accepted HADDMS Code Equivalent EN Code (not to be used) Defect type Description / quantification (CSA = cross-section area) Roots Fine 1 RF(J) BBAB Tap 5 RT(J) BBAA Guidance note on drainage data formats, Issue 1. August 2012 52 of 93 Description / quantification (CSA = cross-section area) Defect type Score Mass: Attached deposits <5% CSA loss 2 5%–19% 4 20%–49% 6 50%–74% 10 75%+ 20 Encrustation/scale: <5% CSA loss 1 5%–19% 2 20%+ 5 Grease: <5% CSA loss 2 5%–19% 4 20%–49% 10 50%–74% 15 75%+ 20 Fouling (deprecated code) – Other attached deposits: <5% CSA loss 1 5%–19% 2 20%+ 5 Blocked perforations 10 Accepted HADDMS Code Equivalent EN Code (not to be used) RM(J) BBAC DEE(J) BBBA DEG(J) BBBB DEF(J) BBBC DEZ(J) BBBZ DEPB – Settled deposits Settled deposits fine: DES(J) BBCA (same quantification bandings apply to each code) Settled deposits coarse: DER(J) BBCB Settled deposits hard or compacted: DEC(J) BBCC Other settled deposits: DEX(J) BBCZ <5% CSA loss 1 5%–19% 2 20%–49% 5 50%–74% 8 75%+ 10 Guidance note on drainage data formats, Issue 1. August 2012 53 of 93 Defect type Description / quantification (CSA = cross-section area) Intruding lateral Intrusion % diameter: Score <5% 1 5%–19% 2 20%–49% 5 50%–74% 8 75%+ 10 Accepted HADDMS Code Equivalent EN Code (not to be used) CNI BAG Blocked connection/junction Connecting pipe is blocked (and intruding) 20 CXB(I) / JXB BAHE Intruding Sealing Material Sealing ring intruding 2 SR BAIA Visible, not intruding 1 SRN BAIAA Hanging loop above centre 5 SRHA BAIAB Hanging loop below centre 8 SRHB BAIAC Broken 5 SRB BAIAD Other sealant intruding 2 SO BAIZ Ingress of soil Sand: INGS(J) BBDA (same quantification bandings apply to each code) Peat: INGP(J) BBDB Other fine material: INGF(J) BBDC Gravel: INGG(J) BBDD Other material: INGZ(J) BBDZ Other obstacles <5% CSA loss 1 5%-19% 2 20%-49% 5 50%-74% 8 75%+ 10 Brick or masonry in invert 10 OBB(J) BBEA Pipe material in invert 10 OBM(J) BBEB Other object in invert 10 OBX(J) BBEC Obstacle protruding through wall 10 OBI(J) BBED Obstacle wedged in joint 10 OBJ(J) BBEE Obstacle through connection/junction 10 OBC(J) BBEF External pipe/cable 20 OBP(J) BBEG Guidance note on drainage data formats, Issue 1. August 2012 54 of 93 Score Accepted HADDMS Code Equivalent EN Code (not to be used) Obstacle built into structure 20 OBS(J) BBEH Other obstacle 10 OBZ(J) BBEZ Infiltration – Seeping 2 IS(J) BBFA Infiltration – Dripping 5 ID(J) BBFB Infiltration – Running 10 IR(J) BBFC Infiltration – Gushing 20 IG(J) BBFD Exfiltration – visible leakage of flow 5 EX(J) BBG Rat 1 VR BBHA Other vermin 1 VZ BBHZ WLS – Description / quantification (CSA = cross-section area) Defect type Infiltration and exfiltration (not applicable to perforated or porous pipes) Vermin Standing water due to pipe deviation Standing water: <20% CSA 1 (characterised by camera entering and leaving water) 20%-49% 5 50%-74% 8 (not applicable to iphons) 75%+ 10 Table C-4: Inventory observations for pipe surveys Accepted HADDMS Code Equivalent EN Code (not to be used) JN BCAAA Observation type Description Connections Junction (having prefabricated connection) (specify radial position and diameter in mm of incoming pipe) Closed junction JNC BCAAB Connection other than junction CN BCAFA CNC BCAFB LL BBCA Line of drain deviates left and up LLU BCCAA Line of drain deviates left and down LLD BCCAB Line of drain deviates right LR BCCB LRU BCCBA Closed connection Curvature of pipe Line of drain deviates left Line of drain deviates right and up Guidance note on drainage data formats, Issue 1. August 2012 55 of 93 Observation type Accepted HADDMS Code Equivalent EN Code (not to be used) LRD BCCBB Line of drain deviates up LU BCC_A Line of drain deviates down LD BCC_B Manhole MH BCDA Inspection chamber IC BCDB Rodding eye RE BCDC Lamphole LH BCDD Outfall to watercourses OF BCDE Major connection (branch) without manhole BR BCDF Gully GY BCDXA Catchpit CP BCDXB Soakaway SK BCDXC Oil separator OS BCDXD Other node type OC BCDZ Manhole MHF BCEA Inspection chamber ICF BCEB Rodding eye REF BCEC Lamphole LHF BCED Outfall to watercourses OFF BCEE Major connection (branch) without manhole BRF BCEF Gully GYF BCEXA Catchpit CPF BCEXB Soakaway SKF BCEXC Oil separator OSF BCEXD Other node type OCF BCEZ Description Line of drain deviates right and down Node type at start of survey (observation optional for HADDMS – point asset item type must also be recorded in inventory data using codes in Table B-2) Node type at finish of survey (observation required only to confirm completion of CCTV surveys – in all cases, point asset item type must also be recorded in inventory data using codes in Table B-2) Guidance note on drainage data formats, Issue 1. August 2012 56 of 93 C4. Highway Drainage Ditches – Defect-level assessment Inventory item type(s): Ditch (DI), Grip (GP) Table C-5: Defect-level structural condition assessment of ditches Score Accepted HADDMS Code No erosion 1 QDAA Slight erosion – evidence of scour, exposed subsoil 20 QDAB Holes in channel side and loose soil 40 QDAC Slippage – Failure of channel side obstructing channel 80 QDAD Slippage – Failure channel side blocking channel 165 QDAE No evidence of burrowing 1 QDBA Evidence of burrowing, exposed subsoil 20 QDBB Established burrowing with flow slightly impeded 40 QDBC Burrowing – Spoil obstructing flow 80 QDBD Severe burrowing – Failure of channel side blocking channel 165 QDBE No exposure of lining 1 QDCA Lining exposed at top of bank 20 QDCB Displaced joints in lining 20 QDCC Lining (non-concrete) exposed in slope side 40 QDCD Displaced concrete liner 40 QDCE Lining exposed in channel. Lining torn or broken above top water level 80 QDCF Concrete liner broken but not blocking channel 80 QDCG Lining torn or broken below top water level 165 QDCH Concrete liner collapsed and blocking channel 165 QDCI Defect type Description Slope stability Burrows Lining Guidance note on drainage data formats, Issue 1. August 2012 57 of 93 Table C-6: Defect-level service condition assessment of ditches Score Accepted HADDMS Code 0.5 QDPA Minor accumulation causing no loss of flow 1 QDPB Accumulation causing some loss of flow and/or pooling 3 QDPC Heavy sedimentation causing obstruction of flow 5 QDPD Channel full of sediment 10 QDPE Short Grass – Grass stands in flow 0.5 QDQA Long Grass – Grass folds in flow 1 QDQB Weed Growth – Plants established in channel 1 QDQC Reeds – Reeds established in channel 1 QDQD Rushes - Invasive growth in channel 2 QDQE Shrub or brambles – Low growing dense plant >300mm 4 QDQF Sapling Tree <100mm girth 5 QDQG Mature Tree >100mm girth 10 QDQH No debris 0.5 QDRA Litter – Individual pieces 1 QDRB Litter – Accumulation 2 QDRC Debris – Vehicle component 4 QDRD Debris – Building products 4 QDRE Debris – Multiple vehicle components 8 QDRF Debris requiring mechanical removal (vehicle) 10 QDRG Defect type Description Sedimentation No sediment or detritus Vegetation Debris Guidance note on drainage data formats, Issue 1. August 2012 58 of 93 C5. Highway Drainage Enclosed Channels – Defect-level assessment Inventory item type(s): Combined Kerb & Drainage (CK), Combined Pipe & Channel (CS), Linear Drainage Channel (LI) Note: blockage of gratings is recorded as a defect of the relevant point asset, not the channel Table C-7: Defect-level structural condition assessment of enclosed channels Score Accepted HADDMS Code No defects 1 QLAA Surface damage, e.g. chipped, weathered or corrosion. 10 QLAB Cracked kerb-drain unit 40 QLAC Displaced unit 80 QLAD Broken kerb-drain unit 165 QLAE Cracked integral cover 40 QLAF Broken integral cover 165 QLAG Damaged removable cover 40 QLAH Missing removable cover 165 QLAI Defect type Description General Defective integral cover Defective removable cover Table C-8: Defect-level service condition assessment of enclosed channels Score Accepted HADDMS Code 0.5 QLPA Minor accumulation causing no loss of flow 1 QLPB Accumulation causing some loss of flow 2 QLPC Heavy sedimentation causing obstruction of flow 5 QLPD Channel full of sediment 10 QLPE No litter/debris 0.5 QLQA 1 QLQB Defect type Description Sedimentation No sediment or detritus Debris / Vegetation Minor accumulation causing no loss of flow Guidance note on drainage data formats, Issue 1. August 2012 59 of 93 Accumulation blocking individual waterway 2 QLQC Heavy debris obstruction of multiple waterways 5 QLQD Outlets blocked 10 QLQE Guidance note on drainage data formats, Issue 1. August 2012 60 of 93 C6. Highway Drainage Open Channels – Defect-level assessment Inventory item type(s): Surface Water Channel (CH), Drainage Channel Block (DB), Edge Channel (EC) Note: blockage of gratings is recorded as a defect of the relevant point asset, not the channel Table C-9: Defect-level structural condition assessment of open channels Score Accepted HADDMS Code No defects 1 QCAA Surface cracking or weathering 20 QCAB Missing sealant 40 QCAC Displaced pre-cast concrete blocks 80 QCAD Spalling concrete 80 QCAE Fractured / broken 165 QCAF Defect type Description General Table C-10: Defect-level service condition assessment of open channels Score Accepted HADDMS Code 0.5 QCPA Minor accumulation causing no loss of flow 1 QCPB Accumulation causing some loss of flow or ponding in invert 2 QCPC Heavy sedimentation causing obstruction of flow 5 QCPD Channel full of sediment 10 QCPE No litter/debris 0.5 QCQA Minor accumulation causing no loss of flow 1 QCQB Accumulation causing some loss of flow 2 QCQC Heavy debris causing obstruction of flow 5 QCQD Channel blocked with debris 10 QCQE Defect type Description Sedimentation No sediment or detritus Debris Guidance note on drainage data formats, Issue 1. August 2012 61 of 93 C7. Highway Drainage Grassed Channels – Defect-level assessment Inventory item type(s): Grassed Surface Water Channel or Swale (SC) Note: blockage of gratings is recorded as a defect of the relevant point asset, not the channel Table C-11: Defect-level structural condition assessment of grassed channels Score Accepted HADDMS Code No defects 1 QVAA Shallow rut or depression with no impact on flow 20 QVAB Shallow wheel rut or depression slightly impeding flow 40 QVAC Deep wheel rut or depression severely impeding flow 80 QVAD Multiple wheel ruts or displaced soil forming ridges 165 QVAE Displaced or exposed geogrid 80 QVAF Exposed impermeable lining 165 QVAG Defect type Description General Lining and geogrid Table C-12: Defect-level service condition assessment of grassed channels Score Accepted HADDMS Code 0.5 QVPA Minor accumulation causing no loss of flow 1 QVPB Accumulation causing some loss of flow and/or ponding 2 QVPC Heavy sedimentation causing obstruction of flow 5 QVPD Channel full of sediment 10 QVPE No litter/debris 0.5 QVQA Minor accumulation causing no loss of flow 1 QVQB Accumulation causing some loss of flow and/or ponding 2 QVQC Defect type Description Sedimentation No sediment or detritus Debris Guidance note on drainage data formats, Issue 1. August 2012 62 of 93 Vegetation Contamination Heavy debris causing obstruction of flow 5 QVQD Channel full of debris 10 QVQE Short grass 0.5 QVRA Long grass 1 QVRB Intrusive vegetation 2 QVRC Raised grass strip at pavement edge 2 QVRD Patches of bare ground 2 QVRE Extensive bare ground 5 QVRF Completely bare ground 10 QVRG Extensive vegetation blocking flow 10 QVRH No contamination 0.5 QVSA Isolated damage by contamination 5 QVSB* Extensive damage by contamination 10 QVSC* * Note: QVSB and QVSC require immediate action and may not be recorded on HADDMS if this is taken Guidance note on drainage data formats, Issue 1. August 2012 63 of 93 C8. Highway Drainage Filter Drains – Defect-level assessment Inventory item type(s): Combined Surface and Sub-Surface Filter Drain (filter media) (CF), Counterfort Drain (CD), Filter Drain (FD), Soakaway Trench (ST) The tables below relate to assessment of the filter drain media. For a combined surface and sub-surface filter drain where a pipe is present (item type code CF), the filter media is assessed as below, and the pipe as per the tables in Appendix C2. The overall grades for the combined asset are based on the highest scoring structural and service defects. Table C-13: Defect-level structural condition assessment of filter drains Score Accepted HADDMS Code No structural defect 1 QFAA Individual displacement of stones 20 QFAB Individual wheel rut with scattered medium 40 QFAC Encroached earthworks 80 QFAD Surface obscured (no treatment) 80 QFAE Multiple wheel ruts with major displacement of medium onto carriageway 80 QFAF Inappropriate medium (Type A in carriageway drain) 80 QFAG Combined surface & sub-surface drain remote from pavement edge 165 QFAH Defect type Description General Design, construction or maintenance defects Table C-14: Defect-level service condition assessment of filter drains Score Accepted HADDMS Code 0.5 QFPA Sediment present on surface 1 QFPB Sediment build up at pavement edge 1 QFPC Vegetation growth at pavement edge 2 QFPD Vegetation established in medium 4 QFPE Filter medium surface crusted 5 QFPF Standing water 10 QFPG Defect type Description General Clean Guidance note on drainage data formats, Issue 1. August 2012 64 of 93 Filter medium blocked 10 Guidance note on drainage data formats, Issue 1. August 2012 65 of 93 QFPH C9. Highway Drainage Chambers – Defect-level assessment Inventory item type(s): Bifurcation or Storm Overflow (BI), Catchpit (CP), Inspection Chamber (IC), Lamphole (LH), Manhole (MH), Rodding Eye (RE), Soakaway Borehole (SB), Soakaway Chamber (SO, SK), Other Special Chamber (OC) The presence of manufactured holes in the chamber wall signifies a soakaway as could the presence of granular material within the chamber or no evidence of an outgoing pipe. In such cases, these are not defects and should not be recorded as defects. For a chamber containing a fully integrated flow control device such as an orifice plate, the chamber is assessed as below, and the flow control as per the tables in Appendix C12. The overall grades for the asset are based on the highest scoring structural and service defects. Table C-15: Defect-level structural condition assessment of chambers Score Accepted HADDMS Code Equivalent EN Code (not to be used) Defect type Description / quantification Deformed Localised – affecting small proportion of wall 80 DS DAAB General – affecting large proportion of wall 165 DL DAAA Vertical 5 HFV DABAA Horizontal 5 HFH DABAB Inclined 5 HFIN DABAD Multiple 5 HFM DABAC Vertical 20 CV DABBA Horizontal 20 CH DABBB Inclined 20 CIN DABBD Multiple 40 CM DABBC Vertical 40 FV DABCA Horizontal 40 FH DABCB Inclined 40 FIN DABCD Multiple 80 FM DABCC Break – pieces visibly displaced 40 B DACA Missing – missing pieces of wall 80 H DACB Collapse – complete loss of structural integrity 165 XPV DACC / DACCA Surface crack Cracked Fractured Break Guidance note on drainage data formats, Issue 1. August 2012 66 of 93 Defective brickwork or masonry Displaced – units displaced from original position 20 DB DADA Missing – units missing 80 MB DADB Collapse – complete loss of structural integrity 165 XB DADC Missing mortar Missing mortar 20 MM DAE Surface damage Increased roughness 10 SW DAFA Spalling – breaking away of small fragments 20 SS DAFB Visible aggregate 20 SAV DAFC Aggregate projecting from surface 20 SAP DAFD Missing aggregate 40 SAM DAFE Visible reinforcement 40 SRV DAFF Reinforcement projecting from surface 80 SRP DAFG Corroded reinforcement 80 SRC DAFH Missing wall 165 SH DAFI Corrosion products on surface 20 SCP DAFJ Intruding and defective (see service condition for nondefective intruding connection) 20 CXI DAG Position is incorrect 20 CXP DAHA Gap – circumferential gap between wall and pipe 80 CXG DAHB Gap – between end of pipe and chamber wall 80 CXH DAHC Damaged – connecting pipe damaged 80 CXD DAHD Visibly displaced but not intruding 40 SRN DAIAA Hanging but not broken 80 SRH DAIAB Broken seal 80 SRB DAIAC Unpointed joint 10 UJ – Units displaced vertically 20 OJ DAJA Units displaced horizontally 80 JD DAJB Defective connection Intruding seal Defective joint Guidance note on drainage data formats, Issue 1. August 2012 67 of 93 Repair Defects (other) Defective step or ladder Defective cover Angular displaced units – axes of units not parallel 80 AJ DAJC Intact wall repair 10 RPR DCBA Intact localised lining 10 RPL DCBB Injection of sealant 10 RPI DCBC Missing wall (part) 80 RXM DALA Defective patch sealing hole 80 RXP DALB Porous wall 40 PP DAN Soil visible through defect 165 SV DAO Void visible through defect 165 VV DAP Broken plastic encapsulation of step 20 SXP DAQE Step corroded 80 SXC DAQC Step bent 80 SXB DAQD Step loose 165 SXL DAQA Step missing (where provided) 165 SXM DAQB Ladder handrail corroded 20 SXH DAQF Ladder runner corroded 40 SXRC DAQJ Ladder support loose 80 SXSL DAQG Ladder support missing 80 SXSM DAQH Ladder support corroded 80 SXSC DAQI Ladder handrail loose 165 SXHL – Ladder handrail missing 165 SXHM – Defective toe hold 40 SXT DAQK Defective hand hold 80 SXQ – Rocking cover 20 TXCR DARB Cover below surface level 20 TXCD DARG Cover above surface level 20 TXCU DARH Unpointed lifting holes 20 TXCP – Broken cover 80 TXCB DARA Seized cover 80 TXCS – Missing cover 165 TXCM DARC Guidance note on drainage data formats, Issue 1. August 2012 68 of 93 Defective frame Loose frame 40 TXFL DARE Broken frame 80 TXFB DARD Missing frame 165 TXFM DARF Benching Benching defective (e.g. spalling) 40 BEX DCHA Channel Channel defective 40 CHX DCIA Safety feature Safety chain missing – evidence of attachments 165 SFCM DCJB Safety chain defective 165 SFCX DCJC Safety bar missing – evidence of attachments 165 SFBM DCJF Safety bar defective 165 SFBX DCJG Pit is defective 165 CPX DCNB Silt pit Table C-16: Defect-level service condition assessment of chambers Score Accepted HADDMS Code Equivalent EN Code (not to be used) Defect type Description / quantification (CSA = cross-section area) Roots Independent fine roots 2 RF DBAB Tap root 5 RT DBAA Complex mass 10 RM DBAC Encrustation 1 DEE DBBA Grease 1 DEG DBBB Fouling – organisms attached to wall 2 DEF DBBC Attached deposits Settled deposits Settled deposits fine: DES DBCA (excluding soakaways and catchpits) Settled deposits coarse: DER DBCB Settled deposits hard: DEC DBCC Other settled deposits: DEX DBCZ OBDA – (same quantification bandings apply to each code) <5% CSA loss (and below invert of outgoing pipe) 1 5%–19% 5 20%–49% 8 50%–74% 10 2 75%+ Settled deposits Sediment filling catchpit sump 4 Guidance note on drainage data formats, Issue 1. August 2012 69 of 93 (catchpits only) Sediment obstructing flow <50% pipe diameter 8 OBDB – Sediment obstructing flow 50%+ pipe diameter 10 OBDC – Soakaway part full of sediment 5 OBDD – Soakaway full of sediment and/or water 10 OBDE – Intruding above flow level 0.5 CNIA – Intruding into flow or through benching 10 CNIX – Blocked connection Blocked – connecting pipe blocked 10 CXB DAHE Obstacles and debris Dislodged brick/masonry in invert 2 OBB DBEA Pieces of broken pipe in invert 2 OBM DBEB Other object or debris in invert 2 OBZ DBEC Protruding through wall 1 OBI DBED Wedged in joint 1 OBJ DBEE Entering through connection 5 OBC DBEF External pipe or cable through chamber 5 OBP DBEG Built into structure 5 OBS DBEH Sweating 1 IS DBFA Dripping – non-continuous flow 2 ID DBFB Flowing – continuous flow 5 IR DBFC Gushing – entering under pressure 5 IG DBFD Sealed pipe through chamber Access cover missing 20 SPM DCLC Contamination Presence of oil or other contaminant 10 OIL – Settled deposits (soakaways) Intruding connection Infiltration Guidance note on drainage data formats, Issue 1. August 2012 70 of 93 C10. Highway Drainage Gullies – Defect-level assessment Inventory item type(s): Gully (GU, GY) The following codes also apply to sumpless gullies and kerb inlets. Table C-17: Defect-level structural condition assessment of gullies Score Accepted HADDMS Code Intact 1 QGAA Rocking 20 QGAB Missing frame bedding 20 QGAC Cracked 40 QGAD Void forming adjacent to frame 80 QGAE Failure of surrounding pavement 80 QGAF Missing bar 80 QGAG Broken or collapsed 165 QGAH Missing grating 165 QGAI Missing frame and grating 165 QGAJ Grating bars parallel to road 165 QGAK Intact 1 QGBA Missing mortar 20 QGBB Broken/fractured brick 40 QGBC Missing brick 80 QGBD Broken section 80 QGBE Unsuitable material (slate, timber etc) 80 QGBF Intact 1 QGCA Missing stopper 20 QGCB Cracked 20 QGCC Fractured pot 40 QGCD Broken 80 QGCE Leaking 80 QGCF Collapsed 165 QGCG Defect type Description Grating & Frame Adjustment Course Pot Guidance note on drainage data formats, Issue 1. August 2012 71 of 93 Position In channel 1 QGDA Above surface level 10 QGDB >25mm below surface level 40 QGDC 75–125mm in front of kerb line / channel edge 80 QGDD >125mm in front of kerb line / channel edge 165 QGDE Table C-18: Defect-level service condition assessment of gullies Score Accepted HADDMS Code 0.5 QGPA Partially obstructed leaves/litter 2 QGPB Completely obstructed leaves/litter 5 QGPC Blocked by sediment 10 QGPD No sediment 0.5 QGQA Some sediment within sump 1 QGQB Sediment filling sump, outlet not obstructed 2 QGQC Sediment obstructing outlet, no water 5 QGQD Sediment to grating level, pot full 10 QGQE Thick layer of oil 5 QGRA Defect type Description Grating Clear Sediment Contamination Guidance note on drainage data formats, Issue 1. August 2012 72 of 93 C11. Highway Drainage Inlets & Outlets – Defect-level assessment Inventory item type(s): Grip Inlet (GI), Inlet (IT), Outfall (OU, OF), Outlet (OL) For an inlet/outlet containing a fully integrated flow control device such as a flap valve, the inlet/outlet is assessed as below, and the flow control as per the tables in Appendix C12. The overall grades for the asset are based on the highest scoring structural and service defects. Some observations are only applicable if the inlet/outlet has a formal structure. The asset’s inventory attribute data should clearly indicate the presence or lack of this. Table C-19: Defect-level structural condition assessment of inlets/outlets Score Accepted HADDMS Code No defect 1 QIAA Settlement of piped outlet 20 QIAB Fractured pipe 40 QIAC Displaced pipe (differential settlement) 80 QIAD Collapsed pipe 165 QIAE Structure – formal Cracked mortar joints 20 QIBA (e.g. headwall) Loose concrete 20 QIBB Insecure bagwork or gabions 40 QIBC Cracked brickwork 40 QIBD Missing bricks 40 QIBE Spalling concrete 40 QIBF Exposed reinforcement 40 QIBG Displaced bagwork or gabions 80 QIBH Fractured structure (concrete or brickwork) 80 QIBI Differential settlement 80 QIBJ Collapsed structure 165 QIBK Erosion of bank 40 QICA Collapsing bank around pipe 80 QICB Failure of bank 165 QICC Inclined screen to outlet 1 QIDA Inclined screen to inlet 10 QIDB Defect type Description Structure – general Structure – banks Screen (where required) Guidance note on drainage data formats, Issue 1. August 2012 73 of 93 Loose fixing 20 QIDC Free standing 20 QIDD Deformed bars 40 QIDE Vertical screen to outlet 40 QIDF Buckled / broken 80 QIDG Vertical screen to inlet (unsafe) 165 QIDH Safety features No safety-related defects 1 QIEA (where required) Remote from structure 10 QIEB Loose rails 80 QIEC Loose or insecure guardrail posts 80 QIED Missing guardrail 165 QIEE Missing fence 165 QIEF Missing harness eye 165 QIEG Missing access path 165 QIEH Table C-20: Defect-level service condition assessment of inlets/outlets Score Accepted HADDMS Code 0.5 QIPA Sediment to pipe invert 1 QIPB Sediment <25% pipe diameter 2 QIPC Sediment 25%–75% pipe diameter 5 QIPD Sediment >75% pipe diameter 10 QIPE No debris 0.5 QIQA Light debris not obstructing flow 1 QIQB Debris causing standing water 2 QIQC Debris obstructing flow or partially blocked screen 5 QIQD Flow blocked or surcharging by debris 10 QIQE Defect type Description Sediment No sediment, and invert is above channel bed Debris Guidance note on drainage data formats, Issue 1. August 2012 74 of 93 C12. Highway Drainage Miscellaneous Assets – Defect-level assessment Inventory item type(s): Flow Control Device (FC), Interceptor (IN), Oil Separator (OS), Pumping Station (PS). As noted in previous sections for specific point asset types, the following codes are also used to record the condition of flow control devices that are fully integrated within another point asset, for example a chamber with an orifice plate. The overall grades for the asset are based on the highest scoring structural and service defects. Where the flow control device is a standalone device with an item type code of “FC”, the asset is assessed against the below items only. Table C-21: Defect-level structural condition assessment of miscellaneous assets Score Accepted HADDMS Code No defect 1 QAAA Surface cracking or rust 10 QAAB Structural crack 40 QAAC Fractured 80 QAAD Poor condition but functioning 80 QAAE Broken 165 QAAF Unsafe 165 QAAG Damaged baffle 40 QABA Loose or unseated flap valve 80 QABB Missing vent pipe from oil separator 80 QABC Missing weir sections 80 QABD Seized flap valve 165 QABE Missing handrail where required for safety 165 QABF Defect type Description General structure Component specific Table C-22: Defect-level service condition assessment of miscellaneous assets Defect type Description Score Accepted HADDMS Code General service condition Unobstructed 0.5 QAPA 1 QAPB Minor accumulation of litter, sediment and/or vegetation Guidance note on drainage data formats, Issue 1. August 2012 75 of 93 Contamination Flow slightly impeded 2 QAPC Flow severely impeded 5 QAPD Blocked causing surcharge 10 QAPE Asset poses a pollution risk 10 QAQA Oil separator full (applies to item type code “OS” only) 10 QAQB Guidance note on drainage data formats, Issue 1. August 2012 76 of 93 C13. Region items Inventory item type(s): Detention Pond (DP), Infiltration Basin (IB), Pollution Containment Pond/Tank (PC), Reed Bed Treatment System (RB), Retention Pond (RP), Sediment Pond (SP), Wetlands (WL), Pond (undifferentiated) (PU) Table C-23: Defect-level structural condition assessment of region items Score Accepted HADDMS Code No slope defects 1 QPAA Slope disturbance 20 QPAB Slope movement 40 QPAC Slope slippage 80 QPAD Slope failure 165 QPAE No burrowing 1 QPBA Minor burrows on non water-retaining slope 40 QPBB Large burrows on non water-retaining slope 80 QPBC Burrows on water-retaining slope 165 QPBD Liner visible in bank 40 QPCA Liner exposed and displaced in bank 80 QPCB Liner torn or split 165 QPCC Vehicular access unobstructed 1 QPDA Vehicular access difficult 20 QPDB Vehicular access track obstructed 40 QPDC No constructed vehicular access 80 QPDD No vehicular access 165 QPDE Intact and secure fence 1 QPEA Insecure security fence 80 QPEB No security fence 165 QPEC Missing safety equipment (e.g. lifebelt) where required 165 QPED Pond has vertical sides with no means of escape 165 QPEE Defect type Description Stability Burrows Liner Access Safety Guidance note on drainage data formats, Issue 1. August 2012 77 of 93 Table C-24: Defect-level service condition assessment of region items Defect type Description Score Accepted HADDMS Code Sediment No sediment 0.5 QPPA Visible sediment not impeding flow 1 QPPB Sediment levels <25% inlet/outlet diameter 2 QPPC Pools formed due to sediment 5 QPPD Sediment levels >25% inlet/outlet diameter 5 QPPE Sediment blocking inlet/outlet 10 QPPF No invasive vegetation 0.5 QPQA Invasive vegetation on banks 1 QPQB Invasive vegetation in pond 2 QPQC Invasive shrubs in pond 5 QPQD Tree(s) in pond that may affect liner 10 QPQE No litter or debris 0.5 QPRA Litter – accumulation 2 QPRB Debris – building products 4 QPRC Debris – other large items not requiring mechanical removal (e.g. due to flytipping) 8 QPRD Debris – items requiring mechanical removal (due to size or location) 10 QPRE Vegetation Debris Guidance note on drainage data formats, Issue 1. August 2012 78 of 93 Appendix D. HADDMS Piped Drainage CCTV Survey Records D1. Introduction D1.1 This Specification details the requirements for compiling, locating, indexing and submitting drainage survey data in WinCan 7 or DrainageXML format to the HADDMS team for upload to HADDMS, together with the supporting photographs. D1.2 The tasks outlined in this Specification are intended to be readily achievable, by utilising an existing data format and a series of simple indexing tools. D1.3 Submission of the data follows different routes depending on the version of WinCan used: D1.3.1 WinCan 7.6/7.7 – instructions for indexing and providing the data by email or on DVD or hard disk given in section D3. D1.3.2 WinCan 8 or later (subject to availability of the necessary software) – instructions for conversion to DrainageXML format and direct upload by the Service Provider given in section D4. D1.4 Video data is to be retained by the Service Provider. Storage location of this is to be noted by the Service Provider in the corresponding scheme details on HADDMS. D2. WinCan data processing D2.1 This section describes the work required on the WinCan data, prior to compiling it for indexing and submission. D2.2 Observation codes are to be in accordance with those shown in the tables in section C2. Data conversion and location D2.3 Existing drainage CCTV data held in WinCan format must be imported into WinCan version 7.6 (dated 30/01/2009) or later. The WinCan database also needs to be converted from the standard MSCC 4th Edition Highways format to that required for HD43 utilising a conversion macro. If a database has been split it must be re-joined such that one database file contains all data for one scheme. D2.3.1 Older databases captured to MSCC 3rd Edition are not coded to the requirements of SD15/03 and will require extensive re-working to convert to an acceptable HA format, in addition to the above. D2.3.2 For newly recorded data in WinCan 8, the HD43 template must be used from the outset. D2.4 It is essential that every WinCan survey run is located with the OS grid coordinates of the start and end manholes being embedded in the WinCan database. Most historic WinCan data will be missing this information and it may need to be sourced from other records. In order to add this information the WinCan HD43 extension module is required. Without these location coordinates, the data cannot be fully utilised within HADDMS. Guidance note on drainage data formats, Issue 1. August 2012 79 of 93 D2.4.1 The only permitted exception to the above is a survey run in a pipe such as a gully connector that ends at another surveyed pipe, part way between manholes. • The start point coordinates and reference (e.g. for the gully) must be specified as normal; however, the end point of the run must be specified as the Pipe Length Reference (PLR) of the main pipe. • No coordinates are to be provided for this end point. • The survey of the main pipe must include a connection observation that specifies the reference of the start point of the connecting pipe as the “upstream reference”, by using the Connector Form facility in the WinCan HD43 module. • The main pipe with the connection must have OS coordinates at both ends; surveys of a connector pipe running into another connector pipe and then into a main pipe are not supported. D2.4.2 Survey runs in pipes ending at a connection into a sewer where there is no manhole, only fall into the exception of D2.4.1 if the sewer has also been surveyed in the same set of WinCan data with all required matching connections recorded. Otherwise, coordinates of the connection must be determined in the field, for example using a sonde. A unique point reference must be given for this connection such that any other pipes ending at different connections to the same sewer do not have the same end reference. If this end point marks the exit from HA-owned drainage, this point would be coded as an “outfall”. D2.4.3 “Satellite” survey runs, which start at a connection within a main pipe, are not currently supported. D2.5 References shall be assigned to the survey run start and end manholes that must be unique within the scheme. Only where two surveyed pipes meet at the same manhole, or at connections into the same main pipe, should the same reference be used. There is no restriction on the format of these references, and HD43 style references should be avoided as these may conflict with HADDMSassigned references. Where a survey ends at a connection with a main pipe, the end reference must match the PLR of the main pipe. D2.6 WinCan automatically assigns a PLR to a survey run based on the upstream node reference, with an “X” appended. If multiple survey runs in either or both directions are present in a single pipe, the PLR for each run must match. HADDMS will aggregate such surveys together onto a single pipe, retaining the latest surveyed observations for each part of the pipe. D2.6.1 Survey runs in other pipes must not use the same PLR. A survey run is considered to be in a different pipe if it has a different pair of start and finish node references (not including where a pair is swapped due to surveys in each direction, as per D2.6). Where two pipes share the same upstream node, it is common for the same PLR to be assigned to both – this is incorrect, and in such cases the final “X” character must be changed for the PLR of all survey runs in one of the pipes. D2.6.2 If there are two separate pipes between the same pair of start and finish nodes, these must have a different PLR so that runs in both pipes are treated separately. D2.6.3 A PLR of a pipe must not match either its start or finish node reference. Guidance note on drainage data formats, Issue 1. August 2012 80 of 93 D2.7 Where a survey run was abandoned (e.g. due to a blockage), the end manhole that the survey was aiming for should be given in the header. The termination point will be recorded as an observation with chainage from the start as normal. D2.8 Only the OS locations of start and end manholes are required. Do not calculate OS locations of the various observations along each pipe, or connections between pipes. D2.9 It is not anticipated that any other data in the WinCan database will need to be added, amended or removed in order to comply with HADDMS requirements. D3. Submission procedure – WinCan 7.6/7.7 data D3.1 Once processed to the requirements of section D2, data in WinCan 7.6/7.7 format is submitted for upload to HADDMS as follows: D3.1.1 Zip the MDB file only and attach to a covering email. The ZIP file must contain only one MDB file, which contains all survey data for the scheme. If multiple schemes are to be submitted, these must be sent separately. D3.1.2 In the covering email, state clearly the following: • HA Area or DBFO number/name • HA Road(s) • HADDMS scheme reference • HADDMS scheme title • Advise whether photographs referenced in the WinCan database are available, and their total filesize – these should not be provided at this stage. D3.1.3 Send the email with attached zipped MDB file to support@haddms.com, including the HADDMS scheme reference in the Subject line. D3.1.4 The HADDMS team will check the data for compliance with section D2 and either return a check report to the sender, or advise that it complies. If data is not fully compliant (e.g., not all survey runs in a historical survey could be located) it may still be uploaded to HADDMS with non-compliant survey runs omitted; however, this will be with agreement of the Service Provider for the Area/DBFO. D3.1.5 Following the above checks and if photographs for the CCTV survey are available, the HADDMS team will liaise directly with the data supplier on how to provide these prior to upload of the CCTV data to HADDMS, based on their total filesize. Video files are retained by the Service Provider. D3.1.6 The HADDMS team will upload the data to the system together with any photographs, using the scheme information originally provided. Alternatively, the HADDMS team can convert the data to DrainageXML format and provide this to the data supplier for upload with any photos by the Service Provider. D3.1.7 The Service Provider is responsible for providing further scheme information on HADDMS; only the minimum data above will be entered by the HADDMS team. D3.1.8 Once the CCTV data has been uploaded to HADDMS, it may be downloaded by the Service Provider in the standard SHP format as defined in Appendix A. Any revisions to such data (for example to add above ground assets) should be carried out in accordance with general guidance in Appendix A and the specific guidance in section D5. Guidance note on drainage data formats, Issue 1. August 2012 81 of 93 D4. Submission procedure – WinCan 8 data D4.1 To produce WinCan 8 format data with OS eastings and northings of manholes, the following software is required. • WinCan version 8 • WinCan 8 HD43 extension module This is subject to availability of both of the above; until such time, the procedure for WinCan 7.6/7.7 must be followed instead. Alternatively, other software that can produce DrainageXML format data for HADDMS may be used. D4.2 Once processed to the requirements of section D2, WinCan 8 should be used to convert the data to DrainageXML format. D4.3 Contractors may not upload data into the HADDMS database, but may carry out a check upload before submission to the Service Provider. D4.4 The DrainageXML format data is zipped, together with any photos and uploaded directly to HADDMS in the same way as for SHP data by the Service Provider. Video files must not be included in the ZIP file. The “XML” option must be selected when uploading the data. D4.5 The data will be checked by HADDMS in the same way as for SHP files for uniqueness of referencing, etc. It is advised that you carry out similar checks on the data before uploading it to HADDMS. D5. Combining CCTV data with non-CCTV SHP files D5.1 Service Providers are recommended to follow the procedure below for provision of data combining CCTV and non-CCTV surveys of the same drainage system. D5.1.1 Provide the CCTV data as described in sections D3 or D4 as appropriate. D5.1.2 Once the CCTV data has been imported into HADDMS it can be downloaded in SHP format by the Service Provider, where the remainder of the assets can be added to create a single set of SHP files containing all assets. D5.1.3 “System” fields (as described in Appendix A) must not be changed in the data that has been downloaded from HADDMS. In particular: • Supplier’s and HADDMS asset references must not be edited or deleted. • For assets being added, HADDMS asset reference fields must be left blank, as these will be generated by HADDMS. D5.1.4 The observations recorded in the CCTV survey will be present in a file named “observation.dbf”, the format of which is described in Appendix A7. This file should not normally be altered and it must be re-uploaded with the SHP file data. D5.1.5 The Service Provider should then re-upload the whole scheme to HADDMS as a new version of the same scheme. D5.2 It is strongly recommended that SHP files are not manually created from WinCan data, as this procedure is fully automated by HADDMS including scoring and grading of assets from the survey observations. Manual production of SHP files Guidance note on drainage data formats, Issue 1. August 2012 82 of 93 from WinCan data has historically resulted in the condition data being omitted, although this may be manually prepared in the format described in Appendix A. D5.3 Further guidance and recommendations for producing data suitable for HADDMS are available from the contact email address at the beginning of this document. D5.4 Further assistance with uploading suitable data to HADDMS is provided in the HADDMS User Manual, available from the Downloads page of HADDMS. Guidance note on drainage data formats, Issue 1. August 2012 83 of 93 Appendix E. HADDMS Drawing Records Specification E1. Introduction E1.1 This Specification details the requirements for compiling and submitting electronic drawing records as Located Files and Georeferenced Drawings to the HADDMS team for upload to HADDMS. E1.2 Example datasets and file indexes are available for download from the download page of HADDMS. E1.3 Located Files and Georeferenced Drawings are grouped into Drawing Sets, comprising all drawings for a maintenance or construction project, survey or other scheme. E1.4 Each Drawing Set can be categorised as drainage, geotechnical or archaeological. E1.5 This Specification supersedes the previous requirement to group Located Files and Georeferenced Drawings within a “drainage scheme”. Drainage drawings can continue to be submitted in the format published in the Draft IAN on HADDMS Data Population (April 2008). Drawings provided in this format will be automatically imported to HADDMS as Drawing Sets with a “drainage” category. However, some additional flexibility including resolution of several known issues is available by using the method described in this updated Specification. E1.6 Non-drawing files may not be included in Drawing Sets and must be uploaded to the relevant database on HADDMS, e.g. as a report, document attached to a drainage scheme or asset, etc. E2. Summary of requirements E2.1 Service Providers are required to compile, index and submit their drawing records in the structure and format described in this specification. There are two main types of files that will be uploaded to HADDMS: Located Files and Georeferenced Drawings, which are defined below. E2.2 Located Files can be any type of drawing file related to any aspect of the HA drainage or geotechnical asset or relevant archaeological drawings. They are located as a rectangle on the map, covering either the entire extent of a Drawing Set or a specific section of road within it. HADDMS users can click the index rectangle to download the file. An example of this functionality on HADDMS is shown in Figure E-1. Guidance note on drainage data formats, Issue 1. August 2012 84 of 93 Figure E-1: Display of Located Files on HADDMS E2.3 Georeferenced Drawings are the map-based elements of drawings that have been included as Located Files, and will be displayed as a mapping layer on HADDMS, aligned with the background Ordnance Survey MasterMap mapping. These will be provided as monochrome GeoTIFF files. An example of this functionality on HADDMS is shown in Figure E-2. Figure E-2: Display of georeferenced drawings on HADDMS E2.4 It is anticipated that many records will only currently exist on paper; therefore, these will require scanning. Where electronic versions already exist, there is no requirement to produce paper versions. E2.5 A Drawing Set Index will list the Drawing Sets to which the provided files relate. Guidance note on drainage data formats, Issue 1. August 2012 85 of 93 E2.6 A File Index will list and provide coordinates for all of the Located Files, and specify the Drawing Set to which they relate. Located Files may only belong to a single Drawing Set. E2.7 Files are to be arranged in a single folder per Drawing Set with no subfolders, and submitted on DVD or USB hard disk as described later in this specification. Each Service Provider is encouraged to submit batches of Drawing Sets as their work progresses, to ensure that file formats and structures are correct. E3. Scanning of paper records requirements E3.1 Where records only exist as paper, they will need to be scanned prior to indexing. There is no requirement to print and scan records that are held electronically, provided they can be submitted within the requirements of this Specification. E3.2 Standard scanning specification for paper records is as follows: • 300 dots per inch • monochrome (black and white) Following scanning, they will be indexed and provided as described in the rest of this Specification. Guidance E3.3 Monochrome scanning is anticipated to be sufficient except in the following cases: • Colour scanning shall only be used where this is essential for interpreting the records. Documents must not be scanned in colour for the sole purpose of displaying company logos etc correctly. • Greyscale scanning shall only be used for poor quality paper drawings where elements of the original drawing may be so faint that they would disappear if scanned correctly in monochrome. Greyscale should otherwise not be used, as this will considerably increase file sizes, will enhance imperfections in the paper surface, and will not improve quality when a reasonable quality original is re-printed on black-and-white equipment. E4. Drawing Set index requirements E4.1 Each Service Provider’s records will be split into “Drawing Sets”. A Drawing Set is a self-contained set of drainage records, originating from a single source and at a given point in time. A Drawing Set could comprise records from a CCTV survey, or paper as-built records for a maintenance or construction project. It is each Service Provider’s responsibility to determine appropriate Drawing Sets for their data. E4.2 Within each submission of records, there must be one Drawing Set Index, which is a CSV file generated from a Microsoft Excel spreadsheet containing one record (row) per Drawing Set in that submission. E4.3 A Microsoft Excel spreadsheet is provided to generate the Drawing Set Index. Further instructions for using the spreadsheet are included in it. Data is validated by the spreadsheet before the Drawing Set Index can be generated. Guidance note on drainage data formats, Issue 1. August 2012 86 of 93 File naming and location E4.4 The filename of the Drawing Set Index is to be drawing_set_index.csv, and it must be located in the root folder of the first DVD of each submission, or hard disk. E4.5 The spreadsheet used to create the Drawing Set Index and File Index must also be included in the root folder. Index contents E4.6 The following index data is required for each drawing set: E4.6.1 Area (number of current HA Area or DBFO contract). E4.6.2 Drawing Set Number (this must be unique within an Area or DBFO contract). E4.6.3 Drawing Set Reference (automatically generated by the spreadsheet as Area_DrawingSetNumber, e.g. “06_1234”, “12_0123”). See “Import Action” for how drawing sets are processed if they already exist on HADDMS. E4.6.4 Category (archaeology, drainage or geotechnics) E4.6.5 Roads (can be a comma separated list, e.g. “M4,A34,A329M” or a single road – these must be in the same format as the HAPMS definition; do NOT enter any other detail, e.g. road name or location in this box) E4.6.6 Drawing Set Title (e.g. ‘A123 Sometown By-pass’). E4.6.7 Location From (descriptive location e.g. “M25 Junction 6”). E4.6.8 Location To (descriptive location e.g. “Mile End roundabout”). E4.6.9 Drawing Type (one of “Design”, “As-built”, “Field survey”). E4.6.10 Original Creator (name of organisation who produced original records, such as the designer or construction contractor named on the drawings, or the survey contractor named on the CCTV survey). E4.6.11 Original Year (original year records were produced in ‘YYYY’ format, e.g. “1981”). E4.6.12 Submission Date (date of submission of data to HADDMS team in ‘YYYY-MMDD’ format, e.g. “2008-01-31”). E4.6.13 Submitted By (name of submitting organisation). E4.6.14 Contact Person Name (name of contact person in submitting organisation). E4.6.15 Contact Person Email (email address of contact person in submitting organisation). E4.6.16 Contact Person Phone (telephone number of contact person in submitting organisation). E4.6.17 Notes (any other relevant information - optional). E4.6.18 Import Action “new”, “merge”, “replace” or can be left blank: • if left blank: if a Drawing Set with the same reference already exists on HADDMS, the submitted data will be merged with it (see below); if it does not exist then it is treated as a new Drawing Set. In most cases leaving this field blank will suffice and is the safest option. Guidance note on drainage data formats, Issue 1. August 2012 87 of 93 • new: assumes the Drawing Set does not already exist on HADDMS and will be created from scratch. If a Drawing Set with the same reference does already exist, it will be completely replaced with the newly submitted data. • merge: the Drawing Set already exists on HADDMS and drawings will be added to it if their filename does not match an existing Located File. If the filename does match an existing Located File, the existing file and any georeferenced drawings associated with it will be replaced. • replace: the Drawing Set already exists on HADDMS and will be completely replaced by this submission. Guidance E4.7 Drawings from different sources and of different ages for the same section of road should be treated as separate Drawing Sets. For example, a section of road may have the following records available: • original design drawings from 1968 • as-built drawings from 1971 • refurbishment records from 2002 • condition survey from 2005 Each of the above four sets of data should be treated as a separate Drawing Set, even if they cover the same, or overlapping sections of road. E5. Located Files requirements E5.1 Located Files can be any type of drawing related to the HA asset, including inventory and condition records. E5.2 Each Located File must be related to one Drawing Set only. E5.3 No executable files (.exe, etc) will be accepted, including “self-extracting” files. File naming and folder structure E5.4 Located Files for a Drawing Set must be provided in a folder named with the Drawing Set Reference, e.g. “12_0123”. No subfolders within Drawing Set folders are permitted – files in such subfolders will not be imported. E5.5 Located Files may have any reasonable filename, which may be a number or a more descriptive title. In addition to the standard forbidden characters in filenames, located files must not use the underscore (“_”) character in their filename. The correct file extension (e.g. .tif, .pdf) must be included. Located Files within a Drawing Set must have different filenames. Indexing E5.6 Each submission is to have a File Index, which is a CSV file generated from a Microsoft Excel spreadsheet containing one row per Located File within each Drawing Set being submitted. Guidance note on drainage data formats, Issue 1. August 2012 88 of 93 E5.7 The filename of the File Index is to be located_files_index.csv, and it must be located in the root folder of the first DVD of each submission, or hard disk. E5.8 A Microsoft Excel spreadsheet is provided to generate the File Index. Further instructions for using the spreadsheet are included in it. Data and files are validated by the spreadsheet before the File Index can be generated. E5.9 The spreadsheet used to create the Drawing Set Index and File Index must also be included in the root folder. Index contents E5.10 All fields (columns) are mandatory unless indicated and must be completed for each Located File, as follows: E5.10.1 Drawing Set Reference (as given in the Drawing Set Index, e.g. “12_0123”). E5.10.2 File Name (name of the file as submitted on the disk including the file extension, e.g. “00040.tif”) E5.10.3 File Type (e.g. “TIF”, “PDF” – if submitting a ZIP file, the file type will be the main type of file contained within the zipped file, e.g. “DWG”, “SHP”). E5.10.4 Title (title of drawing). E5.10.5 Reference (any reference shown on the original record, e.g. drawing number – leave blank if no reference available). E5.10.6 Number of georeferenced drawings (the number of submitted georeferenced drawings derived from this located file; if none enter ‘0’) E5.10.7 MinE (the minimum OS Easting covered by the record as a six-figure number – i.e. the westernmost point, e.g. “409995”). E5.10.8 MinN (the minimum OS Northing of the record as a six-figure number – i.e. the southernmost point, e.g. “182632”). E5.10.9 MaxE (the maximum OS Easting of the record as a six-figure number – i.e. the easternmost point, e.g. “410488”). E5.10.10 MaxN (the maximum OS Northing of the record as a six-figure number – i.e. the northernmost point, e.g. “183198”). Guidance File format guidance E5.11 If Located Files can be generated electronically as PDF/DWF, then this is preferable to printing and scanning. E5.12 It is expected that the majority of the files will be of the following formats: • Scanned drawings in TIFF format, with each drawing as a separate file – multi-page TIFFs are not permitted. • Drawings in an “electronic paper” format such as PDF or DWF. PDF files may be electronically created from the original files, or be scanned drawings. • Drawings in original CAD format such as AutoCAD DWG should be provided as an “e-Transmit / pack-and-go” ZIP file, containing all xrefs etc to Guidance note on drainage data formats, Issue 1. August 2012 89 of 93 allow the drawing to be correctly re-opened. However, licensed or copyright files such as Ordnance Survey map tiles, must NOT be included. • ZIP – If providing a set of files that must be used together, these should be zipped together and submitted as one located file. Where files can be used separately, e.g. separate DWF drawings, PDFs or TIFFs, these must be submitted as separate located files. Files within ZIP files can be named as required, and only the ZIP file itself is included in the file index. Indexing guidance E5.13 The MinE, MinN, MaxE and MaxN fields shall give a rectangular area, oriented north-south, east-west, which covers the section of HA network to which the file relates. As the Drawing Set Index specifies the Road(s) the Drawing Set relates to, it does not matter that this rectangle may overlap other roads, nor that it may overlap other drawings. E5.14 Due to the anticipated number of files, data is processed by automated methods and it is not possible to process individual items manually. Therefore, do not add comments regarding handling individual items, as these will either be ignored or the submission rejected. E6. Georeferenced Drawings requirements E6.1 Georeferenced Drawings are a processed sub-set of the drawings that have been provided as Located Files. Only those drawings containing map-based elements are included as Georeferenced Drawings; furthermore, any nonmapping segments (borders, title blocks, etc.) of those drawings are excluded. E6.2 Where a Located File contains two or more mapping segments, each segment is provided as a separate Georeferenced Drawing. Georeferencing requirements E6.3 Georeferenced Drawings may originate from scanned paper records, or be generated using software from the original CAD drawing files. In either case, Georeferenced Drawings are to be submitted as single-page monochrome TIFF files only, compressed using a method such as ‘FAX – CCITT 4’ and with a separate world file (TFW). Where Located Files are scanned and submitted in colour or greyscale, these must be converted to monochrome for the Georeferenced Drawings. E6.4 Drawings must be georeferenced to OS Grid such that they align to OS MasterMap including Positional Accuracy Improvement (PAI). E6.5 When georeferencing scanned drawings to the OS mapping, sufficient control points must be used to give accurate matching. Normally a first rough match is carried out using four points, then improved with approximately 15 points and ‘rubber-sheeting’ techniques to obtain the best match possible across the area of the mapping, with the objective of locating each asset to within 1m accuracy. However, it is accepted that with older as-built records it may not be possible to achieve 100% precise registration with current OS mapping, and in such cases the objective should be to achieve the best fit possible of the maximum area of the map. Guidance note on drainage data formats, Issue 1. August 2012 90 of 93 File naming E6.6 A Georeferenced Drawing derived from the located file “12345.tif” will be named “12345_1.tif”, with an accompanying world file, “12345_1.tfw”. (Please note in section E5.5 that located files must not contain the underscore character in their filename.) E6.7 Where a single drawing uploaded as a Located File contains multiple segments of map, each shall be cropped and georeferenced separately. Such Georeferenced Drawings will be named “12345_1.tif”, “12345_2.tif”, etc, each accompanied by a similarly named world file. Indexing E6.8 Georeferenced Drawings do not require separate indexing, as they must relate to files also submitted as Located Files. E6.9 The Located File Index contains a column for entering the number of Georeferenced Drawings derived from each located file. The indexing spreadsheet will check that these files are present before generating the file index. Examples Georeferenced drawing: 12345_1.tif 12345_1.tfw Located file: 12345.tif Figure E-3: A Located File with one mapping segment Guidance note on drainage data formats, Issue 1. August 2012 91 of 93 Georeferenced drawing: 00521_1.tif 00521_1.tfw Located file: 00521.tif Georeferenced drawing: 00521_2.tif 00521_2.tfw Figure E-4: A Located File with two mapping segments E7. Data submission requirements E7.1 Service Providers responsible for more than one Area or DBFO must submit each Area or DBFO’s data separately. E7.2 Data is to be submitted on DVD or USB external hard disk as appropriate to the volume of data. For large datasets, hard disk submissions are preferred to multiple DVDs. E7.3 Each submission must stand-alone; data and indexes for the Drawing Sets within a submission must be complete. Georeferenced Drawings must not be submitted without their accompanying Located File. E7.4 The data for a single Drawing Set must not span across multiple DVDs; Drawing Sets that are too large for a single DVD must be submitted on hard disk instead. E7.5 Service Providers are encouraged to make staged submissions; in particular, Service Providers should initially submit a sample of two smaller Drawing Sets together as soon as possible, so that any problems can be fed back quickly. E7.6 Repeat submissions of a Drawing Set will be dealt with as described in E4.6.18. If errors are present in submitted data the HADDMS Team will, as far as possible, only reject single Drawing Sets within a submission, so Service Providers should not need to resubmit Drawing Sets that were correct. Checking prior to submission E7.7 All data must be checked for viruses and other malicious software with appropriate software using the latest updates available prior to submission. E7.8 All DVDs must be verified for errors prior to submission. Some DVD burning software allows verification immediately after recording. Guidance note on drainage data formats, Issue 1. August 2012 92 of 93 Labelling E7.9 DVDs must be clearly marked with “HADDMS Data”, the Area or DBFO number, Batch Number, and Disc number within the batch (e.g. “HADDMS Data – Area 10 – Batch 1 – Disc 1 of 9”). E7.10 Hard disks must be clearly labelled with “HADDMS Data”, the Area or DBFO number and Batch Number. E7.11 All submissions must include contact details of the originating person and organisation. E7.12 All submissions are to be sent by Special Delivery or courier to: HADDMS Data Team (FNG) Mott MacDonald Ltd Mott MacDonald House 8–10 Sydenham Road Croydon CR0 2EE Guidance note on drainage data formats, Issue 1. August 2012 93 of 93