Migration Handbook
Migration Preparation of Current Content and
Data Structures
UWP
UWP009
Document Version: 1.1
Date: 08/07/2014
UWP009: Migration Preparations of Current Content and Data Structures
1
Migration Handbook v1.1
Executive Summary
The migration handbook is a living document that will be enhanced as further Drupal developments
are made. It will also be refined for purpose as lessons are learned during the migrations.
This document provides a workflow which covers the technical coding, personnel structure and
stage to stage running of a migration. This end to end plan can be adapted to each individual
migration. This adaptability is recommended by both the project team and Drupal consultants
Wunderroot.
The auto migration of content has been run for testing purposes on three occasions during our
project work with issues or additional developments resolved each time before the next run.
UWP006 will review this code (and the process) and transfer it into planning for 14/15 where the
migration process will be versioned as UoE Drupal develops.
2
UWP009: Migration Preparations of Current Content and Data Structures
Migration Handbook v1.1
Contents
1
Executive Summary ......................................................................................................................... 2
2
Introduction .................................................................................................................................... 7
3
The Roles within Each Migration .................................................................................................... 8
4
Stage One – Planning ...................................................................................................................... 9
4.1
4.1.1
Who is involved ............................................................................................................... 9
4.1.2
Guidance ......................................................................................................................... 9
4.1.3
Output ............................................................................................................................. 9
4.2
Assign the Migration Roles ................................................................................................... 10
4.2.1
Who is involved ............................................................................................................. 10
4.2.2
Guidance ....................................................................................................................... 10
4.2.3
Output ........................................................................................................................... 10
4.3
Site Report Provided to the Migration Team (Except the Client Roles) ............................... 12
4.3.1
Who is involved ............................................................................................................. 12
4.3.2
Guidance ....................................................................................................................... 12
4.3.3
Output ........................................................................................................................... 12
4.4
Estimate the Sub Site Migration ........................................................................................... 13
4.4.1
Who is involved ............................................................................................................. 13
4.4.2
Guidance ....................................................................................................................... 13
4.4.3
Output ........................................................................................................................... 13
4.5
5
Schedule the Migration........................................................................................................... 9
Expectation Sign Off .............................................................................................................. 15
4.5.1
Who is involved ............................................................................................................. 15
4.5.2
Guidance ....................................................................................................................... 15
4.5.3
Output ........................................................................................................................... 15
Stage 2 – Client Liaison, Editorial and Pre Auto Migration Sign Off ............................................. 16
5.1
Deliver a Site Report to the Client to Guide them on Actions they MUST Take ................... 16
5.1.1
Who is involved ............................................................................................................. 16
5.1.2
Guidance ....................................................................................................................... 16
5.1.3
Output ........................................................................................................................... 16
5.2
Client to be Given Time to Make Editorial Changes ............................................................. 17
5.2.1
Who is involved ............................................................................................................. 17
5.2.2
Guidance ....................................................................................................................... 17
5.2.3
Output ........................................................................................................................... 17
3
UWP009: Migration Preparations of Current Content and Data Structures
5.3
6
Migration Handbook v1.1
Pre Auto Migration Sign Off .................................................................................................. 18
5.3.1
Who is involved ............................................................................................................. 18
5.3.2
Guidance ....................................................................................................................... 18
5.3.3
Output ........................................................................................................................... 18
Stage 3 – Migration of the Content .............................................................................................. 19
6.1
Polopoly Content Extract ...................................................................................................... 19
6.1.1
Who is involved ............................................................................................................. 19
6.1.2
System Dependencies ................................................................................................... 19
6.1.3
File Structure ................................................................................................................. 19
6.1.4
Installation .................................................................................................................... 20
6.1.5
Configure ....................................................................................................................... 20
6.1.6
Running the Code.......................................................................................................... 20
6.1.7
Using the Generated XML Files ..................................................................................... 21
6.1.8
Output ........................................................................................................................... 21
6.2
Editorial Tasks in XML Stage ................................................................................................. 22
6.2.1
Who is involved ............................................................................................................. 22
6.2.2
Guidance ....................................................................................................................... 22
6.2.3
Output ........................................................................................................................... 22
6.3
Drupal Migration Module Implementation .......................................................................... 23
6.3.1
Who is involved ............................................................................................................. 23
6.3.2
Migration Grouping....................................................................................................... 23
6.3.3
System Dependencies ................................................................................................... 23
6.3.4
File Structure ................................................................................................................. 23
6.3.5
Output ........................................................................................................................... 23
6.4
Image Migration .................................................................................................................... 24
6.4.1
Who is Involved ............................................................................................................. 24
6.4.2
Class UOE_UWS_IMAGES ............................................................................................. 24
6.4.3
Output ........................................................................................................................... 24
6.5
Content Migration................................................................................................................. 25
6.5.1
Who is involved ............................................................................................................. 25
6.5.2
Class UOE_UWS_Articles .............................................................................................. 25
6.5.3
Class UOE_UWS_Folders............................................................................................... 25
6.5.4
Output ........................................................................................................................... 26
6.6
Menu Structure Migration .................................................................................................... 27
4
UWP009: Migration Preparations of Current Content and Data Structures
6.6.1
Who is involved ............................................................................................................. 27
6.6.2
Class UOE_UWS_Structure ........................................................................................... 27
6.6.3
Output ........................................................................................................................... 27
6.7
7
Who is involved ............................................................................................................. 28
6.7.2
Guidance ....................................................................................................................... 28
6.7.3
Output ........................................................................................................................... 29
Stage 4 – Editorial Tasks and UWP Site Sign Off ........................................................................... 30
Editorial within Drupal .......................................................................................................... 30
7.1.1
Who is involved ............................................................................................................. 30
7.1.2
Guidance ....................................................................................................................... 30
7.1.3
Output ........................................................................................................................... 30
7.2
Deployment Checklist ........................................................................................................... 31
7.2.1
Who is involved ............................................................................................................. 31
7.2.2
Guidance ....................................................................................................................... 31
7.2.3
Output ........................................................................................................................... 31
7.3
Deployment Review .............................................................................................................. 31
7.3.1
Who is involved ............................................................................................................. 31
7.3.2
Guidance ....................................................................................................................... 31
7.3.3
Output ........................................................................................................................... 31
Stage 5 – Client Training and Objectives Checklist ....................................................................... 32
8.1
Drupal Training...................................................................................................................... 32
8.1.1
Who is involved ............................................................................................................. 32
8.1.2
Guidance ....................................................................................................................... 32
8.1.3
Output ........................................................................................................................... 32
8.2
9
Creating New Migrations ...................................................................................................... 28
6.7.1
7.1
8
Migration Handbook v1.1
Objectives Checklist .............................................................................................................. 32
8.2.1
Who is involved ............................................................................................................. 32
8.2.2
Guidance ....................................................................................................................... 32
8.2.3
Output ........................................................................................................................... 32
Stage 6 – Launch ........................................................................................................................... 33
9.1
Go Live................................................................................................................................... 33
9.1.1
Who is involved ............................................................................................................. 33
9.1.2
Guidance ....................................................................................................................... 33
9.1.3
Output ........................................................................................................................... 33
5
UWP009: Migration Preparations of Current Content and Data Structures
9.2
10
Migration Handbook v1.1
Migration Closure ................................................................................................................. 33
9.2.1
Who is involved ............................................................................................................. 33
9.2.2
Guidance ....................................................................................................................... 33
9.2.3
Output ........................................................................................................................... 33
Document Appendix ................................................................................................................. 34
10.1.1
Task 1.2 – Assign the Migration Roles .......................................................................... 34
10.1.2
Task 1.3 - Site BO Analysis............................................................................................. 35
10.1.3
Task 1.4 – Estimate the Migration ................................................................................ 36
10.1.4
Task 1.5 – Expectation Document................................................................................. 37
10.1.5
Task 2.3 - Site Report – For Clients ............................................................................... 38
10.1.6
Task 2.3 – Pre Auto Migration Sign Off Doc .................................................................. 40
6
UWP009: Migration Preparations of Current Content and Data Structures
2
Migration Handbook v1.1
Introduction
This document will guide you through the planning and implementation involved in migrating a UoE
Polopoly sub site from the Polopoly CMS into the University Website Programme’s Drupal
development.
The will be a living document that will be amended in tandem with the development progress of the
Drupal project. We will signpost where the migration process will require additional development at
the following wiki page (https://www.wiki.ed.ac.uk/pages/viewpage.action?pageId=222574660).
This early version of the migration process is based on the content within the test version of the
UWP Polopoly sub site and has been developed on a virtual machine provided by the UWP006 team.
7
UWP009: Migration Preparations of Current Content and Data Structures
3
Migration Handbook v1.1
The Roles within Each Migration
Each migration will be run as an individual project. As such the roles within the project have been
defined as below. It may be that one single person could be responsible for more than one role, but
all roles will require a responsible person to be assigned to them.
Role
Description
Client Senior Manager
This is the person identified as the Sponsor of the
migration from the client’s perspective.
Client Tech Rep
This is the person identified as the main technical
representative for the running of the client’s
website
Client Editor
The person on the client side who performs the
majority of the site edits, or has been assigned as
the lead editor by the Client Senior Manager.
Site Migration Manager
The UWP person in charge of the migration project
for the particular sub site.
(Must be cc’d into all communications)
Editor
The UWP person given editorial sign off for the
particular sub site
Developer
The UWP person responsible for the code within
each particular migration. Including delegating out
or creating the additional migration coding required
as Drupal expands.
CMS Support
A Drupal CMS Support person who can coordinate
the resolution of any Drupal CMS support issues not
related to the
Infrastructure
The person within the infrastructure team identified
to switch the sites between the Polopoly and Drupal
servers.
User Support
A user support person identified within the IT
helpdesk who can liaise with the migration team.
Trainer
A UWP person who is responsible for organising the
Drupal Training.
Migration Officer
The UWP person who is the owner and
administrator of the Migration Schedule. The
Migration Officer will also be in charge of the
Migration Handbook/Process and will be required to
organise any additional development time required
to update the migration process.
1- Table of Migration Roles
8
UWP009: Migration Preparations of Current Content and Data Structures
4
4.1
Migration Handbook v1.1
Stage One – Planning
Schedule the Migration
4.1.1 Who is involved
The ‘Migration Officer’ and the ‘Client Senior Manager’
4.1.2 Guidance
The Migration Officer (who is responsible for maintaining the schedule of all the sub site migration
projects) will liaise with the Client Senior Manager to schedule a suitable time window for the
migration.
The Migration Officer can use the Polopoly reports produced using the Polopoly search
enhancements to make sure the Polopoly sub site’s migration to Drupal is realistic when matched
against the current Drupal release.
4.1.3 Output
An updated sub site migration schedule to run alongside the schedule of Drupal development
releases.
9
UWP009: Migration Preparations of Current Content and Data Structures
4.2
Migration Handbook v1.1
Assign the Migration Roles
4.2.1 Who is involved
The ‘Migration Officer’, ‘Site Migration Manager’ and the ‘Client Senior Manager’
4.2.2 Guidance
The three people mentioned above must assign all the migration roles. The assigned person will
then be responsible and accountable for their part in the migration process. Also as mentioned in
section 2 one person can be assigned more than one role however the Site Migration Manager must
take care not to unduly pressurise one person.
The record of who has been assigned to the roles must be held on file for the duration of the sub site
migration project and must be signed by each of the three persons involved in the task.
Should the person(s) assigned to the task change this must be reflected in the creation of an
updated doc with associated notes.
4.2.3 Output
The output will be a signed off document detailing the migration roles. A blank copy of the ‘Assign
the Migration Roles’ document can be found in the Appendix under ‘Documents 1.2’. An image is
below.
2- Assign the Migration Roles
10
UWP009: Migration Preparations of Current Content and Data Structures
Migration Handbook v1.1
The document will require the sign off of the Migration Officer and the Site Migration Manager. The
Client Senior Manager will be involved and may sign off for transparency should the Migration
officer feel it’s appropriate or necessary.
11
UWP009: Migration Preparations of Current Content and Data Structures
4.3
Migration Handbook v1.1
Site Report Provided to the Migration Team (Except the Client Roles)
4.3.1 Who is involved
The ‘Editor’ and the ‘Site Migration Manager’
4.3.2 Guidance
The Editor and the Site Migration Manager will take ownership of the initial Polopoly Site analysis
used by the Migration Officer. They will do a brief confirmation of the analysis within this report and
share it with those filling the non-client roles within the particular sub site migration project.
4.3.3 Output
The output is to be determined and is reliant on the work being carried out by the UWP team
around the use of their Polopoly search enhancements.
Our current vision is to have a one page briefing document sent from the Editor (with approval by
the Site Migration Manager) to give the team performing the migration an early overview of the site
being migrated.
12
UWP009: Migration Preparations of Current Content and Data Structures
4.4
Migration Handbook v1.1
Estimate the Sub Site Migration
4.4.1 Who is involved
The Full Migration Team (Except the Client roles)
4.4.2 Guidance
Now that the team know their roles and are familiar with the basics of the site they are migrating an
estimation of the process must be performed.
Ideally all the participants in the work will also participate in the migration estimation. Even if a role
seems to be a straightforward and known element (such as when the site is swapped at an
infrastructure level towards the end) it will still be beneficial to get a confirmation that the task is
scheduled and the resource is available.
We have provided a simple estimation sheet that can be talked through with the team. It may be
that depending on the size of the site and the complications of the content that certain elements will
require additional code development to be done. This must also be estimated and so it can mean
that one site estimate can be very different in time and structure from another.
Note: If doing a full estimation is not time conducive we suggest doing the estimations in bulk.
Perhaps several at a time can be estimated in one session therefore relieving the pressure to get
everybody in the same room.
4.4.3 Output
The output will be a document with dates and hours effort marked against the tasks. Should you
wish to move some tasks around you should update the document to show the tasks in date order.
Below is an example image of the document. The up to date version will be held in the document
appendix of this handbook.
13
UWP009: Migration Preparations of Current Content and Data Structures
Migration Handbook v1.1
3- Estimation Sheet
14
UWP009: Migration Preparations of Current Content and Data Structures
4.5
Migration Handbook v1.1
Expectation Sign Off
4.5.1 Who is involved
The UWP ‘Site Migration Manager’ and the ‘Client Senior Manager’ will curate and sign off this
document.
4.5.2 Guidance
Each expectation document will require top be individually edited subject to the needs and structure
of the site (and the people) being migrated.
This handbook supplies a basic template for the Expectation document which should be built on by
the Site Migration Manager and the Client Senior Sign Off.
As will all documents within the migration process they should be versioned with appropriate notes
and dates.
4.5.3 Output
Below is an example image of the document. The up to date version will be held in the document
appendix of this handbook.
4- Expectation Document
15
UWP009: Migration Preparations of Current Content and Data Structures
5
5.1
Migration Handbook v1.1
Stage 2 – Client Liaison, Editorial and Pre Auto Migration Sign Off
Deliver a Site Report to the Client to Guide them on Actions they MUST Take
5.1.1 Who is involved
‘Editor’, ‘Client Editor’, ‘Client Tech Rep’ and the ‘Client Senior Manager’
5.1.2 Guidance
Having used the Polopoly BO reports to analyse the site for estimation and providing expectation of
effort to the clients in Stage 1 here we deliver a site report to guide the clients through the editorial
tasks they must undertake.
In part this document will remind clients of the editorial commitments they should already have
been adhering too and also (where possible) give the clients assistance in getting their site into order
for the Auto Migration.
This document will be tailored to the expectations laid out in task 1.5. The ‘Editor’ will then be the
point of contact for the clients while they make the editorial changes.
5.1.3 Output
Below is an example image of the document. The up to date version will be held in the document
appendix of this handbook. An email address should be assigned for migration support.
5- Site Report - For Clients
16
UWP009: Migration Preparations of Current Content and Data Structures
5.2
Migration Handbook v1.1
Client to be Given Time to Make Editorial Changes
5.2.1 Who is involved
The ‘Client Editor’, ‘Client Tech Rep’, ‘Client Senior Manager’ and the ‘Editor’
5.2.2 Guidance
During this time the Editor should be on hand for support calls from the Client Editor. It would be
helpful if all communication was channelled at this point was between the UWP Editor and the
Client Editor only. This would allow a clear support audit trail to form and promote clear
communication at a time when the Clients may feel under pressure to complete the tasks asked of
them.
5.2.3 Output
JIRA is a fine client to monitor support issues as they can be separated into separate Sub Site
projects just like the Migration process.
Unidesk could also be used which would have the benefit of maintaining a consistent support shell
for each issue.
Otherwise (and most simply) an email address could be used and a rule placed on every email that
comes in from the Client Editor only. This rule would forward the email to a folder with the Sub Site
Migration name so it is easy to monitor the issues the clients are having.
17
UWP009: Migration Preparations of Current Content and Data Structures
5.3
Migration Handbook v1.1
Pre Auto Migration Sign Off
5.3.1 Who is involved
The ‘Site Migration Manager’, the ‘Editor’, ‘Client Editor’ and the ‘Client Senior Manager’
5.3.2 Guidance
Taking a combination of the site report (task 2.1) and the Expectations Document (Task 1.5) all tasks
within those documents must be reviewed and signed off by the Site Migration Manager and the
Client Senior Manager.
Editor’s time from both sides will be required for any questions the Managers may have. Ideally the
Pre Migration Sign off will be done with a face to face meeting.
5.3.3 Output
The output will be a single, short document that will state agreement that the clients understand
what is being migrated and that UWP are satisfied that the site is in the correct state for migration.
This will be the last point where editorial effort is the responsibility of the Clients. After this all
expectation will be on UWP and the migration process so any and all doubts held by the Site
Migration Manager MUST be detailed before sign off of this section.
An example is below and contained within the Appendix.
6- Pre Auto Migration Sign Off Document
18
UWP009: Migration Preparations of Current Content and Data Structures
6
Migration Handbook v1.1
Stage 3 – Migration of the Content
6.1
Polopoly Content Extract
6.1.1 Who is involved
The ‘Developer’
6.1.2


System Dependencies
PHP 5.3.10 (or later)
o With OCI8 (i.e. oracle support)
A web server configured to run PHP (unless you want to run the scripts in a CLI)
6.1.3 File Structure
The extraction code is based on section 4 of the document:
https://www.projects.ed.ac.uk/system/files/projectotherdocuments/UWP%20Migration%20Analysis
%20and%20BO%20Enhancements%20v1.2.docx

polopoly_extract
o find_url_path.php
 recursively generates the URL of a node from the database
o 19polopoly_article_grab.php
 generates an XM file describing the contents of a Polopoly sub-site
o 19polopoly_element_translator.php
 The main loop translation BODY elements found in an Polopoly article
o 19polopoly_site_grab_to_array.php
 Identifies the extents of a site from ISG_POLOPOLY_TREE and returns the
values for the site from this DB table in an array
o 19polopoly_structure_grab.php
 generates an XM file describing the structure of a Polopoly sub-site
o settings.php
o element_handlers
 Handlers that generates HTML from Polopoly contents. Some of these are
reasonably fully implementations but most are just stubs providing minimal
functionality
 handle_header.php
 handle_def_list.php
 handle_event.php
 handle_featurebox.php
 handle_rssfeed.php
 handle_table.php
 handle_paragraph.php
 handle_unknown.php
 handle_quote.php
 handle_listitem.php
 handle_article_image.php
 handle_article_media.php
 handle_infolist.php
 handle_link_list.php
 19polopoly_include-article_grab.php
 handle_media.php
19
UWP009: Migration Preparations of Current Content and Data Structures


Migration Handbook v1.1
handle_include_image.php
handle_include_article.php
6.1.4 Installation
- Create a sub-directory on your web-server configured so that PHP files can be run from a browser
- Copy/extract the Polopoly extraction code to this directory (maintaining the file structure)
- Confirm that the files have the correct permissions to be run from the web (read and execute)
- Create a directory where the XML files that are generated by the process can go. Make sure that
the user the web-server runs as can write to this directory
6.1.5 Configure
All configurable settings can be found in the file ‘settings.php’ where each variable has a detailed
description of what values they hold.
These settings need to be set for each migration:





$root_department
$parent_of_root_department
$drupal_url_root
$structure_file
$article_file
These settings only need to be set once per environment (TEST/LIVE)







$image_url_prefix
$polopoly_read_access_user
$polopoly_read_access_passw
$polopoly_read_access_dbstr
$bopolostats_read_access_user
$bopolostats_read_access_passw
$bopolostats_read_access_dbstr
6.1.6 Running the Code
The code can be run either from a web browser or a CLI. The following example shows it running in a
web browser. It assumes that the webserver is ‘localhost’ and that the directory on ‘localhost’ that is
configured for PHP is ‘/php/’ and that ‘polopoly_extract’ has been extracted/copied to this directory.
Edit the settings.php file. Make sure that values set to the variables $structure_file (e.g. struct.xml)
and $article_file (e.g. site.xml) are in the directory set up in point 4 in ‘Installation’ above and that all
the other settings are correct for your environment (TEST/LIVE) and migration.
Point your browser to:
http://localhost/php/polopoly_extract/polopoly_article_grab.php
20
UWP009: Migration Preparations of Current Content and Data Structures
Migration Handbook v1.1
A file containing XML data describing article content and any images in those articles will be
generated I the location specified in the variable: $article_file (e.g. site.xml)
Point your browser to:
http://localhost/php/polopoly_extract/polopoly_structure_grab.php
A file containing XML data describing article content and any images in those articles will be
generated I the location specified in the variable: $structure_file (e.g. struct.xml)
6.1.7 Using the Generated XML Files
The code for the migration module requires 3 XML files
Site.xml (as generated above), struct.xml (also as generated above) and a third file which currently is
called sitexxx.xml. This third file should contain the contents of site.xml and struct.xml and it is used
to generate the menu-links structure in Drupal. The order of the content in structxxx.xml is also
important. The folder/department data need to come before the article data. One way of creating
this file is to:






Open a blank document in a text editor
Open the files struct.xml and site.xml in an editor as well
Copy and paste; first the contents of struct.xml followed by the contents of site.xml into the
blank document.
Find where the 2 pasted in XML files meet in the new document
Delete the xml:
</site>
<site>
found at this location
Save this document as sitexxx.xml
6.1.8 Output
The 3 XML files can now be put where the Drupal Migrate module expects to find them.
21
UWP009: Migration Preparations of Current Content and Data Structures
6.2
Migration Handbook v1.1
Editorial Tasks in XML Stage
6.2.1 Who is involved
The ‘Editor’ with support from the ‘Developer’ and the ‘Site Migration Manager’
6.2.2 Guidance
It should be noted that the recommendation when our original code was reviewed was that all
editing should happen in Drupal itself. That being said we have maintained this task in the migration
handbook as the Site Migration Manager may still wish to allocate work to this section for identified
easy fixes.
Estimating of how long editorial tasks in Polopoly will take have not yet been established. Work to
do this will be picked up by the 14/15 UWP Migration project work.
6.2.3 Output
Edited XML
22
UWP009: Migration Preparations of Current Content and Data Structures
6.3
Migration Handbook v1.1
Drupal Migration Module Implementation
6.3.1 Who is involved
The ‘Developer’
6.3.2
Migration Grouping
Group: Scald
Group: Content
UOEPolopolyScaldMigration
UOEPolopolyArticleMigration
Processed before
Group: Structure
UOEPolopolyOverviewMigration
Processed before
extends
extends
UOE_UWS_IMAGES
extends
UOEPolopolyMenuMigration
extends
UOE_UWS_Folders
extends
UOE_XXX_IMAGES
extends
UOE_UWS_Articles
UOE_XXX_Folders
UOE_XXX_Articles
UOE_UWS_Structure
UOE_XXX_Structure
The migration of a site is split into 3 migration groups which have been configured so that they
appear in the order of dependencies in listings with the migration that needs to happen first at the
top of the migration list in the Migrate UI or in drush.
6.3.3 System Dependencies
The system uses the following Drupal modules: migrate, migrate_ui and migrate_extras. All 3 can be
downloaded with drush.
6.3.4 File Structure
The files are found in: /var/www/sandbox/src/sites/all/modules/custom/migrate_polopoly . The file
structure in that directory is:







uoe_migrate_polopoly.module
uoe_migrate_polopoly.info
uoe_migrate_polopoly.migrate.inc
uoe_scald_polopoly.inc
uoe_article_polopoly.inc
uoe_overview_polopoly.inc
uoe_menu_polopoly.inc
6.3.5 Output
XML (this directory contains XML files for migration. It can be placed anywhere)
23
UWP009: Migration Preparations of Current Content and Data Structures
6.4
Migration Handbook v1.1
Image Migration
6.4.1 Who is Involved
The ‘Developer’
6.4.2
Class UOE_UWS_IMAGES
6.4.2.1 function_construct
parent::__construct takes 2 arguments:
1: The location of the XML file containing the article migration. This will be the site.xml file
generated by the Polopoly extract. E.g.:
/var/www/sandbox/src/sites/all/modules/custom/migrate_polopoly/xml/site.xml
2: A string describing the migration. E.g.:
Migrate images found in the UWS sub-site
6.4.2.2 function_prepareRow
Is used to extract the filename of an image from the URL used to extract the image from Polopoly.
The regular expression used to generate the contents of the variable $t1 will have to be changed to
match the environment (LIVE/TEST)
6.4.3 Output
All images connected to the Polopoly sub site will be ready for migration into the Scald areas of
Drupal.
24
UWP009: Migration Preparations of Current Content and Data Structures
6.5
Migration Handbook v1.1
Content Migration
6.5.1 Who is involved
The ‘Developer’
6.5.2
Class UOE_UWS_Articles
6.5.2.1 function_construct
parent::__construct takes 2 arguments:
1: The location of the XML file containing the article migration. This will be the site.xml file
generated by the Polopoly extract. E.g.:
/var/www/sandbox/src/sites/all/modules/custom/migrate_polopoly/xml/site.xml
2: A string describing the migration. E.g.:
Migrate articles found in the UWS sub-site
This is also where the Drupal group ‘owning’ the content is set:
parent::$this->addFieldMapping('og_group_ref')->defaultValue(11);
where the value in defaultValue is the group ID of the group.
6.5.2.2 function_prepare
This is used to change the format of the field_slug part of the node to a format that uoe_pathauto
recognises. No user configurable parts.
6.5.2.3 function_prepareRow
This is used to change the format of the content I the XML input file to something the Migrate
module recognises: It’s major function in article import is to generate Scald atom mark-up for images
from the text in the article_content field in the XML source. To implement this a lookup of the
mapping for the image migration corresponding to the article migration is needed. This is done by
looking up data in the migration table generated by the image migration. The prepareRow function
will have to be manually edited for each class extending UOEPolopolyArticleMigration. The part of
the code in question is:
$querymms = Database::getConnection('default')
// the migrate map needs to be changed to match the image migration for each migration
// the format is 'migrate_map_'<name_of_img_migration>
->select('migrate_map_uoe_uws_images', 'mms');
$querymms->fields('mms', array('destid1', 'sourceid1'))
->condition('sourceid1', $lookup_id , '=');
6.5.3
Class UOE_UWS_Folders
6.5.3.1 funtion_construct
parent::__construct takes 2 arguments:
1: The location of the XML file containing the structure of the migration. This will be the struct.xml
file generated by the Polopoly extract. E.g.:
/var/www/sandbox/src/sites/all/modules/custom/migrate_polopoly/xml/struct.xml
25
UWP009: Migration Preparations of Current Content and Data Structures
Migration Handbook v1.1
2: A string describing the migration. E.g.:
Migrate folders found in the UWS sub-site
This is also where the Drupal group ‘owning’ the content is set:
parent::$this->addFieldMapping('og_group_ref')->defaultValue(11);
where the value in defaultValue is the group ID of the group.
6.5.4 Output
Content will be prepared for use in the migration module.
26
UWP009: Migration Preparations of Current Content and Data Structures
6.6
Migration Handbook v1.1
Menu Structure Migration
6.6.1 Who is involved
The ‘Developer’
6.6.2
Class UOE_UWS_Structure
6.6.2.1 function_construct
parent::__construct takes 2 arguments:
1: The location of the XML file containing the combined structure and article migration, with the
structure before the article. This will be the sitexxx.xml file generated by the Polopoly extract. E.g.:
/var/www/sandbox/src/sites/all/modules/custom/migrate_polopoly/xml/sitexxx.xml
2: A string describing the migration. E.g.:
Migrate The structure found in the UWS sub-site
6.6.2.2 function_prepare
This sets the parent Drupal parent node in Drupal’s menu-link structure (i.e. the existing point in
Drupal you will connect the migrated Polopoly content to)
//use the mlid of the node you want to be the parent
//found in the menu_list db table and other places
if (!isset($entity->plid)) {
$entity->plid = '1322';
}
6.6.2.3 function_createStub
This function needs to be configured so that no stub creation is attempted for the parent of the topnode in the import. This will be the parent_id for the first entry in the struct.xml file. The value is set
in this code:
// if it is the top node in the export, that means it has no parent, so don't create a stub
if ($current_nid=='2.392') {
return FALSE;
}
The value you set here MUST be the same as the one configured for $parent_of_root_department in
the Polopoly content extract.
6.6.3 Output
The Polopoly menu structure will be prepared for migration.
27
UWP009: Migration Preparations of Current Content and Data Structures
6.7
Migration Handbook v1.1
Creating New Migrations
6.7.1 Who is involved
The ‘Developer’
6.7.2 Guidance
To create new migrations first create new classes extending:




UOEPolopolyScaldMigration
UOEPolopolyArticleMigration
UOEPolopolyOverviewMigration
UOEPolopolyMenuMigration
Following the OUE_UWS_* example. This can be done by copying the UOE_UWS_* classes, giving
the new classes descriptive names and configuring the values described in the sections above in the
new classes so that they describe the new migration. The following needs to be configured:
For all classes:


Location of XML file
Description of migration
For the classes extending UOEPolopolyArticleMigration and UOEPolopolyOverviewMigration

The field mapping for og_group_ref need to be set to the group_id of the Drupal
organic group who will ‘own’ the content after migration.
For the class extending UOEPolopolyArticleMigration

The table name for the table describing the migrate map for the image migration in the
migration will have to be configured. This can be found in the database after the image
migration have been registered with the Migrate module. The name of the table will be
of the form: 'migrate_map_'<name_of_img_migration>.
For the class extending UOEPolopolyMenuMigration


In the function ‘prepare’ the menu-link id (mlid) of the existing node in Drupal that will
be the parent of the migrated content needs to be set.
In the function ‘prepareRow’ the Id of the parent of the top node in the structure
extracted from Polopoly needs to be set so that a stub is not generated for this node
(which is outside the migrated content)
Add the new migrations t the function uoe_migrate_polopoly_migrate_api () found in the file:
uoe_migrate_polopoly.migrate.inc. Make sure you follow the template:
‘<unique machine name>’ => array(‘class_name’ =>’<name of class you created for the migration>’
The migration system will get very confused if you either fail to use a name that does not already
exist for the <unique machine name> or a class_name it cannot find for <name of class you created
for the migration>
28
UWP009: Migration Preparations of Current Content and Data Structures
Migration Handbook v1.1
Finally: flush all Drupal caches to load the new migration
6.7.3 Output
An individual migration will be created for each site.
29
UWP009: Migration Preparations of Current Content and Data Structures
7
Migration Handbook v1.1
Stage 4 – Editorial Tasks and UWP Site Sign Off
7.1
Editorial within Drupal
7.1.1 Who is involved
The ‘Editor’ and the ‘Site Migration Manager’
7.1.2 Guidance
This editorial check will look at the major elements discussed with the clients in Stages 1 and 2. The
editor will check:




Structure
Content pages (a random selection if the site is too large for full review)
Images
Authors and Owners
Note: When our initial code was reviewed we were given advice that all editing should be done in
Drupal.
7.1.3 Output
The Drupal site will need to be signed off by both the Editor and the Site Migration Manager
30
UWP009: Migration Preparations of Current Content and Data Structures
7.2
Migration Handbook v1.1
Deployment Checklist
7.2.1 Who is involved
‘Editor’
7.2.2 Guidance
Taking the Site Report and the Expectation Document the Editor will review each statement within
those documents and make any appropriate notes that should be highlighted to the Site Migration
Manager.
7.2.3 Output
An annotated Site Report and Expectation document where elements of those documents are
‘checked’ as approved or notes are made to not approved elements that need consideration.
7.3
Deployment Review
7.3.1 Who is involved
The ‘Site Migration Manager’, ‘Editor’ and the ‘Migration Officer’
7.3.2 Guidance
Taking the Deployment Checklist the three people review that the objectives have been reached.
Any objectives that have outstanding elements will need to be discussed where appropriate and
action list generated. Once this list is satisfied, the migration can be signed off.
7.3.3 Output
An appended Deployment Checklist where all actions have been resolved.
31
UWP009: Migration Preparations of Current Content and Data Structures
8
8.1
Migration Handbook v1.1
Stage 5 – Client Training and Objectives Checklist
Drupal Training
8.1.1 Who is involved
‘Trainer’ and the Site Migration Manager’
8.1.2 Guidance
The training should be of a standard form. It may be that this will be a good time for the Content
Owners to see their new site for the 1st time. Any feedback can then be taken forward to the next
task to keep surprises limited.
8.1.3 Output
Feedback Forms from the Training
8.2
Objectives Checklist
8.2.1 Who is involved
‘Site Migration Manager’, ‘Client Senior Manager’ and if possible the two editors
8.2.2 Guidance
This will be a high level discussion taking the Expectation Document and a Summary of the Feedback
from the training. Any action points will be discussed/raised and if required a second sign off will be
scheduled after those action points have been resolved.
8.2.3 Output
A site sign off completed by the ‘Site Migration Manager’ and the ‘Client Senior Manager’.
32
UWP009: Migration Preparations of Current Content and Data Structures
9
Migration Handbook v1.1
Stage 6 – Launch
9.1
Go Live
9.1.1 Who is involved
‘Site Migration Manager’, ‘Migration Officer’ and the ‘Infrastructure’ person
9.1.2 Guidance
The Go Live instruction is sent to infrastructure in the form of a support call with a date specified.
The Development Technology team have given UWP the following guidance:





We will be using Varnish to proxy contents between the 2 systems.
By default the content will be directed to Drupal unless a url path is present in the proxy
configuration which proxy it to Polopoly.
A list of url path will be created and pre populated in the proxy file. Any time a url path has
been migrated to Drupal, this path will be commented out.
A load test will be done using the test systems for Polopoly and Drupal. Ultimately a load
test will also be performed on live cms.
Riky Harris will take the lead on this
9.1.3 Output
A successful site switch from Polopoly to the Drupal infrastructure
9.2
Migration Closure
9.2.1 Who is involved
‘Site Migration Manager’ and the ‘Migration Officer’
9.2.2 Guidance
The migration will be formally closed after feedback from the two people involved on any Lessons
Learned. These lessons will then be fed back in the migration project by the Migration Officer.
9.2.3 Output
Closure sign off and a Lessons Learned summary.
33
UWP009: Migration Preparations of Current Content and Data Structures
Migration Handbook v1.1
10 Document Appendix
10.1.1 Task 1.2 – Assign the Migration Roles
Name of Sub Site
Base URL
Role
Version
Name
Client Senior Manager
Client Tech Rep
Client Editor
Site Migration Manager
Editor
Developer
CMS Support
Infrastructure
User Support
Trainer
Migration Officer
Sign Off
Migration Officer
Version
Changes
Site Migration Manager
Client Senior Manager
Date
34
UWP009: Migration Preparations of Current Content and Data Structures
Migration Handbook v1.1
10.1.2 Task 1.3 - Site BO Analysis
Notes from UWP
Overview:
> A database running on webhost1t contains the BOXI output for the full Polopoly content tree.
> We query this database either through a website interface, or using an Excel workbook and
download CSV files for particular sub sites.
> These CSV’s contain a list of every page in that sub site, with the option to exclude pages in child
sub sites.
> We analyse over 300 of these sub sites to build a picture of the current Polopoly website.
>> We track things like total published/unpublished pages, number of database/proxy/XML articles
> All of these analysed sub sites are exported into another Excel workbook that we will use to decide
which sites are ‘eligible’ for migration based on certain criteria (which will include factors like total
size and article types).
Future expansion:
> Intelligent automation to minimise refresh period and errors. The current refresh process is semiautomated, and takes about 45 minutes to complete with human interacting at key points. I’d like to
remove the human factor to allow an ‘on demand’ refresh with a quick turnaround time.
> Excel macros are susceptible to errors, especially when generated CSV files are not completely
processed server side. This requires a manual correction which takes about 5 minutes per sub site.
Lessons learned:
> BOXI report specifications should not have been predetermined without a test implementation. I
found that if I had access to article types, I could save two editors over a month of laborious
Polopoly searching each to manually determine article types. This change took a couple of weeks to
come into effect.
> VBA is still useful when enough blood, sweat, tears and threats are offered to it. :)
35
UWP009: Migration Preparations of Current Content and Data Structures
10.1.3 Task 1.4 – Estimate the Migration
Ref
Task
1.1
Assign Migration Roles
1.2
BO Report with Brief Notes
1.3
Estimate the Migration
1.4
Expectation Document and with Sign Off
1.5
Deliberation Time
2.1
Site Report
2.2
Client Editorial
2.3
Pre Auto Migration Sign Off
3.1-
Auto Migration
3.2
Editorial Task in XML
3.3
Individual Migration built within the
Migration GUI
3.4
Rollback of Drupal Content Set
3.5
Content Auto Migration
3.6
New Site Editorial Check
4.1
UWP Edit
4.2
Editorial Deployment Checklist
4.3
Deployment Sign Off
4.4
Inform teams that switch is imminent
5.1
Drupal Training
5.2
Client Objectives Checklist
6.1
Go live instruction
6.2
UWP Sign Off
Total
All Tasks
Start Dates
Migration Handbook v1.1
Estimated
Effort
36
UWP009: Migration Preparations of Current Content and Data Structures
Migration Handbook v1.1
10.1.4 Task 1.5 – Expectation Document
Expectation: Senior Sign Off
Editorial effort to be
completed by the Content
Owners.



Published/Unpublished Page Review
Image Review
Menu Structure Review (URL Structure)
Final date for edits
What we will Auto Migrate
All published content will be Auto Migrated including images.
However please note that due to the way Polopoly handles images
some of these images may not be suitable for Drupal. Therefore
the Image review to be performed by the Content Owners must
be seen as crucial work.
Start date for the auto
migration
Will my site be offline?
?
Drupal Training Dates
Content Owner Drupal Site
Review Date
Proposed Go Live Date
Client Senior Sign Off
UWP Senior Sign Off
-
-
37
UWP009: Migration Preparations of Current Content and Data Structures
Migration Handbook v1.1
10.1.5 Task 2.3 - Site Report – For Clients
Our aim through the migration
We are aiming to automate as much content migration from
Polopoly to Drupal as possible.
What we require from you
To make this an efficient process, minimising editorial effort for
both yourself and the University Website Programme, you should
undertake the following steps with your published content.
Published/Unpublished Content
We will be migrating Published content only. All content that is
not published will be archived so that it can be recovered at a
later date should this be necessary. This is a great opportunity to
slim line your content for current accuracy, need and relevance.
Be harsh with your content. Websites should be up to date and
adding content into the new user friendly Drupal website will be
much easier.
Published pages: ???? (pages and Authors attached)
Structure
Your current Polopoly site had strict editorial guidelines that you
will have been adhering too. As per the regulations only content
that matches those guidelines will be eligible for migration. The
following is a reminder of the pertinent guidelines.
News
If you publish a latest news section on your site, make sure that
this content is published under a url including the distinct
subsection "news".
e.g. www.ed.ac.uk/schools-departments/my-unit-name/news/...
Staff Profiles
If you publish a section on your site consisting of staff profile
pages, make sure that this content is published under a url
including the distinct subsection "staff-profiles".
e.g. www.ed.ac.uk/schools-departments/my-unit-name/staffprofiles/...
Includes
If you use include articles in your website, make sure that these
are all located in a section hidden in website navigation with the
CMS name "INCLUDES".This section may contain sub-sections.
Single Contact Boxes
If you publish contact details for people or services using the grey
box presentation through an include, make sure these are
delivered using the Single Contact Article.If your website was built
before 2010 it could be that you are still using the old approach
using HTML code in the Special HTML Article. These must be
updated, as advised at the time of the new article's launch.
Documents
If you host documents and similar files in the Polopoly Media
Bank, make sure that they are all stored in a section called 'XXXX'.
This section can contain subsections.
Any documents that are no longer published on the website, and
do not need to be migrated for information retention purposes
should be moved to another subsection in your area of the Media
Bank named 'YYYYY'
Images
All images will be migrated
38
UWP009: Migration Preparations of Current Content and Data Structures
Migration Handbook v1.1
Other Notes
Final Date for Changes
39
UWP009: Migration Preparations of Current Content and Data Structures
Migration Handbook v1.1
10.1.6 Task 2.3 – Pre Auto Migration Sign Off Doc
Pre Auto Migration Sign Off
This document is the final sign off before auto migration. It is essential that all aspects of the
‘Expectation Document’ and the ‘Site Report’ have been acted upon. Once satisfied that all those
criteria have been met the document should be signed by the ‘Site Migration Manager’ and the
‘Client Senior Manager’ as identified at the beginning of the Migration project.
In the space below outline any outstanding issues from those two documents and their current
state (open/resolved/Won’t Fix). Once no more issues are open the document should be signed
off and the project can move into the Auto Migration Phase.
Issue
Notes
Open/Resolved/Won’t Fix
Client Senior Sign Off
UWP Senior Sign Off
-
-
40