Summative Evaluation of the
Prairie Grain Roads Program (PGRP)
- Final Report -
Audit and Evaluation Team
September, 2006
Summative Evaluation of the Prairie Grain Roads Program
1.0
Final Report 1
INTRODUCTION
Over the past decade, transportation policy reform has led to a dramatic restructuring of rail and truck haul traffic in western Canada. A reduction in rail branch lines, and a consolidation of grain elevators, caused 80% of the grain to shift from 645 delivery points to 178 between 1995 (the end of the Western Grain Transportation Act, WGTA) and 2005. As a result, the distance that producers transport grain from farm to delivery point has lengthened, with producers using much larger trucks that carry substantial grain haul weights. In addition, more agricultural diversification has taken place leading to substantially greater truck transportation of grain to and from livestock and special crop processing facilities. The result has been a transformation in grain handling from a system formerly dependent on ‘rail’ to a system now dependent on ‘road and rail’. The increase in grain haul distances and truck weights has contributed to an overall deterioration in the surface and safety of municipal and provincial roadways. Damage to roads also affects the quality of life for prairie residents and creates an increased cost burden to provincial and municipal governments who are responsible for the construction and maintenance of a major transportation network. In response to this situation, a Cabinet Decision on May 9, 2000 provided the policy authority for Transport Canada to help address the increased pressure on rural roads caused in part, by reforms to the grain handling and transportation system. The decision was cited in the Order in Council’s proposal as a response to the recognition that grain handling and transportation reforms will continue to increase pressures on grain roads and that road safety and integrity must be maintained as the sector makes the necessary adjustments. On May 10, 2000, joint announcements from three Ministers (Agriculture and Agri‐Food Canada, Transport Canada, and Natural Resources) cited PGRP as one of several measures to improve the western grain handling and transportation system. The $175 million, five‐year, Prairie Grain Roads Program (PGRP) has operated from April 2001 to March 2006, with a focus on strategic grain roads. AAFC’s Prairie Farm Rehabilitation Administration (PFRA) agreed to deliver the program on behalf of Transport Canada and Treasury Prairie road network
existed before WGTA
reforms . . .
. . . most roads not
built to heavy truck
weight standards.
Reform to WGTA and
restructuring have
resulted in larger,
heavier grain trucks,
traveling longer
distances . . .
. . . costly damage results
to provincial and
municipal roads. . .
. . . road safety
decreases.
PGRP was intended to
help the sector make the
necessary adjustments
caused in part by grain
handling and
transportation system
reforms.
Summative Evaluation of the Prairie Grain Roads Program
Final Report 2
Board provided AAFC with the authority to enter into individual agreements with provincial governments to implement the program. PGRP represents an umbrella program that encompasses separate agreements between the Government of Canada and each of Alberta, Saskatchewan, and Manitoba. While PGRP is a single federal program, it is operated under different circumstances in each province. A particular province’s history with road development programming and evaluation, stage of development in strategic road planning, and dependency on grain haul routes has played a role in how the program was implemented in each province. PGRP operated
differently in each
province.
Each individual provincial program had an Agreement Management Committee (AMC) responsible for the development of its project selection criteria, the selection of projects, and the collection of project information. The federal government was represented by the Prairie Farm Rehabilitation Administration (PFRA) which provided program oversight and administration. This summative evaluation describes the progress made by the PGRP towards improving municipal grain roads and provincial secondary highways in the prairie region. The evaluation report provides answers to specific program evaluation questions relative to program results and outcomes, adequacy of program design and delivery, program relevance, and program cost effectiveness /alternatives. 2.0
PGRP’s scope covers
80% of Canada’s
agricultural land with a
road network that would
encircle the equator ten
times.
RESULTS ACHIEVED BY THE PROGRAM
The PGRP’s primary objective was to assist provincial and municipal governments to improve road infrastructure that had deteriorated an accelerated pace over the past ten years. Overall, PGRP contributed to the reconstruction or improvement of an estimated 3,235 km of grain haul roads in the Prairie Provinces. Average project road length was 8 km. Total cost of projects funded was $169 million. Of this investment, 54% of PGRP assistance went to provincial highways and 46% was directed at municipal or county roads. Focus on Strategic Grain Corridors
At the on‐set of the program, policy‐makers and program designers realized that the level of program funding was small relative to the size of the road network and the rate of deterioration. Therefore, a Limited financial
resources warranted a
strategic approach.
Summative Evaluation of the Prairie Grain Roads Program
Final Report 3
strategic approach to project investment was intended. Upon program implementation, the program adopted a strategic approach in Saskatchewan, but not in Alberta or Manitoba. In Saskatchewan, PGRP succeeded at developing and using a strategic grain haul plan. Saskatchewan used a consultative and analytical process to develop a recommended Saskatchewan Grain Haul Corridor Network. This plan recommends a strategic priority for 23,476 km (provincial highway 11,500 km and municipal 12,000 km) from the existing network of 189,000 km. Saskatchewan used the plan to select PGRP funded projects. As a result, Saskatchewan located approximately 96% of its provincial road improvements and 86% of its municipal road improvements on the identified strategic grain corridor. A strategic approach
was intended at a
program level . . . but
was not used in Alberta
and Manitoba . . .
Although Alberta and Manitoba did not use a strategic approach they did support roads involving grain haul. Even so, Alberta and Manitoba chose projects based on the individual merit of the project at the local level, rather than on its fit to an overall plan. Some of these projects have likely fallen on important grain haul routes. Considering this inevitability, analysis of all provinces’ funded projects indicates that approximately 65% of the total PGRP program spending was focused on strategic grain roads. -
65% of PGRP
funding was
strategic,
-
76% of projects
increased weight
capacity of roads,
-
24% of projects
increased safety or
other improvements.
Contribution to Integrated Planning
PGRP fostered improved communication at the AMC level between federal, provincial, and municipal representatives. In Saskatchewan, PGRP demonstrated a significant contribution to fostering integrated planning through the inclusion of rural municipalities in the development of a recommended municipal grain corridor network. Rural municipalities in Saskatchewan reported a shift from individual municipal planning to more multi‐rural municipality planning and investment. In Saskatchewan there is evidence of integrated projects that involve municipal, provincial and federal spending on joint projects. Interviews with successful applicants from all three provinces indicate that approximately half of the respondents reported a greater degree of collaboration with other municipalities after the PGRP was implemented. Integrated planning and the
use of a strategic plan
occurred in Saskatchewan.
Summative Evaluation of the Prairie Grain Roads Program
Final Report 4
Improved Grain Haul Capacity of Roads
In Saskatchewan, 7%, or 844 km of provincial highways were improved to a standard that facilitates heavy grain haul. At the municipal level, 10% or 1,202 km were improved to a standard that facilitates heavy grain haul on municipal gravel roads. Saskatchewan reports that this highway improvement is an important achievement because its secondary highways were predominantly ‘Thin Membrane Surface’ (TMS) at the end of the WGTA. These TMS highways could not carry a heavy truck weight. PGRP has made progress toward converting these highways to weight bearing roads for heavy trucks. At the municipal level, PGRP contributed to discovering cost‐effective methods of using gravel roads to carry heavy loads (primary weights). On average, Saskatchewan municipal projects forecasted a 40% increase in truck tonnage, on these PGRP up‐graded roads. Approximately 10% of
Saskatchewan strategic
grain corridor upgraded to heavy grain
haul standard by PGRP.
Since the end of the
WGTA, Saskatchewan
has up-graded 25% of
its TMS secondary
highways to heavy
weight standards.
Manitoba PGRP up‐graded 705 km of municipal roads to a minimum municipal standard established by PGRP. The program also contributed to 122 km of provincial highways built to heavy grain haul standards. Alberta’s program contributed to improvement of 352 km of municipal roads with increased road weight capacity. Overall, PGRP allocated 76% of its project funds to road reconstruction to accommodate heavy grain roads. Pavement recapping was the next most popular improvement at 18% of project numbers. Other Impacts
¾ Successful applicants (90%) reported an increase in safety on PGRP project roads. ¾ PGRP did not lever new money for road infrastructure but rather shifted provincial and municipal spending to grain haul roads where PGRP funding was available. ¾ Client satisfaction with PGRP is very high. ¾ Eighty‐nine percent (89%) of PGRP recipients reported that projects would not have proceeded without PGRP assistance. PGRP shifted financial
resources to grain roads
but did not cause
incremental funding.
Impact important to
beneficiaries . . .
…small compared to size
of the problem.
Summative Evaluation of the Prairie Grain Roads Program
3.0
Final Report 5
PROGRAM DESIGN AND DELIVERY
The criteria used to select PGRP projects were different in each province. All provinces were effective at selecting grain haul roads. Saskatchewan’s PGRP program utilized a set of project selection criteria that included location on strategic grain haul routes, contribution to integrated road networks, and other relevant factors related to the intent of the PGRP. Manitoba’s criteria for project selection had some similar characteristics to Saskatchewan’s criteria. However, the lack of a strategic grain haul plan, and limited communication between municipal and provincial road owners, meant that the decision‐making process was less strategic and more reactive to the needs of individual municipalities. Alberta’s PGRP supported projects that were of primary importance to the municipality and did not follow a provincial strategic plan. 4.0
Different criteria used to
select projects in each
province.
PROGRAM RELEVANCE
Continued Need for Public Investment in Road Infrastructure Several indicators point to a continued need for public to address the wear and tear on provincial and municipal roads in the Prairie provinces: 1. The cause of road damage will continue into the future. Grain production is forecast to continue at the same or at greater levels of production. As a result, grain will continue to move by truck on prairie roads with on‐going damage being caused over time. 2. Road infrastructure costs will continue. ‘Grain Transportation in Saskatchewan’, a transportation study completed in 2005 for PGRP, has estimated the required investment demand to address the impact of agricultural haul on Saskatchewan roads resulting from WGTA reform. Changes since 1995, due to policy reforms and industry consolidations, has led to an incremental effect on road damage compared to before 1995. The study estimates the incremental trucking effect on Saskatchewan roads to be 3 billion tonne kms per annum resulting in an annual on‐going road cost of $89.1 million. Increases in the
movement of grain and
other commodities by
truck will continue…
. . . resulting in an
ongoing need for large
investments to upgrade
and maintain roads
Summative Evaluation of the Prairie Grain Roads Program
Final Report 6
3. Construction work has just begun. PGRP has initiated improvements in the capacity of grain roads to carry heavy truck hauls. Approximately 10% of the required work has been completed during the PGRP period. While road infrastructure management strategies, such as load limits or traffic restrictions on certain roads, may contribute to addressing the problem and to reducing costs, significant, on‐going investments will be required to upgrade and maintain roads in the prairie provinces. 4. Future road investments in all provinces will be better targeted when a strategic plan is created for provincial grain haul routes. Furthermore, provincial plans need to be linked together to form a prairie‐wide plan because of the inter‐provincial trucking of grain. There is continued relevance for a program that fosters strategic planning and prioritizing of prairie rural roads. PGRP in Saskatchewan has provided a template for planning and prioritizing grain road investments. However, the overall planning process in western Canada is not complete. Prairie-wide strategic
plan is needed because of
inter-provincial trucking
of grain
Continued Need for Federal Government Involvement 5. Stakeholders identify continued need. Virtually all (99%) of PGRP provincial and municipal program participants report a need for on‐going federal involvement in road infrastructure programming. Eighty‐nine percent (89%) of grain industry participants surveyed reported a need for federal involvement in road improvement. 6. Adequacy of provincial and municipal budget. The cost of up‐
grading the prairie grain roads, is generally considered by provincial and municipal stakeholders to be beyond their resources. An indicator of this situation can be seen over the past 10 years in Saskatchewan. In 1997, Saskatchewan announced a $2.5 billion, ten‐year investment program for roads and highways. The budget covered all operating and capital investment amounts required to support Saskatchewan’s road system. After ten years, the result has been a conversion of 2,200 kms, or 25% of TMS secondary highways, to granular pavement that supports heavy truck weight. This illustrates that most of the $2.5 billion investment was used for operations, maintenance and costs to replace aging roads. PGRP participants point
to an on-going need for
federal involvement to
assist provinces and
municipalities address the
cost of road up-grading.
Summative Evaluation of the Prairie Grain Roads Program
Final Report 7
During PGRP, Saskatchewan upgraded 815 kms of grain highways – 70% were up‐graded with federal assistance, 30% were up‐graded without. This appears to indicate that even with a $2.5 billion ten‐
year road investment program, federal assistance was needed to increase the amount of roads up‐graded to heavy weight standards. Fit with Federal Government Policy
While it could be argued that the PGRPis consistent with the Agricultural Policy Framework (APF) at the highest level, contributing to the “APF’s ultimate goal of sustainable sector growth and profitability, it does not directly contribute to the APF’s strategic objectives as infrastructure development and investment have not been a part of the APF. It should be noted, however, that Transport Canada was the lead department for the PGRP and that AAFC was delivering the program on behalf of Transport Canada, with whose mandate PGRP is more compatible. PGRP’s greatest contribution appears to be in how it fills a need for road infrastructure development in the Prairie Provinces that cannot be met by the provincial and municipal governments alone. PGRP provides a strategic focus to cost‐shared investment with provinces and municipalities in prairie grain roads. It has had a specific focus on up‐grading roads between the farm gate and the point of delivery or processing facility. Stakeholders generally feel that the scope of PGRP has been appropriately focused on grain roads. Some believe it could have been expanded to include other non‐grain agricultural products on prairie roads. They reason that expansion would have allowed good road projects that support agricultural value‐added initiatives (e.g. food processing), which are now excluded under the PGRP’s criteria. PGRP is more consistent
with Transport Canada’s
mandate than with
AAFC’s….
…in any case, it fills a need
for road infrastructure
investment in the prairie
provinces that provincial and
municipal governments
cannot meet
. . . some suggest expansion
from grain roads to include
other agri-related truck
traffic.
Should the federal government decide to continue to invest in road . . . but a better approach
infrastructure in the praire provinces, a future oriented approach could would be to support
select roads for up‐grading based on all heavy haul truck traffic, not “economic corridors.”
just on the basis of grain hauls. The rationale for this is that some roads, such as secondary highways, are used for other purposes such as resource industries (e.g. oil, gas, forestry, and mining) and tourism. Inclusion of non‐grain traffic would recognize all traffic (not just grain and grain value‐added industries) and lead to the creation of Summative Evaluation of the Prairie Grain Roads Program
Final Report 8
“economic corridors” and the strategic use of public funds. Roads with grain traffic would still be supported, but the approach would recognize other non‐grain truck traffic. Cost Effectiveness/Alternatives
Almost all (89%) of PGRP municipal applicants reported that their road projects would not have gone ahead without PGRP assistance. In Saskatchewan, PGRP had the effect of focusing provincial and municipal spending on a strategic grain corridor. No new or incremental spending was reported by stakeholders. Rather, a shift in priority to grain roads because of PGRP was reported. Applicants report that 89%
of projects would not have
proceeded without PGRP.
Overlap and/or duplication of other federal programs
PGRP was successful at avoiding overlap or duplication with other federal or provincial programs. This is due to the composition of the Agreement Management Committee (AMC) which included federal representatives from Western Diversification and provincial government agencies. This provided a safeguard against overlap and duplication of program spending. Alternative approaches to delivery
The partnership between the federal government, the province and the municipalities is seen by participants as the best and preferred approach. This approach is appropriate for roads that cut across municipalities or which connect to secondary provincial highways. A strategic grain haul planning approach can be described as a ‘top‐
down’ approach because projects are selected, based on their overall fit to a strategic plan. Municipalities are most familiar with a “bottom‐
up” approach to government funding of infrastructure projects. This approach is characterized by the municipality selecting projects based on individual municipal priorities. This approach works well for site specific infrastructure like water or sewage, however it is not as effective for grain haul roads that cross multiple municipalities. For this type of road infrastructure, a broader, strategic plan is required to maximize investments and impacts. Some municipalities may
resist a strategically planned
approach . . .
. . . preferring to make local
choices on spending.
Summative Evaluation of the Prairie Grain Roads Program
Final Report 9
There is potential to deliver the PGRP from another Department, outside of AAFC, however PFRA has established good working relationships with municipalities and provincial Departments and has built on past experience to provide program delivery efficiencies. A results‐based performance management system was lacking in PGRP. Program management measured program activity through number of projects approved and costs approved. Some baseline measures and administrative data were developed that compiled applicants’ estimates of truck activity and grain volume on roads before and after project completion. However, no post‐construction measurements were collected by PGRP to substantiate the estimates and to identify the actual road length completed, traffic volume and tonnes of grain hauled on roads. The absence of post-construction data impairs an assessment of
program impact. This is a significant short-coming in the PGRP
performance management system.
5.0
EMERGING THEMES AND CONSIDERATIONS
The evaluation findings provide important insight into the overall PGRP from the perspective of results/outcomes, adequacy of design, relevance and program cost effectiveness. A review of the evaluation results and interview findings identifies a number of “emerging themes” and considerations that would be relevant if future programming is to continue in this area. The on‐going need to upgrade rural prairie roads is present. How the need is addressed will be determined by policy makers’ at all three levels of government. Theme 1 Foster Strategic Planning Through its funding of PGRP, the federal government has created an opportunity for provincial and municipal governments to implement integrated transportation planning for strategic grain routes. Strategic planning was intended to lead to the creation of priority grain haul routes and was expected to be utilized as a tool to target program funding. The creation and use of such a strategic plan was anticipated, PFRA has managed PGRP
effectively and efficiently…
. . . but a lack of resultsbased performance
management focus was
noted.
Summative Evaluation of the Prairie Grain Roads Program
Final Report 10
and was recommended in the Results‐Based Audit Framework (RBAF), but was not identified as a requirement of the PGRP. Consideration 1.1 If the federal government considers involvement in road‐based infrastructure programming in the future (at present there are no plans for such involvement), it should require the establishment of an Integrated Strategic Grain Corridor Plan in each province prior to the funding of individual projects. In addition, these strategic plans should consider the need for planning outside of traditional provincial boundaries to recognize that transportation of grain and agri‐food products cuts across borders. Participants in the strategic planning should include provinces, municipalities and agri‐industry/grain value‐added stakeholders in order to ensure that projects of the highest priority, with the greatest strategic impact, are targeted for funding. The project selection criteria used in each province can further foster integrated planning by attaching a greater number of points to projects that demonstrate collaboration amongst municipalities and stakeholders. Under this situation, collaborative projects would have a greater chance to receive funding. Management Response: Agree. This program terminated on March 31, 2006 with a wind‐down year for project completion to March 31, 2007. No action by program management is required in respect of these considerations. These recommendations will be examined if a new program is proposed. Consideration 1.2 If a decision was made by the federal government to participate in future programming of this, consideration should be given to broadening the scope of the program to include roads with non‐grain related truck traffic. Grain traffic on municipal prairie roads will continue to be a large component of heavy truck traffic. While grain haul could remain the main focus of future programming, inclusion of non‐grain heavy haul traffic will ensure that roads are prioritized and selected based on their overall economic value. This would not likely change the prioritization of municipal roads which are predominantly grain‐based, but may lead to a different and better prioritization for secondary highways that carry grain and non‐grain traffic. Summative Evaluation of the Prairie Grain Roads Program
Final Report 11
Management Response: Agree. Theme 2 Continued Need for Road Infrastructure Investment PGRP has contributed to road infrastructure investment to partially compensate for the adjustment from the WGTA to a restructured GHTS and to a more diversified agricultural economy. Abandonment of branch lines and consolidation of elevators is still occurring but at a much reduced rate compared to previous periods. The effects of transportation adjustments are on‐going and will continue for several years until selected roads in the prairies have been built to a standard that can support heavy grain haul trucks. The need to rebuild prairie grain roads is not currently being addressed by other programs at Transport Canada and Infrastructure Canada. The need for road infrastructure will continue in the Prairie provinces; it is not clear that provincial and municipal governments will have the resources to address this need. Consideration 2.1 Road infrastructure development must be weighed against other government (federal, provincial, and municipal) priorities. Continued support for road infrastructure will need to be considered to address the required upgrades and construction of grain haul roads to standards that can support heavy grain haul trucks. These roads provide a broader benefit to a diversified agricultural production base and to other economic activity in the Prairie Provinces. Policy makers and planners should establish a plan for road infrastructure and determine the roles and responsibilities appropriate for each level of government. Management Response: Agree Consideration 2.2 If the federal government should consider future involvement in road infrastructure programming, consideration should be given to allocating program funds to support the cost of research, development, management, and monitoring projects such as: 1) Performance Management System ‐ develop indicators and measurement systems to effectively track program results and impacts. Summative Evaluation of the Prairie Grain Roads Program
Final Report 12
2) Innovative Solutions to Road‐Building and Road Deterioration – support research into new technologies, alternative traffic management methods, alternative road construction techniques, use of a global positioning system (GPS) to track truck traffic, etc. Other, more cost effective alternatives may be developed that reduce the need to up‐grade roads to heavy weight standards. 3) Project Information Gathering and Measurement of Truck Traffic – conduct customized traffic counts to measure before‐project and after‐project heavy truck haul traffic levels so that better information is available to show program results. 4) Delivery Point Trucking Information – develop a research project to collect data at the point of delivery (grain elevators, processing plants, etc.) on length of truck haul, route traveling from farm to destination and types and weights of trucks. Such information will provide improved data and information for monitoring transportation from farm to delivery point. 5) Formalize Post‐Construction Review ‐ a formalized procedure should be implemented to certify that municipal road projects have been built to project specifications. An informal certification process for road inspection was utilized by the PGRP. A third‐
party, technical assessment and certificate of completion should be completed for each project that receives PGRP funding to ensure project specifications have been met. 6) Build on Existing Capabilities - any future programming in grain and agri‐industry road infrastructure should consider building on PFRA’s experience, capabilities and working relationships. Stakeholders report a high degree of satisfaction with the implementation and administration of the PGRP. PFRA at AAFC has responded to suggestions arising from the final evaluation of the Canada Agriculture Infrastructure Program (CAIP) by initiating greater communication, implemented a more comprehensive administrative data base and placing a high priority on integrated planning (particularly in Saskatchewan). PFRA has demonstrated experience in project management and are well qualified to provide valuable input on construction related projects from an engineering and environmental perspective. Project applicants and PFRA have made a significant effort to collect data based on pre‐construction information and to Summative Evaluation of the Prairie Grain Roads Program
Final Report 13
estimate post‐project traffic. Most importantly, PFRA has established a good rapport with municipalities and is building strong relationships with provincial Departments involved in road infrastructure development. Management Response: Agree Theme 3 Create a “Whole System” Approach to GHTS The present Grain Handling and Transportation System (GHTS) in Canada operates as two disconnected components. Farm‐to‐ elevator trucking and related road costs are considered in isolation from elevator‐to‐port logistics. The federal government (AAFC and Transport Canada) has created policies and monitors the patterns and costs associated with the elevator‐to‐port monitoring component to help improve the efficiency of the GHTS. But this is a partial measurement of the overall system. As a result, efficiency gains in this component can come at the expense of the other component. Spring road bans, weight restrictions on key grain roads and uncoordinated permitting requirements between municipalities can influence the efficiency and cost of grain delivery to elevators. This component can also influence the GHTS’s ability to meet customer delivery requirements. Efficiency improvements in the elevator to port component of the GHTS can create cost consequences for the provinces and municipalities and increase trucking costs to grain producers. Consideration 3.1 The federal government should consider creating a ‘holistic approach’ to the GHTS. This approach should encourage coordination of road planning among provincial, municipal, and industry stakeholders and measure the efficiency and performance of the GHTS from farm to port. This could initially begin in two ways. Firstly, strategic planning of grain road corridors be carried out with the involvement of grain and agricultural value‐added industries, together with federal, provincial and municipal governments. Secondly, the federal government could consider expanding the Grain Monitor’s terms of reference to include performance monitoring from the farm to the delivery point. This should factor in costs related to grain haul impact on municipal provincial roads and costs of trucking. This will require new data being collected at delivery points that could include, for example, truck haul distances, type of truck, Summative Evaluation of the Prairie Grain Roads Program
Final Report 14
truck route to delivery point (or strategic/non‐strategic corridor route), type of grain hauled, etc. This will allow an accurate and complete assessment of the costs and efficiency of the overall GHTS. Management Response: Agree