Here - Nicro
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
NICRO PROGRAMME TO HELP ACHIEVE THE CREATION OF A SAFE
SOUTH AFRICA
BY CONTRIBUTING TO SIGNIFICANT PENAL REFORM
D R R ICHARD A G RIGGS .
MONITORING AND EVALUATION SPECIALIST ( EXTERNAL )
© NICRO 2013 1
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
TABLE OF CONTENTS
Introduction To The NICRO Programme To Help Achieve The Creation Of A Safe South Africa
Annexure 1.2: DFID Achievement Ratings Applied To 7 Intended Outputs For An Overall
Annexure 1.3 DFID Log Frames On Outputs With Detailed Comments ................................ 187
© NICRO 2013 2
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
FINAL EVALUATION OF THE NICRO PROGRAMME TO HELP ACHIEVE THE CREATION OF A
SAFE SOUTH AFRICA BY CONTRIBUTING TO SIGNIFICANT PENAL REFORM 01 JANUARY 2009
TO 30 JULY 2012 1
GTF Number GTF 112
Short Title
Programme of Non-Custodial Sentencing Project
Name of Lead
Institution
National Institute for Crime Prevention and the Reintegration of
Offenders
Start date 2 08 September 2008
End date: 31 March 2013
Amount of DFID
Funding:
£4,916,108
All countries where activities have taken or will take place
South Africa only
Implementing partners
Lead Author
Only NICRO
Target groups- wider beneficiaries
Departments of: Justice and Constitutional Development,
Correctional Services, Social Development, National Prosecuting
Authority. South African Police Services and traditional leaders, diverted and sentenced offenders, families of offenders, communities, social workers, magistrates, Legal Aid
Dr Richard A Griggs, 5 Little Beach, Bakoven, Cape Town richgriggs@telkomsa.net
072-624-8687
Other contracted undertake
Evaluation people to the
No one (K&S Accounting sub-contracted to review expenditures)
1 The period of the evaluation January 2009-30 July 2012 = 3 years 7 months or 43 months
2 Date the grant agreement was signed but the project began in 2009.
© NICRO 2013 3
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
LIST OF ABBREVIATIONS AND ACRONYMS
AS Alternative Sentencing (also refers to NCS; see below)
CBO Community Based Organisation
CFMC
CJS
CPA
CSO
DCS
DFID
Case Flow Management Committee
Criminal Justice System
Certified Public Accountant
Community Service Orders
Department of Correctional Services
Department for International Development (UK Government)
DoJCD Department of Justice and Constitutional Development
DSD Department of Social Development
GTF
JCPS
Governance and Transparency Fund
The Justice, Crime Prevention and Security Cluster
LASA
LSC
Legal Aid South Africa
Local Steering Committee
MTR Mid Term Review
NCS Non-custodial sentencing (sometimes referred to as AS or alternative sentencingsee above)
NICRO National Institute for Crime Prevention and the Reintegration of Offenders
NGO Non-Government Organisation
NPA
PSP
PM
PSDI
National Prosecuting Authority
Policy and Procedures Manual
Project Manager
A Professional Skills Development Initiative (PSDI)
SA South Africa
SANCA South African National Council on Alcoholism and Drug Dependence
SAPS
SAW
SW
South African Police Service
Social Auxiliary Worker
Social Worker
© NICRO 2013 4
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
NICRO’S FORMAL RESPONSE TO THE IMPACT EVALUATION
Output Final Impact Evaluation Conclusions
Output indicators below) = 2
1: fully-fledged alternative
A referral system developed for sentencing at 45 magistrate’s courts
Overall rating
(explanation given in two
The reasons for low referrals:
1. A tension exists between the magistrate’s desire to complete cases backlogs (owing to high court rolls) and NICRO’s professional requirement thorough assessments requiring up to two weeks or more, which are perceived to cause court delays
2. A magistrates are making the referrals-- (sometime one or two targets
Welkom quickly small and to evaluation presents reduce conduct percentage of per court), which produces insufficient referrals to achieve
3. 50% of magistrates surveyed said speedier assessments would improve the referral rate
4. Some magistrates see NCS as an unproven risk and want more evidence of success such as this
5. Some magistrates reported not getting feedback on cases such as at Tembisa, East London and
6. That some magistrates did not receive feedback was confirmed by SWs (Annexure 7, question
20) and by supervisors
(Annexure 8, Section 8.2, and
Question 12). There was also widespread confusion amongst
NICRO Management
Response
1. Here lies the philosophical tension between (i) following what research has indicated is the best practice for offender reintegration versus (ii) a managerial approach to criminal justice, which focuses on statistics and timeframes.
2. We cannot influence this any more than we have done.
3. See point no. 1.
4. To date NICRO has made all our statistics available. We have two books of offender testimony. It is possible that this is an excuse not to use NCS. The same absence of data indicating success does not put judicial officers off using prison as a sentencing option.
5. NICRO social workers do give feedback when specifically requested to provide follow up reports. Few magistrates have ever asked. When we have asked some magistrates whether they want automatic feedback (unasked), they said no.
6. A magistrate can ask for feedback at any point, not just after
© NICRO 2013 5
Output
© NICRO 2013
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
Final Impact Evaluation Conclusions
SWs, as to when feedback is required. After assessment and after treatment are correct and only 9 of 31 (29%) SWs could offer such a reply.
7. Some magistrates lack confidence in a particular SW
(this occurs when magistrates are not given feedback on cases or when there has been SW turnover)
8. SW turnover in terms of reducing magistrate confidence is a big challenge owing to SW shortages and NICRO’s inability to offer better salaries and benefits while some other employers competing for the same SWs offer more pay and benefits
9. The LSCs were originally designed as therapeutic case management forums but this did not occur owing to time constraints and skills deficits; instead members cooperated on ensuring the smooth delivery of the services (although this did not match the original function, it proved to be important)
10. Certain LSC partners reported in focus groups and interviews at a sample sites visited and on surveys that they were not receiving the feedback on client outcomes via the NICRO SWs (it has improved since the MTR but these reports were confirmed by the SWs themselves, 40% of whom indicated on question 20
Annexure 7 –they were not offering appropriate feedback on
NICRO Management
Response assessment and treatment. This issue is one of the purposes of the LSC
– to sort these matters out and to get clarity on what the magistrates want.
7. NICRO has attempted to address NICRO skills and capacity through ongoing staff training and skills development.
Senior staff are also available to provide support where needed.
8. NICRO has attempted to address the high social worker turnover, where possible, and continous efforts are underway.
Government departments offer excellent packages and benefits which attract social workers.
9. The LCS did not function as they were meant to.
Guidance and support from NICRO Head Office was provided on a monthly basis and some
LSC meetings were also attended by project staff.
A guide for staff on how the LSC should operate was also developed and sent out. There was also the guide in the
NCS Toolkit.
10. See point no. 6.
11. No comment.
12. In a limited way and not at all NCS sites.
13. Neither DCS nor DOJ
6
Output
© NICRO 2013
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
Output
Therapeutic
2: programmes available for in utilization alternative sentencing
Overall rating
(given in three indicators below) = 1.5
Final Impact Evaluation Conclusions client outcomes)
11. All protocols, tools, processes and guidelines in place
12. The Department of Justice was making space and facilities available for NCS services at selected courts around the country
13. A ‘fully fledged’ system requires monitoring, aftercare and tracking and this this was not always occurring; a partnership with DCS and the Judiciary on appropriate systems and sentencing for NCS might help to address this
1. NICRO demonstrates a high level of professionalism in terms of programme design and development
2. No significant shortage of programme offerings were detected while a more significant problem was detected in terms of SW capacity to deliver certain programmes.7
3. 54% of the supervisors thought that all the available intervention offerings were adequate. A little under half thought that some additional offerings were
‘needed’. However, the definition of what is most needed varied greatly from site to site. In analysis, intervention tools that help to address anger issues, poverty, and substance abuse were viewed as needed by those who said no.
However, based on all data,
NICRO Management
Response have expressed any keenness in assisting with NCS, except here and there at the local level. Less than 4% of
DCS’s budget goes towards Community
Corrections and they are understaffed and undercapacitated. It is not likely that they will have resources available to help with NICRO aftercare.
1. No comment
2. NICRO has attempted various methods and approaches for staff training and development, including national training, provincial cascading training, training, onsite coaching, online coaching etc. 1 287 people have been trained as of Feb 2013
(obviously more than once per person). Senior staff are also available to provide support where needed. Work is now underway to develop skills related manuals, as opposed to only offender programmes manuals.
3. See above point.
4. The NICRO social workers tend to not use the Matrix programme
7
Output
© NICRO 2013
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
Final Impact Evaluation Conclusions training in the existing offerings is more important than developing new offerings.
4. Matrix was only utilised for about 10 clients —it is potentially the best of the programmes but it is regarded as too lengthy and not all SWs have the skills required
5. Still in development on 30 July
2012 were Journey, Adult
Lifeskills (content development underway); Community Service
(90% complete); Critical Thinking
Manual (was very close to completion —mainly formatting)
6. SW turnover disrupted services at many sites
7. Magistrates convinced of the worth of the services refer but too few magistrates who want to refer were happy with the time period for the assessments
8. 6000 was also a revision downward from original targets @
14,640 offenders enrolled in programmes by March 2013
— these targets were unrealistic for a variety of reasons but also for the above bulleted point
9. The drop in Year 5 owed to
1. SW turnover
2. NICRO restructuring Nov
2011- April 2012, which included a moratorium on filling SW vacancies;
3. Retrenchments at NICRO from November 2011; and
4. Relocating sites with low referrals to new sites in the
NICRO Management
Response and send people to
SANCA instead because they state the programme is too long and demanding on staff.
Training and skills development on the
Matrix programme has been provided.
5. The programme design and development is behind schedule. The reason for the work being behind schedule is that the staff with the programme design and development skills also have other functions to fulfill (service delivery, project management etc).
6. See earlier point about staff turnover.
7. See ealier points about the tension between best practice and bowing to court pressure.
8. As of Feb 2013, we are
77% target attainment and should get very close to 100% by
September 2013.
9. Relocating sites is necessary. If NICRO has done its best and still the court does not refer, NICRO cannot continue to spend money on a site that does not function.
8
Output
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
Final Impact Evaluation Conclusions hopes of improved referrals
(this was a calculated risk because the start-up time at a new site can be very slow and this changeover can also dampen impact on the improving referrals)
Output
Professional
3:
SW expertise available at 45 sites
Overall rating
(given in two indicators below) = 3
1. “All 45 sites staffed by one fully trained SW by project end” is a perfunctory measure of output and cannot be rated as fully achieved because there are strong indications that one SW is not sufficient since of sites surveyed.
2. Annexure 7, Section 7.3, question 6 shows that at only
10 of the 31 sites (32%) was there a SW devoting more than 20 hours to NCS. This demonstrates the high degree to which SWs were multitasking and that the time available to them for developing referrals to NCS services was compromised by other obligations.
3. Annexure 7, Section 7.3, questions
5 and 6 show that most SWs were multitasking between service streams-- diversion, NCS, offender reintegration, schools and community work) and 61% of SWs found this difficult or very difficult
4. Annexure 8, question 4 on the
Supervisor Survey shows that
62% of supervisors thought the SWs should NOT be multitasking between service streams, rather the SWs on
NCS should be full-time NCS
NICRO Management
Response
© NICRO 2013
1. There is divided opinion on whether more than one SW per site is needed. There is an opinion that it is more a question of conducting proper planning and time management.
2. It is inevitable that SW will multitask as and where needed in service delivery.
3. See above point.
4. The supervisors have the authority to allocate cases to different social workers. If multitasking is perceived as a problem, then examining case management at the site level would be part of the solution.
5. See earlier point on staff turnover.
6. See earlier point on staff turnover.
7. See earlier point on staff turnover.
8. See earlier point on staff turnover.
9. DCS and DSD are unlikely to assist NICRO in this regard. They want the same social workers as we do, so they are, in effect, our competitors. DSD has
9
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
Output
© NICRO 2013
Final Impact Evaluation Conclusions specialists
5. There were times when SW turnover was a very serious threat to the project
—in March 2012 there were 16 vacancies (36% of sites) — and while it can be stated that by
August 2012 there were only 2 vacancies it remains that turnover is a significant programme threat
6. Of the 34 sites surveyed
(supervisors), more than half (18 or
53%) had challenges of SW turnover. Eight of those 18 sites or
44% were on the 3 rd or 4 th replacement which makes it very difficult to develop magistrate trust and to provide the continuity of service required to achieve the targeted referrals. .
7. There is a national SW shortage and turnover of SW at sites can slow the pace of referrals as
NICRO looks for a new SW. This partly explains the failure to achieve minimal referral targets
(the supervisor must slow down or suspend referrals while training a new SW; magistrate confidence is weakened; the LSC is weakened)
8. NICRO cannot compete with the
DSD in terms of salaries, benefits such as bursaries and this caused frustration amongst certain NICRO supervisors and provincial managers who sometimes described themselves as ‘training
SWs for the DSD’
9. For the reasons above and more, the availability of SWs must be considered in a broader context than filled posts at a given point in time.
NICRO Management
Response even thought about bringing social workers from overseas, such is the dire need.
10. The state of NICRO’s aftercare service is an issue of long standing.
Some people insist that only social workers can do aftercare (see earlier points about SW shortages) and others feel that lay volunteers can help out. Sites can request volunteers (they do get paid a stipend) if they motivate to Head
Office: Human
Resources. The Human
Resource Manager said that few sites have requested volunteers. In other words, there are other options available, but sites need to plan properly and make the necessary motivations.
More attention will be given to address the aftercare service gap.
11. See above point.
12. See above point.
13. See above point.
14. See above point.
15. See above point.
16. See earlier point about the likelihood of DCS and DSD assisting
NICRO.
17. See above. NICRO has made numerous and ongoing efforts to meet with state officials from various departments at
10
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
Output
© NICRO 2013
Final Impact Evaluation Conclusions
This is but one of many lines of evidence that NICRO will have to develop a joint strategy with the
DSD, DCS and other stakeholders to respond to such challenges as these.
10. Aftercare is essential to describing the delivery a professional service because it creates a support structure for the client to address further needs after completion of the initial programme but it was not regularly provided because it requires social auxiliary workers in support of the SW and that was seldom available (this is why 2
SWs per site are needed as stated above)
11. Client tracking refers to follow up for monitoring purposes and only
13% of clients were tracked more than 7 months
12. Client tracking and aftercare persists as the weakest areas of delivery and yet these areas are critical components of a fullyfledged NCS service (see Figure below)
13. In interviews and surveys SWs and supervisors admitted aftercare was not occurring and explained the reasons--see Figure below for reasons
14. The challenge of insufficient
Aftercare and Tracking persisted throughout the intervention, was identified at the time of the MTR, and might be described as the
Achilles Heel’ of the project
15. In the 1990s when DSD ran a
Pretoria-based diversion
NICRO Management
Response all levels. Many times state officials are simply not available to NICRO.
When meetings take place, the clear message is that the state departments (i) lack a comprehensive funding policy that would coordinate funding of
NGOs for service delivery (ii) that it will take time to develop and implement such a funding policy (iii) they like what NICRO does and see the benefit of it but they cannot make a decision. (iv) They advise us to meet with another state official (v) they tell us it is not within their department’s mandate. This has gone on for years. The political will and budgetary resources to support the NCS programme at National
Government levels appears to be lacking.
11
Output
© NICRO 2013
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
Final Impact Evaluation Conclusions programme for youthful offenders, the programme collapsed because there was no monitoring and followup with consequences. some serious
16. DCS must be a partner in the delivery of NCS services (a vital recommendation of this report) but they cannot cope with their policing/correctional services/monitoring role. The old system described collapsed this way and little has changed — basically, Correctional Services has an insufficient budget for a monitoring and follow-up role.
Correctional services cannot simply be relieved of their duty and are ultimately responsible for monitoring and yet have proven over decades that they cannot deliver without support
17. A DCS/NICRO partnership from national level is essential to NCS services but this was not consolidated even though there has been dialogue making this a critical focus for NICRO
Output 4: Expert alternative sentencing programme interventions
Overall rating
(given in two indicators below) = 2
1. The log frame indicator is not sufficient for exposing the challenges including gaps in delivery owing SW turnover (they do not stay in positions long) and posts not filled or a new SW starting up — all of which compromise the quality of the intervention. Those that were trained and became familiar with the intervention offerings departed with sufficient frequency that some
NICRO sites were being drained of capacity constantly
NICRO Management
Response
1. See earlier point on staff turnover
2. See earlier point on staff training.
3. See earlier point on staff training
4. See earlier point on staff training
5. See earlier point on staff training
6. NICRO requires sponsorship for staff to go on external training, and in the South African criminal justice sector,
12
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
Output
© NICRO 2013
Final Impact Evaluation Conclusions
2. The supervision model and the superbly developed PSDI can still leave gaps as identified by both supervisors and SWs in their surveys
3. Annexure 8 shows that 77% of supervisors did not think the SWs had all the requisite skills and training to deliver in all needed areas (Perpetrators of Intimate
Partner Violence, Sexual Offences and Matrix are 3 examples)
4. Annexure 8, question 2.5 shows 13 supervisors, identifying a challenge with either: (a) new recruits lacking sufficient training (having missed formal opportunities) or (b) SWs who were skilled but could not advance to provide certain needed programme through self-training only (sexual offences, matrix, programmes were cited) perpetrator
5. Annexure 7 shows that the SWs themselves wanted formal training opportunities to advance their skills so that they could address all criminogenic risk factors identified amongst clients (see questions 2, 4, 7
8 and 18)
6. While the NICRO professional skills development initiative is an example of a good management effort in the face of very limited resources, it is not sufficient to replace all formal training opportunities away from the workplace. These interactive opportunities are needed to produce a cadre of well skilled SWs who can be retained to provide a large range of needed services (an essential strategic component when it comes
NICRO Management
Response such training can cost up to R5000 per person
(over £300).
7. Government does not have the expertise to train NICRO staff. When government departments have funds, they often approach NICRO for training. Since the beginning of the project,
NICRO has invited all government stakeholders at national and local level to participate in training offered in conjunction with the NCS project.
Conducting joint training with state departments for NCS would be difficult to coordinate.
8. Procurement equipment of and resources is dealt with according to stringent policies and procedures in terms of good governance. Within this context, requests were dealt with as quickly as possible. Not all site requests were granted.
9. Court personnel have expanded far more quickly than the court infrastructure. It is common for staff to work in container offices etc.
NICRO has little choice in the offices made available to us at court.
10. There is little that
13
Output
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
Final Impact Evaluation Conclusions to marketing the services)
7. Rather than formal training as an
NICRO internal offering, it would be better to provide training with all involved departments and stakeholders as part of an integrated strategic effort to deliver services as recommended in this study (see recommendations)
8. Not all sites received equipment and furnishings in a timely way but by the time of the evaluation they had what was required
Output 5: A national media and marketing campaign: increase public awareness of
NCS and restorative practices
Overall rating
(rating given in
5 components
10. The biggest gap was the lack of landlines in several offices. The SWs tried to use cell phones but these are a more difficult way to communicate and more expensive and compromised some components of delivery including Aftercare and
Tracking.
1. There was never a point in the intervention in which against NCS services such that it can be said that services were welcomed by communities the community resisted or protested
2. The support ratings are coming mainly form poor communities as do most NICRO beneficiaries
(mainly young, male and poor)
© NICRO 2013
9. Wherever NICRO utilised facilities provided by the courts, there was less of a therapeutic atmosphere than might be desired (no waiting areas, poor design, and minimum space) —sometimes-shipping containers were used (turned into offices).
NICRO Management
Response
NICRO can do about the absence of landlines at the offices in the courts.
It is not a NICRO building and getting permission to install landlines is difficult.
Often landlines at courts were abused and difficult to monitor. The decision was taken in
2010/2011 that all
NICRO staff will function from the NICRO node where all the infrastructure and resources are kept. All the nodes are fully equipped. NICRO does not have the means to then, in addition, equip all the satellite offices at the courts community centres. and
1. No comment.
2. No comment.
3. No comment.
4. No comment.
5. It is not clear that conducting additional research into this issue would add value to the
NCS service.
6. No comment.
7. No comment.
8. No comment.
14
Output below) = 2
© NICRO 2013
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
Final Impact Evaluation Conclusions and this suggest that amongst family members in these poor communities there is little overt resistance to NCS services (at least less than anticipated in project design).
3. Those being surveyed are mainly responsible citizens interested in programmes that can alleviate crime and therefore there can be a bias in the figures
4. Owing to the above it would be beneficial to identify in more precise terms the characteristics or traits of those opposed to restorative justice.
5. It could be hypothesised based on the general data in support of
NCS in poor communities that resistance is found in greater numbers amongst the middle class and wealthy (often the decision makers and those allocating budgets) than amongst the poor (the segment most subject to arrest)
6. The NICRO media and marketing work was comprehensive (broadcast, print, and online) and of such a quality that it had impact (as evaluated by Newsclip) and assisted both primary stakeholders and the public to better understand the services.
7. NICRO first commissioned some professional support for marketing and then gradually assumed responsibility for it. Total Media
NICRO Management
Response
9. This is a debatable point. It was clear that some social workers with substantial workloads also did their media and marketing work. It is likely to be an issue of time management, planning and will.
10. See point. 9.
11. This belies the data contained in the monthly site reports which stated, throughout the life of the project, that stakeholders ongoing require education workshops. Court personnel have a high turnover, and many magistrates rotate to a new court every six months. Project staff site visit interviews with court stakeholders indicated that most court stakeholders wanted more workshops.
12. Local workshops cannot address the systemic problems, but they can address the issue of not knowing that NICRO is available at that court.
13. NICRO cannot do this.
See earlier point.
14. No comment.
15. The Lobbying and
Advocacy Manager addresses systemic criminal justice matters in her submissions.
Focusing only on NCS is
15
Output
© NICRO 2013
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
Final Impact Evaluation Conclusions initiated a campaign Apr 2009-
Feb 2010 starting with media roundtables that introduced selected media to the NCS concept and generated a media conversation, and then NICRO worked on a pro-rata basis in
Years Two and Three with
Redline with NICRO assuming marketing responsibility in the final years. NICRO demonstrated the capacity to run a good campaign and the stakeholder and public videos produced by the project were of a very high standard —high enough to air on national television. This suggests that had NICRO assumed responsibility earlier (rather than contracting out) some expenses might have been saved.
8. At some sites the media published stories about the services NICRO renders
(sometimes following community engagement activities) and supervisors reported receiving positive comments from the public following these publications
9. Considering that nearly all sites had but one SW who normally multi-tasked between projects
(NCS and others) and one supervisor multi-tasking between sites, the human resources are insufficient at each site for marketing activities to the strategic level required to move the national from punitive thinking about justice to thinking of justice in more restorative terms —NICRO needs either
NICRO Management
Response counterproductive at the
Parliamentary level. At that level they deal with sectoral issues. NICRO will continue to seek opportunities at
Parliamentary level to promote NCS.
16. See above No. 15.
17. See above point No. 15.
18. See earlier points on partnerships with government.
19. Not all NCS offenders are sentenced simultaneously to DCS
Community Corrections.
See earlier points on possible funding from the state. See earlier points on the networking work done by NICRO to date, which is not acknowledged by state stakeholders in the impact evaluation interviews and focus groups.
20. See earlier points on this issue.
16
Output
© NICRO 2013
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
Final Impact Evaluation Conclusions more staff at each site for this engagement and more partners
(see the bullet below as well)
10. Scaling back from 45 NCS sites to about half as many or even fewer; and mainstreaming NCS into a regular NICRO offerings; should release some energies for marketing NCS as part of the range of NICRO offerings and building community level understanding and participation
11. As early as the MTR and persisting into the evaluation, magistrates stated that no more educational opportunities, workshops or publications for magistrates are required and such activities are unlikely to change referral levels:
12. The rate of magistrate referrals cannot be improved with educational or outreach activities because the challenge that needs to be addressed is systemic: magistrates are under pressure to reduce the court roll and this requires that they speed up case processing times and therefore they want NICRO to speed up the assessment time owing demands on them
(79% of magistrates surveyed said this; See Annexure 10, question 7; See also the Basil
King interview Annexure 25, question one )
13. Most magistrates want the assessment back in 2 weeks or less and in one day in some cases
—see question 6 on the
NICRO Management
Response
17
Output
© NICRO 2013
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
Final Impact Evaluation Conclusions
Magistrate Survey, Annexure 10
14. Magistrates and court stakeholders are so overworked and pressurized that most of these project materials end up in a drawer
15. Submission were not so specific to NCS
16. These were made to parliamentary portfolio committees and framed within an overall advocacy effort to see more of the budget allocated to prevention as opposed to police, courts and jails
17. Findings from the project were not integrated into an advocacy effort but advocacy was more general (as described above) and appeared to have had little impact on sustaining the project or increasing referrals
18. The particular logframe indicator appeared incomplete. NCS services need to be sustained more in terms of partnerships with government departments such that a strategy to lobby the
Parliamentary Portfolio committees (mainly Correctional
Services) did not go to the issue of sustainability.
19. Ultimately perpetrators on NCS become a DCS responsibility and it is also possible that in terms of a sustainability plan, they might fund NICRO. Achieving this outcome requires advocacy and lobbying national level
NICRO Management
Response
18
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
Output
Output 6: A newly developed indigenousbased diversion programme available
(Eerstehoek)
Overall rating
(given in 2 components below) = 2
Final Impact Evaluation Conclusions departments but this was not done sufficiently as clearly indicated in the focus group with
DCS National (see Annexure 18)
20. See Annexures 18, 19, 20 and 21
–interviews at national--which clearly informed the evaluator that by late 2012 a partnership had not been consolidated with the most important decision-makers. Only
DSD national appeared to have no interest in NCS for ‘adults’
(short-sighted thinking) but
DOJCD, DCS, JCPS, and DSD
Provincial stated they were very interested in NICRO approaching them with some formal partnership proposals.
1. Target not attained
2. Most of the target attainment was NOT via the traditional courts but owed to school, community and magistrate referrals
3. Most referrals started in the third year of the programme (very slow start up)
4. NICRO dropped work in the traditional courts that did not refer including eNikwakuyengwa
(1 referral) and concentrated on the ones that did refer —
Eerstehoek Magistrate Court and the tribal courts of eMbhuleni,(3 referrals)
Mandlamakhula (1 referral) and
Mpisikazi (5 referrals)
5. The school referrals owed to a crime prevention programme in
© NICRO 2013
NICRO Management
Response
1. This is correct, depsite ongoing efforts to address the low rate of referrals from the traditional courts.
2. See above.
3. See above.
4. See above.
5. NICRO found the schools to be more responsive in referring at risk youths than the traditional courts.
6. No comment.
7. No comment.
8. No comment.
9. This is the reality of doing rural work in South
Africa. Distances are vast and there is no public transportation.
10. This is the approach that NICRO subsequently adopted, and which resulted in the
19
Output
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
Final Impact Evaluation Conclusions two schools 3 which then referred at-risk youth to NICRO and then these youth became mentors to other at-risk youth
6. NICRO’s original hypothesis that traditional practices of restorative justice could be incorporated into diversion proved false —most of the traditional leaders and courts had little knowledge of restorative justice principles
7. The failure of the indigenous justice model does not mean that
NICRO failed since upon identifying the challenges in 2011, the organization then focused on schools and community work, which was effective in increasing referrals to NICRO programmes
(this locates an appropriate model for working in such areas — community based work).
8. Despite a high acceptance rate, this is a perfunctory measure of performance that involves only
10 referrals such that a 2 rating does not capture the challenges
9. The area was too big in both population and distances for
NICRO to provide an effective intervention either in terms of
NCS or diversion from traditional courts. NICRO served 3 traditional courts but there are 12. School and community interventions provide a better way in rural communities to make impact on
3 The High Schools: Bantfwabetfu and Dlomodlomo
© NICRO 2013
NICRO Management
Response referral rates increasing.
11. No comment.
20
Output
Output
Develop
7: a knowledge base of NCS systems and practices to further public and sector understanding and to promote awareness
Overall rating
(given in 5 components below) = 1
© NICRO 2013
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
Final Impact Evaluation Conclusions more people.
10. The LSC thought NICRO should rather intervene in schools and communities to prevent crime and divert people out of the system and to advocate that national and provincial departments provide training in the Traditional Affairs Act
11. The magistrate and the case flow manager (NPA) thought that court cases had been reduced owing to the schools work and strongly recommended that this be the focus
1. The traffic is was relatively low in terms of number but the forums attract researchers and media interest such that there are ‘spinoffs’ for disseminating findings
2. A large number of research products were generated that enable other stakeholders and organizations to learn about the practice
3. The data from the NCS services offers very good value internationally and within South
Africa
4. All materials were of a very high standard
5. It cannot be said that all promotional materials
(especially bracelets and bags) offered high value but in general most materials were of high value and effectively utilized —
NICRO Management
Response
1. The low
3. No comment.
4. No comment.
5. This is an ongoing point of vigorous debate in
NICRO. The service delivery staff insist these items are needed. The uncertainty appears to lie in whether or not these promotional items have impact. traffic is disappointing, but those who use the sites have indicated they are of value (both facebook and NING).
2. No comment.
21
Output
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
Final Impact Evaluation Conclusions see Figure 3.8 Below)
NICRO Management
Response
RECOMMENDATIONS
Impact Evaluation
Recommendations
1. Reduce the number of sites to
‘sites of excellence’ so as to design a service that can be presented as a model for national roll-out
2. Design a long-term DRAFT strategy for providing sustainable and replicable services nationwide as a discussion document and distribute it to key stakeholders, national departments and especially the JCPS Cluster
Body
3. Follow up on the above discussion document by meeting with the JCPS Cluster and each department at Deputy
Director level with the objective of integrating stakeholders into a long-term strategic plan for the delivery of NCS services
4. Enter into a partnership agreement with SAPS and the
DCS (Community Corrections) that includes control groups for measuring the impact of the
NCS services on recidivism
5. Design with other stakeholders a magistrate-to-magistrate programme of sharing information about NCS services and a programme to educate new magistrates at the
Justice College
NICRO Management Response
We have to do this in Year six (2013) as there are insufficient funds in the Year 6 budget to run 45 sites.
Having been unsuccessful in obtaining state support, we have reduced the number of sites from 45 to 28.
NICRO will have to carry the costs of these 28 sites where the DFID budget is insufficient.
NICRO has begun drafting a long term strategy for taking the NCS service forward. NICRO will provide more details on the strategy in the Post Completion
Report. In addition, NICRO is hosting a seminar to release the results of the NCS impact evaluation.
NICRO will also begin a discussion about the NCS strategy at this seminar as part of the way forward.
We are inviting senior national government stakeholders to the event.
This has been done in the past. NICRO can follow up with the JCPS cluster again once the strategy document has been completed (see above point).
NICRO can explore these possibilities as part of the above-mentioned strategy.
NICRO has approached the Justice College and the
Association of Regional Magistrates in South Africa.
However, there are various adjustments and developments at these organisations, and this needs to settle down before we can engage in any meaningful work there.
© NICRO 2013 22
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
Impact
Recommendations
6. Design at all sites
Evaluation with other stakeholders a joint strategy for monitoring offenders, providing aftercare and tracking them consistently
NICRO Management Response
Before such a joint strategy can be explored and discussed, the various government role-players must, amongst themselves (i) finalise their funding policy for
NGOs (ii) agree upon their own and other departments’ mandates and (iii) are willing to allocate sufficient resources to this. NICRO is currently working on a review of its Offender Reintegration
Services, and joint tracking and aftercare will form part of that review.
We have done this already from an organisational point of view. This is not a NCS problem, but a social welfare problem.
7. Design a comprehensive strategy for providing sufficient
SWs for NCS services in view of the SW shortage in South
Africa
8. Design a community-based model for working in indigenous areas and drop the existing indigenous based model of working with traditional courts
9. Design a time-based model for various kinds of assessments so that magistrates can understand a specific time range for completing different kinds assessments
NICRO can explore this option. In a sense, this is what happened when we moved our focus from the traditional courts to the community and schools. It is now more a question of writing up the model and lessons learned. This is planned for 2013
– 2014.
NICRO cannot do this.
Each case must be dealt with on its own merits. Each offender is dealt with as an individual. There are professional standards in place for social workers.
People can be struck from the register and no longer be allowed to practice if these standards are not met.
Our standard time range is 4 weeks, and this is what we communicate to the magistrates.
This is not feasible. NICRO cannot do this every time a pilot or new service is rendered, which NICRO does on a regular basis.
10. When Developing A New
Service, Develop And
Implement A Change
Management Strategy So As
To Manage And Monitor The
Transition In Organisational
Culture
11. Budget Sufficiently
12. Ensure The Magistrates
Receive Feedback On The
Treatment Plan And Outcomes
13. All Evaluations Should Begin
With A Pre-Intervention
NICRO now has greater experience in drafting a budget for such a large-scale and varied project.
This matter will be discussed with NICRO staff and magistrates.
NICRO did run a a one year NCS pilot funded by the
DOJCD (2007-2008), and based the DFID planning
© NICRO 2013 23
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
Impact
Recommendations
Baseline Study
Evaluation NICRO Management Response and indicators on those results. Despite this, most of the challenges experienced in the DFID project only emerged in the DFID project, because of the (i) length of the project and (ii) the scale of the project.
ACKNOWLEDGEMENTS
This was a participatory evaluation designed by an evaluator but with NICRO and NICRO partners providing significant support to all research processes. Of special note, was a highly supportive Project Manager, Regan Jules-Macquet who commissioned this study, funished project data, supported or undertook many of the logistical arrangements, distributed and ensured the return of surveys, arranged for a focus group with key stakeholders in Cape Town and was most responsive to questions and requests. Deputy Executive Director Celia Dawson and Advocacy Director Venessa Padayachee must be thanked for significant insights into project effectiveness, activities, impact and possible ways forward.
Participatory evaluations take up staff time and even disrupt normal working days but uniformally and without exception all NICRO staff from social auxiliary workers to provincial managers were gracious, supportive and instrumental in ensuring the success of the evaluation process. Certain provincial managers like Claudine Mare (Mpumalanga), Alida
Boshoff (Gauteng), Nthabiseng Molwantwa-Mogotsi (Limpopo) and Marita Van Kraayenberg
(Free State and Northern Cape) were instrumental in enabling the field research, which provided the information and understanding for designing surveys. Supervisors at five sites described the accomplishments and challenges of delivering services, arranged for client interviews and showed the evaluator how the NCS services worked in practice (Annexure 9, illustration 9.1, lists all the supervisors and provincial managers who assisted). In terms of the
Indigenous Justice Pilot Site at Eestehoek, Claudine Mare (who was doubling as supervisor and area manager) drove the evaluator considerable rural distances and was instrumental in explaining the impact of the pilot.
© NICRO 2013 24
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
Magistrates, prosecutors, and members of various national, provincial and local departments also assisted. Hundreds of clients who are not named in this report to retain their privacy are acknowledged as important contributors who endured problems of transport and distance to ensure that the evaluation process was supported. Meeting a client who has reformed has a certain quality about it that cannot be captured in statistics but is very convincing in terms of impact.
EXECUTIVE SUMMARY
The NICRO Programme to Help Achieve the Creation of a Safe South Africa by Contributing to
Significant Penal Reform consisted of Non-Custodial Sentencing (NCS) services (therapeutic programmes to reform offender behaviour) and an Indigenous Justice Diversion Pilot Project at
Eerstehoek to see how NICRO diversion services could be adapted to the traditional courts of the country. It was predicted that both these offerings would contribute to significant penal reform by providing the means to address the contributing factors producing the behavioural problem and thereby prevent reoffending.
A critical sign of programmatic success was demonstrating that the risk factors for reoffending were reduced by providing NCS services. Of 3,960 offenders on programmes by 31 July 2012
(the cut off date for the evaluation), a sample of 936 (24%) clients took diagnostic tests on individual risk factors, which were then compared to SW diagnostic of the same clients using case file data (this tested the veracity of client self-assessments). The self diagnostic scores and the SW scores were very close —a 2-3% difference overall--such that it can be stated conclusively that the risk factors for clients were reduced (see Chapter 3 or Annexures 5 and
6). This was corroborated by other data including a check on a smaller sample of client to see if they reoffended. Surveys of magistrates, social workers, supervisors and provincial managers also identified impacts (along with some challenges). Site inspections included interviews and focus groups with a broad range of informants including client at five sites.
Interviews in Pretoria with directors of key national departments and a focus group in Cape
Town with key stakeholders and experts in criminal justice issues (see Annexures 7-10; 13; 15-
25) provided data on sustainability and replicability.
© NICRO 2013 25
© NICRO 2013
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
The project increased accountability and responsiveness mainly through Local Stakeholder
Committees working together at 37 of 45 sites (the situation at the cut-off date for evaluation data
—31 July 2012) to reform perpetrators by diagnosing and then addressing through therapy the causal factors producing the criminality. It is neither accountable nor responsive to lock people up with no analysis of what led to the crime as that contributes to repeat offending.
Accountability and responsiveness increased in another measurable way owing to community workshops which used before-and-after polling to identify a considerable shift in thinking at 44 targeted sites from retributive justice or punishing the offender toward restorative justice, which seeks to repair the harm caused by crime (see Annexures 28 and 29).
The Indigenous Justice Pilot Project did not work as designed (the referrals from the traditional courts were too low and it was too costly and time consuming per referral, partly owing to rural distances. There was also controversy and national debate during implementation over the
Traditional Courts Bill which seemingly ‘legitimised’ authoritarianism in such courts. This did not focus on NICRO but promoted reflection (see Annexure 15). If traditional courts were still in the process of finding legitimacy in terms of state laws, then the state should address this disjuncture rather than NICRO. Most importantly, it was learnt that by including traditional leaders on the Local Stakeholder Committees for the NCS services and engaging pro-actively in rural schools and communities to prevent crime provided a model that reached more people efficiently and effectively. Piloting tested a model, the model failed, but a better model was generated.
The overall achievement rating was a 2 (see Chapter 3, subsections 2.3.1-2.3.7; 2.4 or the log frames in Annexure One). The evaluator ’s ratings in terms DFID log frames were corroborated by: (a) supervisor surveys (see Annexure 8, Section 8.2, question one); and (b)
33% weightings placed on budget and expenditure data; quantitative data; and qualitative data such that the ‘2’ rating objectives largely achieved with a few shortcomings was well evidenced.
In the final analysis, most shortcomings relate to the lack of an interdepartmental , interagency strategy from national level that linked all stakeholders into the provision of services.
26
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
Cooperation could be found at all levels from local to provincial to national but this did not generate what is needed: a multidisplinary, interdepartmental team from national to local level working according to a single, integrated and measurable plan to address major challenges such as:
ï‚·
Addressing the material circumstance of an offender (e.g., NICRO alone cannot do this effectively)
ï‚·
Establishing a system for measuring the rate of recidivism accurately (NICRO would need to work with DCS but a partnership for this never consolidated)
ï‚·
Monitoring offenders effectively from the time on programmes through to a year or more after completion (cooperation is needed with DCS, SAPS, and possibly community-based organisations like the community police fora) 4
ï‚·
Addressing the shortage of SWs in the country so that many sites can be services (the DSD, DCS and civil society organisations compete for SWs and the demand for them is such that NICRO experience high turnover)
ï‚·
Generating agreement or understanding around the time period for an assessment since ineffective communication around this problem prevented magistrates who would like to refer from referring
The inconsistent monitoring of offenders inclusive of aftercare and tracking and the SW shortages for delivering the service adequately are two serious shortcomings in terms of providing a consistent and safe service. SW turnover disrupted services at more than half the sites according to 53% of surveyed supervisors. The SW becomes familiar with site-based processes, clients and partners and then departs with negative impacts (e.g., intake might stop while employing and training the new SW and this turnover can cause magistrates and LSC members to lose trust in the model —see Annexure 8.1 section 8.24 and 8.2.5). The monitoring issue did not cause a major crisis but holds that potential such that one senior magistrate with significant NCS experience characterised the components of monitoring,
4 The inadequate monitoring of offenders explains why NCS failed in the 1990s (long before the NICRO offered the NCS services).
© NICRO 2013 27
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013 aftercare and tracking as the ‘Achilles heel’ of the intervention (see Section 2.3.3 of Chapter 3 or Annexure 25).
Most challenges are beyond NICRO alone and find their solutions in formal partnership building (contractual agreements) from a national to local level, a task that cannot be avoided if
NCS services are to become a truly national offering that one might find in any court. Many national departments and stakeholders said they hold such a vision and wanted NICRO to approach them formally (see Annexures 17-25) but felt NICRO had not (NICRO thought it had). It appeared to be the method of approach that matters. At present, the services were found sustainable only in terms of a NICRO offering at a reduced but significant number of sites in the years ahead with no integrated national plan for replicating this offering.
The impact evaluation showed that NCS is much more cost effective than incarceration.
Cost Per Offender For The GTF 112 Project As Of March 2012 5
DFID Budget
Cost of project in Pounds: £4 916 108
Cost per Offender
£1 355
Cost of project in ZAR: R 58 993 298 6
© NICRO 2013
R 16 260
The value for money calculations below are based on the following data:
1. Cost of R243 per inmate per day (from the Judicial Inspectorate of Prisons)
2. Incarceration costs are calculated by converting the sentence lengths to prison sentences.
3. Cost savings are calculated by subtracting the cost of incarceration from the cost of the
NCS project.
Raw
No. MINIMUM MAXIMUM MINIMUM MAXIMUM
Less than 6 months sentence 598 R 4 359 420.00 £1 816 425.00
6 months
7-12 months
13-18 months
19-24 months
432 R 18 895 680.00
420 R 21 432 600.00
170 R 16 110 900.00
99 R 13 712 490.00
R 21 797 100.00 £363 285.00
R 0.00 £1 574 640.00
R 36 741 600.00 £1 786 050.00
R 22 307 400.00 £1 342 575.00
R 17 321 040.00 £1 142 707.50
NA
£3 061 800.00
£1 858 950.00
£1 443 420.00
5 The calculations by the NICRO project manager were found in an addendum to the GTF 112
Annual Report 2011-2012 to DFID
6 Using a R12:£1.00 exchange rate
28
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
25-30 months
31-36 months
37-48 months
49-60 months
More than 5 years
No sentence details available yet
Cost Savings
Raw
No.
36
MINIMUM
R 6 561 000.00
MAXIMUM
R 7 873 200.00
MINIMUM
£546 750.00
MAXIMUM
£656 100.00
653 R 147 571 470.00 R 171 373 320.00 £12 297 622.50 £14 281 110.00
440 R 118 681 200.00 R 153 964 800.00 £9 890 100.00 £12 830 400.00
364 R 130 024 440.00 R 159 213 600.00
£10 835 370.00 £13 267 800.00
51 R 26 768 880.00 R 111 537 000.00 £2 230 740.00 £9 294 750.00
697 NA NA
3960 R 504 118 080.00 R 702 129 060.00 £42 009 840.00 £58 510 755.00
R 445 124 781.38 R 643 135 761.38 £37 093 731.78 £53 594 646.78
INTRODUCTION TO THE NICRO PROGRAMME TO HELP ACHIEVE THE CREATION OF A
SAFE SOUTH AFRICA BY CONTRIBUTING TO SIGNIFANT PENAL REFORM
The above named programme concerns both the NCS services and the Indigenous Justice
Pilot Project: two separate activities aimed at contributing to a reduction in reoffending based on cognitive behavioural therapy or diagnosing the cognitive thinking patterns behind the offenses and then offering the offender appropriate therapeutic interventions. The difference between the activities is that the NCS services were sentences upon conviction at 45 magistrate courts around the country whereas the pilot was diversion from traditional courts at one site, Eerstehoek, a rural traditional area in the province of Mpumalanga. Each utilised the same types of therapeutic programme offerings although at Eerstehoek there was an effort to tailor the content of the offerings to indigenous people.
These two efforts will be introduced under the follow sub-headings:
ï‚·
The theory behind NCS Therapy
ï‚·
A brief background to the NICRO
’S provision of the services
ï‚·
A description of the Indigenous Justice Pilot Project (and the theory behind it)
ï‚·
The basic components of delivery under review
ï‚·
Chapter Outline (an explanation of the DFID requirements for presentation)
© NICRO 2013 29
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
THE THEORY BEHIND NCS SERVICES
United Nations Resolution 45/110 of 14 December 1990 (the standard minimum rules for noncustodial sentencing also known as the ‘Tokyo Rules’) calls upon member states to model ways to address the underlying thinking patterns producing criminal behaviour and report on its implementation. This resolution also calls upon non-government organisations to be involved with modelling such programmes.
7 NICRO responded by pioneering the attachment of therapeutic interventions to NCS or sentences served outside of prison (sentences that are postponed, suspended or under correctional supervision) in an effort to reduce the chances of reoffending.
Non-custodial sentencing has long existed in South Africa but before NICRO it was NCS without any therapy-- suspended sentences, fines or community service (labour) under various sections of the Criminal Procedure Act (e.g., Sections 2761h and 2761i). With no therapy attached and no measure of change and very little monitoring of offenders, magistrates lost faith in this older NCS system. NICRO
’s NCS services made use of the same sentences under the same Criminal Procedure Act but offered individually tailored cognitive behavioural therapy to affect change along with pre-tests and post-tests to monitor and identify progress.
This made NCS more appealing to magistrates as a sentencing option although some struggled with certain aspects such as delaying court for an assessment as will be explained in this study.
NCS services provide an alternative to many problems associated with prison wherein: (a) the risk factors are undiagnosed and unaddressed; and (b) perpetrators of non-violent and minor crimes are regularly exposed to the influence of hardened criminals. Those are two factors among many but probably best explain why the reoffending rate is above 50% and perhaps as high as 90% in South Africa —the factors producing criminal behaviour are not addressed but reinforced and even deepened. The excessive use of incarceration has long been discussed internationally as a factor in reoffending and this is why in 1990 the United Nations called upon member states to consider prison as the last and final resort in justice while NCS services
7 United Nations (1990) United Nations Standard Minimum Rules for Non-Custodial Measures, http://www.un.org/documents/ga/res/45/a45r110.htm
© NICRO 2013 30
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013 should be considered first.
8 Cognitive Behavioural Therapy is not just an idea to model but it has been found effective in addressing problems associated with criminal behaviour amongst juveniles, adults, substance abusers, violent offenders, probationers and parolees by improving social skills, moral reasoning, critical thinking, impulse management, self-control, means-end problem solving and self-efficacy.
9
The use of NCS services is also fairly low risk since most magistrates will only sentence suitable offenders if their case files suggest they are low risk and will likely benefit from appropriate therapeutic interventions. In other words, the process starts with a suitability assessment by a professional social worker to see if their thinking and behaviour can respond to therapy. The suitability assessment then goes to the magistrate, who can sentence to
NICRO ’s NCS services where attendance is mandatory such that non-attendance can result in a prison sentence. Even if an offender is eligible for the programme according to NICRO ’s rigorous assessment process, magistrates do not accept every recommendation (72% was the acceptance rate to NICRO
’s NCS services). In other words, many restraints are applied before an offender is actually sentenced to NCS services. Once sentenced, a NICRO SW develops a programmatic intervention tailored to the specific criminogenic needs of the offender. This might be individual counselling combined with some therapeutic programmes (e.g., anger management).
Reforming criminal behaviour via NCS services such as NICRO has modelled offers other significant benefits for the criminal justice system and society by:
ï‚·
Preventing crime (in terms of each offender whose risk profile for reoffending is reduced —see Chapter 3, Figure 3.2)
ï‚·
Keeping low-risk offenders away from the harmful influences of high risk prisoners such that the criminal tendencies do not worsen this way
ï‚·
Reducing state expenses for incarcerating offenders (see Chapter 3, Section
6.1)
8 United Nations (1990) United Nations Standard Minimum Rules for Non-Custodial Measures, http://www.un.org/documents/ga/res/45/a45r110.htm
9 Clark, Patrick (2012) Preventing Future Crime with Cognitive Behavioural Therapy. National
Institute of Justice, US Department of State, Washington, D.C.
© NICRO 2013 31
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
ï‚·
Engaging departments, communities and local stakeholders at each site in local stakeholder committees; community outreach activities and public information campaigns to promote restorative justice principles and practices
(see Chapter 3, Sections 2.3.1, 2.3.5 and 2.3.7)
A BRIEF BACKGROUND TO NICRO
’S PROVISION OF THE SERVICES
The history of this project might be dated from 2006 when NICRO collaborated with the
Department of Justice and Constitutional Development (DoJCD) to host a national colloquium on alternative sentencing options (other than prison sentences). The colloquium made it clear that strong stakeholder support was available for such initiatives and this led to NICRO establishing five initial sites for NCS with one year of funding from the DoJCD. Owing to the experience at these sites, NICRO then applied to DFID in late 2007 to design, develop and implement alternative sentencing (AS) options at magistrate courts around South Africa (this included NICRO
’s initial five pilot sites).
10 In April 2008, NICRO received the news that DFID would be granting them nearly
£5 million or some R55 million for the roll out and entrenchment of NCS at 52 sites (the targets were later revised with DFID to 45) over a five year period.
NICRO had planned a smooth transition from 5 pilot sites to establishing some 8-10 sites yearly from the first tranche in early 2008 but the first tranche of funding was not released until
October 2008. The Year One and Year Two sites had to start together in 2009, making that a much pressurised year for the organisation. NICRO had been efficient during 2008, using the time to design therapeutic programmes including manuals, materials and training programmes that could be utilised for NCS clients and ran workshops for the regional and area managers who would be implementing the programme. However, shorn of resources and equipment allocations, the new sites could not open until January 2009. The 2010 Mid-Year Review
(undertaken by the same evaluator) found that processes had been rushed whereas the 2012 evaluation found NICRO delivering services in 8 provinces and 45 sites across the country
(see FIGURE 1.1 below) with some notable successes (and some challenges) as will be described in this study.
10 The funding for this constituted 5% of the £100 million Governance and Transparency Fund
(GTF) set aside by DFID to help citizens worldwide hold their governments accountable for delivering services in a transparent manner.
© NICRO 2013 32
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
Figure 1.1: 45 Sites delivering NCS Services Nationwide
1
5
1
3
1
4
GAUTENG
↓
1
Site
Soweto
2 Attridgeville & Hatfield
3 Pretoria
Odi
Year/replacements
1
4 (replaced Springbok)
1
2 4
5 Vaal 3
1
0
1
1
6 Germiston (Ekurhuleni)
7 Randburg
Tembisa 8
9 Daveyton
Katlehong
Benoni
Johannesburg
3
3
4
4 (replaced Rustenberg)
4 (replaced Krugersdorp)
4
4 (Replaced Roodepoort)
1
2
Pretoria North Moved in year 4 to replace
BRITS
Soshanguve
Boksburg /Alberton
Meadowlands
4
5
Replaced George
4 (Replaced Uppington)
Mamelodi Moved Year 3
Replaced Mafikeng
LIMPOPO
Site
Mankweng
Year
1
Sekgosese
Thohoyandou
Moved Year 3
Replaced Lebowakgomo
4
KWAZULU-NATAL
↓
Site
Pietermaritzburg
Year/replacements
2
Durban
Chatsworth
Pinetown
Site
Queenstown
Gelvandale
Port Elizabeth
East London
Mdantsane
Zwelitsha
Site
Wynberg
Kuils River
Khayelitsha
4 (replaced Wentworth)
EASTERN CAPE
WESTERN CAPE
Cape Town
Mitchell’s Plein
Simons Town
1
5
Year
1
1
2
2
3
3 (Replaced Scottburgh)
Year
1
3 (Replaced Bellville)
3
3
5, (replaced Oudtshoorn)
3
© NICRO 2013 33
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
MPUMALANGA
Site
Nelspruit
Kabokweni
Eerstehoek (both an NCS site and the indigenous justice pilot site)
Year
2
1
2
Site
Kimberley
Site
NORTHERN CAPE
FREE STATE
Year
2
Year
White river 4 Bloemfontein (Batho court)
Welkom
1
Barberton 4 (Replaced Middleburg) 1
Kanyamazane 4 (Replaced Witbank)
NICRO encouraged magistrates to refer certain kinds of offenders to a NICRO SW for an assessment and modelled local stakeholder committees (LSCs) showing that offenders could be monitored appropriately and that various departments can cooperate in promoting NCS.
Successfully modelling the programme, effectively implementing it in 45 sites around the country and seeing a reduction in criminogenic thinking and behaviour describes the NICRO achievement in the main but as important was the effort to change both the thinking and practices of government leaders, magistrates and the public. In other words, this was also an advocacy effort and the public was exposed to media reports and information about how NCS works to avoid any misperceptions that NCS ‘soft’ on crime (NCS services were aimed at both rehabilitation and crime reduction). This is a big effort in a nation of 50 million people wherein shifting national thinking about justice (from punitive to restorative) was a key aim. NICRO embarked on this knowing that it had five years of funding within a very long term effort of perhaps the 10-20 years required to institutionalise such services, which also makes sustainability and replicability key issues for analysis.
A BRIEF DESCRIPTION OF THE INDIGENOUS JUSTICE PILOT PROJECT
This document also reviews the Indigenous Justice Pilot Project which sought to locate an applicable restorative justice model for diversion in which the concepts of justice belonging to traditional societies were incorporated into NICRO offerings. The hypotheses that: (a) restorative justice was embedded in traditional societies--that the two are synonymous; and
(b) that traditional leaders in Eerstehoek could refer to NICRO services directly from traditional
© NICRO 2013 34
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013 courts in much the same way as a regular court were both nullified by the data. NICRO could certainly draw upon certain indigenous concepts of justice and reconciliation for its programme offerings (e.g., storytelling) in indigenous areas but the targeted traditional courts themselves did not turn out to be models of indigenous justice and were not particularly effective for referrals either. All the reasons why the referral system failed need not be itemised here since this was well covered in the Chapter 2 findings (see Sections 2.2, 2.3.6, 2.5.4 – 2.5.6 and Annexure 16) but in brief traditional courts were not as traditional’ as anticipated. Unlike regular courts, traditional African justice is not codified and is subject to inappropriate fining and corruption. This was so embedded that NICRO had no hope of changing it. The enormity of the task is illustrated by the huge national debate that arose in the press during the intervention with regard to the Traditional Courts Bill. Bad press was not the reason the pilot failed but in the words of the Deputy Director Celia Dawson-- “it became a political football”
(see Annexure 15, question 7) and this just illustrates the enormity of the task.
NICRO responded appropriately to the challenges at Eerstehoek by increasing its school and community outreach work, which appeared much more effective than the original model when measured in terms of numbers on programmes (see Chapter 3, Sections 2.2 and 5.2-5.4).
There is also testimony from magistrates; the DSD and the local LSC that NICRO even helped to lower crime rates in schools (see Chapter 3, Section 2.3.6). Community outreach also allows for the kind of broad, economic and political impact that was desired by both NICRO and the GTF fund (reaching out to the broad spectrum of disempowered and the impoverished members of indigenous society). This was a pilot with findings that clearly established for
NICRO the kind of strategy that can work in traditional communities (see Annexure 15, question 7 for instance). NICRO sometimes spoke of this as a ‘failure’ but from an evaluator’s perspective, a nullified hypothesis is as good as a confirmed one and NICRO now knows how to work in such communities (outreach combined with traditional leaders represented on the local steering committees) and can take such a programme forward into new indigenous sites based on the learning.
© NICRO 2013 35
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
THE CRITICAL DELIVERY COMPONENTS UNDER EVALUATION
An evaluation needs to be based on the objectives of delivering a programme or services. The objectives allow an evaluator to measure whether or not they were attained and how well or completely. The specific objectives that can be analysed include:
1. Developing and promoting successful NCS services at 45 magistrate courts countrywide.
2. 60% non-re-offending rate amongst sentenced offenders 80% non-re-offending rate amongst indigenous justice pilot offenders
3. Positive perception of the NCS services amongst targeted communities
4. 70% of recommendations for sentencing accepted by magistrates in 45 courts by end of funding period
5. A total of 6000 sentenced offenders sentenced to non-custodial sentencing options over a five year period 11
6. One trained SW at each of 45 courts by the 30 June 2012 (the cut-off date for analysing data)
7. 300 offenders in the indigenous justice pilot over four years 12
While the bulleted items can be statistically measured using project documentation, NICRO was also seeking to understand and document:
1. What were the positive and negative impacts of the NCS Services?
11 This was revised from the MTR period target for NCS of 14,600
12 This was revised downward since the MTR period (in consultation with DFID) from 1,100 diverted offenders to 300
© NICRO 2013 36
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
2. Did the Indigenous Justice Pilot Project Develop an impactful and replicable model for indigenous areas?
3. What lessons emerged from delivering these services?
4. What was achieved in a broad context?
5. Can information from this project be used to improve programme design and management of other projects of this kind?
6. Were the funds used effectively and efficiently to deliver results?
7. Did this project increase government accountability and responsiveness?
CHAPTER OUTLINE
The order of the chapters and the basic content is according to DFID standard requirements for all recipients of the Global Transparency Funds. Figure 1.2 (below) illustrates the content by chapter and basic contents within each:
CHAPTER TITLE CONTENTS
0 Title pages and executive summary
 Cover note (from NICRO)
 Title Page including Programme identification
Details

Table of Contents
 Abbreviations/acronyms page
 Executive Summary (maximum 3 pages)
1 A introduction short the programme to
 The theory behind NCS Therapy
 A brief background to the NICRO ’S provision of the services

A description of the indigenous Justice Pilot Project
(and the theory behind it)
© NICRO 2013 37
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
CHAPTER
2
3
4
5
6
TITLE
The evaluation methodology
Findings relation in to standard review criteria
Innovation and lessons learnt
Summary of recommendations
Annexures
CONTENTS
 The basic components of delivery under review
 Chapter Outline (an explanation of the DFID requirements for presentation)
 Overview of the evaluation methodology

Explanation of methodology
 Challenges to the research
 The timetable for evaluation activities

Relevance to GTF
 Impact
 Efficiency (measuring productivity)

Effectiveness (in terms of meeting objectives)
 Value for money
 Sustainably
 Replicability
 How NCS Services promoted innovation in South Africa
 Lessons learnt in how to offer such services
 Recommendations in terms of delivering and sustaining the service
 This is the data that informs much of the analysis.
The annexures are referenced throughout the 5 main chapters. Some are annotated and make
© NICRO 2013 38
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
CHAPTER TITLE good reading.
CONTENTS
 Annexures 1-5 include the achievement rating scales (these are also used in the Chapter 3 findings); terms of reference, the research time table, list of people met, documents consulted and then the detailed data from research processes.

The annexures include a table of contents.
The standard review criteria furnished by DFID situates the detailed findings of this evaluation in subsection 2.3 of Chapter 3 (‘Findings’). In that chapter, there is a summary in broad terms of the contribution that NICRO made to the overall Governance and Transparency Framework.
Following that, each project objective is discussed and measured in terms of an achievement rating scale (a table that assigns a numeric achievement rating to each programme objective).
The findings are also illustrated in tables although most detailed statistical data enters into the annexures. Some of the achievement rating scales (those in terms of objectives) are in the main body of the report and others are referenced to the annexures. It was found that the log frames provide a way to clearly and succinctly express findings in a way that links them to the evidence but on occasion the log frame indicator can be questioned as it might not be the best measure of impact, in which case some data is provided to explain why.
© NICRO 2013 39
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
EVALUATION METHODOLOGY
The evaluation focused on the achievements, challenges and impacts (up to 31 July 2012 13 ) of the ‘NICRO Programme to Help Achieve the Creation of a Safe South Africa by Contributing to
Significant Pen al Reform’. This refers to two activities funded by DFID and under review:
(a) The NCS services
(b) The Indigenous Justice Pilot Project at Eerstehoek
By 30 July 2012, NCS services were fully instituted at 37 of 45 courts in South Africa whereby magistrates were referring perpetrators to NICRO for pre-sentencing assessments to see if they could benefit from a NICRO intervention. If the perpetrator is found to be suitable and the magistrate agrees to a non-custodial sentence, he or she may be sentenced to NICRO for therapy. Eerstehoek was one site amongst the 45 developed for the delivery of NCS services but was also the site for piloting the Indigenous Justice Diversion Project, a system in which traditional courts (administered by traditional leaders) could refer perpetrators direct to NICRO.
Would such a model work in traditional areas? If so, what adaptations are required for servicing traditional court systems? If not suitable, how might such areas might be best served by NICRO?
The methodology applied to this study will be discussed under these subsections:
A. Overview of the evaluation methodology
B. Explanation of methodology
C. Challenges to the research
D. The timetable for evaluation activities
13 This was the cut-off date for returning data from the sites for analysis. All research tools
(schedules, surveys, site-based data) were designed for comparative analysis based upon project data returning up to 31 July 2012 but much returned in late September and early
October. This was a participatory methodology involving the whole organisation but restructuring within the organisation occurred during the evaluation such that the timelines were not met (most data returned about 60 days later than anticipated; some even later).
© NICRO 2013 40
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
INTRODUCTION TO THE EVALUATION METHODOLOGY
The methodology was broadly characterised by:
ï‚·
One evaluator engaged in all research processes to ensure an integrated analysis of the data
ï‚·
The development of indicator tables to ensure that all the specific objectives of the services given in the terms of reference were incorporated into the design of research instruments
ï‚·
The incremental design of research instruments whereby the findings from one method were integrated into the next method (to progressively deepen analysis and ensure that statements satisfying the objectives were well supported with sound evidence)
ï‚·
Multiple lines of evidence that could be triangulated (surveys, questionnaires, focus groups, site inspections, project documentation could be inspected to see if the data corroborated)
ï‚·
A survey of 936 clients to identify the whether or not 20 key risk factors for reoffending were reduced in and then surveys of SWs who used case file data to offer their own assessment on same client according to the very same factors
(the two surveys were then compared as further explained below)
ï‚·
Site inspections with attendant interviews and research processes at Tembisa,
Soshanguve, Sekgosese. Welkom, Pretoria, and Eerstehoek involving NICRO social workers, clients, supervisors, area managers and partners
ï‚·
Participatory research (the evaluator openly shared learning as it developed and
NICRO supported research processes)
On the last point, participatory methodologies increase the flow and feedback of information to generate good data at an affordable cost. For instance, indicators and methods were workshopped with NICRO both at national and during interviews at the sites (particularly with supervisors and managers) which preceded and influenced the design of the survey questions.
The evaluator’s questions, early findings and observations, are openly shared during
© NICRO 2013 41
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013 participatory processes, in order to build organisation capacity and stimulate thinking about practice so that changes can take place even while the evaluation is underway. Participatory methodologies are cost-saving because the organisation uses available resources to assist with research activities (facilities, vehicles and personnel). For instance, the evaluator did not rent a car or employ translators but partnered with NICRO, which provided these services from its own resources (this is also part of the cost effectiveness of the project).
The most critical component of the evaluation was identifying whether or not NICRO reduced the offender’s risk for repeating crime through cognitive and behavioural therapy. The SWs completed risk assessments of clients prior to the provision of the service, designed a programme for rehabilitation according to client risk factors and afterwards identified if there had been change. The evaluation research was modelled on this and made use of NICRO case file data (without invading client privacy) to identify whether or not the therapeutic interventions reduced the risk factors (criminogenic thinking patterns and some environmental factors) that led to the criminal behaviour. The methodology involved the design of a two-part investigation of impact: (a) client self-assessments--asking the clients directly about a series of individual risk factors; and (b) SW assessments of client risk according to the same questions asked of the client but with reference to confidential project data (that the evaluator could not access). In other words, the SW assessment of the same client based by inspecting each case file was used to determine the validity of the client self-assessment. The focus on risk factors trends toward honest and frank self-evaluations since there is no benefit, reward or threat in answering the question one way or another but it is possible that clients could overestimate or underestimate their own progress. The two surveys together helped to achieve an accurate measure of impact.
The described method of focusing on risk factors to measure impact was also designed to mitigate the challenge of analysing impact in terms recidivism data, which is highly problematic in South Africa:
ï‚·
There is no official data in South Africa on recidivism rates to enable a comparison to the NICRO programme (it is unreliable and really amounts to approximations--figures like 50-90% are used out of hand but these are not based on scientific studies)
© NICRO 2013 42
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
ï‚·
The Department of Correctional Services is still working on a system to capture recidivism (and advised the evaluator during the Pretoria visit that any statement on recidivism rates would be challenged)
ï‚·
NICRO clients are screened and thus one could argue that they might axiomatically be at lower risk of recidivism owing to this (thus, random sampling of those not on the programme does not compare to random samples released from prison) 14
ï‚·
Self-reporting on recidivism rates can be subject to many criticisms (the client surveys did not do this but were specific to individual risk factors)
ï‚·
Measuring recidivism rates would require a control group but none were available (attempts were made by NICRO to gain access to a comparative group of prisoners but these negotiations failed).
ï‚·
NICRO also offers little internal tracking of clients owing to challenges explained in other sections of the report (see subsection 2.3.3 of Chapter 3, which is the logframe rating and description of findings).
So, with regard to measuring recidivism, NICRO will have to follow up on clients for a much longer period and create partnerships for that task
—at minimum working with the Department of Correctional Services to develop randomised control groups. The operative question with this evaluation was “how well did the intervention reduce the risk factors for criminal behaviour, especially in terms of cognitive behavioural factors and the criminogenic thinking patterns that produce criminal behaviour ?’ This allowed for a clear measure of impact without engaging in a debate about recidivism rates.
15 This approach is also better aligned to the theory behind the
NCS services--NICRO services focus on changing offender thinking and behaviour. The tools for measuring this were:
14 A control group might have been utilised or a baseline undertaken but neither was in the original project plan
15 Clear, Todd (November 2010), “Understanding Recidivism and Reoffending,” in Report on the Open Society Foundation for South Africa (OSF-SA) conference on recidivism and reoffending in South Africa,” 29 and 30 November, Sandton Sun Hotel, Johannesburg
© NICRO 2013 43
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
ï‚·
A self-assessment survey of 936 clients according to individual risk factors 16
ï‚·
SW assessments of the same 936 clients according to the same risk factor
(for correlation with the above)
ï‚·
Spot-checking a group of 237 surveys (25%) with SAPs data on reoffending
(
‘SAPS 69 data’ --did any of them actually reoffend?)
ï‚·
Identifying from project participants (local stakeholders committees, magistrates, social workers and partners) whether or not they observed that a reduction in risk factors was being achieved (e.g., clients at risk no longer returning to court; changed behaviour etc.)
In other words, multiple lines of evidence were compared to identify how well the data corroborated (and a high level of corroboration obtained). The surveys on risk factors provide the best measure of impact because each question on the client survey corresponded to a risk and the SW researched each client using the case files to decide if risk was lowered.
17 Thus, agreement between the client and the SW as to whether or not the risk factors were reduced is very good evidence of impact. In terms of outcomes, the 936 client self-assessment surveys
(24% of the 3,960 on programmes) on particular risk factors for reoffending matched the results of 936 SWs referencing cases files. The variance between all the client and SW surveys fell within a small 1-3% range in terms of 3 categories of risk (low, medium, high).
Furthermore 96% of clients were found to be in the range of very low risk to low risk.
Some verification was obtained using the national data system on offenders (the CRIM system) and it is not weighted heavily as this system is not error free (it is based on ID books and criminals being rearrested might use fake IDs) but 203 clients (23% of those surveyed) could be located on the CRIM system to show that between 2008 and 2012, only 9 NICRO clients reoffended (4%). One can say that police records on reoffending were consistent with the evaluation findings that risk factors for reoffending were reduced amongst NICRO clients
16 The original target was 2000 clients surveyed but this target was not achieved as explained further below.
17 The case file on each client is private and the evaluator would not have access to this but the SW would.
© NICRO 2013 44
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013 but no finding or conclusion relies solely on this data. It is just a supplemental indicator because it did not contradict the findings.
Estimates for reoffending amongst those released from SA correctional facilities are estimated in the range of 50%-90% but this data was not used for any direct comparisons either because those on NICRO programmes have a different profile
—perhaps at a much lower risk than the general prison population owing to the assessment process for admission.
OVERVIEW OF THE METHODOLOGY
FIGURE 2.1 (below) provides an overview of the research methodology including the data collection instruments that were utilised. Below this illustration, research activities are further detailed and described under sub-headings.
FIGURE 2.1: OVERVIEW OF THE METHODOLOGY
ACTIVITY DESCRIPTION
1 DEVELOPING
INDICATORS AND
DEFINING MEASURES
All available project materials were collected, reviewed and assessed in order to develop indicator tables to inform the design of data collection instruments. The evaluator then work-shopped the indicators with NICRO to identify additional ones and to identify and refine the suggested measures. This led to additional data requests and generating data collection instruments. A CPA was also contracted for the project expenditure review, satisfying certain indicators.
2 DESIGNING CLIENT
AND SOCIAL
WORKER SURVEYS
The evaluator conducted web-based research into ways to analyse offender risk factors, designed a survey and then circulated it for discussion. The result was original: two identical surveys —one for the client and one for the social worker to verify the client findings.
The forms were piloted in Cape Town and then sent to sites with a cover letter to NICRO social workers and supervisors. NICRO also paid for surveyors when it was found that NICRO personnel did not have the available time to engage with the surveys.
© NICRO 2013 45
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
3
4
ACTIVITY
WORKSHOP ON
INDICATORS AND
SURVEYS
SURVEYING
DESCRIPTION
The indicator tables and initial designs were reviewed with NICRO for feedback
The client and social worker surveys were piloted early in the research process and distributed to sites. This was supplemented by a request (through NICRO) for SAPS 69 forms on reoffending.
5 SITE INSPECTIONS,
WITH INTERVIEWS
AND FOCUS GROUPS
The evaluator visited 7 sites for various research processes but engaged in intensive interviews, focus groups and inspections at 5 of these (Eerstehoek Shoshanguve, Welkom, Tembisa and Sekgosese) where interviews or focus groups were held with SWs, supervisors,
LSC members, magistrates, key local partners and traditional leaders. The other two sites visited were Cape Town (piloting) and
Pretoria to discuss sustainability issues with key national department
(some clients and staff were engaged too).
6 E-MAIL SURVEYS For project supervisors, social workers, magistrates, and regional managers, surveys were designed based on both the original indicator template (which encompasses all research objectives) and informed by the findings from the site inspections.
7 CAPE TOWN
RESEARCH
PROCESSES
Semi-structured interviews at NICRO National and a focus group with
NICRO and partners were designed from a review of findings to date.
Includes: programme designer, steering committee, PM, Project
Owner, Magistrates. A workshop was also held with key NICRO partners based in Cape Town.
8 DATA CAPTURE All data was captured for analysis. Client surveys and verification of client responses had to be captured on spread sheets and the remainder was captured directly into the report as tables to illustrate the report.
9 DATA REVIEW AND
ASSESSMENT
All data sets were organised into tables and diagrams for comparative analysis and to establish levels of corroboration and for weightings so that a succinct and illustrated report could be issued.
10 ANALYSIS AND
DRAFT REPORT
© NICRO 2013
The draft evaluation report was written in accordance with the NICRO and DFID reporting structure to respond to all objectives and to detail
46
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
ACTIVITY
WRITING
11 SUBMISSION OF
DRAFT REPORT
DESCRIPTION the evidence; submitted by mid-February
The original deadline was 15 November but much data was not available until 01 November, delaying submission until December
12 WORK-SHOPPING
DRAFT REPORT TO
FINAL SUBMISSION
Preparation and discussion of a PowerPoint Presentation capturing findings to supplement discussion with NICRO National and reference committee on the report; record critical feedback on findings that enter into the report
13 FINAL SUBMISSION The findings from the above workshop enter into the final submission of the report
14 REPORT
PRESENTATION
Preparation and delivery of a PowerPoint presentation to the public and invited guests
EXPLANATION OF METHODOLOGY
The basic research processes as outlined above are further detailed below.
1. Developing Indicators And Defining Measures
The evaluator took receipt of and reviewed the following documentation to generate a comprehensive table of indicators that can satisfy all project objectives, the TOR for the study and the report writing requirements for DFID. This included:
ï‚·
All service statistics
ï‚·
Project reports
ï‚·
Internal evaluations
ï‚·
NCS booklets, success stories and client testimony
ï‚·
Public attitudes research reports
ï‚·
Monthly steering committee minutes
© NICRO 2013 47
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
ï‚·
Policy and Procedures Manual (this is the ‘PSP’ manual that guides the entire organisation in terms of implementing projects and programmes and furnishes monitoring tools)
ï‚·
Relevant news reports and clippings pertaining to the project
ï‚· Stakeholder’s toolkits
ï‚·
Quality assurance documents
ï‚·
Target attainment statistics
ï‚·
Client materials (e.g., two videos, pamphlets, testimonials)
ï‚·
Project budgets and expenditures for the five years
ï‚·
A sample of completed pre- and post- assessment forms and data on the availability of this documentation
ï‚·
A contact list of key national stakeholders who have participated in the project and may be significant to interview or who might otherwise assist in research processes
ï‚·
Details on NICRO ’s Pretoria-based national partners in the project SAPS,
Justice, DSD and other Departments
This initial indicator template was constructed and analysed to determine: a) What can the available data explain? b) What are the gaps? c) What data collection tools can capture the additional data?
In the course of this analysis the evaluator also:
ï‚·
Contracted an accountant to analyse data on budgets and expenditures (a project expenditure review analysis)
ï‚·
Engaged in a very broad but initial analysis of impact employing service statistics and other data (overview)
ï‚·
Engaged in e-mail, telephonic and personal communications with the PM who clarified questions and helped with terminology in the early design of research tools
© NICRO 2013 48
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
ï‚·
Contracted an individual for data capture
ï‚·
Engaged in some initial data capture and assessment (tables; what will available data be able to explain in terms of project objectives?)
ï‚·
Reviewed the pre- and post-assessment forms to see what has already been captured and what can be captured regarding impact on risk factors
Careful and methodical design work in the early phases was significant for achieving the objectives of the research and the evaluator would return to this same project documentation and especially the indicator tables many times in the course of research. Once all the data was in, a fully integrated analysis in terms of all project objectives and requirements could occur.
Design Of Client And Social Worker Surveys
Research, consultation and piloting were required as the survey had to be in a very easy-tounderstand format owing to a big range in literacy rates amongst offenders (57% had not completed high school) and yet address some significant and telling risk factors. Furthermore, it had to be designed so that it was possible to data capture about 40 surveys an hour.
FIGURE 2.2 below illustrates the design process:
FIGURE 2.2: THE DESIGN AND DELIVERY OF CLIENT SURVEYS, REFINEMENTS AND FOLLOW-
UP
Informant DESCRIPTION TL
C LIENTS A survey based on:
ï‚·
Project objectives
ï‚·
Self-assessed risk of reoffending according to multiple risk factors,
ï‚·
Client perceptions of the project (successes and shortcomings, and recommendations
Literature
Review
Hours
80 hours
Design
Hours
Hours To Review And
Engage In Discussion On
The Design
16 8
Telephonic and inperson, and electronic consultations with experts to review the design
104
© NICRO 2013 49
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
The client surveys were designed for 2000 clients (as per agreement) but this was very ambitious as this was 51% of intake (3960 as of July 2012). Many had not yet completed programmes and the personnel, costs and logistics required for the surveying under the conditions found at the sites were underestimated. Instead there were 936 returns in terms of client and SW surveys on each of those clients.
The method of the SW utilising case file data (assessment materials, pre- and post-test, notes) to complete a survey on client risk (the same questions asked of the client but involving reference materials) would reveal if client self-assessments of risk according to individual factors would corroborate with SW assessments of same client.
SAPS data on reoffending (SAPS 69 forms) was intended to check on 697 clients (74%) but
66% could not be found on the system (the system for monitoring offenders is poor in South
Africa). However, 203 clients (22% of those surveyed) could be found on the system to provide some indication that clients had reformed (2% of these reoffended per year). The downside of this ways of measuring impact (and it is not weighted when it comes to measuring overall impact) is that the numbers that completed programmes were not many and large percentages were still on programmes. In terms of a timeline for measurement, the starting event would be described as entering therapy and recidivism would be described as an arrest.
The longer the time since the intervention was completed without returning to crime
(demarcated by arrest records), would have more weight as a finding but locating clients who had completed the programme proved to be too time consuming and costly for the organisation such that:
ï‚·
43.5% of clients surveyed were still on programmes
ï‚·
No clients were surveyed (although some exist) who had been in aftercare more than a year (this owes to the challenge of tracking clients--their details change--as discussed in various section of the report)
ï‚·
Only 13% (126) of those surveyed were in a range of 7-12 months post therapy.
For these reasons, it was useful that measurements of impact were not based on recidivism rates or reoffending as this was too short of a time to measure any decisive impact in those terms. Identifying that risk factors were reduced in course of these programmes was a much
© NICRO 2013 50
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013 better method but not perfect and it would be ideal to measure this again after clients had been off the programme for say two years.
It can be said that all indications are that the risk factors for reoffending were established to have been reduced owing to a combination of evidence of which the first two carry significant weight while four and five are also included in the weightings (see Section 2.4 on weightings and note that the recidivism data is not utilised although it has certain limited value):
1) The client survey data verified by social workers with reference to project documentation on each client 18
2) Routine pre- and post-assessments by social workers, which entered into the verification process above
3) Low recidivism figures for a random sample of 203 clients (the value is that it does not contradict the findings)
4) Large numbers and percentages of partners, participants and observers of the project perceiving lower recidivism figures amongst NICRO clients (based on observations in the court such as magistrates not seeing these clients return to court and attributing this to programme activities)
5) Interviews with a sample of 37 clients at sites.
Workshop On Indicator Tables, Site Selection And Surveys
After the project materials had been researched, reviewed, and indicators developed; then measures were proposed and two surveys drafted for discussion. A workshop was then held in which evaluator met with NICRO to review the evaluation matrix and discuss the planned surveys. The PM was particularly helpful with refining the language of the instruments so as to fit with organisational culture.
18 The SW used court reports, programme pre-tests and post-test, process notes, anecdotal data from clients, aftercare and tracking reports and group work reports
—these are confidential documents but the SW referred to these to rate client progress on the survey (this could then be checked against the client self-assessment)
© NICRO 2013 51
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
Surveying Clients And Social Workers
The evaluator designed surveys for clients (a 24% sample obtained--936 of 3,950 clients) on some 20 risk criminogenic factors (thinking factors in Section B and situational factors in
Section C). In addition, a survey was designed for SWs with instructions to verify from case files each of 20 client responses (the SW survey and client survey had an identical set of 20 questions). In practice, th e SW would independently consult the individual’s case file and on that basis make a professional judgement based on some 20 risk factors as to whether or not the intervention had made impact on lowering risk. The SWs could refer to this kind of data in the case files:
ï‚·
Process notes
ï‚·
Progress Reports
ï‚·
Forms that document client progress on programmes
ï‚·
Pre and post tests
If there was insufficient data in the case file to make a judgment, the SW would write on the form ‘insufficient data available’ and this did occur on about 7% of the surveys, especially in
Section C concerning the life situation of the client post-NCS.
19
It was found in analysis of the data and during site interviews that it is better to collapse the categories of agree and strongly agree as ‘agree’ (the tables of this report illustrate this in both ways) since most clients and many SWs reported in site interviews that they had difficulty in distinguishing the two (otherwise a difference might be noted that owes to language or understanding of the question rather than a finding). Site interviews took place after surveying had started (a necessity owing to time-lines), otherwise the survey might have been changed after site visits but this is not at all problematic in terms of describing the outcomes. In fact, the additional choice on the survey might have assisted some SWs and clients with their replies
19 The main reason there were gaps on the life situation of the client mainly owed to a lack of aftercare and tracking. Occasionally SW turnover at sites could also produce a situation where one SW had difficulty understanding the process notes of another. That SWs indicated insufficient data was in the case file was actually helpful because it demonstrated that the responses were based on consulting the case files (as designed).
© NICRO 2013 52
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013 and confused others and then the collapsing ‘agree and strongly agree’ and ‘disagree and strongly disagree’ for measurement and analysis helped to make more accurate statements.
Site Inspections, Interviews And Focus Groups
The ToR required that the evaluator visit five project sites. One of these had to be Eerstehoek, the indigenous justice pilot site. The remaining four were randomly sampled from 33 out of 45 sites (12 were not suitable for inspection —newly established sites). The Project Manager then used a random sampling programme to select the following:
ï‚·
Welkom (Free State)
ï‚·
Tembisa (Gauteng)
ï‚·
Soshanguve (Gauteng)
ï‚·
Sekgosese (Limpopo)
Two of these were poorly performing sites and this helped to create some balance in terms of the data being captured since only the top performing sites would have the numbers to justify expenditures on surveying.
In addition, informal client and social worker interviews took place in Cape Town in the process of piloting the client and SW surveys (the first site visited). Similarly, the evaluator interviewed two clients in Pretoria since that office was the base of operations for visiting Tembisa,
Shoshanguve and for meeting with both key government departments (DCS, NPA, Judiciary,
DoJCD and SAPS) and key informants (i.e., an expert in restorative justice outside of NICRO).
Thus, one can say the evaluator made site visits for research activities at 7 of the 45 sites
(16%) and thoroughly inspected 5 of them (11%). These processes are further described in
FIGURE 2.3 (below) and FIGURE 2.4 (further below) shows that 142 informants participated in site-based research processes (a good-sized sample for qualitative research processes at sites).
© NICRO 2013 53
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
FIGURE 2.3: THE SEVEN INSPECTED SITES AND THE FIVE SITES FOR INTENSIVE
RESEARCH PROCESSES
1 (CAPE TOWN)
2
3
4
5
6
SITE
EERSTEHOEK
WELKOM
SEKGOSESE
TEMBISA
SOSHANGUVE
7 (PRETORIA)
CATEGORY PROVINCE
URBAN
RURAL
INDIGENOU
S
URBAN
RURAL
PERI-
URBAN
PERI-
URBAN
URBAN
WESTERN
CAPE
MPUMALANG
A
LOCATION
Regional NICRO office in Cape Town
85 km from Nelspruit;
300 km from Pretoria
VALUES
The pilot site for testing research tools
A first year site which offers some value for qualitative analysis owing to longevity on the programme (see data in the MTR)
This is the only indigenous Justice Pilot
Project site and had to be included in the study
FREE STATE
LIMPOPO
GAUTENG
GAUTENG
GAUTENG
164 km to Bloemfontein
337 km to Pretoria
Remote/rural —one hour drive from Polokwane
Mid-Rand just north of
Kempton airport
Park and
ONE OF THE FIRST
YEAR SITES
Randomly selected for inspection
432,000 people
At the Sekgosese
Magistrates Court
(secure, maintained) well-
Served by Polokwane
NICRO Office 3 days a week
Randomly selected
512,000 residents
Tembisa is a Zulu name meaning ‘Give Hope’’
Randomly selected
25 km north of Pretoria 600,000 RESIDENTS
Randomly selected
Capital Base
Tembisa for
Soshanguve visiting and and national departments
2.5 million people
© NICRO 2013 54
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
FIGURE 2.4: RESEARCH PROCESSES DURING SITE INSPECTIONS
PROCESS
1 Site-based observatio ns
MINIMUM
TARGET
TARGETS
ACHIEVED
5
(1 per site)
EXPLANATION
7
(140%)
In addition to the 5 targets sites, the evaluator inspected Cape
Town and Pretoria sites although research processes were more intense at Eerstehoek, Shoshanguve, Tembisa, Welkom and
Sekgosese
2 Client interviews
25
(5 per site)
34
(136%)
Formal interviews with a mix of 33 diverted and sentenced offenders — 4 in Cape Town (pilot), 7 in Sekgosese; 6 AT Welkom;
5 each at Eerstehoek, Shoshanguve and Tembisa, 2 in Pretoria,
3 Interviews with SWs and
NICRO staff
10
(2 per site)
12
(120%)
Formal interviews with one NICRO SWs per site (minimum) and additional NICRO staff members were interviewed as available; this also helped to pilot the national survey forms
4 Project
Supervisor interviews
5
(1 per site)
5 Focus
Group:
LSC members
25
(5 per site)
5
(100%)
One interview per site to inform the design of 18 supervisor surveys and to generate some data on particular sites including Eerstehoek
41
(164%)
LSC members were convened at sites for focus groups and some were met individually to meet targets per site
11
(220%)
The evaluator aimed to interview at least one magistrate per site (in interview or via focus groups). Some of the interviewees were
Pretoria based magistrates who provided a national perspective.
6 Magistrate s
5
(1 per site)
7 Key local partner interview
8 Traditional leaders interviews
5
(1 per site)
08
12 9 Pretoriabased interviews
© NICRO 2013
6
(120%)
This referred to any opportunity that arose during inspections at the five targeted sites to interview additional project partners —these included prosecutors, NGOs, and probation officers, and Legal Aid
08
(100%)
This occurred mainly at Eerstehoek: 3 at Mpisikazi and 3 at
Embuleni; 1 at NICRO Eerstehoek Office) but one traditional leader was also interviewed at Tembisa
18
(150%)
Pretoria-based interviews and focus groups with DCS, DSD,
Judiciary, DoJCD, SAPS and NPA and the Restorative Justice
Centre provided insight regarding sustainability issues —who would
55
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
PROCESS MINIMUM
TARGET
TARGETS
ACHIEVED with experts
EXPLANATION partner to sustain the services?
TOTAL 100 142 70% more informants than the minimum target
E-Mail Surveys Based On The Findings Of Site Visits
The research design was incremental —building on findings--such that e-mail surveys were designed after field research for: (a) magistrates; (b) social workers; (c) supervisors; and (d) provincial managers. The aim was to develop a more nuanced, qualitative understanding of impact and to generate some data that could be quantified. These surveys are listed in
FIGURE 2.5 (below). The only returns that were fewer than targeted were those from magistrates who were a difficult group to engage in any research processes.
FIGURE 2.5: FOUR E-MAIL SURVEYS
INFORMANT DESCRIPTION TARGET RECEIVED %
1 SUPERVISORS Survey of supervisors: perceptions of successes, shortcomings and recommendations
10 13 130%
2 SOCIAL
WORKERS
SW perceptions of the NCS services, successes and shortcomings and their recommendations.
15
3 80
MAGISTRATES
E-mail surveys were sent to some 80 participating magistrates (16 returned); 6 interviews at 5 sites and 2 interviews in
Pretoria supplemented the magistrate data
80
31
24
207%
30%
4 PROVINCIAL
MANAGERS
© NICRO 2013
An e-mail questionnaire form sent to all provincial managers
7 7 100%
56
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
Cape Town Research Processes
Following the early return of data, interviews and focus groups also took place in Cape Town to reflect on the findings so as to deepen analysis. The processes are illustrated in FIGURE 2.6 below:
FIGURE 2.6: INTERVIEWS AND FOCUS GROUPS IN CAPE TOWN
INFORMANT NAME DATE in 2012 DESCRIPTION
1 PROJECT
MANAGER
Regan
Macquet
JulesContinuous support throughout the period of the evaluation; 09 October
Workshop, 05 November
Focus Group
Induction meeting, e-mail correspondence, telephonic interviews, meetings, provided initial review documentation and documentation on request, support to all research processes
2 PROJECT
OWNER
Celia Dawson
3 NICRO
ADVOCACY
Venessa
Padayachee
28 October; 05 November,
09 October
Formal Interview, Focus Group at
National
05 November,
November
10 Focus group at national, interview on lobbying and advocacy activities
4 F OCUS
GROUP WITH
PARTNERS
17 attendees
See Annexure 17 for list of participants
09 October A 2-hour workshop on the quality and value of the NCS services and sustainability issues
Data Capture
The main survey to measure client risk (clients and social workers) required data capture onto spread sheets. The rest of the data was captured by the evaluator directly into tables as found in the annexures and in various parts of the report. Owing the quantity of the data, many weeks were expended in capturing the data seen in the annexures. Some of the annexures were also designed with annotations so that the source of the data could be clearly linked to the conclusions being drawn by the evaluator. Much interview data is also in the annexures.
© NICRO 2013 57
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
Data Review And Assessment
Data capture (above) led to an assessment of the data to first determine whether or not the broad objectives of the project had been met and a comprehensive analysis of all the captured data and with respect to the indicators these satisfied (the scope given to the impact evaluation and in terms of the contractual arrangements are in the indicator tables). FIGURE 2.7 (below) illustrates the data that was assessed and reviewed for analysis, illustration and report writing including the weightings given to each kind of data.
FIGURE 2.7: TYPE OF DATA FOR ANALYSIS AND WEIGHTINGS
TYPE OF DATA WEIGHTING SOURCES
1 BUDGET
EXPENDITURE
AND
DATA; VALUE FOR
MONEY
33%  Review of Service Statistics 2008-2012
 Budget Analysis and expenditures (5-year-budget, project expenditure reports)
 CPA report
2 QUANTITATIVE
DATA
3 QUALITATIVE DATA
33%
33%
 Service Statistics
 Analysis of Target Attainment Statistics
 936 Client Surveys
 936 SW surveys on client risk
 Social worker surveys
 Supervisor Surveys
 Magistrate Surveys
 Provincial Manager Surveys
 Site based observations
 Client interviews,
 Staff interviews
 Focus groups at sites
© NICRO 2013 58
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
TYPE OF DATA WEIGHTING SOURCES
 Interviews with departments in Pretoria
 Focus Group/Workshop in Cape Town with project partners
 Focus Group with National Managers
 Project Reports 2008-20012
Analysis And Draft Report Writing
DFID has very specific structure for conveying the findings in the evaluation report, which are explained further below. This involves 11 key features and underlined are the three chapters with the critical findings and recommendations:
1. Cover note (from NICRO)
2. Title Page including Programme identification Details
3. Table of Contents
4. Abbreviations/acronyms page
5. Executive Summary (maximum 3 pages)
6. A short introduction to the programme
7. The evaluation methodology
8. Findings in relation to standard review criteria including Achievement Rating Scales
9. Innovation
10. Summary of recommendations
11. Annexures for the detailed statistical data, documents —some key data is annotated
Notable is that the standard review criteria furnished by DFID situates the critical findings of this evaluation study in Chapter 3 (‘Findings’). In that section, there is summary discussion of the contribution that NICRO made to the overall Governance and Transparency Framework.
Following that, each objective is discussed in terms of an achievement rating scale that clearly assesses in a table the success of programme in terms of these findings. Much of the detailed statistical data enters into the annexures and those who want a greatly detailed understanding
© NICRO 2013 59
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
(presumably NICRO and others seeking to implement similar projects) would find it worthwhile to consult these including the annotations provided.
Submission of Draft Report (end February)
Owing to delays in data returning from the field (the methodology was participative and SWs and supervisors were facing stresses at sites with restructuring and other NICRO activities that delayed returns by about two months), the DRAFT report was initially due by for review by
NICRO (permitting a response) by 15 November but this did not occur until mid-March.
Circulation Of DRAFT Report With NICRO Reference Team (End Feb 2013)
Shortly after the delivery of the draft report, it was intended that the reference team would meet at national for feedback.. The evaluator would offer a presentation and opportunity for the findings to be reviewed and discussed. It was found to be more productive to circulate the draft from comments and then the evaluator took these into account when writing up the final submission.
Final Submission (March 2013)
The comments of the reference team members through discussion of the draft were integrated into the final report.
Report Presentation (scheduled for June 2013; preparation and presentation of the findings on a PowerPoint)
The evaluator will present the findings of the report at a public presentation of the paper and utilise photographs and other materials to create a PowerPoint Presentation.
© NICRO 2013 60
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
CHALLENGES
The challenges owed mainly to data returning later than anticipated. NICRO started restructuring the organisation (staff changes) during data collection for the evaluation study
(post 30 July 2012) and this was participatory research involving NICRO staff. This led to such challenges as:
ï‚·
Limited NICRO staff availability for surveying (budget cuts affected staff numbers and activities; they were too pressurised; fieldworkers had to be contracted)
ï‚·
A reduction in sites planned for surveying (originally all 45, then 22 and finally
16)
ï‚·
A focus on surveying clients longest on the programme or in aftercare (the original research plan) could not be maintained (the focus became obtaining the required numbers targeted and using group settings, home or work settings, or as a last resort phoning clients just to obtain these)
ï‚·
Survey data returning about 60 days later than anticipated
The surveying focused on reaching a large number of client initially (but a sufficient number were reached for making statements) and particularly those who had completed aftercare. The vast majority were still on programmes at the time of the evaluation and those who had completed aftercare were difficult to reach. The numbers reached with surveys per site are indicated by province in FIGURE 2.8 below:
FIGURE 2.8: SEVEN PROVINCES AND 16 SITES RETURNING CLIENT AND SOCIAL WORKER
SURVEYS
PROVINCE THE 16 SITES AND NUMBERS SURVEYED PER SITE
1 EASTERN CAPE East London (170), Port Elizabeth (130), Queenstown (90),
Gelvandale (106), Mdantsane (120), Zwelitsha (40)
2 FREE STATE Bloemfontein (83) , Welkom (40)
RETURNS
230
106
© NICRO 2013 61
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
PROVINCE
3 GAUTENG
THE 16 SITES AND NUMBERS SURVEYED PER SITE
Soweto (250), Pretoria (150), Randburg (110), Odi (100)
4 KWA-ZULU-
NATAL
5 MPUMALANGA
Pietermaritzburg (70), Durban (110)
Eerstehoek (30), Nelspruit (10)
Kimberley (180) 6 NORTHERN
CAPE
7 WESTERN CAPE Cape Town (Athlone, Bellville, Kuils River, Cape Town, Wynberg combined)
TOTAL
RETURNS
389
67
40
180
38
936
TIME-FRAME
The time frame detailed according to more specific activities is given in FIGURE 2.9 (below)
FIGURE 2.9: RESEARCH ACTIVITIES ACCORDING TO A TIMETABLE
A CTIVITY D ESCRIPTION O RIGINAL
T ARGET
D ATE
A CHIEVED
1 Identifying
Indicators
Evaluator reviews and analyses project documentation to identify the indicators for measuring progress.
May May
2 NICRO Forms
A Reference
Group
NICRO PM and possibly other head office managers form a reference group to review indicators and support the site selection process so as to enable research processes to proceed in a timely way.
May May
3 Workshop
Indicators
Work-shopping the indicators and measures —an evaluation matrix developed by the evaluator is refined with input from the PM and/or a reference group; finalise site selection based on the data that needs to be recovered
Late May Late May
4 Schedule Site
Visits
NICRO PM and evaluator work together to ensure that the site visits can take place with all identified informants available.
June June
© NICRO 2013 62
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
5
A CTIVITY
Design Of
Data
Collection
Instruments
D ESCRIPTION
Data collection instruments based on the refined indicators are designed (e.g., questionnaires, focus groups and site observation forms). These are also reviewed with the PM
O RIGINAL
T ARGET
June
D ATE
A CHIEVED
June
6 Issue Letters
Of Introduction
The interview participants at sites are informed of the process with letters and a sample of data collection instruments so that they can think about and prepare for the interview process.
June June
7 Return Of
Survey Data
This was delayed: the sites were granted extensions owing to personnel shortages
June/July 30 August
July/August 7 Site Visits In-person interviews are conducted at five sites with clients, social workers, supervisors, LSC members, magistrates, prosecutors and key project partners
July
Oct-Nov 8 Interviews Interviews with NICRO national and available regional managers and three high court judges follow from site visits (preceded the design of e-mail questionnaires)
July 2012
9 The Design Of
Surveys
All previous research processes contribute to the design of these surveys which will also be pre-viewed in detail with the project manager
July/Aug
2012
July/Aug
10 E-Mail
Surveys
E-mail surveys: project supervisors, SWs, clients, magistrates, regional managers
Aug 2012 Sep 2012
11 Data Capture Five surveys require data capture onto spread sheets; the surveys did not return until end October owing to extensions granted (sites were under pressure owing to restructuring)
Sep 2012 Nov 2012
12 Data
Assessment
This is the analysis of all data sets and the generation of tables for analysis using all project materials
Sep 2012 Nov 2012
13 Report Writing Report writing of the data is anticipated as the longest period owing to satisfying the TOR and DFID objectives (a late start owing to the late return of all data for assessment)
Sep/Oct/No v
Dec/Jan
© NICRO 2013 63
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
14
A CTIVITY
Review
D ESCRIPTION
The evaluator meets with NICRO PM on the initial findings to anticipate feedback and expedite processes
O RIGINAL
T ARGET
Nov
A
D ATE
CHIEVED
End Feb 2013
15 Draft Report A draft report is issued for discussion and refinements 15 Nov
2012
End Feb 2013
Feb/Mar 16 Reference
Group Meeting
The evaluator circulated the DRAFT report from comment. 30 Nov
2012
17 Final
Evaluation
The final report is issued 31 Jan
2013
March
18 Public
Presentation
The evaluator presents the results of the impact evaluation
Apr 2013
© NICRO 2013
June
64
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
FINDINGS IN RELATIONSHIP TO STANDARD REVIEW CRITERIA
What has been learnt and achieved in terms of the NCS services at 45 sites and the
Indigenous Justice Pilot Project at Eerstehoek? The findings on these questions are presented in 8 sections conforming to the DFID standard review criteria, which are as follows 20 :
1. RELEVANCE to GTF : Did the programme give voice to those who could not speak out for themselves? Did the programme make government more accountable? Did the programme fit in with local, national or international priorities?
2. IMPACT:
What was the programme’s overall impact? What were the broader economic, social and political consequences of the programme? How did it contribute to the overall objectives of the Governance and Transparency
Fund
—increased capability, accountability and responsiveness and poverty alleviation? What was the programme’s overall impact and how does this compare with what was expected? Did the programme address the intended target group or what was the actual coverage? Who were the direct and indirect wider beneficiaries of the project? What difference has been made to the lives of those involved with the programme?
3. ECONOMY: What goes into providing NCS services (e.g., social workers, facilities, training, etc.) and is it cost-effective? Could the same inputs have been achieved for less money? Were salaries and expenditures appropriate to the context?
4. EFFICIENCY: This requires a measure of productivity. How much did funding, personnel, regulatory, administrative time, and other resources and procedures contribute to or hinder the achievement of planned outputs? Are there obvious links between significant expenditures and key programme outputs? How well did partnership and management arrangements work?
How did they develop over time? How well did the financial systems work?
20 ‘Equity’ was also given as a DFID review criteria --gender, age, race, and disability—but this is not appropriate to NCS services. First, NICRO has no control over who gets sentenced to the programme (it is the magistrate who decides on the sentence). Second, in SA the majority of offenders are young black males. Third, the services are for adults. The programmatic impacts on the vulnerable are more in terms of addressing poverty (see relevance to the GTF below in Section 1) and by maintaining the integrity of the family unit and addressing substance abuse (see Impact sections 2.5 and 2.6).
© NICRO 2013 65
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
How were local partners involved in programme management and how effective was this and what were the benefits of and difficulties of this involvement? Were the risks properly identified and well managed?
5. EFFECTIVENESS: How effectively were the objectives achieved in terms of the original targets (as set in the original logical framework)? Were the objectives met or are they likely to be met? How effective was the programme approach? (e.g., one measure might be how the services can be leveraged into something more; or how replicable it is). With hindsight how could it have been improved?
6. VALUE FOR MONEY : Were resources used optimally to offer good value for money? Was a balance achieved between economy, efficiency and effectiveness? Did the programme represent good value for money (best results for money spent)?
7. SUSTAINBABILY : What is the potential for the programme continuing after the withdrawal of DFID support? Will the programme be sustained? Will collaboration, networking or the project’s effort to influence public opinion on
NCS support sustainability?
8. REPLICABILITY: How replicable is the process that introduced the changes/impact? What specific innovative aspects are going to be replicable? What is replicable elsewhere (outside South Africa)? What are the circumstances or contexts in which this programme is replicable?
Owing to a critique of the MTR as overly detailed, much of the evidence is provided in annexures in which the data from all research processes is clearly displayed in tables and often annotated to show how that data is linked to the conclusions of the report. Should the reader be seeking more detailed insights into the reasons behind the findings or the sources for them, the annexures offer a fairly thorough but easy way to review the findings. Section
Two on impact (further below) is the lengthiest section in this chapter and particular, Section
2.3, subsections 2.31-2.3.7 rate each objective of the project with an achievement rating scale with explanati on/validation in terms of the evaluator’s findings and analysis. Section 2.4 shows that even if weighted differently the data returns with the same achievement rating. Surveyed
Supervisors also provided the same ‘2’ rating (see Annexure 8, Section 1).
© NICRO 2013 66
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
RELEVANCE TO GTF
This refers to three main questions:
ï‚·
How well did the programme relate to governance priorities at local, national of international levels?
ï‚·
Did the NCS services increase voice, accountability, and responsiveness within the local context?
ï‚·
Did the programme make government more accountable?
On the first bullet, the programme was highly relevant to both international and national priorities. First, since 1990 the United Nations has promoted the ‘Standard Minimum Rules for
Non-custodial Meas ures’ better known as the ‘Tokyo Rules.’ These ‘rules’ call on member states (South Africa is a member and signatory) to view prison as a measure of the last resort and to treat on non-custodial sentences that encourage the reintegration of the offender into society as the first priority in policy and practice.
21 South Africa’s Criminal Procedures Act also views prison as the choice of last resort and further identifies (in a large number of subsections of the Act) how NCS can be applied in terms of sentencing.
22 Despite the Act, this is not normal practice and South Africa has the ninth highest population of detainees the world.
23
One of several reasons for that was the lack of credible non-custodial sentencing services, which prevented the application of the law. Thus, NICRO ’s modelling of a credible range of
NCS services enabled governance priorities long-held and described at national and international levels to be applied (see also Annexure 23, question two where a member of the magistrate commission makes such a statement).
Still on the first bullet, breaking the cycle of crime and poverty is a priority at national and local levels in South Africa and it is this that will have massive social benefits. An example of the relationship is that crime is costing the business sector about 3.2% of sales, which also
21 United Nations (1990) United Nations Standard Minimum Rules for Non-Custodial
Measures, http://www.un.org/documents/ga/res/45/a45r110.htm
22 The Criminal Procedure Act 51 of 1977 has numerous subsections that define how noncustodial sentencing can be applied.
23 Schroeder, Fatima (25 Nov 2012) ‘Crisis as jails bursting at the seams’ Sunday Argus, p. 4
© NICRO 2013 67
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013 reduces the tax base for poverty alleviation programmes.
24 NICRO tried to break the cycle by reducing criminogenic risk factors amongst offenders and to also keep them away from the influence of hardened criminals through non-custodial sentencing. This could help unchain
South Africa from the 140 billion rand systems of personnel and infrastructure for arrests, trials and incarceration (police, courts and 42 prisons containing 150,000 detainees occupying
119,000 beds), which erodes the tax base for fighting poverty while providing hardly any evidence of preventing crime.
25 Imprisonment is more of a contributor to a cycle of poverty and crime than preventative and many social scientists studying the phenomenon identified from a review of hundreds of studies that it increases recidivism.
26 The offender loses his job and is exposed to institutionalised violence in the form of gangs and criminal behaviour which reinforce rather than address the criminogenic factors (thinking patterns, behaviours) that led to the crime. Offenders, as often as not, have children who then grow up in poverty without positive adult role models (serious risk factors that contribute to that young person becoming an offender too).
27 Household poverty is usually deepened in the absence of a breadwinner and exacerbated as the family have the added burden of visiting and supporting the prisoner.
On the second and third bulleted points, the NCS services were evidenced to be increasing voice, accountability and responsiveness in both the local and national context in these major ways (see also the log frame indicators on impact that are found in Sections 2.3.1-2.3.7 further below):
24 Seria, Nasreen (29 Jul 2010) ‘Crime in South Africa is Businesses Biggest Constraint;
Bloomberg, http://www.bloomberg.com/news/print/2010-07-29/crime-in-south-africa
25 Spending on defence, public order and safety was R140 billion for 2012/2013 and is projected to be R158 billion for 2014/2015 Gordhan,Pravin (2012) Budget Speech, RSA, http://www.info.gov.za/speech/DynamicAction?pageid=461&sid=25270&tid=57402l see also
Hoffman, Sandy (08 Oct 2012) Rehabilitation of Prisoners in Transforming South Africa, http://csvr.org.za/wits/confpaps/hoffman.htm
26 Clark, Patrick (2012) Preventing Future Crime with Cognitive Behavioural Therapy. National
Institute of Justice, US Department of State, Washington, D.C; Van der Westhuizen, AEM and
A Lombard (2005) A Model for Community Corrections Residential Centres in South Africa from A Social Work Perspective, Acta Criminologica 18(3) pp 101-109
27 Republic of South Africa (1996) National Crime Prevention Strategy; Republic of South
Africa (1996); National Crime Combating Strategy, The Presidency (2011) National
Development Plan: Our Future Make it Work; Republic of South Africa
© NICRO 2013 68
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
ï‚·
Departments and service providers were found to be working together in an integrated fashion on Local Steering Committees (or otherwise NICRO serving on the Case Flow Management Committees at the courts) to provide
NCS services to 3,960 offenders at 37 or 45 sites around the country as of 30
July 2012 (See Annexure 32 and/or Annexure 7, question 21 that details the participation levels of the departments and stakeholders that reviewed sentencing and promoted NCS services)
ï‚·
The DoJCD was making space and facilities available for various aspects of the NCS services at 45 magistrate courts around the countries (they were also a partner in the original pilot
—some 5 years of partnering with NICRO)
ï‚·
In an August 2012 focus group with evaluator, the 5 top Deputy Directors of
DCS expressed willingness to work with NICRO from 2013 to try to sustain the services at sites throughout the country (see also Annexure 18)
ï‚·
The programme included community engagement workshops from July 2010 with before-and-after surveying at 44 sites which identified a shift in thinking from retributive justice--punishing the offender--toward restorative justice-repairing the harm caused by crime. NICRO measured a rise in public support in targeted areas from a 2009 baseline indicator of only 44% in favour of rehabilitation programmes and 54% in support of more prisons, to 70% agreeing that certain kinds of offenders can be rehabilitated without going to prison (see Annexures 28 and 29 or discussion in Chapter 4 on Innovation)
IMPACTS
Impacts pertaining to the NCS Services will first be discussed in three sections:
ï‚·
The broad, economic, social and political impacts of the NCS services
ï‚·
The broad, economic and social impacts of the Indigenous Justice Pilot Project
ï‚·
The measured impacts (according to an achievement rating scale)
The first pertains to general statements and findings on the contributions to the broad GTF objectives such as increased capability, accountability and responsiveness, and poverty alleviation and in terms of general impacts on perpetrators, the intended target group and then direct and indirect wider beneficiaries. The main findings on impact are given in easy-to-read
© NICRO 2013 69
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013 tables that include an achievement rating according to all project objectives (see Annexure 1.1 for the broad purpose log frame).
THE BROAD ECONOMIC, SOCIAL, AND POLITICAL IMPACT OF THE NCS SERVICES
The broad purpose impact rating was a 2 (see Annexure 1, Log frame 1.1) and this was mainly achieved by the NCS Services meeting these broad purpose indicators (with a few shortcomings):
ï‚·
Create a safer South Africa by contributing to penal reform
ï‚·
Change offender behaviour and reduce crime
ï‚·
Develop, implement and promote the use of non-custodial sentencing in magistrate courts
These bulleted items can be discussed in an integrated way because they were part of one broad delivery plan: introduce and entrench non-custodial responses to crime at magistrate courts and within the South African criminal justice system by introducing appropriate, empowering and therapeutic interventions that could be attached to sentences to address risk factors in terms of criminogenic thinking (cognitive-behaviour), which in turn would change offender behaviour and reduce crime. The evidence is that risk factors for offenders on programmes were reduced and while it is too strongly worded to say that this system was entrenched in the justice system, it promoted new thinking and new sentencing practices that will be sustained wherever NICRO decides to sustain it as a standard NICRO offering. NICRO estimates there will be 15-20 sites where the volume of referrals justifies it. Advocacy to expand such services will also continue. This was evident because by 30 July 2012 (the cutoff date for the study but with some 8 months of DFID funding remaining), NICRO had local, provincial and national government representatives at 37 sites around the country engaged in an effort to attach therapy to 3,960 sentences. The LSCs had not yet met at 8 sites on 31 July
2012 and with one exception these were Year 4 and 5 sites but start-up times were indicated as a challenge (trust building and other processes to get referrals going can be very slow).
© NICRO 2013 70
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
At the top of FIGURE 3.1 (below) is a numeric list of five broad GTF objectives and in the table beneath this list are some major outcomes of the project. The tick boxes to the right of each listed outcome are coded to the numeric list of objectives at the top, to succinctly show how each major outcome met 4-5 major GTF objectives by 30 July 2012.
FIGURE 3.1: HOW NICRO MET FIVE BROAD GTF OBJECTIVES
1. Capacitate local leaders to perform their functions better and in ways that are sensitive to the poverty-related causes of crime
2. Increase accountability to citizens and communities
3. Sensitise government leaders to operate with increased respect for human rights and the rule of law
4. Make direct impacts on the intended target group
5. Make impact on direct and indirect wider beneficiaries (criminal justice sector; communities)
BROAD IMPACTS BY 31 JULY 2012 IN TERMS OF 5 GTF OBJECTIVES ↓
Meeting DFID objectives as bulleted above ↓
1 2 3 4 5
23 Local Stakeholder Committees (LSCs) and 14 Case Flow Management Committees around the country (37 of 45 sites or 82%) were reviewing and promoting NCS sentencing practices.
X X X X X
3,960 offenders on NCS programmes (66% target attainment by 30 July 2012) to prevent re-offending by both addressing the aberrant behaviour and by not exposing them to hardened prisoners who have a strong influence on fellow inmates in terms of increasing the skills and propensity of commit crimes
X X - X X
936 clients (a 24% sample) were given a diagnostic test and their SWs completed a survey on each of those clients to find that the risk factors for reoffending had been reduced (only a 1-3% variance on a scorecard for reoffending between SW and client) indicating that the core behaviours that led to the offense were diagnosed and addressed through cognitive behaviour therapy (see Figure 3.2 below and Section 2.6; see also footnote 10 below)
- X - X X
Magistrates at 45 sites were presented with a more credible range of sentencing options so that they can implement existing policy appropriately 28
X X X X X
28 There were no therapeutic programmes for NCS until NICRO provided them, which enabled magistrates to refer knowing that part of that benefit, is increased respect for human rights and the rule of law. NCS had been available in South Africa but fell into disuse because: (a) the alternatives were not therapeutic —fines, suspended sentences, or reduced sentences; (b) the
© NICRO 2013 71
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
BROAD IMPACTS BY 31 JULY 2012 IN TERMS OF 5 GTF OBJECTIVES ↓
Meeting DFID objectives as bulleted above ↓
1 2 3 4 5
Public and official awareness that there were viable and better alternatives to imprisonment was indicated to by a R10-million-value media campaign and outreach activities at 45 sites involving magistrates and public figures
X X X X X
An increase in the capacity of government leaders and departments (found at local level and representing local, provincial and national departments) to deliver justice in a way that resolves the behavioural issues producing the crime (evidenced by the efforts of 37 Local
Stakeholder Committees)
X X X X X
FIGURE 3.2 (below) presents the general findings on reducing the risk for reoffending based on a sample of 936 clients (24% of all clients). Clients self-rated the individual risk factors for reoffending (strongly agree, agree, not sure, disagree, strongly disagree) AND for verification
SWs evaluated each client’s self-rating with reference to project documentation.
29 Would SW project documentation find agreement with the client self-scores? Of 936 NCS clients (a 24% sample conforming to the age, race and gender profiles of all NCS clients), 94% either strongly agreed or agreed that the NCS services helped them to make changes such that they will not reoffend again.
30 The SWs concurred within a range of 0-6% difference depending on the question (to average less than 3% difference overall). In other words, the majority of offenders with rare exceptions experienced a reduction in criminogenic thinking patterns owing to NCS therapy. The detailed findings on this are presented in Section 2.6 (further below) but in
FIGURE 3.2 (immediately below) the findings are presented in terms of a score card
—how many NICRO clients were at low risk and how many still at risk? At most 4-6% of them were public did not support these; and (c) monitoring by Correctional Services of these cases was so weak that many magistrates simply quit supporting it.
29 The SW used court reports, programme pre-tests and post-test, process notes, anecdotal data from clients, aftercare and tracking reports and group work reports —to rate client progress on the survey (this could then be checked against the client self-assessment)
30 It was found in analysis of the data and during site interviews that it is better to collapse agree and strongly agree as ‘agree’ since most clients and many SWs reported in site interviews that they had difficulty in distinguishing the two (otherwise a difference might be noted that owes to language rather than a finding).
© NICRO 2013 72
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013 still at risk (and this corroborates with some limited recidivism data on NICRO clients (2% per year).
FIGURE 3.2: SURVEY RESULTS SCORECARD FOR 936 OFFENDERS
Percentage
Range of
Criminogenic
Risk Factors
(Scoring Card)
↓
0-20% (very low risk) →
21-40% (low risk) →
41-60%
(medium risk)
→
61-80% (high risk) →
Client Self-
Assessment
N=931
# in range
% in range
SW Assessment of client
N=928
# in range
% in range
651
246
15
13
70%
26%
2%
1.5%
539
335
44
09
58%
36%
5%
1%
Percentage variance between client and SW assessment
Per Row
12%
10%
0.5%
Across shaded areas
3%
2%
1%
81-100% (very high risk) →
06 0.5% 01 --- 0.5%
TOTALS →
Incomplete →
Blank →
Spoilt →
931 100% 928
01
02
02
--
--
--
06
01
01
100%
--
--
--
SUMMARY: % IN EACH RISK CATEGORY ↓
Risk ↓
Low →
Medium
→
High →
Selfassessed
96%
2%
SWassessed
94%
5%
TOTALS IN
SURVEY
936 -- 936 -- 2% 1%
100% 100%
The evidence of a reduction in risk factors amongst 936 clients (a 24% sample of 3960 clients on programmes as of 31 July 2012) is significant for establishing the broad value of the service which aimed to ensure that perpetrators account for and address their offending behaviour. If it were not for the NICRO programmes, 3,960 perpetrators (to July 2012 with more to come) would either have been: (a) released with only a fine, a suspended sentence or remanded to
Correctional Supervision with NO therapeutic programmes to reduce the risks that they would commit a crime again; or (b) gone to prison. In neither case would rehabilitation be addressed
© NICRO 2013 73
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013 very well if at all.
The country’s recidivism rate is in the 50-90% range and this owes, in part, to not addressing the behaviour that led to the crime. The NCS services address the risk factors for reoffending (which were lowered based on the data) while in prison the criminogenic risk factors are not addressed and can worsen owing to the conditions and people found there.
Reduced risk factors do not tell us about recidivism rates and while this was not the methodology used for this report it can be documented for future reference that some limited recidivism data that was available did not contradict the findings in any way (the sample is consistent with and in the same range as the broader findings of reduced risk). Some project data on 65 clients whose criminal records were checked in 2010 and another 127 in 2011 to see if they reoffended found that of 192 clients (a 2% sample of all 3,960 clients), five (2.6%) reoffended in 2010 and four in 2011 (2%).
31 Data was also located for 2008 and 2009 but found too small to draw any conclusions (8 clients; none reoffended). In fact, none of this data is utilised for any weightings because: (a) NICRO clients and the general recidivism rates cannot be compared as the risk profiles for prison populations are not the same; (b) the length of time off the programme is not yet sufficient; and (c) monitoring and evaluation with control groups should be a continuous part of the NCS services for next 10 years or more to obtain the longitudinal data required.
THE BROAD ECONOMIC, SOCIAL, AND POLITICAL IMPACT THE INDIGENOUS JUSTICE
PILOT PROJECT
The broad goal of the Indigenous Justice Pilot Project was to draw upon the indigenous concepts of justice and reconciliation and pilot a model of diversion that fit with the cultural framework of traditional courts. This was not achieved in terms of many lines of evidence, which are itemised in Annexure 16 (see also Section 2.3.6 in terms of Logframe Ratings and
Explanations below; Sections 5.4 – 5.6 further below on the overall effectiveness of the pilot project; and Section 7.2 on sustainability where it was determined that this was not sustainable as designed owing to a large number of factors including distance). Many other lines of evidence corroborated with those listed above to establish definitively that the original project
31 NICRO (2012) Electronic Communications with the Project Manager
© NICRO 2013 74
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013 design of diversion from the traditional courts to NICRO programmes was not working including these:
ï‚·
The referrals from the traditional courts were too low (only 10 referrals between November 2010 and July 2012; 13% of all referrals) to justify the expense of the service
ï‚·
Most of the traditional court cases turned out to be inapplicable for referral to
NICRO and yet there were three traditional courts involved all sitting different days of the week (Monday-Friday), which wanted NICRO present during the hearings (the time, petrol, and funds for this were not warranted since one of them did not refer; another lacked the caseload to warrant many referrals and altogether referrals were too few)
ï‚·
Related to the above bullet, traditional leaders stated in focus groups that they liked the NICRO programme but said it would not work unless the SW was posted at the traditional court but this turns out to be an expensive use of
SW time compared to other activities that would have more impact on crime reduction.
ï‚·
Some traditional courts were imposing fines beyond what the law allows
ï‚·
A full national debate on the future of the traditional courts and the Traditional
Justice Bill erupted during the intervention and got very bad press providing strong indications that the bill is flawed (not useful for NICRO to engage at this time or be viewed as taking one side or another in a political debate)
ï‚·
Relating to the above bullet point, there is a need to define the relationship between Traditional African Justice and the criminal justice systems (it needs to be properly codified)
ï‚·
LSC members perceived was that the area was too big in both population and distances for NICRO to provide an effective intervention either in terms of
NCS or diversion from traditional courts and testified that NICRO should rather intervene in schools and communities to prevent crime and divert people out of the system and leave the traditional courts with national and provincial departments (e.g., these departments need to provide training in the Traditional Affairs Act).
© NICRO 2013 75
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
ï‚·
The SWs thought that rural distances provided a challenge such that some thought that more prevention work in the schools and communities was a better tactic.
ï‚·
Crime was perceived to have decreased not owing to work in the traditional courts but owing to NICRO work in the schools (the probation officer, the magistrate and an NPA member in case processing at the court held this perception —see Annexure 16 for additional details)
NICRO monitored referrals, observed them as low and engaged in appropriate research to analyse the problem to find that ‘the capacity for traditional leaders to engage with the pilot in the manner originally envisaged did not work’.
32 The response was to increase school and community work. Direct referrals from schools amounted to 28 (37% of 76 referrals; more than double the referrals from traditional courts). More importantly, NICRO workshops with school learners, which focused on 157 students (see further details in Sections 5.2-5.4) led to a drop in crime rates in schools (i.e., the DSD and the magistrate said that their caseload dropped from the particular schools in which NICRO worked —see Annexure 16). Magistrate Chirwa also stated on the magistrate survey that ‘Bantfwabethu Secondary School used to apply for search warr ants but this stopped.’ The magistrate court (also an NCS site) accounted for 50% of the referrals at Eerstehoek.
Based on the data in Annexure 16, it appeared that the twin strategy of including traditional leaders on the Local Stakeholder Committee for the court and pro-active community engagements to prevent crime was more effective than diversion into programmes from traditional courts. Community outreach can make broad economic, social and political impact and overcome the obstacles bulleted further above. For instance, where distances are an obstacle to delivery, it is cost-effective to target youth in the schools (prevent criminality by preventing youth from dropping out of school; develop life skills that prevent crime in the first place) since schools are centrally located--a meeting point where impact on greater numbers can be made (cost-effective) with little or no political fallout.
32 NICRO (2011) ‘Research into the Role of Traditional Leaders and the Possibility of
Integrating SiSwati Practices into NICRO
Services’, Traditional Leaders Research Paper, Cape
Town
© NICRO 2013 76
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
THE IMPACT OF THE NCS SERVICES MEASURED AGAINST OBJECTIVES (LOG
FRAMES ON MEETING INTENDED OUTPUTS)
The project had seven objectives. The targets relevant to each of them are measured against outcomes in log frames and tables that include analysis and evaluation weightings as follows:
ï‚·
FIGURE 3.3 (below) provides the Achievement Rating Scale (provided by DFID) that was utilised to rate the overall outcome and the outcomes according to each objective:
Rating
1
2
3
4
5
FIGURE 3.3 DFID ’S ACHIEVEMENT RATING SCALE
Definitions
Fully achieved-very few or no shortcomings
Largely achieved, despite a few shortcomings
Only partially achieved, benefits and shortcomings finely balanced
Very limited achievement, extensive shortcomings
Not achieved
ï‚·
FIGURE 3.4 illustrates the DFID Achievement Ratings on outputs applied to seven project objectives to issue an overall achievement rating of 2 33
ï‚·
SUBSECTIONS 2.3.1 TO 2.3.7 explain the achievement rating in detailed log frames for each objective that include details on the successes and shortcomings for each objective (along with Figures 3.5-3.9, which help to illustrate certain points)
ï‚·
SUBSECTION 2.4 and Figure 3 offer a different way of weighting the data in terms of three components--budgetary data (weighted 33%), quantitative data
(weighted 33%) and qualitative findings (rated 33%) — this also produced an achievement rating of 2 (this cross-checks and validates the log frame indicators to validate the overall achievement rating)
33 Surveyed supervisors also provided ratings that average 2 overall (see Annexure 8, question
1 in which supervisors are surveyed on impacts
© NICRO 2013 77
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
FIGURE 3.4 : DFID ACHIEVEMENT RATINGS APPLIED TO 7 INTENDED OUTPUTS FOR AN OVERALL
PROJECT RATING
Subsection Objective Rating
2.3.1 A fully-fledged referral system developed for alternative sentencing at 45 magistrate’s courts
2
2.3.2
2.3.3
2.3.4
2.3.5
2.3.6
2.3.7
Therapeutic programmes available for alternative sentencing
Professional social worker expertise available at 45 courts
Expert alternative sentencing programme evaluations
A national media and marketing campaign that increases public awareness of AS sanctions and restorative practices
A newly developed indigenous justice diversion project available
A developed knowledge base of non-custodial sentencing systems and practices to further public and sector understanding and promote awareness
2
2
1.5
3
2
1
2 Overall Project Rating = largely achieved despite a few shortcomings
© NICRO 2013 78
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
Subsections 2.3.1 to 2.3.7 (the detailed log frames)
The significant findings are detailed in log frames (below), which explain and justify the ratings in FIGURE 3.4 (above).
A fully-fledged referral system developed for alternative sentencing at 45 magistrate’s courts
Overall rating (explanation given in two indicators below) = 2
1-5 rating
Log frame
Indicators
Baseline and Target for
Indicators
Actual achievements by
31 July 2012
Comments on final results including unintended impacts
2
Partially achieved
45
Magistrate
Courts referring offenders to
AS interventions no less than
10X per month by project end
Baseline:
0-5 referrals
(0 if the
DoJ pilot sites from
2008-2009 are discounted)
© NICRO 2013
 A referral system for
NCS services was developed, established and functional at 37 of 45 courts-82%
(Annexure
32)
The reasons for low referrals:
 A tension exists between the magistrate’s desire to complete cases quickly and reduce backlogs (owing to high court rolls) and NICRO ’s professional requirement to conduct thorough assessments requiring up to two weeks or more, which are perceived to cause court delays (see Annexure 10, question 6)
 NCS services were provided to
3,960 offenders by 31 Jul
2012 (66% of target)
 A small percentage of magistrates are making referrals (sometimes one or two per court): insufficient referrals to achieve targets

2 sites (4%) achieved the monthly target of 10 referrals per month by 31 July
2012 (see
Annexure
 50% of magistrates surveyed said speedier assessments would improve the referral rate (see Annexure 10; question 6)
 Some magistrates see NCS as an unproven risk and want more evidence of success such as this
79
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
A fully-fledged referral system developed for alternative sentencing at 45 magistrate’s courts
Overall rating (explanation given in two indicators below) = 2
1-5 rating
Log frame
Indicators
Baseline and Target for
Indicators
Actual achievements by
31 July 2012
Comments on final results including unintended impacts
31) evaluation presents
 It appeared that 7 sites
(16%) at most might meet the
10X a month target for magistrat e referrals by project end (see
Annexure s 3.1 and
3.2 for additional details on target attainmen t)
 Some magistrates reported not getting feedback on cases such as at Tembisa, East London and
Welkom (the ones at Welkom stated on 29 Aug that they were not referring owing to this) —see question 1 to magistrates,
Annexure 10, Section 10.3)
 That some magistrates did not receive feedback was confirmed by SWs (Annexure 7, question
20) and by supervisors
(Annexure 8, Section 8.2, and
Question 12). There was also widespread confusion amongst
SWs, as to when feedback is required. After assessment and after treatment are correct and only 9 of 31 (29%) SWs could offer such a reply.
 A very small number of magistrates lacked confidence in a particular SW (this occurs when magistrates are not given feedback on cases or when there has been SW turnover)
 SW turnover in terms of reducing magistrate confidence is a big challenge owing to SW shortages and NICRO ’s inability to offer better salaries and benefits while some other
© NICRO 2013 80
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
A fully-fledged referral system developed for alternative sentencing at 45 magistrate’s courts
Overall rating (explanation given in two indicators below) = 2
1-5 rating
Log frame
Indicators
Baseline and Target for
Indicators
Actual achievements by
31 July 2012
Comments on final results including unintended impacts employers competing for the same SWs offer more pay and benefits
2
Partially achieved
45 LSCs established and functional
(as either
LSCs or as
NICRO participating in case flow management committees)
Baseline: 0  As of 30 July
2012: 37 of
45 (82%) sites were meeting either as 23
LSCs or as
NICRO serving on
14 case flow management committees
(Annexure 2)

As of 30 Jul
2012: 8
LSCs (18%) had not yet met —one
Year 5 site
(Pinetown); five Year 4 sites —
Attridgeville,
Katlehong,
Boksburg,
Meadowland s, and
Kanyamazan e; and two
Year 3 sites

The LSCs were originally designed as therapeutic case management forums but this did not occur owing to time constraints and skills deficits; instead members cooperated on ensuring the smooth delivery of the services
(although this did not match the original function, it proved to be important)
 Certain LSC partners reported in focus groups and interviews at a sample sites visited and on surveys that they were not receiving the feedback on client outcomes via the
NICRO SWs (it has improved since the MTR but these reports were confirmed by the
SWs themselves, 40% of whom indicated on question
20 Annexure 7 –they were not offering appropriate feedback on client outcomes)

All protocols, tools, processes and guidelines in place
© NICRO 2013 81
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
A fully-fledged referral system developed for alternative sentencing at 45 magistrate’s courts
Overall rating (explanation given in two indicators below) = 2
1-5 rating
Log frame
Indicators
Baseline and Target for
Indicators
Actual achievements by
31 July 2012
Comments on final results including unintended impacts
(Vaal and
Mamelodi) see
—
Annexure 2
 The Department of Justice was making space and facilities available for NCS services at selected courts around the country
 A ‘fully fledged’ system requires monitoring, aftercare and tracking and these were not always occurring; a partnership with DCS and the
Judiciary on appropriate systems and sentencing for
NCS might help to address this (see also Annexures 18,
23, 25)
Therapeutic programmes available for utilization in alternative sentencing
Overall rating (given in three indicators below) = 1.5
1-5 rating
Log frame
Indicators
Baseline and Target for
Indicators
Actual achievements by
30 July 2012
Comments on final results including unintended impacts
1 Programm es developed
, reviewed and available
NICRO designs, reviews and updates therapeutic programme s continuousl y
Programmes developed and reviewed during
2009-2012:
 Adapt

Adult Life
Skills
 Community
Service
 Matrix
 PIPV facilitator
 NICRO demonstrates a high level of professionalism in terms of programme design and development

No significant shortage of programme offerings were detected while a more significant problem was detected in terms of SW capacity to deliver certain programmes
—see
Appendices 7 and 8;
© NICRO 2013 82
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
Therapeutic programmes available for utilization in alternative sentencing
Overall rating (given in three indicators below) = 1.5
1-5 rating
Log frame
Indicators
Baseline and Target for
Indicators
Actual achievements by
30 July 2012
Comments on final results including unintended impacts and assessment guide
(resource manual and participant workshops still outstanding)
 Parenting
Development
 RGC
 Yes
Programme
 Professional
Skills
Development
Initiative —this includes onsite coaching, video library,
NING
Knowledge
Forum,
Information telecons
 Critical
Thinking
Manual
 An introduction to bullying: what educators need to know
 Diversion systems training especially question 8.7
 54% of the supervisors thought that all the available intervention offerings were adequate.
A little under half thought that some additional offerings were ‘needed’.
However, the definition of what is most needed varied greatly from site to site. In analysis, intervention tools that help to address anger issues, poverty, and substance abuse were viewed as needed by those who said no. However, based on all data, training in the existing offerings is more important than developing new offerings.

Matrix was only utilised for about 10 clients —it is potentially the best of the programmes but it is regarded as too lengthy and not all SWs have the skills required

Still in development on 30
July 2012 were Journey,
Adult Lifeskills (content development underway);
Community Service (90% complete); Critical
Thinking Manual (was very close to completion —mainly
© NICRO 2013 83
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
Therapeutic programmes available for utilization in alternative sentencing
Overall rating (given in three indicators below) = 1.5
1-5 rating
Log frame
Indicators
Baseline and Target for
Indicators
Actual achievements by
30 July 2012
Comments on final results including unintended impacts
 NCS systems training (90% complete) formatting)
2 6,000 offenders enrolled in programm es as part of alternative sentences by project’s end
Baseline = 0 to enrolments per month
(the latter if the 2008-
2009 pilot is considered as baseline)
3 the
 3,960 offenders enrolled in programmes
(66% of target or 2/3rds of the anticipated level of 6000)
 The project will come closer to target by project end but is likely to fall just short
 SW turnover disrupted services at many sites

Magistrates convinced of the worth of the services refer but too few magistrates were happy with the time period for the assessments
 6000 was also a revision downward from original targets @ 14,640 offenders enrolled in programmes by March
2013 —these targets were unrealistic for a variety of reasons but also for the above bulleted point
1 70% recomme ndations for of alternative sentencin g accepted by magistrate s in 45 courts by end of funding
---  72% of SW recommendati ons for AS accepted by magistrates
(yearly average to 31
July 2012 for years 1-4)
 The sentence recommendatio n acceptance rate was generally high but dropped to
45% during the
The drop in Year 5 owed to

SW turnover
 NICRO restructuring Nov
2011- April 2012, which included a moratorium on filling SW vacancies;
 Retrenchments at NICRO from November 2011; and

Relocating sites with low referrals to new sites in the hopes of improved referrals
(this was a calculated risk because the start-up time at a new site can be very slow and this changeover
© NICRO 2013 84
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
Therapeutic programmes available for utilization in alternative sentencing
Overall rating (given in three indicators below) = 1.5
1-5 rating
Log frame
Indicators
Baseline and Target for
Indicators
Actual achievements by
30 July 2012
Comments on final results including unintended impacts
2012-2013 budgetary year while the evaluation was underway (see
Figure 3.5 below) can also dampen impact on the improving referrals)
FIGURE 3.5: YEARLY RATE OF ACCEPTING NICRO
’S SENTENCING
RECOMMENDATIONS 2008-2013 34
Sentence Recommendation
Acceptance Rate
2008-
2009
2009-
2010
2010-
2011
2011-
2012
2012-
2013
TOTAL
NCS →
100% 85% 85% 73% 45% 72%
Indigenous Justice →
NA NA 100% 94% 100% 98%
Finding: if a magistrate requests an assessment, there is a high acceptance rate. The challenge is that not enough magistrates refer for an assessment per court (often just one or two referring since most magistrates are too consumed with reducing the court roll to want to delay for an assessment as indicated in interviews and Magistrate Survey Data (see Appendix
10; see also Annexure 7, 7.3 question 7 wherein most SWs thought they could reduce the assessment time) .
34 Source: NICRO (July 2012) National Statistics (NCS statistics extracted by Regan Jules-
Macquet, project manager)
© NICRO 2013 85
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
Professional SW expertise available at 45 sites
Overall rating (given in two indicators below) = 3
1-5 rating
Log frame
Indicator s
Baseline and
Target for
Indicators
Actual achievement s by 30 July
2012
Comments on final results including unintended impacts
2
Partiall y
Achiev ed
All sites
45 staffed by one fully trained
SW by project end —one per site
0  43 SW posts filled by
40 social workers
(4 sites share posts)
“All 45 sites staffed by one fully trained SW by project end” is a perfunctory measure of output and cannot be rated as fully achieved because there are strong indications that one SW is not sufficient since of staff surveyed (31 in terms of SW surveys and 34 in terms of supervisor surveys) this was found:


2 sites vacant
96% capacity
 Annexure 7, Section 7.3, question 6 shows that at only 10 of the 31 sites
(32%) was there a SW devoting more than 20 hours to NCS. This demonstrates the high degree to which SWs were multitasking and that the time available to them for developing referrals to NCS services was compromised by other obligations.
 High
SW turnover
—see commen ts
 Annexure 7, Section 7.3, questions 5 and 6 show that most SWs were multitasking between service streams-- diversion, NCS, offender reintegration, schools and community work) and
61% of SWs found this difficult or very difficult
 44% of
SWs on surveys say they do not have all the training required
(Annexu re 7, question
18)
 Annexure 8, question 4 on the Supervisor
Survey shows that 62% of supervisors thought the SWs should NOT be multitasking between service streams, rather the SWs on NCS should be fulltime NCS specialists
 There were times when SW turnover was a very serious threat to the project —in March 2012 there were 16 vacancies (36% of sites) —and while it can be stated that by August 2012 there were only 2 vacancies it remains that turnover is a significant programme threat
 Of the 34 sites surveyed (supervisors), more than half (18 or 53%) had challenges of SW turnover. Eight of those 18 sites or 44% were on the 3 rd or 4 th replacement which makes it very difficult to develop magistrate trust and to provide the continuity of service required to achieve the targeted referrals. See Appendix 8,
Figure 2 for example.
 There is a national SW shortage and high
© NICRO 2013 86
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
Professional SW expertise available at 45 sites
Overall rating (given in two indicators below) = 3
1-5 rating
Log frame
Indicator s
Baseline and
Target for
Indicators
Actual achievement s by 30 July
2012
Comments on final results including unintended impacts turnover at sites impacts on referral rates and partly explains the failure to achieve minimal referral targets (the supervisor must slow down or suspend referrals while training a new SW; magistrate confidence is weakened; the LSC is weakened)
 NICRO cannot compete with the DSD in terms of salaries, benefits such as bursaries and this caused frustration amongst certain NICRO supervisors and provincial managers who sometimes described themselves as ‘training
SWs for the DSD’
 For the reasons above and more, the availability of SWs and SAWs must be considered in a broader context owing to national shortages.
NICRO will have to develop a joint strategy with the DSD, DCS and other stakeholders.to respond to the challenge if NCS services are to be offered widely.
4 Aftercare and tracking
© NICRO 2013
0  Survey data on
936 clients
(Append ix 5) shows that only
13% of clients were tracked more than 7 months
 See
Figure
3.6 below as of
791 clients surveye d and of these
 Aftercare is essential to describing the delivery a professional service because it creates a support structure for the client to address further needs after completion of the initial programme but it was not regularly provided because it requires social auxiliary workers in support of the SW and that was seldom available (this is why 2 SWs per site are needed as stated above)
 Client tracking and aftercare persist as the weakest areas of delivery and yet these areas are critical components of a fully-fledged NCS service (see
Figure 3.6 below)
 In interviews and surveys SWs and supervisors admitted aftercare was not occurring and explained the reasons--see Figure 3.7 below
 The challenge of insufficient Aftercare and
Tracking, which persisted throughout the intervention, was identified at the time of the MTR, and might be described as the Achilles Heel’ of the project:
ï‚· First, a magistrate’s greatest fear with regard to NCS is that there will be insufficient
87
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
Professional SW expertise available at 45 sites
Overall rating (given in two indicators below) = 3
1-5 rating
Log frame
Indicator s
Baseline and
Target for
Indicators
Actual achievement s by 30 July
2012 up to six months, and
15% in the range of
7-12 months.
This shows that tracking seldom occurre d and more than half
(52%) were not tracked at all.
Of the remainin g 48%,
12% were tracked up to 3 months,
21% for when it did it was for a short period only.
Comments on final results including unintended impacts mechanisms for monitoring the offender (see question 10, Annexure 10)
ï‚·
Secondly, in the 1990s when DSD ran a
Pretoria-based diversion programme for youthful offenders, the programme collapsed because there was no monitoring and followup with some serious consequences (see
Basil King Interview, Annexure 25).
 DCS must be a partner in the delivery of NCS services (a vital recommendation of this report) but they cannot cope with their policing/correctional services/monitoring role. The old system described collapsed this way and little has changed — basically, Correctional Services has an insufficient budget for a monitoring and follow-up role.
Correctional services cannot simply be relieved of their duty and are ultimately responsible for monitoring and yet have proven over decades that they cannot deliver without support
 A DCS/NICRO partnership from national level is essential to NCS services but this was not consolidated even though there has been dialogue making this a critical focus for NICRO (see
Annexure 18)
© NICRO 2013 88
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
FIGURE 3.6: SW SURVEY DATA: HOW MANY MONTHS WAS CLIENT TRACKED AFTER
COMPLETING NCS?
35
Months
→
0 1-3 4-6 7-12 1+years 2+years 3+years Blank Spoilt
# →
% →
408
44
93
10
164
17
126
13
0
0
0
0
0
0
129
14
16
02
Findings: Excluding 145 blank and spoilt responses, there is data for 791 clients and of these more than half
(52%) were not tracked at all. Of the remaining 48%, 12% were tracked up to 3 months, 21% for up to six months, and 15% in the range of 7-12 months. This shows that tracking seldom occurred and when it did it was for a short period only. Figure 3.7 (below) explains why.
FIGURE 3.7: SW AND SUPERVISOR DATA: WHY TRACKING AND AFTERCARE NOT
OCCURRING 36
Reason aftercare was weak
Communications
Explanation
Most perpetrators are poor and have pay-as-you-go phones with changing numbers and not much airtime; landlines are also pay-as-you-go and frequently disconnected
Locating the client No permanent address given for tracking
Insufficient time and The SW has little or no time and an insufficient budget for visiting clients in the funds community
SW turnover
Transport
No assistance to
SW
The SW resigns and there is no follow-up with the client
The client does not have transport money to visit the office
The NICRO SW does not have a driver’s license (NICRO does have cars)
A shortage of social auxiliary workers for this support function
35 The survey data is given in Appendix 5; see also Chapter Two on methodology
36 Extracted from Interviews, surveys and focus groups with supervisors, social workers
© NICRO 2013 89
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
Expert alternative sentencing programme interventions
Overall rating (given in two indicators below) = 2
1-5 rating Log frame
Indicators
Baseline and
Target for
Indicators
Actual achievements by
30 July 2012
Comments on final results including unintended impacts
2
Largely achieved with a few shortcomings
All SWs trained and supervised according to the NICRO
PSP manual requirements and able to deliver a broad range of NICRO therapeutic programmes according to the supervision model
0
No suitably qualified
SWs per court who had been oriented, trained and supervised
NICRO training was indicated to be to the standards of the social profession: work

There is a strong supervisi on model that is part of the
Policies,
Standard s and
Procedur es
Manual
 A
Profession al Skills
Developme nt Initiative
(PSDI) was launched in
2010 with many skills building opportunitie s including intensive onsite
© NICRO 2013
 The log frame indicator is not sufficient for exposing the challenges including gaps in delivery owing SW turnover (they do not stay in positions long) and posts not filled or a new SW starting up
—all of which compromise the quality of the intervention. Those that were trained and became familiar with the intervention offerings departed with sufficient frequency that many NICRO sites were being drained of capacity.
 The supervision model and the superbly developed PSDI can still leave gaps as identified by both supervisors and SWs in their surveys
 Annexure 8 (see
Figures 6 and 7) shows that 77% of supervisors
90
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
Expert alternative sentencing programme interventions
Overall rating (given in two indicators below) = 2
1-5 rating Log frame
Indicators
Baseline and
Target for
Indicators
Actual achievements by
30 July 2012 coaching, video libraries, the NING knowledge forum, materials, reviews and information,
20 telecons to date reaching
299 people
Comments on final results including unintended impacts did not think the SWs had all the requisite skills and training to deliver in all needed areas (Perpetrators of
Intimate Partner
Violence, Sexual
Offences and Matrix are
3 examples) 37
 Staff were provided with NCS toolkits and trained on site by supervisors in NCS
(mentoring, training weekly to monthly) involving assessmen ts, review of programme
 Annexure 8, question 2.5 shows 13 supervisors, identifying a challenge with either: (a) new recruits lacking sufficient training (having missed formal opportunities) or
(b) SWs who were skilled but could not advance to provide certain needed programme through selftraining only (sexual offences, matrix, perpetrator programmes were cited)
 Annexure 7 shows that the SWs themselves wanted formal training opportunities to advance their skills so that they could address
37 Under the supervision model, the supervisors are to provide the skills but as indicated this did not occur to the level anticipated either
© NICRO 2013 91
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
Expert alternative sentencing programme interventions
Overall rating (given in two indicators below) = 2
1-5 rating Log frame
Indicators
Baseline and
Target for
Indicators
Actual achievements by
30 July 2012 content, facilitation, skills developme nt, critical thinking, restorative justice principles, legislation and lobbying and advocacy
The NICRO SWs demonstrated expert capacity since the risk factors that produce criminal behaviour were reduced through therapy:
 Of clients
936 surveyed
(24% of the
3,960 on programmes) on particular risk factors most will not reoffend
 A second survey of
SWs on the
Comments on final results including unintended impacts

 all
18) criminogenic factors risk identified amongst clients (see questions 2, 4, 7 8 and
While the NICRO professional skills development initiative is an example of a good management effort in the face of very limited resources, it is not sufficient to replace all formal training opportunities away from the workplace. These interactive opportunities are needed to produce a cadre of well skilled
SWs who can be retained to provide a large range of needed services (an essential strategic component when it comes to marketing the services)
Rather than formal training as an NICRO internal offering, it would be better to provide training with all involved departments and stakeholders as
© NICRO 2013 92
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
Expert alternative sentencing programme interventions
Overall rating (given in two indicators below) = 2
1-5 rating Log frame
Indicators
Baseline and
Target for
Indicators
Actual achievements by
30 July 2012 same 936 clients found agreement a (within small variance) that the risks were reduced
 A random sample of
237 clients
(25% of those surveyed;
6% of all clients) showed that only reoffended
9 between
2010-2012
(2% year) — per another indicator of expert care
 The reduction in risk factors
(Annex 6) for
96.3% of all
NICRO clients
Comments on final results including unintended impacts part of an integrated strategic effort to deliver services as recommended in this study recommendations)
(see
© NICRO 2013 93
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
Expert alternative sentencing programme interventions
Overall rating (given in two indicators below) = 2
1-5 rating Log frame
Indicators
Baseline and
Target for
Indicators
Actual achievements by
30 July 2012 suggests that most of those exposed to the programme received expert care
Comments on final results including unintended impacts
The biggest gap in terms of SW expertise was supervision of which these are indicators:
 Only 39% of the SWs
(or 40% if the blank response is discounted) really understood the clauses in the criminal procedure act relevant to their work. See
Annexure
7.3 question 4.
 On admission
© NICRO 2013 94
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
Expert alternative sentencing programme interventions
Overall rating (given in two indicators below) = 2
1-5 rating Log frame
Indicators
Baseline and
Target for
Indicators
Actual achievements by
30 July 2012 of the SWs in surveys
(Annexure
7, Section
7.3, question
20), 40% of the courts are not getting feedback on the treatment plan — a serious gap.
There was also widespread confusion amongst
SWs, as to when feedback is required.
After assessmen t and after treatment are the closest to correct and only 9 of 31
(29%) respondent s offered
© NICRO 2013
Comments on final results including unintended impacts
95
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
Expert alternative sentencing programme interventions
Overall rating (given in two indicators below) = 2
1-5 rating Log frame
Indicators
Baseline and
Target for
Indicators
Actual achievements by
30 July 2012 such replies.
Comments on final results including unintended impacts
2 All sites with resources and equipment
 All data and evidence corroborate including surveys, site inspections and project reports that the sites were resourced by 30 July
2012
 Not all sites received equipment and furnishings in a timely way but by the time of the evaluation they had what was required

Wherever NICRO utilised facilities provided by the courts, there was less of a therapeutic atmosphere than might be desired (no waiting areas, poor design, and minimum space) — sometimes shipping containers were used
(turned into offices).

The biggest gap was the lack of landlines in several offices (a situation not always in
NICRO control — courthouses did not always allow for this).
The SWs tried to use cell phones but these are a more difficult way to communicate and more expensive and this
© NICRO 2013 96
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
Expert alternative sentencing programme interventions
Overall rating (given in two indicators below) = 2
1-5 rating Log frame
Indicators
Baseline and
Target for
Indicators
Actual achievements by
30 July 2012
Comments on final results including unintended impacts contributed to the difficulty of providing
Aftercare and Tracking.
A national media and marketing campaign: increase public awareness of NCS and restorative practices
Overall rating (rating given in 5 components below) = 2
1-5 rating
Log frame
Indicators
Baseline and
Target for
Indicators
Actual achievements by
30 July 2012
Comments on final results including unintended impacts
2 1. A national media and marketing campaign that measures public attitude and opinion toward AS
Baseline was estimated at
44% public support for
NCS and target was
55% or more
First Phase
NICRO Public
Attitudes
Survey of 253 people: 54% of the public support NCS and rehabilitation
Note: and figures fluctuate the workshops and surveys were continuous the
 The public attitude surveys showed that public support for NCS exceeded targets and rose as high as 70% agreeing that ‘nonviolent’ offenders can be rehabilitated.
ï‚·
Second Phase:
740 surveys show
60% support
ï‚·
Third Phase surveys show 59-
67% support
ï‚·
Support ratings exceeded targets and were in the range of 59--70% public support
 There was never a point in the intervention in which the community resisted or protested against NCS services such that it can be said that services were welcomed by communities
 The support ratings are coming mainly form poor communities as do most
NICRO beneficiaries (mainly young, male and poor) and this suggest that amongst family members in these poor communities there is little overt resistance to NCS services (at least less than anticipated in project design).
 Those being surveyed are mainly responsible citizens interested in programmes that can alleviate crime and therefore there can be a
© NICRO 2013 97
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
A national media and marketing campaign: increase public awareness of NCS and restorative practices
Overall rating (rating given in 5 components below) = 2
1-5 rating
Log frame
Indicators
Baseline and
Target for
Indicators
Actual achievements by
30 July 2012
Comments on final results including unintended impacts bias in the figures
 Owing to the above it would be beneficial to identify in more precise terms the characteristics or traits of those opposed to restorative justice.
 That resistance is found in greater numbers amongst the middle class and wealthy was suggested by some informants and might be a hypothesis to test in the future
1 2. A national media and marketing campaign that
(b) promotes a culture of human rights a within criminal context, educates primary stakeholders and assists the public to understand the of benefits restorative justice
Target:
Entrench restorative justice concepts in the national psyche and educate public
NICRO arranged for the
Newsclip Media
Monitoring Service to provide a review of NCS coverage (see Appendix
11) which identified a nearly R10 million value in radio, TV and newspaper coverage 2008-2013 as follows :



R4,210,150.82 broadcast
R2, 902,050.17 in print
R2, 663,059.44 online
 Total value:
R9 775 260.43
 Reach =
2,118,793
 Print coverage
(alone) could
 See Annexures 11 (11.1. and
11.2) and Annexure 11.3 for more details
 The NICRO media and marketing work was comprehensive (broadcast, print, and online) and of such a quality that it had impact (as evaluated by Newsclip) and assisted both primary stakeholders and the public to better understand the services.
 NICRO first commissioned some professional support for marketing and then gradually assumed responsibility for it.
Total Media initiated a campaign Apr 2009-Feb 2010 starting with media roundtables that introduced selected media to the NCS
© NICRO 2013 98
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
A national media and marketing campaign: increase public awareness of NCS and restorative practices
Overall rating (rating given in 5 components below) = 2
1-5 rating
Log frame
Indicators
Baseline and
Target for
Indicators
Actual achievements by
30 July 2012
Comments on final results including unintended impacts also be valued in terms of advertising cost and the PR
Institute of
South Africa estimated this at more than R4.6 million in advertising value with a consumer reach of 2, 118,793 people
 NICRO used its own networks (45 sites and the national office) to distribute booklets explaining NCS services —
Moments of
Growth, Voices of
Change
(sentencing stories) concept and generated a media conversation, and then
NICRO worked on a pro-rata basis in Years Two and Three with Redline with NICRO assuming marketing responsibility in the final years.
NICRO demonstrated the capacity to run a good campaign and the stakeholder and public videos produced by the project were of a very high standard —high enough to air on national television. This suggests that had NICRO assumed responsibility earlier
(rather than contracting out) some expenses might have been saved.
 Brochures and
NCS cards explaining the service distributed from both national and all 45 sites
2 3. Community
Engagement
Activities
No indicators other than to engage the community through events and activities
(baseline = 0)
© NICRO 2013
 8,800 community members reached by 31
July 2012
 The Supervisor survey showed that community
 At some sites the media published stories about the services NICRO renders
(sometimes following community engagement activities) and supervisors reported receiving positive comments from the public
99
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
A national media and marketing campaign: increase public awareness of NCS and restorative practices
Overall rating (rating given in 5 components below) = 2
1-5 rating
Log frame
Indicators
Baseline and
Target for
Indicators
Actual achievements by
30 July 2012
Comments on final results including unintended impacts engagement workshops were conducted at all sites except for
Durban (44 of
45 sites)
 Community outreach events commonly occurred at schools, malls,
CPF meetings, during public holiday events where brochures,
NCS cards and other materials were distributed following these publications
 Considering that nearly all sites had but one SW who normally multi-tasked between projects
(NCS and others) and one supervisor multi-tasking between sites, the human resources are insufficient at each site for marketing activities to the strategic level required to move the national from punitive thinking about justice to thinking of justice in more restorative terms —NICRO needs either more staff at each site for this engagement or more partners
(see the bullet below as well)
 Annexure 8;
Section 8.2.9 shows that 11 of
13 supervisors
(85%) were convinced or highly convinced that the described events and activities were promoting a culture of human rights and restorative justice because there would be an increase in referrals after such presentations or members of the
 Scaling back from 45 NCS sites to about half as many or even fewer; and mainstreaming NCS into a regular NICRO offering; should release some energies for marketing NCS as part of the range of NICRO offerings and building community level understanding and participation
© NICRO 2013 100
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
A national media and marketing campaign: increase public awareness of NCS and restorative practices
Overall rating (rating given in 5 components below) = 2
1-5 rating
Log frame
Indicators
Baseline and
Target for
Indicators
Actual achievements by
30 July 2012
Comments on final results including unintended impacts public would come to visit the
NICRO office but the only hard evidence of impact was given above in terms of Public
Attitude Surveys where the majority of the public indicated support for NCS services
 Annexure 7, question 16: 31
SWs identified the top two indicators of improved public attitudes: (a) increased referrals with
34% of SWs saying this; and
(b) positive verbal feedback from either the community or the LSC with
30% saying this
1 4. Develop and implement education, information and outreach programmes for magistrates
No previous materials
 Magistrate toolkit developed (2009-
2010)
 4,175 court stakeholders trained —
(sensitising court stakeholders supports the magistrate
 As early as the MTR and persisting into the evaluation, magistrates stated that no more educational opportunities, workshops or publications for magistrates are required and such activities are unlikely to
© NICRO 2013 101
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
A national media and marketing campaign: increase public awareness of NCS and restorative practices
Overall rating (rating given in 5 components below) = 2
1-5 rating
Log frame
Indicators
Baseline and
Target for
Indicators
Actual achievements by
30 July 2012
Comments on final results including unintended impacts referrals change referral levels:
 The rate of magistrate referrals cannot be improved with educational or outreach activities because the challenge that needs to be addressed is systemic: magistrates are under pressure to reduce the court roll and this requires that they speed up case processing times and therefore they want
NICRO to speed up the assessment time owing demands on them (79% of magistrates surveyed said this;
See Annexure 10, question 7;
See also the Basil King interview Annexure 25, question one )
 Most magistrates want the assessment back in 2 weeks or less and in one day in some cases —see question 6 on the
Magistrate Survey, Annexure
10
 Magistrates and court stakeholders are so overworked and pressurized that most of these project materials end up in a drawer
3 5. Develop and implement a
Lobbying
Programme aimed at the
Parliamentary
Portfolio and
Standing
No previous materials on
NCS
Seven submissions made to parliamentary portfolio committees (mainly for
Correctional Services):
 2009-2010 —four submissions
 Submissions were not so specific to NCS: these were made to parliamentary portfolio committees and framed within an overall advocacy effort to see more of the budget allocated to prevention as opposed to police,
© NICRO 2013 102
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
A national media and marketing campaign: increase public awareness of NCS and restorative practices
Overall rating (rating given in 5 components below) = 2
1-5 rating
Log frame
Indicators
Baseline and
Target for
Indicators
Actual achievements by
30 July 2012
Comments on final results including unintended impacts
Subcommittees  2010-2011-Three submissions courts and jails
 2011-2012--
Lobbying and advocacy (written or oral) to portfolio committees to promote NCS
 The advocacy approach described above was well executed but appeared to have had little impact on sustaining the project or increasing referrals such that this particular logframe indicator appeared insufficient to the task of sustaining and replicating NCS services
 NICRO had approached the
DOJCD at national level and had worked together with that department on piloting NCS in
2007 such that there is evidence of partnership building efforts — but the final consolidation into an inter-departmental, intersectorial, multidisciplinary and sustainable plan was not achieved (see Annexures 18,
19, 20 and 21 –interviews at national--which suggest that a partnership had not yet been consolidated with some important decision-makers)
 DCS might help fund NCS services but achieving this outcome requires advocacy and lobbying at national level but this was not done sufficiently as indicated in the focus group with
DCS National (see Annexure 18)
 Only DSD national appeared to have no interest in NCS for
‘adults’ (short-sighted thinking) but DOJCD, DCS, JCPS, and
© NICRO 2013 103
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
A national media and marketing campaign: increase public awareness of NCS and restorative practices
Overall rating (rating given in 5 components below) = 2
1-5 rating
Log frame
Indicators
Baseline and
Target for
Indicators
Actual achievements by
30 July 2012
Comments on final results including unintended impacts
DSD Provincial stated they were very interested in NICRO approaching them with some formal partnership proposals.
A newly developed indigenous-based diversion programme available (Eerstehoek);
Overall rating (given in 2 components below) = 2
1-5 rating Log frame
Indicators
Baseline and
Target for
Indicators
Actual achievements by 30 July 2012
Comments on final results including unintended impacts
4
Very limited achievemen t, extensive shortcomin gs
330 offenders diverted through the indigenous based diversion programm e projects by end
0
No indigenous justice diversion programme had been available before the
NICRO initiative
 23% (N=76) of target attained by end July 2012 as follows:
 Traditional courts: 10
(13%)
 Target not attained
 Most of the target attainment was
NOT via the traditional courts but owed to school, community and magistrate referrals
 Magistrate courts: 38
(50%) including 2 via the DSD
 Most referrals started in the third year of the programme (very slow start up)


Schools: 17
(22%)
Community:
11 (14%)
 NICRO dropped work in the traditional courts that did not refer including eNikwakuyengwa (1 referral) and concentrated on the ones that did refer —Eerstehoek
Magistrate Court and the tribal courts of eMbhuleni,(3 referrals)
Mandlamakhula (1 referral) and
Mpisikazi (5 referrals)
 The school referrals owed to a crime prevention programme in two schools 38 which then referred at-risk youth to NICRO and then these youth became mentors to
38 The High Schools: Bantfwabetfu and Dlomodlomo
© NICRO 2013 104
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
A newly developed indigenous-based diversion programme available (Eerstehoek);
Overall rating (given in 2 components below) = 2
1-5 rating Log frame
Indicators
Baseline and
Target for
Indicators
Actual achievements by 30 July 2012
Comments on final results including unintended impacts other at-risk youth
 While some traditional practices of restorative justice could be incorporated into programmes — the intellectual understanding requisite for traditional courts to be referring onto NICRO programmes was limited
 The failure of the indigenous justice model does not mean that
NICRO failed since upon identifying the challenges in 2011, the organization then focused on schools and community work, which was effective in increasing referrals to NICRO programmes
(this locates an appropriate model for working in such areas — community based work).
2 70% of recommen dations for indigenous based diversion accepted by the courts/site s by end of funding
0
No indigenous justice diversion programme had been available before the
NICRO initiative
 The sentence recommendation rate of acceptance was
98% (see Figure
3.5 further above)
 Despite a high acceptance rate, this is a perfunctory measure of performance that involves only 10 referrals such that a 2 rating does not capture the challenges
 The area was too big in both population and distances for
NICRO to provide an effective intervention either in terms of
NCS or diversion from traditional courts. NICRO served 3 traditional courts but there are 12.
School and community interventions provide a better way in rural communities to make impact on more people.
 The LSC thought NICRO should rather intervene in schools and
© NICRO 2013 105
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
A newly developed indigenous-based diversion programme available (Eerstehoek);
Overall rating (given in 2 components below) = 2
1-5 rating Log frame
Indicators
Baseline and
Target for
Indicators
Actual achievements by 30 July 2012
Comments on final results including unintended impacts communities to prevent crime and divert people out of the system and to advocate that national and provincial departments provide training in the Traditional Affairs
Act (see Annexure 16).
 The magistrate and the case flow manager (NPA) thought that court cases had been reduced owing to the schools work and strongly recommended that this be the focus (see Annexure 16)
Develop a knowledge base of NCS systems and practices to further public and sector understanding and to promote awareness
Overall rating (given in 5 components below) = 1
1-5 rating
Log frame
Indicators
Baseline and
Target for
Indicators
Actual achievements by 30 July 2012
Comments on final results including unintended impacts
1
1
Develop accessible online forums for the collection and dissemination of research on sentencing and offender reintegration
0
No online forums regarding NCS
Research into NCS systems and related practices
0
No such research in SA specifically
A Facebook Group
(278 members)
Ning
Justice
Criminal
Online
Forum(@153 members; active)
30+
Newsletter: reach of 390 people
 Indigenous Justice
Research (Apr
2009-2010)
 Stakeholder
Seminar on NCS and conference
 The traffic is was relatively low in terms of numbers but the forums attract researchers and media interest such that there are
‘spinoffs’ for disseminating findings
 A large number of research products were generated that enable other stakeholders and organizations to learn about the practice
© NICRO 2013 106
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
Develop a knowledge base of NCS systems and practices to further public and sector understanding and to promote awareness
Overall rating (given in 5 components below) = 1
1-5 rating
Log frame
Indicators
Baseline and
Target for
Indicators
Actual achievements by 30 July 2012
Comments on final results including unintended impacts reports (Apr 2009-
Mar 2010)
 GTF DFID reports
(annual and quarterly 2009-
2012
 The data from the NCS services offers very good value internationally and within South
Africa
 3 phases of Public
Attitude Research-
One-July 2010;
Two--July 2011-
2012 Three--2011-
2012
© NICRO 2013
 Mid-Term Review
(Sep 2010)
 Telecon
Knowledge
Sessions Report
(Oct 2010)
 NCS Sentencing
Stories Booklet
(Dec 2011)
 SW attitudes and values research
(Feb 2011)
 NCS Colloquium
Transcripts (Mar
2011)
 Traditional Leaders
Research (Aug
2011)
 NCS Noncompliance research (2011)
 Case Studies
Research (Jan
2012)
 Magistrate
107
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
Develop a knowledge base of NCS systems and practices to further public and sector understanding and to promote awareness
Overall rating (given in 5 components below) = 1
1-5 rating
Log frame
Indicators
Baseline and
Target for
Indicators
Actual achievements by 30 July 2012
Comments on final results including unintended impacts
Attitudes and
Values Toward
Sentencing (Jun
2011-July 2012)
1 Materials produced
No previous history of this
Baseline = 0
 NCS Toolkit (Apr
2009-March 2010)
 NCS Brochure
Redesign (Aug
2010)
 NCS Videos (Nov
2010-May 2011)
 NICRO promotional materials---posters, bracelets and bags
 Moments of
Growth booklet
(Feb 2012)---letters and poems from
NCS offenders
 SiSwati Indigenous
Justice Brochure
(Jan 2012)
 All materials were of a very high standard
 In general most materials were of high value and effectively utilized —see Figure 3.8
Below)
 It is not certain that bracelets and bags offered high value for promoting services (some of these items are requested and desirable but do not necessarily advance delivery in the most needed ways)
1 Print
Media and This was covered and rated as ‘1’ in
2.3.5 above and Annexures
11.1 - 11.3 offer a detailed listing of the achievements in print and media
 This was covered in 2.3.5 above and
Annexures 11.1 -
11.5 offer a detailed listing of the achievements in print and media
 This was covered in 2.3.5 above and need not be repeated but entered into the ratings
 Annexures 11.1 - 11.5 offer a detailed listing of the achievements in print and media
© NICRO 2013 108
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
FIGURE 3.8: Some NICRO Publications And Materials That Facilitated The Delivery Of
NCS Services
Date Publication Explanation Value
Mar
2010
NCS Toolkit Offers participating stakeholders an explanation of how the services work
Useful for introducing the services to magistrates, LSC members and any stakeholders who might participate in delivery
Aug
2010
2010-
2011
NCS
Videos
Useful for visualizing the kind of impact NCS can have and the expertise that goes into it
Dec
2011
NCS
Sentencing
Stories
Booklet
Case studies of how NCS services changed lives and prevented reoffending
These are success stories that really help to flesh out the value of the services
FIGURE 3.9: Formal Research Publications And An Explanation Of Value
Date Publication Explanation Value
Mar
2010
A series of videos in which participants, judges, magistrates and offenders explain what the
NCS services are and their impact
Jul
2010-
2012
Indigenous
Justice
Research
Public Attitude
Research
Desktop research into understanding justice issues in traditional areas
Understanding public attitudes toward NCS services
Useful as an initial introduction as to how to develop NCS services in a traditional area
The is research in phases to identify the issues that might arise when trying to develop the services in a community and also to monitor changing attitudes
Sep
2010
Sep
2010
NCS
Brochure
Offender Non
Compliance
Research
Mid-Term
Review
Explains the services in more general terms
Understanding the issues around Non compliance
The first review of the
NCS services
Useful for eliciting public understanding of the services
This would help those setting up NCS services to understand how and why non-compliance occurs
Focuses on the challenges when first setting up NCS services and some of the early achievements of NICRO
Feb
2011
Social Worker
Attitudes and
Values
© NICRO 2013
How do the attitudes and values of the Social
Workers impact on an
Valuable for understanding the kind of training that is required to deliver the
109
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
Date Publication
Research
Aug
2011
Research into the role of traditional leaders
2011 Offender Non
Compliance
Research
Ongoing from
Jun
2011
Magistrate
Attitudes and
Values toward
Sentencing
Jan
2012
Case Study
Research
Explanation offender’s opportunity to change
Value service
Analyses the workings and challenges of traditional courts in terms of NCS
Valuable for understanding the fit between NCS services and indigenous justice practices and why the project shifted toward more community work
Understanding the issues around Non compliance
Explains the range of magisterial attitudes toward sentencing
Can help those setting up NCS services to understand why noncompliance arises
Helps one to understand why some magistrates refer to NCS services and not others
An in-depth examination of four offenders on NCS programmes
Generates a detailed understanding of offering NCS services to four offenders including how and why they offended and what was required to reform them
WEIGHTING THE FINDINGS
The evaluation data was also categorised into (a) budget, expenditure and value for money data (33%); quantitative data from the project (33%); and qualitative data (33%). If a DFID
Achievement Rating Scale applied to these three different forms of groupings, would it produce a different kind of conclusion? They weighted very closely but here scoring a fraction of a percentage better than in the previous section to score a 1.83 rating here as illustrated in
FIGURE 3.10 (below). The main reason for the modest difference is the strong weighting on budget, expenditure and value for money data which shows that good value for money was delivered —South Africa was presented with a new NCS service with social workers, programmes, resourced sites, local steering committees, and other components on a budget that the government could not have managed in terms of salaries alone.
© NICRO 2013 110
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
FIGURE 3.10: Weighting The Findings
TYPE OF
DATA
1 BUDGET,
EXPENDITURE
AND VALUE
FOR MONEY
DATA
WEIGHTING RATING
33% 1.5
TYPE OF DATA
 An independent budget and expenditure analysis by a certified public accountant indicates no shortcomings (see certification in Annexure 12)
 Good value for money was delivered in terms of the number of outputs, which was beyond what might be expected from a government department in terms of salary expenditure alone
 See Annexure 26 for Value for Money Calculation
 Rather than over expenditure, some project costs and overheads were not included in the funding
(NICRO made certain voluntary contributions such as the analysis of public opinion surveys)
 The cost of aftercare and tracking, which is essential to delivery, appears to have been underestimated (resources, partnership building activities, research and training costs) although it is complicated by other issues (e.g., SWs without driver’s licenses, etc.)
 Target attainment was not achieved for most sites
(only two had reached 10X a month referral at the time of this study and at most 7 might by project end)
2 QUANTITATIVE
DATA
33% 2  936 clients (a 24% sample) were given a diagnostic test and their SWs completed a survey on each of those clients to find that the risk factors for reoffending had been reduced (only a 1-3% variance on a scorecard for reoffending between
SW and client) indicating that the core behaviours that led to the offense were diagnosed and addressed through cognitive behaviour therapy
 The service statistics indicate a significant shortfall in terms of target attainment: NCS services were provided to 3,960 offenders by 31 Jul 2012 (66% of target and unlikely to reach more than 75-80% by project end)
 A referral system for NCS services was developed, established and functional as: (a) 23 Local
Stakeholder Committees (LSCs) and (b)14 Case
© NICRO 2013 111
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
TYPE OF
DATA
3 QUALITATIVE
DATA
© NICRO 2013
FIGURE 3.10: Weighting The Findings
WEIGHTING RATING TYPE OF DATA
33% 2
Flow Management Committees around the country.
 37 of 45 LSCs or 82% were actively reviewing and promoting NCS services
 Only 2 sites (4%) achieved the monthly target of
10 referrals per month by 31 July 2012 (see
Annexure 3) and 7 sites (16%) at most might meet the 10X a month target for magistrate referrals by project end (see Annexures 3.1 and
3.2 for additional details on target attainment)
 Public and official awareness that there were viable and better alternatives to imprisonment was indicated by a R10-million-value media campaign and outreach activities at 45 sites involving magistrates and public figures
 Survey data from magistrates and NICRO SWs, supervisors and provincial managers indicated a conundrum with the assessment time (NICRO requires up to 30 days and the magistrates want it within 2 weeks and some as little as a day)
 82% of surveyed and interviewed magistrates
(Annexure 10) indicated that they had been presented with a more credible range of sentencing options so that they can sentence to NICRO
 Site based observations, client interviews, client stories, staff interviews, focus groups at sites and client interviews indicated significant impact on reforming clients
 Focus groups with local stakeholder committees at five sites showed that despite any problems the capacity to meet and engage in support of NCS sentencing options had occurred
 Site visits showed that all sites were functional but some were lacking in resources that would have facilitated betters aftercare and tracking (landlines, vehicles)
 SW turnover was never well addressed complicating and slowing delivery at many sites and most available SWs were multitasking between NCS services and other projects (at only
112
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
TYPE OF
DATA
SCORE
FIGURE 3.10: Weighting The Findings
WEIGHTING RATING TYPE OF DATA
1.83
10 of the 31 sites was a SW devoting more than
20 hours to NCS).
 Most supervisors thought the SWs should not be multitasking between service streams, rather the SWs on NCS should be full-time NCS specialists (suggesting challenges with providing a consistent service from intake to aftercare and tracking)
 In interviews (and on some surveys) magistrates made it clear that they do not want to delay long for an assessment —most wanted assessments within 2 weeks and some within one day since they are under pressure to reduce the court roll)
 Interviews with Pretoria-based departments made it clear that partnership building at national level was weak (see Annexures 17-25)
DID THE PROGRAMME ADDRESS THE INTENDED TARGET GROUP? WHO WERE THE
DIRECT AND INDIRECT BENEFICIARIES?
The intended target group and the direct beneficiaries were perpetrators who were provided with therapy to reduce the risk of repeat offending. NICRO has no control over the sex, race or ages of the sentenced--only a magistrate can sentence to NCS. However, in South Africa it is well known that the majority of offenders are young (85% under 40), black (75%), poorlyeducated (58% with less than a high school education), and male (80%). See FIGURE 3.11
(below):
© NICRO 2013 113
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
FIGURE 3.11: Racial, Gender And Age Composition Of NCS Clients To 31 July 2012 39
SEX AGE RACE GRADE COMPLETED
(EDUCATION)
M F 15-
18
19-
29
30-
39
40-
49
50
+
Afric an
Asi an
Colou red
Whi te unkno wn
0 1-
11
HS
Diplo ma
Terti ary
80
%
20
%
13
%
49
%
22
%
10
%
7
%
75% 01
%
18% 6% 24% 2
%
56
%
14% 4%
Other primary beneficiaries included the household members of the perpetrators. According to
NICRO intake statistics, about 40% of the perpetrators contributed to the family income through casual, formal or self-employment while 50% were unemployed and the rest were scholars (mostly too young to contribute). Amongst the poor, even an unemployed male member of the household usually contributes to the family pool of labour (e.g., gardening, livestock) and once imprisoned there are costs instead of benefits as the incarcerated is then dependent on the family for outside support. The NCS Services also help with long term crime prevention since the reformed perpetrator is a better role model for children, family members and friends (e.g., an incarcerated parent or an unreformed parent disposed to crime are risk factors for the child offending later in life).
The secondary beneficiaries include:
ï‚·
Local communities
—who take receipt of ex-offenders with a much lowered risk for repeat reoffending (2-4% per year of NICRO clients returned to crime while normal recidivism rates are in the range of 50-90%)
ï‚·
All South Africans —taxpayers, government departments and citizens who were provided with alternatives to the high social and economic costs of imprisonment
ï‚·
The Global Community, which benefits from NICRO modelling best practice that can be shared.
39 NICRO (July 2012) Integrated National Statistics (NCS statistics extracted by Regan Jules-
Macquet, project manager)
© NICRO 2013 114
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
On the last two points, NICRO promoted new thinking (reconciliation) and modelled a way to change offender behaviour that is being shared both within SA and internationally. As to SA, evidence has already been provided in this report that a cultural shift toward communities taking more responsibility for its own societal dysfunctions that are producing criminals has begun such as LSCs at 45 sites working to reform perpetrators (Section 2.3.2 above) or through public outreach (TV, radio, publications, and social media) as shown in Section 2.3.7 above. Further to this, 936 clients (24%) were asked 10 questions on the value of NCS services for South Africa to find that 98% of them said that such services are needed in South
African since they help to (percentages saying this indicated):
ï‚·
Improve government service delivery (90%)
ï‚·
Prevent crime through rehabilitation (91%)
ï‚·
Alleviate poverty (82%; this is mainly by holding families together)
ï‚·
Enable rehabilitation (94%)
ï‚·
Benefit the disadvantaged (87%; referring mainly to the poor, uneducated and those disadvantaged by history) 40
ï‚·
Increase respect for human rights (95%)
ï‚·
Increase respect for the law (96%)
ï‚·
Create a safer South Africa (95%)
ï‚· Improve on South Africa’s sentencing options (93%).
41
40 See Appendix 5 for the spread sheet of responses on this survey; note also that the majority of NICRO clients are the disadvantaged
41 See Appendix 5 for the spread sheet of responses on this survey
© NICRO 2013 115
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
WHAT DIFFERENCE HAS BEEN MADE TO THE LIVES OF THOSE INVOLVED IN THE
PROGRAMME?
Some of the impacts noted by clients were given in terms of Section Three of the Client Survey
(as discussed in Section 2.5 immediately above). This section provides more precise details on client impacts in terms of:
ï‚·
Survey One (clients with SW verification): 10 questions concerning impacts on reducing client’s criminogenic risk factors
ï‚· Survey Two (clients with SW verification): 10 questions on the client’s life situation owing to NCS
ï‚·
Evaluator interviews with 34 clients at sites (additional detail)
ï‚·
Findings on Eerstehoek Clients (diversion at Eerstehoek)
In brief, 94% of the offenders on NCS services experienced a reduced risk for reoffending based mainly on criminogenic thinking and behaviour patterns (Survey One) but in terms of some situational factors (Survey Two).
Findings On Survey One: 10 Questions On Criminogenic Risk Factors For Reoffending
First, refer to FIGURE 3.2 (further above) that presented the general findings on 18,720 responses to the diagnostic survey on criminogenic thinking patterns for 936 clients (24% of intake) and again with their SWs but the latter referenced their responses to extensive project documentation.
42 Survey One showed that the SW findings correlated to the client selfassessments to within a 0-6% range of variability dependent on the question (2.7% on average). The high level of corroboration was such that it is strongly indicated that the criminogenic risk factors for reoffending were reduced.
42 The SW used court reports, programme pre-tests and post-test, process notes, anecdotal data from clients, aftercare and tracking reports and group work reports
—these are confidential documents but the SW referred to these to rate client progress on the survey (this could then be checked against the client self-assessment)
© NICRO 2013 116
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
FIGURE 3.12 (below) is illustrates that owing to the programmes, counselling or therapy, 90% of surveyed clients were able to:
ï‚·
recognise negative triggers;
ï‚·
take more responsibility for their behaviour;
ï‚·
employ skills to get along with people better;
ï‚·
handle daily problems better;
ï‚·
overcome behavioural problems;
ï‚·
take better decisions (including one to not reoffend);
ï‚·
And to make changes such that they will not reoffend.
In addition, 85% of clients felt more emotionally stable, which is 5-10%% fewer reporting this benefit than some of the benefits but that fits with the findings since this is a more difficult area in which to make an impact given the history of arrest, trial and conviction and the emotional instability that attends to it.
FIGURE 3.12: Client And SW Agreement Levels: The Impact Of NCS Services 43
CLIENTS AND SWs THAT AGREE WITH THE
The programmes, counselling or therapy that I attended… clients agree
STATEMENT↓
% sws agree %
1 …were of benefit to me (929) 877 94% (929) 874 94%
% difference↓
0
2 …helped me to recognise negative triggers in my life
(931) 875 94% (925) 851 92% 2%
3 …helped me to take responsibility for my behaviours
(934) 887 95% (930) 855 92% 3%
4 …taught me skills that I can use to get along with people better
(933) 880 94% (930) 846 91% 3%
43 Agree and strongly agree are collapsed as ‘agree’ in this table and this is a better way to express the findings since clients and SWs had difficulty distinguishing the two and this accounts for much of the variation in the tables in the Appendix.
© NICRO 2013 117
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
The programmes, counselling or therapy that I attended…
5 …helped me to handle daily problems better
CLIENTS AND SWs THAT AGREE WITH THE
(933) 876
STATEMENT↓
94% (930) 817 88%
% difference↓
6%
6 …helped me identify ways to overcome behavioural problems
(932) 868 93% (929) 837 90% 3%
7 …helped me to feel more emotionally stable
(934)
8 …helped me to make better decisions
(932)
9 …helped me to take the decision to not reoffend again
(932)
794 85% (927) 794 86%
888 95% (929) 856 92%
879 94% (930) 850 91%
1%
3%
3%
1
0
…helped me to make changes such that I will NOT reoffend
(934) 881 94% (927) 841 91% 3%
Findings On Survey Two : 10 Questions On The Client’s Situational Risk Factors Owing
To NCS
Annexures 6.1 and 6.2 provide details on the survey of 936 clients (24% of all clients) and their SWs, which included two main sections, one of which was not presented in the general findings (further above) because it was designed to estimate impact in terms the situational risk factors for reoffending (e.g., poverty, drugs, negative relationships, unemployment etc.). A client at reduced risk of reoffending in cognitive terms should be managing their life more responsibly and this warrants an effort to measure this. However, such factors cannot be measured precisely since they are a consequence of therapy and not addressed directly by it or measured precisely in project documentation. As in the preceding section, the clients selfevaluated and the SW’s made use of project documentation to complete the identical survey questions. Predictably, the ‘not sure’ responses and ‘incompletes’ were higher in this data set
(indicative of the explanation above and owing to weak tracking and monitoring as discussed elsewhere). The range of difference between client and SW responses was in a small range of 0-16% dependent on the question but when the scores are converted to categories of low,
© NICRO 2013 118
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013 medium and high risk for reoffending (a scorecard), the range in scores between client and SW is very small (0.5% to 9%). See findings in the bottom right hand corner of FIGURE 3.13.
FIGURE 3.13: Did The NCS Services Reduce Situational Risk Factors?
Percentage
Range of
Criminogenic
Risk Factors
Client Self-
Assessment
N=928
# in range
% in range
SW Assessment of
Client N=902
# in range
% in range
Per cent difference:
Client self-assessment Vs. SW assessment
Per Row Across shaded areas
(Scoring Card)
↓
0-20% (very low risk) →
630 68% 466 52% 16%
9%
21-40% (low risk) →
256 28% 318 35% 7%
41-60%
(medium risk)
→
61-80% (high risk) →
81-100% (very high risk) →
TOTALS →
30
7
5
3%
0.5%
0.5%
109
8
1
12%
0.5%
---
0%
0.5%
9%
0.5%
Incomplete →
Blank →
Spoilt →
928 100%
(4)
(2)
(2)
--
--
--
902
(31)
(1)
(2)
100%
(3%)
--
--
FINDINGS: % IN EACH RISK CATEGORY
↓
Risk Selfassessed
SWassessed
Low →
Medium
→
High →
96%
3%
1%
87%
12%
0.5% TOTAL
PARTICIPANTS
IN SURVEY
(936) (936)
-- 100% 100%
That client self- scores were consistently higher than the SW scores on all ten questions indicates that client views of their own progress (in terms of situation) is more inflated than SW
© NICRO 2013 119
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013 views of clients. This was predicted (and affirming) since a professional assessment is more nuanced and the SW assessments were referenced to case files and project materials. A further indication that client self-assessments are more inflated than SW assessments of clients is that few clients offered ‘do not know’ responses (see ‘not sure’ column in Annexure
6.1) but when data or information was not available to assess the client; the SW indicated this as not available. For the reasons just cited, the SW assessment of the 936 survey clients will be utilised to express the findings in terms of percentages of clients at reduced situational risk for crime:
ï‚·
80% were engaged in more productive activities
ï‚·
64% were better able to manage their finances
ï‚·
84% experienced improved relationships
ï‚·
89% were taking more responsibility for their actions
ï‚·
82% did not think that society owed them a better life
ï‚·
76% were not abusing alcohol
ï‚·
79% were not abusing drugs
ï‚·
85% were no longer abusing relationships
ï‚·
90% were no longer thinking about engaging in criminal behaviour
ï‚·
78% did not need additional support to prevent reoffending
FIGURE 3.15 (below) illustrates both client and SW responses (see Annexure 6.1 for the complete spread sheet): 44
44 Agree and strongly agree are collapsed as ‘agree’ in this table and this is a better way to express the findings since most clients and many SWs reported in site interviews difficulty in distinguishing the two and therefore this method better described the variation.
© NICRO 2013 120
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
FIGURE 3.15: The Impact Of NCS Services On Life Situation Of Clients And SW
Verification
My situation owing to NCS is such that…
CLIENTS AND SWs THAT AGREE WITH
THE STATEMENT
CLIE
NTS
AGR
EE
% SWS AGR
EE
%
% differen ce
↓
1 I am engaged in productive activities
2 I am better able to manage my finances
(929) 799 86% (910) 728 80%
3 My relationships have improved (family, friends, coworkers…)
(932) 857 92% (908) 759 84%
6%
(930) 771 83% (910) 581 64% 19%
8%
4 I can accept responsibility for my actions
5 I do not think society owes me a better life
6 I am not abusing alcohol
7 I am not abusing drugs
(931) 907 98% (908) 809 89%
(927) 817 88% (909) 747 82%
9%
6%
(935) 832 89% (908) 691 76% 13%
(928) 865 93% (907) 716 79% 14%
8 I am not abusing relationships
9 I no longer think about engaging in criminal behaviours
(931) 885 95% (909) 776 85% 10%
(932) 897 96% (907) 813 90% 6%
7% 1
0
I do not need additional support to prevent reoffending
(925) 786 85% (907) 703 78%
FINDINGS ON EVALUATOR’S SITE-BASED INTERVIEWS WITH 34 CLIENTS
Prior to or during surveying at sites, the evaluator visited 7 of 45 project sites (16%) that included research processes with 34 clients with more intensive formal interviews with 30 of these (see asterisks):
1. Cape Town (4 clients helped to pilot survey forms)
2. Welkom, Free State (6 client interviews)*
3. Tembisa, Gauteng (5 client interviews)*
4. Shoshanguve, Gauteng (5 client interviews)*
5. Sekgosese, Limpopo (7 client interviews)*
6. Eerstehoek, Mpumalanga (5 client interviews)*
7. Pretoria (2 client interviews)+
© NICRO 2013 121
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
Five of the six sites with an asterisk were those targeted for 5 client interviews each. In addition, two clients were interviewed in the Pretoria office, which was the base of operations for meeting with key government departments. Of the 34 interviewed clients, 15% had completed programmes and 53% were in aftercare such that 68% had significant experience with programme. The sample was also typical of those on the programme (mainly poor, uneducated, young, black males):
ï‚·
2 Caucasians, one Indian, 2 Coloured and the rest Black (85%)
ï‚·
Females (30%) and Males (70%)
ï‚·
79% with less than a high school education
Annexure 13 details the comments and findings from these 34 interviews on client impact including data per site, all of which corroborate the client survey findings. FIGURE 3.16
(below) offers a summary table of the findings and then bulleted below are some particular findings within the data set that furnish a more nuanced understanding.
FIGURE 3.16: Summary Of Client Interview Data (30 Clients)
QUESTION RESPONSES (ALL SITES)
What is your risk of reoffending (30 respondents)?
What factors led to offending in the first place (30 respondents; each could identify more than one factor)?
Do you have more respect for human rights and rule of law (30 respondents)?
No risk--20 (67%) Low Risk —10 (33%)
A life skills deficit —28 (93%) Poverty-07 (23%)
Substance Abuse-04 (13%) Self Defence: 01 (3%)
No = 4 (13%) Yes = 26 (87%) (those that said ‘no’ had a bad experience with the law, especially upon arrest)
No = 1 (3%) Yes = 27 (97%) Did NICRO increase your access to justice (28 respondents; 2 were no asked this question)?
Did NICRO increase access to justice for your family members or friends (30 respondents)?
Yes = 30 (100%)
Bulleted below are some additional details from those interviews or insights owing to them.
Both the table above and the comments below establish (along with most data sets) that the
NCS services reduce criminogenic thinking patterns and provide the life skills that can prevent crime but are not sufficient to address all the factors that contribute to criminality especially those relating to poverty, joblessness and the necessary adjustments required in terms of how
© NICRO 2013 122
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013 the CJS functions in order to develop an integrated national plan (the NICRO offering needs to be integrated into a national strategic plan):
ï‚·
The interviewees (and the clients replying to surveys) had conceptual difficulty distinguishing ‘low risk’ from ‘no risk” such that these categories might be collapsed into one and doing so also allows for a statement that amongst 100% of the clients interviewed some risk factors for reoffending were self-observed to have been reduced
ï‚·
Life Skills courses or anger management (also a life skill) seemed to target criminogenic risks factors amongst more than 90% of clients interviewed
ï‚·
33 interviewees (97%) stated that the programmes, counselling or therapy reduced the likelihood that they would reoffend and one person in Cape Town
(3%) said that joblessness and poverty might force him to reoffend (and 13 others made similar suggestions in the bullet below)
ï‚·
13 clients (38%) suggested that NICRO needs to address unemployment in its offerings in addition to the cognitive-behaviour therapy (e.g., One client said,
“Please help us with programmes that provide work because this was the motivation for theft —I have no permanent job and a family and I only know how to be a petrol station attendant” 45 )
ï‚·
8 clients (24%) said that the cost of transport to attend NICRO therapy was very high for them and some of these said that on occasion they could not get back home easily
ï‚· 3 clients (9%) suggested that ‘prevention was better than cure’ and that NICRO programme offerings should be in schools (an example: “the programme should expand to help people who are not criminals —help to prevent the problem in the first place because being stigmatised as a criminal is not helpful” 46
45 Client (17 August 2012) Interview for NCS Evaluation, Pretoria
46 A Welkom Client (29 August 2012) Client Interviews at Welkom
© NICRO 2013 123
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
FINDINGS
ON EVALUATORS’ SITE-BASED INTERVIEWS AT EERSTEHOEK
The data set from Eerstehoek corroborate the general data set provided in the section immediately above. However, the table of questions and replies at that site is offered in
FIGURE 3.17 (below) to document those findings and to show that all data sets corroborate to suggest that NICRO reduced the risk factors for reoffending:
FIGURE 3.17: Five Interviewees At Eerstehoek: Testimony On The Impact Of Diversion
Questions To 5
Diversion Clients
At Eerstehoek ↓
What is your risk reoffending? of
Replies
No risk--3 clients
Low risk —2 clients
A Sample Of The Testimony
“NICRO helped me to manage stress”
“I take better decisions now because I know how to look at the steps in a decision and the consequences of each action”
“In circumstance where I get angry now, I walk away and I do not feel it is necessary any longer to express everything that I feel”
“I was quick to get angry and now I am much cooler “
“I know how to manage anger and communicate better”
Circumstance that led to offending
Do have more respect for the human rights and rule of law?
Weak skills-3 life
Poor anger management-
-2
Yes--5
Did NICRO increase your access justice?
Did increase access to
NICRO to justice for those around you?
Yes--5
Yes--5
“I not only behave better but I watch out and help others behave better”
“My behaviour is better and everyone now sees me as a friendly guy”
“I did not have experience of the law before but I am not co nscious not to break the rules”
“The anger management class was close to 9 months and I changed but things would have gone badly without NICRO ”
“If NICRO did not exist I would be in Jail because of the way
I was behaving”
“If NICRO was not there I would have a criminal record and would not have selfcontrol and life skills”
“Being jailed would have hurt my family and now I apply my skills at home, making life easier for my parents”
“It has benefitted my family because they can trust me more and everything turned at alright owing to NICRO
”
“I have a one-year-old child. What would have happened to that child without NICRO? Misery and maybe no schooling because no one there to pay
© NICRO 2013 124
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
ECONOMY
This section concerns the cost-effectiveness of the intervention:
ï‚·
Was the NICRO effort to contribute to significant penal reform cost-effective or could the same outputs have been achieved for less money?
ï‚·
Were salaries and expenditures appropriate to the context?
WAS THE PROGRAMME COST-EFFECTIVE?
There was no evidence of wasted resources in terms of budget lines as certified in two audits
(see Annexure 12). There were some signs of under-expenditure, which will be discussed below but the broad finding is that South Africa was presented with a new NCS service with social workers, programmes, resourced sites, local steering committees, and other components on a budget that no government department could have managed such that this was entirely cost-effective. This is also evidenced in terms of:
ï‚·
The discussion of value (below) in Secti on 6.1 (see ‘Value for Money) also applies to cost-effectiveness
ï‚·
SWs deployed to courts nationwide providing therapeutic services at some 45 courts that reduced the risk factors for repeat offending
ï‚·
37 of 45 Local steering committees (as of 31 July 2012) offering expert oversight of the NCS services (and little was expended on this to offer tremendous value)
ï‚·
NICRO SWs working for salaries and benefits below those of colleagues employed by the DSD and yet delivering more on the ground than one might see when compared to the Department of Social Development (one can argue they were underpaid since it is wastage expenditure to train an SW only to lose them to the DSD or DCS)
ï‚·
Sub-section 2.4 and Figure 3.10 which gives a 1.5 weighting (DFID Achievement rating scale) on budget and expenditure
Rather than wasted resources in terms of reviewing budget lines and expenditures, there were minor indications of some under-budgeting and less than optimal resourcing at sites:
© NICRO 2013 125
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
ï‚·
The supervisors suggested that basic rather than optimal resources for delivery were supplied to sites (Annexure 8)
ï‚·
Related to the above, under spending on sites is also indicated in the budget on
‘securing project resources’ which was 25% underspent as of March 2012 and this related mostly to communications from site to national (sites were not always diligent in requesting needed items and requests from sites could be inappropriate as this line item was for computer equipment, books, DVDs, games, toys, white boards and specific project items for direct delivery of services)
ï‚·
Under-budgeting initially in terms of site planning is indicated-- too many sites, occasional site selection that was poor to start or else turned poor (a change of magistrate; loss of a good SW); slow initial set-up times; just SW per site even in areas with very high referrals
ï‚·
Optimal budgeting might have included an initial baseline study well commissioned (a proper one by an M and E specialist) and that included addressing best site locations
ï‚·
Calculating the reach of all the media coverage was not in the budget and therefore impact exceeds what can be measured (also under budgeting on research)
While the above is enough to indicate some under minor aspects of under-budgeting, less than optimal resourcing or even under spending in one regard, it is more useful to stress how funds could have been utilised differently:
ï‚·
Employing a baseline study could have located fewer, better, sites enabling each to have all the phones, equipment, vehicles and personnel required for the full range of service from intake to aftercare and tracking
ï‚·
Posting two SWs per 20 sites dedicated to NCS services (or a senior SW with assistant) who are happy because they have all the salary, benefits, training and pleasant working conditions that they require would have prevented SW turnover and delivered a superior service
ï‚·
Employing more fully into operational delivery, the senior NICRO SWs, rather than employing them to manage, develop and train younger inexperienced
SWs on-the-job (who then might leave for the DSD) would have increased efficiency and effectiveness
© NICRO 2013 126
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
ï‚·
Implementing the above two bullets would have created a superior platform for advocacy (and this was largely an advocacy project): provide a top-notch service and base all public relations efforts on that fact (a focus on fewer sites of excellence is also costeffective and the ‘excellence’ generates its own media coverage)
ï‚·
Cutting from the budget the focus on portfolio committees (a major yearly expense delivering very little)
ï‚·
The above would have liberated funds to focus advocacy on national departments and especially deputy ministers and ministers
On the first point, rather than tying up time and project money to establish and resource 45 sites (52 originally) only to later close them as funds dry up and also closing and moving many of them to achieve targets (a costly exercise and a gamble), it would have been better if a baseline had taken place to identify sites very likely to deliver (e.g., magistrate buy-in calculated; locating sites where DCS could provide the effective monitoring; proximity to existing NICRO offices). Then perhaps 20 ‘sites could have become the models undergirding a rigorous advocacy campaign. Site location is a major issue and involves many considerations and geographers could even have helped but this point need not be belaboured here because the values attending to such approach are also addressed in Sections 5.2 and
5.3 and this is a major theme underlining the broad findings and recommendations.
On the other bullets: with fewer sites, project money could have been liberated to provide: (a) two SWs dedicated to NCS (and not multitasking) and (b) a model for excellence on the ground, which is more significant advocacy than many sites under-delivering and stressed at the same time. Then this superior model for delivery could have become the focus for advocacy (saving massive funds on advocacy focusing on Parliament). The agreed upon budget lines for lobbying portfolio and standing subcommittees in parliament did not deliver significant results and neither did the information and outreach campaign for magistrates. This is because such funding would have been better directed to advocacy at national with the
JCPS cluster, DCS, DoJCD, NPA, SAPS and key government departments who need to buy in
(see Annexure 17-25 as they feel they were not approached). The national stakeholders have to be consolidated into a single national strategy because this is how problems slowing
© NICRO 2013 127
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013 referrals can be addressed (e.g., magistrates who feel they cannot refer owing to court pressures or simply because they don’t want to both). Not taking a strong national approach partly explains failure to reach targets.
Basically, the project was tied up in delivering and resourcing 45 sites (52 originally) and lobbying at parliament such that the funding for the partnerships and strategic plans for sustaining the effort were not really allocated. It would have been more cost-effective to deliver the services at fewer sites to liberate the energies of the organisation for such a massive advocacy effort, especially in Pretoria to build partnerships with Deputy Directors and
Directors of key departments (see Annexures 17-25).
WERE SALARIES AND EXPENDITURES APPROPRIATE TO THE CONTEXT?
The salaries and expenditures were submitted to a certified public accountant for review to find that all expenditures and salaries were in line with budgets and no irregularities could be found
(Annexure 12). Since this is a service-based activity, salaries are closely tied to programme outputs, Although, little could be done, the salaries and benefits for the SWs were less than desirable. Most of their NCS training was on site (the supervision model) and they did not receive enough opportunities to meet with their colleagues nationally according to both SW and supervisor surveys (this would also be a way for national to really understand and work out all the challenges).
In terms of SW salaries, NICRO might pay a bit more compared to other NGOS but NICRO
SWs work hard and deliver much. More importantly, the general average salary of a NICRO
SW is way below the salaries and benefits for the same professional work in government such that it contributes to NICRO losing staff as SWs seek out the bursaries, higher salaries and benefits at the DSD (see Section 2.3). By comparison to the DSD (and DCS too which also hires SWs), NICRO cannot offer the same level of benefits. Sometimes skills building and the opportunity to do interesting work is enough for some SWs to be attracted to NICRO (see SW surveys in Annexure 7, question 2) but places NICRO in the position of working with young
SWs seeking ‘experience’ (an argument was made by some supervisors that it would be more efficient and effective to hire experienced SWs at a higher wage to do the work owing to
© NICRO 2013 128
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013 turnover). Per annum increases for NICRO SWs are somewhat dependent on funding cycles
(e.g., very few NICRO staff received any salary increases from November 2011 up to March
2012 during NICRO ’s restructuring owing to budget cuts) and the rate of increase is also more in the range of 8% than the 11%, which government offers.
Could NICRO have saved on expenditures in any way?
In general, NICRO was very costefficient because of a limited budget (and not all project costs were anticipated and budgeted-some oversights in planning) and yet NICRO delivered on its objectives making it very difficult to say there was wastage. However, two points can be made. First, contracting a public relations firm was identified in the MTR as a little excessive since NICRO later demonstrated internal capacity to manage its own public relations superbly (the programme manager even narrated a highly professional programme that aired on national television). Secondly, had a proper baseline study at the start been conducted, better value would have obtained in terms of: (a) data on reoffending since control groups could have been identified at the start and the lack of them was problematic at a later stage; and (b) identifying the most suitable sites for
NCS services. On the latter point, moving sites to new locations involved some wastage since sites were tested for their value in the course of a programmatic effort with set up costs, offices and trust-building activities (more expensive and less efficient than a good initial baseline study).
EFFICIENCY
Five subsections (below) address project efficiency in terms of:
ï‚·
Did the procedures and systems for funding, personnel, regulations, administration of time and procedures contribute to or hinder the planned outputs?
ï‚·
Are there obvious links between significant expenditures and key programme outputs?
ï‚·
How well did the partnership and management arrangements work? Were local partners involved in programme management and how effective was this and what have been the benefits or difficulties of this involvement?
© NICRO 2013 129
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
ï‚·
Were the risks properly identified and well managed?
DID THE PROCEDURES AND SYSTEMS FOR FUNDING, PERSONNEL, REGULATIONS,
ADMINISTRATION OF TIME AND PROCEDURES CONTRIBUTE TO OR HINDER THE
PLANNED OUTPUTS?
The project had a dedicated project manager who ensured that all the systems for funding, personnel, regulations, administering time and procedures contributed to the planned outputs.
There was some evidence from the time of the MTR that the PM had to micro-manage processes (which was fine when there were only a few sites) but by the time of the evaluation it seemed that the supervisors and provincial managers had assumed greater levels of responsibility and were working much better with all the project management tools. This growth in organisational skills levels for administering NCS services was evidenced in the data as illustrated in FIGURE 3.19 (below) which draws upon the third question from the supervisor surveys (Annexure 8):
FIGURE 3.19: Did your capacity to supervise the delivery of NCS services increase between the start of the service and now?
# % % CHANGE CAPACITY AT THE
OUTSET
Not capacitated 4 31%
CAPACITY AS OF 30 JULY
2012
Not capacitated 0
#
0 -31%
Weakly capacitated
Capacitated
Strongly capacitated
2 15% Weakly capacitated
7 54% Capacitated
0 0% Strongly capacitated
0 0 -15%
12 92% +38%
1 8% +8%
Findings: the capacity of supervisors to provide and supervise others in the provision of NCS services increased between the start of the project and 30 July 2012 when all supervisors deemed themselves to be fully capacitated whereas at the start 6 of the 13 (46%) did not feel they had all the requisite skills or experience. At that time the PM was micro-managing the project from Cape Town.
© NICRO 2013 130
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
ARE THERE OBVIOUS LINKS BETWEEN SIGNIFICANT EXPENDITURES AND KEY
PROGRAMME OUTPUTS?
No wasted expenditures could be identified since all of it was directed economically to programmatic outputs. This was also the finding of a CPA, subcontracted to review the expenditures (see Annexure 12). Moving sites was costly measured in terms of closing one site and then having to start-up the service at a new one (start-up times can usually be measured in months and slows and delays magistrate referrals into programmes but retaining sites where referrals was not working would not have been cost effective either)r. A baseline study commissioned in the pre-intervention period could have included a study of the best sites for the service perhaps saving some funds in the long-term (it is expensive to test out sites by setting up programmes to see if they work out).
HOW WELL DID THE PARTNERSHIP AND MANAGEMENT ARRANGEMENTS WORK?
WERE LOCAL PARTNERS INVOLVED IN PROGRAMME MANAGEMENT AND HOW
EFFECTIVE WAS THIS AND WHAT HAVE BEEN THE BENEFITS OR DIFFICULTIES OF
THIS INVOLVEMENT?
The stakeholder partnerships were effective at local level in terms of 37 out of 45 stakeholder committees supporting NCS services with obtaining referrals, community outreach and problem-solving. They never functioned as originally designed to review cases because they cannot donate that amount of time. They were functioning in the absence of any consolidated partnership arrangements at national department level (despite some efforts to arrange these) and this was addressed in the Achievement Rating Scales (see 2.3.5.5 above) and again under Section 7.2 (below). It is a main finding of the study that the focus for partnership building needed more work at national level as some critical partners thought they were never formally approached (See Annexure 17). However, this is discussed at length in the sections below on sustainability and replicability and is an underlying factor explaining most shortcomings and this is the main focus of the Chapter 5 recommendations.
WERE RISKS PROPERLY IDENTIFIED AND WELL MANAGED?
The risk management plan was appropriate and identified all risks. It was based on a supervision model that included all the necessary tools such as pre-formatted monthly reports
© NICRO 2013 131
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013 to the PM that she closely monitored and especially significant was the Policy and Procedures manual that identified and explained each risk. 47 For instance, the importance of good quality assessments, programme interventions of sufficient quality to address the problems, and aftercare and tracking were all addressed in the PSP manual.
While risks were fully identified and all the tools were available to manage risks, challenges were indicated in terms of delivery by some managers and supervisors as documented by the
PM in Annexure 27 —a full report on the challenges. Basically the model rests upon managers managing and supervisor supervising in the ways prescribed by the PSP manual but when this was formally reviewed by the PM in early 2012 (mainly 20102011 data), she found that “not one of the Area Managers (now termed Provincial Managers) had been able to ensure full compliance with supervisory requirements and did not seem to be able to develop supervisors to function on an optimal level with highest levels of independence.” 48 For example, there was evidence that SWs would be promoted within a few short months from PM1 to PM2 or PM3 presumably to reduce the need for the supervisor to engage too heavily in supervision. These were challenges that met with responses such as returning the SWs to PM1 level and then engaging all supervisors in service delivery with a targeted number of clients monthly so that were fully engaged and part of the team.
Referrals to NICRO were at a rate less than anticipated as indicated from first annual report to
DFID (June 2009) to this evaluation.
49 The original target for referrals was 14,460 offenders enrolled in programmes by Year Five and this was later revised downward to 6000, which was still a struggle to meet as a target (at least at the time of evaluation). It could have been managed better through a different kind of lobbying and advocacy policy (see Section 7.2 below). NICRO could be better positioned in terms of partnerships for sustaining the effort had partnership building focused on the deputy ministers and ministers of key departments
47 Before the MTR in 2010, NICRO found that “the quality and risk tables were too timeconsuming and laborious to maintain while adding little value to the project and therefore a supervision model was utilised. Jules-Maquet, Regan (6 April 2010) Interview with Project
Manager, Cape Town
48 Jules-Macquet, Regan (2012) Supervision and Professional Development (a review of supervision efficiency by the project manager), Cape Town
49 NICRO Non-Custodial Sentencing Project GTF 112, Annual Report, p. 8, June 2009.
© NICRO 2013 132
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013 especially the DCS (See Annexure 18), DOJCD (see Annexure 21), and the JCPS (see
Annexure 21). A conservative approach by some magistrates led to low referrals but had the partners been in place at national level setting policy and issuing directives aimed at increasing participation, it is likely that there would have been more buy-in on the ground. These deputy ministers may have been approached (NICRO says they were and they say there were not formally approached —see Annexure 18).
Another risk that might have been better addressed was high SW turnover in a situation where
SWs are in short supply. Salaries and benefits are higher at the Department of Social
Development than at NICRO and solutions are required--perhaps sites might have been negotiated with higher magistrate buy-in and then SWs offered a better package for retention.
The project in a sense was far too ambitious in terms of outputs (ultimately 45 sites) and not always focused enough on the quality of delivery per site. Had there been fewer sites, then the budget sufficient to the real training needs of the SWs might have seen better results sooner. The assumption that the SWs were already trained professionals who could simply offer new programmes after self-training with some support, guidance and mentoring of their supervisors is not supported by the data. The supervisor and SW surveys (Annexures 7 and
8), corroborate in terms of a large number of statements that onsite training is not sufficient and formal training opportunities away from the office are required (the challenge: national did not have the budget for it). See:
ï‚·
Annexure 7, question 2, where SWS rated professional development and career pathing as second only to salary and benefits in terms of the improvements they want to see
ï‚·
Annexure 7, question 4 where it is shown that only 40% of SWS understood the clauses in the criminal procedure act relevant to their work.
ï‚·
Annexure 7, question 8 where it was shown that SWs undertaking more than 15 assessments per month are likely to exceed their capacity to deliver and be stressed and 10 SWs (32%) are in such a category
ï‚·
Annexure 7 question 18 where 44% of SWs when asked about ways to improve delivery cite more in-service training and skills building to address their lack of capacity to the full range of services required
© NICRO 2013 133
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
ï‚·
Annexure 8, question 2.5 where of 13 supervisors, three identified problems with either: (a) new recruits lacking sufficient training (having missed formal opportunities) or
(b) SWs who were skilled but could not advance to provide certain needed programme only through self-training (sexual offences, matrix, perpetrator programmes were cited)
ï‚·
Annexure 8, Question 7 where 77% of the supervisors said that the SWs did not have all the training required to implement all available programme offerings and question 6 wherein the supervisors rate additional intervention offerings as less important than training in the existing ones
EFFECTIVENESS
Effectiveness is concerned with the degree to which the intended outcomes were achieved
(the log frame in terms of targets). This will be discussed in term of NCS services and then the
Indigenous Justice Pilot Project at Eerstehoek in these terms:
ï‚·
Did the interventions achieve their objectives (or are they likely to achieve them)?
ï‚·
How effective and appropriate was the programme approach?
ï‚·
With hindsight, how could the intended outputs and results been better achieved?
In brief, the NCS services were found to be effective and meeting objectives while the
Indigenous Justice Pilot Project at Eerstehoek was found to be ineffective in terms of the original project design but it was a pilot and learning took place.
DID THE NCS SERVICES ACHIEVE THE OBJECTIVES?
Section 2 offered a thorough review of the objectives, especially Section 2.1, which showed that the broad objectives were met such that systems were in place for referrals; and (b) courts were referring offenders. The primary failing is not meeting the log frame indicator of 10 referrals per month by end of project. Only two sites met the target (4%) by 30 July 2012 and at most 7 sites were positioned to meet such a target by 30 March 2013. The slow pace owed to multiple factors as given in the Section 2 log frames (see also Annexure 1) but these are three important ones:
© NICRO 2013 134
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013 a. Magistrate perceptions and/or the fact that referring for an assessment delays cases b. SW turnover, which slows the pace of referrals since the new SW must win the trust of magistrates c. Moving sites--a new site requires a start-up time and renewed trust-building.
It is within NICRO
’s power to address ‘b and c’ as discussed in multiple places throughout the report but ‘a’ requires some explanation. Basically, magistrates were convinced that NCS had value (there is no need to try to convince them further) but based on surveying and interviews at least half of all magistrates did not want to adjourn a case for an assessment because this was perceived as delaying their cases. Those few that did refer (1 or 2 per court) accepted
NICRO recommendations 72% of the time but only a small percentage of magistrates per court were referring. The conundrum: NICRO ’s demand for thorough and appropriate assessments
(meaning court delays up to 2 weeks minimum) Vs. magistrates under pressure to reduce the court roll wanting short and quick assessments (one day would be ideal to many of them).
This leaves a systemic problem with the Judiciary unaddressed (the oaths of office are taken to deliver justice; not to get cases off the role which suggests functional/structural problems with
Judiciary —see Annexure 25, question one; see also Annexure 10, question 6). The only way to address this and a given and central to the key recommendations of this report (see recommendations) is to lobby the CJS as a collective body (use the JCPS structure) at national
(Pretoria).
HOW EFFECTIVE AND APPROPRIATE WAS THE PROGRAMME APPROACH (NCS
SERVICES AND INDIGENOUS JUSTICE PILOT)?
In terms of the Indigenous Justice Pilot Project, the broadest spectrum of community members at Eerstehoek benefitted more from NICRO
’s efforts to engage in community based crime prevention with its wider application of resources to the community than the focus on diversion from the traditional court system into NICRO programmes. The latter affected far fewer people, was not cost-effective, and focused the project on a highly contentious system of
© NICRO 2013 135
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013 justice which happens to be a massive national debate at this writing (see the fuller discussion in Section 5.4 and 5.5 below)
In terms of NCS services, the programmatic approach to service delivery at the therapeutic and local level was appropriate with project management and interventions of sufficient expertise that perpetrators reformed and this was the main objective.
If the programme were considered as a long-term advocacy effort to make impact on alternative sentencing practices countrywide, the programmatic approach fell short because the delivery of NCS services requires a formal strategic plan that integrates key departments
(e.g., DCS, DoJCD, DSD, and Judiciary) at national level. This means that NICRO or NICRO working with other stakeholders needs to bring these departments to the table, perhaps as formal partners with NICRO in a national effort to institute NCS services on a scale that has impact on the CJS (e.g., better delivery of justice; prison populations dropping; or costs of incarceration dropping). The programmatic approach to change the attitudes of key decisionmakers within government or getting magistrates to refer (two interrelated challenges since magistrates are working to meet targets set by national decision makers) was far too local.
This lack of an integrated, interdepartmental, cross-disciplinary strategy that linked the national stakeholders occurred despite data from every one of the departments saying they wanted to partner and must partner but had not been formally approached see Annexures 18, 19, 20, 21,
23, 24).
50
It was recommended in the MTR and again in this document that each one of the key departments be individually lobbied (Deputy Directors) and this evaluation adds a strong focus on the JCPS cluster where they are all found. All the evidence shows (e.g., Annexures 17-25) that lobbying and advocacy failed to consolidate stakeholders into a single strategy but this is clearly required to ensure the appropriate delivery of NCS services (e.g., the gap in monitoring offenders must be addressed collectively and many other issues need to be addressed as
50 That it was very difficult for the evaluator to even engage the DSD (and certain other national level stakeholders like the JCPS did not make time available despite scores of attempts, phone calls, and e-mails) is perhaps another indication that they had not been brought aboard as full partners.
© NICRO 2013 136
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013 bulleted in the paragraph below). No single government department or civil society organisation has all the skills, funds or programmes to implement NCS services as is necessary to ensure programme integrity. Hence, the programmatic approach to modelling the service at many sites around the country was excellent and so too were the national and local media efforts but in spite of expert project management and expert programme interventions, the programmatic approach to advocacy at national level was not effective as further explained below.
By advocacy is meant consolidating the partnerships for: (a) the smooth running of services including the huge gap in monitoring that threatens programme integrity; (b) sustainability; and
(c) replicability. The second two items are addressed in Sections 7 and 8 (below). On the first point, some deeply rooted structural flaws in the CJS for delivering NCS services were identified in this evaluation:
ï‚·
Nationally and within NICRO there are insufficient SWs and SAWs for delivery
(one SW per NICRO site was not sufficient such that critical programme components like aftercare and tracking did not take place--see Logframe
Indicator 2.3.3) and the one SW at each site was multitasking between service streams when integrating NCS into the justice system is enough of a task
ï‚·
DCS also lacks the human resources or tools to monitor appropriately (see
Annexures 19, question 3, and Annexures 24 and 25, question 1)
ï‚·
Social Development lacks enough SWs to implement the Child Justice Act let alone help with NCS (see Annexure 21, question 1 and Annexure 24, question
5)
ï‚·
The Judiciary has very limited space and resources in many courts and the
Case-flow management meetings have only a little time to focus on NCS services and some do not work for this purpose
ï‚·
Most magistrates are more focused on reducing the caseloads before them than on achieving long-term solutions to the effective delivery of justice (see
Annexure 25; See Magistrate Survey Data Annexure 10)
ï‚·
The NPA seeks rewards in terms of convictions and jail time (see Annexure
23, question 3)
© NICRO 2013 137
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
ï‚·
Legal Aid needs to be more organized about the available NCS options that can be explored--if the plea is not raised by them, it will not be considered by the magistrate (see Annexure 23, question 3)
ï‚·
Few SAPS members have embraced restorative justice principles thinking the perpetrator must be punished and not ‘get away with school’ for his offense
(see Annexure 22, Question 7) and they evaluate their performance on arrest rates, which congest the courts and prisons, partly explaining why magistrates are too overwhelmed to focus on NCS services
The bulleted items (above) do not comprise a full list of all the structural challenges but are sufficient to demonstrate that the challenges are enormous and cannot be addressed piecemeal but require a single, integrated, interdepartmental and multi-disciplinary strategy.
Just one of those bulleted items
—a judiciary too focused on getting cases off the role to wait for an assessment
—is indicative of a massive structural problem with the judiciary alone and it is linked to arrest rates and a problem with SAPS. Consider the function of training magistrates in the comments of Senior Magistrate Basil King who was a key figure in providing training at Justice College and a reference team member for NICRO who has not been transferred along with his knowledge of training (see his full set of comments in Annexure 25)
51 :
The time pressures on magistrates are great. We have to forget about the statistical chase and concentrate on delivery and have trust in the system but magistrates might sit in class learning about NCS and then get to their office to be told there, we do not work like that here. Four weeks is the total training in the criminal court programme —social context and ethics. Maybe 14 hours are expended on sentencing options —and this is where it is raised and two to three hours cover custodial sentences and 11 hours on non-custodial sentencing.
51 It is a sign of dysfunction that he has been transferred and Justice College Is not so functional as a result because he was a depository of much of the knowledge of what works and what does not in terms of training
© NICRO 2013 138
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
Systemic problems of these kind are simply too huge to address in silo-fashion and the only solution that can match the size of these challenges is a very specific and detailed implementation strategy for NCS services that ultimately defines, assigns, contracts, monitors and evaluates stakeholder roles in implementation. Such an approach is required because it is widely held that the entire CJS system has to shift toward restorative justice and yet there no evidence of any systematic approach to making the shift (the content of Annexures 18-25 should convince anyone that the described situation). What NICRO did was to establish the value of shifting and to demonstrate that a shift is possible but NICRO alone cannot implement
NCS services nationwide and it is insufficient as a NICRO programme offering since NICRO is an NGO with funding cycles. One can say the NCS services were expertly modelled and while there was an advocacy plan targeting certain stakeholders and changing national mind-sets
(all commendable), it did not integrate stakeholders into a national plan for delivery and such a plan is required (the work that lies ahead). This report documents the willingness on the part of departments to so engage (see Annexure 18-24).
WITH HINDSIGHT, COULD THE INTENDED OUTPUTS AND RESULTS HAVE BEEN
BETTER ACHIEVED?
These are areas where in the evaluator’s analysis the intended outputs and results could have been better achieved:
ï‚·
A baseline study, which is not a general study but one specific to identifying the baseline measures for the evaluation, was never undertaken or part of the strategy and is a necessary component of all interventions (it should be required by the funding agency too)
ï‚·
Site selection should involve the M and E specialist and be based upon the findings of the baseline study and include areas likely to offer high referrals -pre-testing of magistrates could have been done as part of an initial baseline study 52 )
52 Baseline studies are normally an integral part of evaluation studies —the two indispensable parts of a monitoring and evaluation system
© NICRO 2013 139
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
ï‚·
Fewer sites (45 sites devoted to much energy to setting up sites, moving them sometimes and promoting referrals when not enough energy was devoted to the kind of advocacy effort that could result in a national plan to roll out the services more widely; instead they will contract)
ï‚·
More opportunities for educational and team building activities are needed for
SWs and supervisors outside the workplace particularly if NICRO is to work with new graduates or relatively inexperienced SWs and this must be budgeted for when setting up projects of this kind (it was indicated in multiple data sets that online and onsite training opportunities are useful but not sufficient to the task
—induction training and various opportunities for SWs to mix provincially if not nationally would help the effort)
ï‚·
A plan to link all key national departments into key performance areas for the delivery of NCS services (monitored and evaluated) and one that addressed the disjuncture between departments in terms of delivering on an integrated strategic plan
ï‚·
An advocacy strategy that achieves the above outcomes (NICRO
’s original advocacy strategy needs to be re-analysed because an interdepartmental, multi-stakeholder and cross-disciplinary plan for sustaining the services from the highest echelons of government never materialised during some 4-5 years of efforts (required: target the JCPS cluster body and all the Deputy Minister in each department in Pretoria with a draft plan)
ï‚·
An example of the challenges pertaining to the above two bullets is that a workable approach to aftercare and tracking, which is a serious programme threat, was never properly addressed and NICRO alone did not have the capacity for this or the full complement of partners and activities to address it nationwide (see Annexure 8, Question 10; see the statement of DCS in Annex
18; see the Basil King Interview in Annexure 25).
DID THE INDIGENOUS JUSTICE PILOT PROJECT AT EERSTEHOEK ACHIEVE THE
OBJECTIVES?
Traditional courts sentencing to NICRO programmes was not really a success in terms of target attainment —from a start date of November 2010 to the 31 July 2012 cut-off date for the evaluation; only 10 of 330 (3%) targeted referrals were attained as shown in Figure 2.3.6 further above. However, this was a pilot to find out how to work in traditional areas such that the failure of a particular indigenous justice model does not mean that NICRO failed since
© NICRO 2013 140
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013 upon identifying this by 2011, the organisation then focused more on schools and community work, which was much more effective in increasing referrals to NICRO programmes. There were strong indications that crime rates in the schools either dropped or were at the very least referred out of the justice system to NICRO: the Control Prosecutor Stephanie Skhonde stated that her caseloads dropped precisely in the schools where NICRO was working and Magistrate
R. M. Chirwa confirmed this on his particular magistrate survey (Annexure 16 and see 16.1.7 and 16.1.10).
What was learnt?
Basically that school and community-based work in the indigenous justice site was a more effective model than working directly with the traditional courts while traditional leaders could benefit through regular participation on the LSC at the court (this helped to facilitate their input into the justice system). In other words, NICRO aimed to promote a culture of human rights and restorative justice in traditional communities and succeeded through schools and community work and giving traditional leaders a voice on the LSC. This means that regular NICRO offerings like diversion, school and community outreach work, NCS, and offender reintegration achieve better value for traditional people than the proposed new model for indigenous justice focusing on traditional courts. It is more effective and more popular with broadest spectrum of indigenous community members to engage in community based crime prevention with its wider application of resources to the community than to focus on diversion from the traditional court system into NICRO programmes, which affects far fewer people and focuses the project on a highly contentious system of justice which happens to be a massive national debate at this writing.
The above findings fit with the perspective of LSC members who stated that the area was too big in both population and distances for NICRO to provide an effective intervention either in terms of NCS or diversion from traditional courts. The court that might have the most cases to refer to NICRO, Mayflower, was 60km distant. NICRO served 3 traditional courts but there are
12. As showing in Annexure 16, the LSC, the magistrate and the NPA thought that NICRO should rather intervene in schools and communities to prevent crime and leave the traditional courts with national and provincial departments (e.g., it is these departments who need to provide training in the Traditional Affairs Act).
© NICRO 2013 141
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
HOW EFFECTIVE AND APPROPRIATE WAS THE INDIGENOUS JUSTICE PILOT
PROJECT?
The Indigenous Justice Pilot Project was not effective as originally theorised and modelled
(see Section 5.4 above) and this explain s the ‘4’ very limited achievement rating as shown in
Log frame 2.3.6. FIGURE 3.19 (below) further elucidates this by illustrating the main sources of referrals up to 31 July 2012 and how low they were despite long-term efforts by NICRO to develop a relationship with the traditional courts (including moving from one court to another in an attempts to locate a traditional leader who would refer). FIGURE 3.20 ( further below) show that the numbers affected by the schools and community work that started later than the work with traditional leaders (from mid-2011 onward) had achieved a more substantial impact in terms of numbers affected.
FIGURE 3.19: Main Source Of Referrals (Listed In Order) At Eerstehoek
3
4
5
1
2
SOURCE OF REFERRALS
NPA/Judiciary
Schools
Community Members
Traditional Courts
Department
Development of Social
NUMBER OF REFERRALS UP TO 31
JULY 2012
36
17
11
10
02
% OF REFERRALS
47%
22%
15%
13%
3%
TOTAL 76
FIGURE 3.20: Community Projects At Eerstehoek And Targets Reached
100%
1
2
Community Project
Safety Ambassadors
Mentorship Programme
FUNDER
First National Bank
DFID
TARGETS
REACHED
29
18 mentors
3
4
Choral Group
Soccer Team
DFID
DFID
30 mentees
12 youth
40 boys
28 girls
157 Total → Four
Projects community 2 funders
( DFID funded 3 projects affecting 128 youth to 31
July 2012)
Analysis: The community projects combined with the NCS services at the court (holistic outreach to an indigenous area) were having a much greater impact than the work at the traditional courts, which were slow to
© NICRO 2013 142
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
Community Project FUNDER TARGETS
REACHED refer and mired in challenges including distances, irregular court proceedings (fines go to traditional leaders instead of courts and the community) and problems and controversies with the Traditional Courts Bill (NICRO would not want to be associated with some of this).
WITH HINDSIGHT, COULD THE INTENDED OUTPUTS AND RESULTS HAVE BEEN
BETTER ACHIEVED (INDIGENOUS JUSTICE)?
The aim of the Indigenous Justice Pilot Project was to learn about what works and this was achieved. The hypothesis was nullified as NICRO learnt about the challenges and pitfalls of working with traditional courts (e.g., justice is not always being served; fines sometimes exceed the limits of the law; systems are not codified but arbitrary; and the costs for petrol, workshops, SW time, strategic development, and partnership building efforts did not justify 10 referrals). Expanding existing NICRO services (including NCS) into more traditional communities achieved better results but piloting produced this knowledge. Perhaps with hindsight (knowing these results in advance), NICRO might have focused on:
ï‚·
Creating more representation for traditional leaders and their representatives on the LSC and Case Flow Management Committees (many parts of rural
South Africa include traditional communities and most of them are not represented)
ï‚·
A developmental approach to such areas to identify cases and social problems (such as in schools) before they reach the traditional court
In fact, by mid-2011, NICRO had taken this approach, which not only improved referrals but was a better fit with the kind of help that these communities actually wanted (these are poor communities that want to see more resource based activities and programmes that benefit schools and communities more widely). So the only benefit of hindsight would have been to work in this manner sooner but NICRO responded in reasonable time.
© NICRO 2013 143
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
VALUE FOR MONEY
Value for money is considered under these sub-headings:
ï‚·
Were resources used optimally to offer good value for money?
ï‚·
Was a balance achieved between economy, efficiency and effectiveness?
ï‚·
Did the programme represent good value for money spent?
WERE THE RESOURCES USED OPTIMALLY TO OFFER GOOD VALUE FOR MONEY?
The intervention was generally very thrifty making use of free venues for workshops and events, using meeting spaces in court facilities for free, and the local steering committees ran for very little expenditure. A value of R10 million was also generated by the media effort as covered in the logframes further above. It is presented in various parts of the report that a baseline study can look at value because it is a pre-intervention activity and this could have saved on some costs. For instance, in the evaluation data it was evident that distances and challenges with the traditional courts could have been picked up in the baseline/scoping study.
In fact, it was covered in the mid-term review in terms of the NCS sites that rural areas have particular challenges for the efficient and effective delivery of services.
The Programme Manager undertook a value for money calculation in March 2012, at which point there were 3,628 offenders on NCS. The total DFID budget for the GTF 112 project of £4
916 108 was divided it by 3 628 offenders to generate a cost per offender of £1 355 (see detailed findings in Annexure 26 and the illustration as FIGURE 3.18 below) and compared it to the daily costs of housing a prisoner (using 2011 rates at R243 a day although at this writing it is @R300 per day) times the days each NICRO client would have spent in prison if not on the programme. The NICRO costs included all project costs in both Rand and Pounds but the prison costs were limited to housing a prison (leaving out major CJS costs —salaries etc.) and the savings is still between R400 million and R600 million. Annexure 26 provides the detailed calculations.
© NICRO 2013 144
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
FIGURE 3.18: Cost Per Offender For The Gtf 112 Project As Of March 2012 53
DFID Budget
£4 916 108
Cost per Offender
£1 355
R 58 993 298 54 R 16 260
The ‘cost per offender’ calculations are also very conservative because:
ï‚·
Costs keep dropping toward project end, however, as more offenders enter the programme (e.g., if one takes the DFID budget £4 916 108 and divides it by the 3,960 offenders on programmes as of 31 July 2012 th e cost of the NCS services drops to £1,
241.44 per offender.
55
ï‚·
The costs of the intervention are inflated by once-off project development costs such as media and marketing activities, training, project management, this evaluation and promotional costs (brochures, videos, materials etc.) that would not be associated with an instituted system.
ï‚·
Imprisonment has many hidden costs such as the cost of trials, police investigations, the social cost of high recidivism rates (prisons are dysfunctional in terms of reforming prisoners and 50-90% reoffend), and lost income to the household (the perpetrator is often a contributor to the household income).
WAS A BALANCE ACHIEVED BETWEEN ECONOMY, EFFICIENCY AND
EFFECTIVENESS?
The services were very economical, efficient and effective. The objective of providing services at so many site (45 sites; 52 originally) appeared to the evaluator as not economical, efficient or effective but that was part of NICRO
’s contractual arrangements with DFID. Nonetheless, it would have been more economic, efficient and effective to focus on demonstrating 25 sites of excellence with superior human and material resources at each site. For instance, NCS really requires two dedicated SWs at each site (or a seasoned, experience SW with support) and certainly not one SW multitasking on several projects of which NCS is just a part. In many
53 The calculations by the NICRO project manager were found in an addendum to the GTF 112
Annual Report 2011-2012 to DFID
54 Using a R12:£1.00 exchange rate
55 The exchange rate on 18 November (the day of this calculation) was 14.0825
© NICRO 2013 145
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013 areas aftercare and tracking appeared to be a workload of its own. From that perspective, the project tried to spread the resources over too many sites and this was not quite in balance with effectiveness if viewed as an advocacy effort. Sites of excellence were required to convince national departments of the value of the services so that they would buy in (see the more detailed discussion under Sustainability and Replicability below).
ARE THERE OBVIOUS LINKS BETWEEN SIGNIFICANT EXPENDITURES AND
PROGRAMME OUTPUTS?
A certified public accountant was utilised to certify the appropriateness of the expenditure (see
Annexure 12). In addition, the evaluator reviews these links in Annexure 1.4 to find significant expenditures are all linked to programme outputs and agreed budget lines for each. There were indications of the opposite kind of challenge: NICRO omitted many expenses in the budget (sufficient funding to retain social workers; research activities including a baseline study; analysis of data returning from community surveying; supplies at site were provided but minimally; no funds to assist clients with transport; limited funding for advocacy efforts with departments at national). It is in the recommendations of the report and an underlying theme throughout this study that fewer, better resourced sites would have delivered more than so many sites and liberated funds for a more rigorous advocacy effort (this is further addressed in
Section 7 and 8 below because this had impact on sustainability and replicability).
SUSTAINABILITY
This section is concerned with three questions:
ï‚·
What is the potential for the NCS services continuing after the withdrawal of DFID support?
ï‚·
What is the potential for the indigenous justice pilot site to continue after the withdrawal of DFID support?
ï‚·
Can collaboration, networking or otherwise influencing opinion sustain the NCS services?
© NICRO 2013 146
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
WHAT IS THE POTENTIAL FOR THE NCS SERVICES CONTINUING AFTER THE
WITHDRAWAL OF DFID SUPPORT?
NICRO planned to sustain the NCS services as one of four main service streams: NCS, diversion, offender reintegration, and school and community work. NICRO will continue to seek and access funding for NCS services as part of NICRO ’s long term business planning
(see Appendix 15, National Manager Survey Data). To contextualise this, NCS services at 45 sites comprise only 12% of the 352 magistrate courts nationwide and in terms of sustainability and replication there are some major challenges since NICRO will be reducing sites for delivery (dropping about 17) owing to reduced funding from 2013. The number of sites delivering services will retract to perhaps 28 sites where the case flow is sufficient to justify the service (Soweto might be an example). Such downsizing is not without its merits since
NICRO needs to put more resources into advocacy efforts and some sites lacked the case-flow to justify the service.
Downsizing to fewer sites has also been the evaluator’s recommendation since the time of the
MTR. Providing NCS at many sites to meet
‘targets’ is less productive than providing the service at a few sites with enough personnel focused on NCS to model excellence. This might include senior SWs engaged in the service with top notch assistants (more productive in terms of outputs) rather than supervising younger SWs seeking ‘opportunities’ for advancing their careers. Otherwise high SW turnover will continue with all the challenges that attend to it.
WHAT IS THE POTENTIAL FOR THE INDIGENOUS JUSTICE PILOT SITE TO CONTINUE
AFTER THE WITHDRAWAL OF DFID SUPPORT?
The Indigenous Justice Project cannot be sustained in terms of the original design as discussed in Sections Section 2.3.6 in terms of Logframe Ratings and Explanations and again in Sections 5.4 – 5.6 on the overall effectiveness of the pilot project. However, this statement must be understood in the context of a pilot by NICRO to understand how to work in indigenous areas. The conclusion that such work needs to be more developmental (school and community outreach for diversion onto NICRO programmes) does not close the door to indigenous areas but opens up a different avenue of working. Furthermore, there is no
© NICRO 2013 147
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013 intention to withdraw from Eerstehoek. Rather, this will continue as one of the sites. NICRO can now work at more indigenous sites now but with the knowledge that its basis for referrals onto diversion needs to include schools, social workers and community sources. They can also support the inclusion of indigenous leaders on local stakeholder committees.
CAN COLLABORATION, NETWORKING OR OTHERWISE INFLUENCING OPINION
SUPPORT SUSTAINABILITY?
Yes, NICRO could do much more to build networks for the delivery of NCS services. First,
NICRO has a role in sustaining its sites since this offers a basis for developing the services further. Then around this, NICRO needs to create a network of partnerships because one
NGO without other national partners will not be able to implement this nationally or to provide for all the inputs required to sustain the services. NICRO would have to view itself as one stakeholder in a national system of NCS (which may not fit with the business model NICRO is actually developing). It can even be argued that it is not appropriate for a civil society organisation to deliver services beyond what is required to model and advocate appropriate government service delivery (CSOs have funding cycles which frequently show them unable sustain services on their own--a fact that was demonstrated in this report; and the philosophy regarding the appropriate role of civil society is also a serious international debate to be considered). Of course, if NICRO moves toward being a business, this would define such matters differently.
Partnerships from national level were never consolidated but efforts were made and certain partners indicated such will: (a) Correctional Services expressed willingness to partner on NCS services nationally (e.g., see Annexure 18 and the extract of their statement a few paragraphs below); and (b) the DoJCD including the Judiciary participated even before DFID funding and provided spaces and facilities in court houses across the country for the services.
Collaboration and networking that will consolidate the partnerships to support sustainability not just as a NICRO offering but as a sustained system of NCS services nationwide include:
ï‚·
NICRO working with a cluster of CJS departments to develop a strategy
(many interviewees suggested the Justice, Crime Prevention and
© NICRO 2013 148
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
Security Cluster (JCPS) at national level as the key locus for collaboration to sustain the services (see Annexures 18, 19, 21, and 23)
ï‚·
Partnership agreements with individual departments like the DCS and
DOJCD to help sustain the service —while no national department had formally partnered with NICRO in sustaining the service as part of their operations and budget, interviews indicated strong interest (e.g., see
Annexure 18)
ï‚·
Partnership agreements with the Justice Department especially in terms of furnishing office space for NCS in court houses; and magistrates serving on the local stakeholder committees such that more work needed to be done to consolidate these partnerships (see Annexure 21 and 23)
ï‚·
Partnerships with DSD national was fairly weak and yet the DSD did provide some subsidies for NICRO pre-assessment reports —more work needs to be done as the evidence is that more progress can be made on this partnership (see Annexure 19)
ï‚·
NICRO leadership in sustaining NCS sites (in the short term NICRO will not be able to maintain 45 sites but was planning to retain more than 20 of these for the purpose of lobbying and advocacy as bulleted above)
ï‚·
NICRO leadership in sustaining the local stakeholder committees at NCS sites (or NICRO representation on Case-flow Management Committees) that consist of key departments, traditional leaders, civil society organisations and court users that together with at least one magistrate review sentencing practices
ï‚·
Working with the Judicial Education Institute where new magistrates are trained to help ensure that NCS services and training in restorative justice principles enter into the culture of the Justice System through presentations to new magistrates
ï‚·
Initiating discussion on ways to develop the capacity of other NGOs
(training) and service providers to join in the delivery of NCS services
(NICRO reached 45 of 366 magistrates courts but support will be needed)
Changing judicial practice so that magistrates refer to the NCS services is tantamount to changing the culture of the justice system and this is such a mammoth task that it cannot be
© NICRO 2013 149
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013 achieved by one NGO and certainly cannot be achieved using only a handful of tactics like modelling good practice, lobbying local magistrates and lobbying portfolio committees. It requires joint strategy, protocols, budgets, and a national team of stakeholders working together to make it happen over a period of 10 to 15 years or more. This kind of effort was not occurring at the time of the evaluation study nor was it certain to trend in that direction either.
The first step that NICRO could take would be concentrating on the particular willing partners at national (often Pretoria-based) and provincial level but this was never achieved for reasons that were not entirely clear but protocol may not have been followed and this is very important in Pretoria. Very specific stakeholders and committees have to be approached one by one and lobbied with a plan (workshops will not work).
The need to collaborate further was also indicated by challenges with monitoring and tracking offenders (see the DFID rating for monitoring offenders in Section 2.3.3; or Cape Town Focus
Group, Annexure 17). Fears (not accusations) were also expressed in some interviews at national that offenders might be on NCS who are not appropriate and such fears can only be resolved by collaborating on mechanisms for implementation (e.g., Magistrate Peet Van Vuuren in response to Interview Question 3 in Annexure 23 suggested NCS sentences should be subject to judicial overview bodies and this would reduce fears and tensions).
The DCS, the DoJCD, SAPS, the Judiciary and even the DSD in terms of Gauteng Provincial) expressed willingness to partner with NICRO and NICRO also achieved some headway at national levels (e.g., DSD helped fund assessments ast some sites). Yes NICRO had approached these stakeholders yearly and never achieved a consolidated partnership (uniting all of these stakeholders into a plan). The issue appeared to the evaluator to be one of formal approaches (see Annexures 18-23). Annexure 18 provides a statement from DCS documented by the evaluator during an August 2012 focus group in Pretoria with the all the
Deputy Directors of the Department of Correctional Services present. The DCS described themselves as a most willing partner that had not to date been formally approached with a plan. Nonetheless they said they would help to sustain NCS services, particularly in terms of a monitoring function, if approached. This is an extract of the DCS statement (see Annexure 18 for full details) that encapsulates the evaluation findings on where partnership building rested as of August 2012:
© NICRO 2013 150
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
One can say that DCS is a natural partner (of NICRO and NCS services) but the partnership needs to be formalised. We do ‘buy in’ and need NICRO but formal relationships are a process and this has not yet been created. We can say there really is no resistance —we just have not consolidated a partnership. We see this as a critical partnership that has not developed. Sharon knows the National
Director (at NICRO) such that this partnership can be easily developed.
Meetings must be held between the big bosses —with the DGs present. Nothing will occur if NICRO does not communicate direct with the Deputy Director
Generals.
DSD in Gauteng issued a similar statement in a focus group that included the provincial director (see the full text in Annexure 19):
The DSD needs to be fully involved but it is not at this time. There does need to be dialogue between DSD and NICRO about a range of issues and not so much about how this works as we are aware of how it works but dialogue is needed.
The DSD must be presented with studies that describe the benefits. There are opinions that there are not enough social workers nationally to even do this although we are increasing the number of them with bursaries. Another problem for NICRO is that the DSD pays better and offers better benefits for social workers. So we are aware of this intervention and its challenges but there has been no dialogue —not a formal one—between NICRO and DSD decisionmakers at the national and provincial level.
In light of the evidence and based too upon the role of civil society to advocate and model the kind of service delivery expected of government, NCS services should ultimately be sustained as government practice and from that vantage point, challenges are indicated. NICRO had a focused advocacy effort that included both the public and government but especially a lobbying of the Parliamentary Portfolio committee on Correctional Services. All of it is useful but it did not achieve a sustainable outcome. Ultimately perpetrators on NCS become a DCS responsibility and it is also possible that in terms of a sustainability plan, they might fund
© NICRO 2013 151
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
NICRO.
56 NICRO says it approached them and they say not via the channels that would result in firm memorandums of understanding (see Annexure 18). Whatever the case, an integrated strategic plan for sustaining the service beyond a NICRO offering was never located.
All the data suggests that the NCS sentencing programmes are essentially owned by NICRO and to some degree have been viewed as NICRO offerings with no plan to pass ownership of such programmes to CJS departments. This may be a valid viewpoint (the CJS Departments would not be ready anyway). At this writing, it is NICRO which must decide on sustaining the
NCS services and growing them in partnership with national stakeholders —all of which with the possible exception of the DSD expressed willingness (see Annexures18, 20, 21, 22 and
23). Strengthening the capacity of government departments to provide NCS services would be a long and serious effort but these initial steps would be required:
ï‚·
NICRO drafting a strategy suggesting what is feasible in terms of activities and roles (taking into account the findings of the evaluation there might be a more developmental strategy for some areas and NCS services in other kinds of areas)
ï‚·
Approaching the Justice, Crime Prevention and Security Cluster (JCPS) at national with such a strategy
ï‚·
Formalising partnerships (DCS, DoJCD, NPA, DSD, SAPS, Legal Aid, and others)
ï‚·
Lobbying DSD at national (logically they cannot focus on child justice only because children are raised by adults and adult criminals do not make good role models)
The project manager said that NICRO had approached the Judicial Education Institute during the time of the evaluation and while both appropriate and helpful as one component of an integrated plan for institutionalising the services, this is insufficient. Pending and needed is the
56 Jules-Macquet, Regan (17 August 2010) Discussion on MTR findings, NICRO National
Offices, Cape Town
© NICRO 2013 152
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013 development of a more intensive and fully strategic advocacy effort that addresses the findings as given in Section 7 above.
REPLICABILITY
This section considers replication with regard to NCS services (since the indigenous justice model was not replicable as designed):
ï‚·
How replicable is the process that introduced the NCS services?
ï‚·
What specific innovative aspects are going to be replicable?
ï‚·
What is replicable outside of South Africa?
ï‚·
What are the circumstances or contexts in which this programme is replicable?
HOW REPLICABLE IS THE PROCESS THAT INTRODUCED THE NCS SERVICES?
First, there are limits to civil society organisations or NGOs like NICRO delivering such services because they operate on funding cycles (e.g., rather than expand, NICRO will cut back from 45 sites to perhaps 20 during 2013) and there are few if any other civil society organisations with a similar capacity to develop and deliver therapeutic programmes. For the programme to be replicated, departments like DSD, DJCD and DCS still need to be lobbied to either pay for the services or provide the facilities for it (this was on-going at least a provincial level). NICRO demonstrated that it can implement such practices but did not capacitate government departments or other stakeholders. Furthermore, government has demonstrated that it requires NICRO to help provide NCS services and build further capacity:
ï‚·
Alternative sentencing practices have been part of the Criminal Procedures Act for decades but never successfully implemented by government (weak monitoring by DCS)
ï‚·
No interview, focus group, survey or evidential fact suggested that capacity exists at national level to really assume full responsibility for NCS services
(without NICRO training, support)
© NICRO 2013 153
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
ï‚·
The Department of Correctional Services has expressed strong interest in NCS and are a willing partner but when comparing their pre-sentence reports to those of NICRO, they do not have the same skills levels
ï‚·
The Department of Social Development is focused on such services for youth offenders only and expressed very limited to no interest in developing such services for Adult offenders but this is extremely short sighted (children come from Adults and are influence by them!) such that they really need to be lobbied more since their focus is not holistic
ï‚·
Government does not have access to the therapeutic programmes (NICRO
’s own therapeutic programmes are copyrighted by the organization).
NICRO could play a role in helping other organisations develop programmatic content or develop programmes for government departments (outsourcing). NICRO had done this before
(e.g., worked with the DSD to develop programmatic content for child offenders). No such arrangement has yet transpired in terms of adult offenders but this could be the outcome in the months and years ahead
WHAT SPECIFIC INNOVATIVE ASPECTS ARE GOING TO BE REPLICABLE?
Theoretically government departments can replicate: (a) the referral system: (b) developing stakeholder committees to discuss sentencing and offenders, (c) training and deploying social workers and (d) monitoring and evaluation systems. Various departments have expressed interest but at this writing there had been very little work at national (this where the work should start) to consolidate a plan. Government cannot easily replicate the NICRO programmes which have been developed over periods much longer than this 5 year project.
WHAT IS REPLICABLE OUTSIDE OF SOUTH AFRICA?
All aspects of the intervention are replicable in wealthy countries (South Africa is a middle income country) where there are sufficient funds, social workers, expert programme developers, and infrastructure (in courts and communities). The restraints in South Africa were mainly in terms of resources (a shortage of social workers, insufficient funds for more than on
NCS SW per site, shortages of vehicles and petrol for aftercare, cramped spaces in courts,
© NICRO 2013 154
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013 etc.). Since South Africa is one of the wealthier African countries and somewhat challenged, it is doubtful if this can be replicated in many other African countries. Some middle income countries in South America like Argentina or Brazil could replicate this. Wealthy countries in
Europe and North America could too. Culturally, there would have to be a strong belief in restorative justice.
WHAT ARE THE CIRCUMSTANCES OR CONTEXTS IN WHICH THIS PROGRAMME IS
REPLICABLE?
The context for making it replicable would be NICRO viewing itself as modelling NCS services
(fewer sites of excellence) and then advocating that government and even other civil society organisations in the years ahead help to roll this out more widely (clear partnerships agreements are required). At the point of authoring this (January 2013), NICRO was negotiating with government departments but also reducing the service points for NCS services. This would be better for advocacy but who can manage and deliver the services better than NICRO? The therapeutic programmes are copyrighted products continuously developed by NICRO over a long period of years but new products can develop and NICRO can play a role in providing these or even selling them.
Politics attend to this to this too. Should products and programmes emerging from a funded civil society organisation be owned or shared? Mike Batley of the Restorative Justice Centre
(see also Annexure 24) commented on this as follows:
A National NPO summit is going on in Boksburg right now and across the social development spectrum we dealt with this question of ownership in a workshop.
One of the dynamics is about ‘haves and have-nots’ in terms of funding for civil society. This gets very political and affects established organizations like
NICRO. There are thousands of emerging organizations and people who cannot obtain government funding and are looking for a way to transfer funding from established organizations to them. Non-custodial sentencing cannot easily be provided by emerging organizations!!! The established organizations respond by closing in –a laager---so maybe one of the roles is to see what
© NICRO 2013 155
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013 partners NICRO can get on board in terms of developing capacity. Can NICRO even handle another 20 sites or even the sites it has now? Do not contain this to NICRO. How do we expand this to other NGOs?
NICRO was developing broader national partnerships but this was not a fait accompli and there were also signs it was moving toward a business model and looking toward advocacy work with other civil society organisations making the NICRO willingness to partner at national somewhat uncertain to the evaluator. Clearer was departmental willingness to partner with
NICRO from national levels, especially in terms of:
ï‚·
The Justice Department (Annexure 21)
ï‚·
The Department of Correctional Services (Annexure 18)
ï‚·
The Magistrate Commission (Annexure 23)
While DSD national takes a narrow view in not expressing any interest in adult based services, much power lies with the provincial DSD (they follow guidelines from national but are quite independent) and the Gauteng Provincial DSD was most interested. The DSD must be credited in many provinces with helping to support the effort both in partnership and in terms of subsidising NICRO to undertake assessments.
NCS services will continue as a NICRO offering in the years ahead and this is now the best and only approach until formal partnerships develop with government departments at national level (dependent on will from all sides). The situation is such that the willingness is there to develop NCS services into a state-owned practice but such a transition would take a very long time. NICRO cannot really lose by sharing its learning and developing organisational capacity in government and amongst other NGOs with the capacity (there are few). Thus, it can be argued that while NICRO ’s approach and methodology included some efforts to develop partnerships capable of sustaining the service, this is not a quick fix. There is time to consolidate partnerships in Pretoria (DCS, DoJCD, DSD, SAPS, Legal Aid, and NPA) and who else is going to sustain the service in the interim but NICRO? Perhaps some partnerships could not be formalised before this point in any case since this is the first formal evaluation of
© NICRO 2013 156
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013 the services
—the findings might be one basis for deciding on sites, partnership roles and other factors.
WHAT ARE THE CIRCUMSTANCES OR CONTEXTS IN WHICH THE NCS SERVICES ARE
REPLICABLE?
The most important context in which the services are replicable as modelled is one in which there are a sufficient number of SWs and also one in which they are paid well and compensated in terms of benefits and learning opportunities for retention as NCS workers
(constant SW turnover is too costly and disruptive). Such conditions do not obtain in South
Africa —the requisite skills levels, numbers and financial resources are available in many
European Countries and in North America. Within South Africa, there are shortages of professional social workers (at the DSD or DCS or at NICRO) and owing to both that and the inability of NICRO to compete with the DSD in terms of salaries and benefits, they tend to come to NICRO for experience and then leave for the DSD or even DCS —both offer better salary packages.
The context for replicating the services in a middle income country like South Africa where there are SW shortages is a single comprehensive strategy that can clearly define the roles of government departments and stakeholder in an interdepartmental and multi-agency strategy.
For instance, NICRO cannot address the material circumstances of an offender and this is something that can lead to recidivism. Civil Society and government departments have to combine efforts and develop a strategic plan in which each stakeholder has a clearly defined role. Memorandums of understanding and even contractual arrangements are needed to ensure that all facilities, services and funding are available on a consistent basis. No formal contractual arrangement of that kind exists even though several departments like DCS and
DoJCD expressed interest.
Another context for replicability is a draft strategic document for discussion —meetings and minutes do not emerge into strategy. Strategy is a meticulous process with objectives, indicators, roles, delivery mechanisms and dates all drafted and then workshopped according
© NICRO 2013 157
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013 to each detailed and integrated component. Who should draft this strategy? DCS thought
NICRO must court and lobby them and NICRO does have the expertise to draft a strategy.
The reality is that NICRO probably needs to initiate the effort and while a civil society organisation doing this is properly termed ‘lobbying and advocacy’, it can be called something else like ‘team-building’ because the will is there amongst most departments and most South
Africans construe the term advocacy as confrontational and fail to see it in a broader context.
Once a draft strategy is constructed, the focus should probably be on the JCPS cluster (the head of this cluster is Pieter du Rand who should be contacted). If this fails the test, then
NICRO will have to turn to stronger advocacy.
Another context for replicability is sufficient funding. This is where civil society so often fails because they have funding cycles that disrupt services and this intervention was just long enough to capture that in the data. For NICRO to even draft plans and engage in strategic development to lobby stakeholders will require funds that it may not have from mid-2013 onward. Fewer NCS sites (e.g., perhaps 20-25 highly effective sites) might be one way to release the energies of the organisation for more intensive and effective lobbying activities but the expertise lies with NICRO and the willingness of the organization to drive a campaign to replicate the services more widely. The reality is that making the service available nationwide requires rigorous planning amongst many partners, much training, and a long-term strategic plan of 10 or even 20 years.
© NICRO 2013 158
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
INNOVATION AND LESSONS LEARNED
Analysis of the data clearly identified innovative practices and certain lessons learnt.
Innovation is discussed in section 4.1 and then Lessons Learnt in Section 4.2.
INNOVATION
Four innovative aspects of the project will be discussed and then a conclusion offered, yielding five sub-sections as follows:
ï‚·
An indigenous based diversion programme
ï‚·
The use of NCS therapeutic interventions in alternative sentencing for adults in South Africa
ï‚·
Various stakeholders led by a magistrate working together in local stakeholder committees across the country to develop, monitor and institute NCS
ï‚·
A national research-based campaign to identify and change the attitudes of both the general public and public officials toward NCS
ï‚·
A conclusion on innovation
AN INDIGENOUS BASED DIVERSION PROGRAMME
Drawing upon concepts of indigenous justice and reconciliation to pilot a model of referrals to
NICRO programmes that fit with the cultural framework of traditional courts was innovative. It was not practical in terms of traditional courts as the best source of referrals (see Section 2.3.6
Logframe Ratings and Explanations of Chapter 3). NICRO then turned to schools and community work for the referrals but included innovative practices since indigenous stories were utilised, which had meaning for the participants from indigenous areas (fitting the learning into the cultural frame of reference for them) and this remains of value. Furthermore, NICRO developed Local Stakeholder Committees for the court and then included traditional leaders
© NICRO 2013 159
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013 where applicable, which is also an innovative practice, since rarely did they service on the case-flow management committees.
THE USE OF NICRO
’S NCS SERVICES IN ALTERNATIVE SENTENCING FOR ADULTS
The use of therapeutic programmes for adult offenders did not exist at all in South Africa before NICRO ’s NCS services (and no evidence could be found of this in other African states either). Previous to NICRO, NCS was comprised of suspended sentences, fines or community service with no attention to what causes offending behaviour or how to address those causal factors. Prior to NICRO, few magistrates wanted to refer to these offerings and despite South African law and international agreements to make prison as the option of last resort, DCS could not monitor offenders effectively and so few magistrates could select this option. Thus, NICRO developed a completely innovative approach through the use of therapy at 45 sites to reform offenders so that existing law could be implemented.
This effort also involves four innovations in terms of delivery: (a) the professional assessment of perpetrators at the pre-sentence stage to determine if a therapeutic intervention can help change his or her behaviour; (b) designing therapeutic programmes; (c) delivering the programmes; and (d) monitoring the change. The data in the Chapter 3 findings showed that a, b, and c were well achieved and d only to a certain degree as most offenders were incompletely monitored during the post-sentence period. Monitoring is a national challenge and not just a NICRO challenge (in fact, DCS holds the responsibility nationally) such that this gap needs to be addressed in concert with a large number of stakeholders (e.g., NICRO,
DCS, SAPS and possibly support from community police forums at the local level) and this gap has been discussed throughout this report. However, the first three components were delivered effectively and these were the innovative components that made it possible to assess behaviour change. This is an appropriate and innovative response to increasing crime levels too--offenders take responsibility for their offending behaviour and undertake therapies that can facilitate a behaviour change.
The NCS Services can be contrasted to a CJS system which more typically incarcerates the guilty without any concern as to whether or not the punishment facilitates a behaviour change.
© NICRO 2013 160
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
Perpetrators receive prison sentences only on the basis of guilt with no data at all on what might reform them, factors which differ tremendously from perpetrator to perpetrator. Thus, prison rarely works as a mechanism to rehabilitate —it is rather society’s mechanism of punishing the guilty but the end result is reoffending and recidivism rates so high (in South
Africa) that estimates range from a very conservative 50% to a high estimate of 95%. Of course, prison is not the place to learn appropriate socialisation skills as hardened criminals mix with first time offenders so that the behaviour of the latter worsens. Criminal skills are imparted during peer interaction in prison that strengthens the resolve to commit crimes. The individuals’ sentence to prison expose them to conditions that are so degrading that respect for human rights lessens rather than increases by the experience.
The delivery of NCS services at 45 sites laid the foundation for more innovation in terms of utilising other kinds of restorative justice practices. The structures are there to do this especially in the NICRO targeted communities where an identifiable shift in thinking from retributive justice to restorative justice has occurred --repairing the harm caused by crime — owing to workshops with communities and before and after polling (from 44% in favour of rehabilitation programmes and 54% in support of more prisons at the start of the project up to
70% agreeing that certain kinds of offenders can be rehabilitated without going to prison--see
Annexures 24 and 25)
VARIOUS STAKEHOLDERS LED BY A MAGISTRATE WORKING TOGETHER IN LOCAL
STAKEHOLDER COMMITTEES ACROSS THE COUNTRY TO DEVELOP, MONITOR AND
INSTITUTE NCS
The Local Stakeholder Committees (LSC) driving the project was a most innovative practice from two perspectives.
ï‚·
First, a magistrate, the sentencing officer of the court, leads the steering committee and this is rare as magistrates in South Africa have never before participated on a multidisciplinary forum focused on therapeutic rehabilitation for offenders
ï‚·
Secondly, NCS promotes a multidisciplinary approach to sentencing practices by bringing together leaders from key government departments such as Justice, the National Prosecuting Authority (NPA), the Department
© NICRO 2013 161
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013 of Social Development (DSD), Correctional Services (DCS) to monitor facilitate and implement the use of therapeutic NCS practices
These two features saw the CJS operating differently than before: departments and service providers were found working together in an integrated fashion on Local Steering Committees
(or otherwise NICRO serving on the Case Flow Management Committees at the courts) to smooth the running of NCS services for 3,960 offenders at 37 or 45 sites around the country as of 30 July 2012
This also occurred at a particularly opportune time: just as the newly promulgated Child
Justice Act demands that courts become involved in therapeutic sentencing options in three phases
—pre-trial, trial and sentencing stage. NICRO is a pioneer and stepped outside of the box again to work with adults in the same manner (many departments like Social
Development only want to consider juveniles for therapeutic sentencing). NICRO did not win over every stakeholder but had CJS professionals working together discussing and monitoring therapeutic sentences for offenders.
A NATIONAL CAMPAIGN TO CHANGE PUBLIC AND OFFICIAL ATTITUDES TOWARD
SENTENCING
NICRO engaged in an advocacy effort to promote the concepts of rehabilitation and restorative justice that had never been undertaken before in South Africa and included:
ï‚·
Community Engagement Workshops at all sites with public attitude surveys
ï‚·
National Media efforts involving all mediums (radio, television, internet)
ï‚·
Lobbying government officials (particularly through parliamentary portfolio committees)
ï‚·
The use of Facebook to promote awareness
ï‚·
Marketing publications, logos, and branding to increase public understanding and awareness
© NICRO 2013 162
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
The employment of every kind of medium to promote the NCS message was innovative and this was the single most comprehensive effort to change sentencing practices ever experienced in South Africa . One can say that this blitz and the national advertising on television caught the attention of the nation and had experts and lay people discussing NCS whereas there was very little discussion in 2007 or 2008.
Most significant to the evaluator in terms of a particularly innovative approach were the
Community Engagement Workshops which took place mainly where the services were being offered (44 out of 45 sites 57 ) and in schools, malls, community halls, community polices forums and public venues where information, booklets, brochures, NCS cards and other materials were distributed. It is innovative because it also identified impact in terms of before and after surveys of participant attitudes: as many as 70% came to agree that offenders could be rehabilitated when at the start of the project (2009) this was in the region of about 45% of the participants thinking this way (Annexures 28 and 29). Supervisors (85%) were convinced that these events and activities were really making a difference particularly indicated by increasing numbers of visitors to the NICRO offices and increased referrals after the public presentations (Annexure 8, Section 8.2.9). The innovative approach was also most revealing because indications are that the public support rating for NCS Services are higher in the targeted communities (mainly poor with offenders who are family members) than amongst professionals in the CJS sector (the nature of this divide should be further investigated as it could be class related).
The described change in public attitudes was not just a product of local efforts either but had also been influenced by a highly successful national media campaign that reached millions of people. Radio, TV and newspaper coverage was conservatively valued at nearly R10 million for 2008-2013 because (a) print coverage alone was calculated as having a R4.6 million advertising value with a reach of over 2 million people —see Annexure 11; and (b) the expense of calculating the reach of all the media coverage was not in the budget and therefore impact exceeds what can be measured. It can be said that millions of South Africans were exposed to messages in newspapers, radio, television, and information online.
57 Durban was the only site at the time of this study where such community engagement had not taken place.
© NICRO 2013 163
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
Making use of Facebook was innovative and generated 278 members discussing the innovative sentencing practices. There was also a Knowledge Forum (Ning Change to
Change) that had 153 members and while the traffic might seem low this attracted research and media interest such that there were spinoffs in terms of disseminating findings. An electronic newsletter reached another 390 people.
CONCLUSIONS ON INNOVATION
NICRO was not only innovative but the innovations were mostly effective (a few shortcomings as described above and in the previous chapter) in terms of outcomes. NICRO established new services and new ways of thinking that could be measured offering offenders therapy and changing public thinking around sentencing practices to enable modelling of these services at
45 sites and sustaining these at perhaps half. There is a long way to go but one can now find average people on the street talking about alternative sentencing practices and even expressing some dissatisfaction with the lack of innovation amongst departments within the criminal justice system which are failing to address the criminogenic thinking and factors that produced the crime. Five or six years ago, this was not occurring. Only NICRO promoted
NCS services and therefore this innovation can attributed to them. The road ahead is long if viewed as shifting the entire culture of a country but that it can be shifted was established.
Ensuring that innovations enter into the CJS system is not complete and it is ultimately more important than even community opinion. The key stakeholders that can bring about change are magistrates, DCS, Legal Aid, the NPA, and SAPS. Each stakeholder needs to be addressed differently too. For instance the data shows that magistrates do not need to be convinced of the value of the services (most are convinced) but the challenge for them is in terms how the Judiciary measures performance (e.g., cases completed; speed of case processing) and then the police perhaps are unconvinced of the value of the services (they think it is about letting the criminal escape punishment) and their indicators for performance are arrest rates (this measure clogs the courts and slows the speed of case processing).
Innovation is still required to address the social worker shortage, a factor which will change very little over the next 20 years (many actually leave SA for higher salaries elsewhere).
© NICRO 2013 164
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
Magistrates need to be able to refer knowing that a SW is there who can handle the referrals and offer feedback. Some magistrates quit referring owing to SW turnover and lack of feedback on cases (there cannot be inconsistency and turnover —an SW needs to be in a post at all times and provide a consistent service).
LESSONS LEARNT
Many of the biggest lessons are addressed in terms of nine recommendations on programme design and 3 others on programme management (next chapter). A challenge as big as high national rates of recidivism (in a 50-90% range) requires a multi-disciplinary, inter-departmental team from national to local level working according to a single, integrated plan with areas of responsibility and measures for monitoring and evaluating progress and outcomes. Given the size of the challenge
—providing all the required strategic components for NCS services nationally--is a huge effort; too enormous for any one department or stakeholder. For instance,
NICRO alone cannot address:
ï‚·
The material circumstances of an offender, which also leads to reoffending (e.g.,
23% of South Africa’s prison inmates have been sentence for economic crimes 58 )
ï‚·
A system for measuring the recidivism rate accurately without working with DCS
ï‚·
Implement a national system of NCS (this is a government job)
ï‚·
The monitoring and tracking of offenders (this job is a statutory responsibility of
DCS; and NICRO fell short on providing this)
ï‚·
Address the national shortage of social workers (this a national and interdepartmental and inter-sectoral challenge)
ï‚·
Provide all the personnel required to deliver a full range of NCS services (NICRO
SWs were actually multi-tasking and had 14,000 offenders been sentenced to
NCS services, it is doubtful it NICRO had the capacity to actually handle this target without dedicated staff)
58 Ndenze, Babalo (11 August 2012) New ministe r ‘faced with mammoth prison task’ Weekend
Argus, p. 11
© NICRO 2013 165
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
ï‚·
Ensure that magistrates refer (this requires support from the Judiciary, the
Magistrate Commission, Justice College and others)
ï‚·
Link all departments into a single strategy (the JCPS cluster is needed to support this impact)
ï‚·
Provide all the research (external evaluators are needed and they must work from a baseline and pre-intervention measurements; DCS and SAPS must help with measuring recidivism rates using randomised control groups)
ï‚·
Provide that the Department of Justice and the Judiciary agree with NICRO on a suitable time period for an assessment
ï‚·
Provide for all the advocacy required to convince key national decision makers of the value of the services (e.g., magistrates must convince other magistrates)
ï‚·
Clarify exactly how the services will be sustained and replicated nationally (there were only hopeful signs such as focus group with the top 5 Deputy Directors at
DCS expressed a willingness to try to sustain the services throughout the country)
So, the key lessons relate to comprehending the scale of the task--not to be daunted by it either--and then having the focus to engage the process of integrating stakeholders at a national scale using a strategy. This is professional work requiring a strategist to create first a
Draft Strategy that shows how national stakeholders can be integrated into a comprehensive strategy from the highest levels of government (Deputy Ministers and Ministers at national and in the President’s Cabinet including the President) to the local level to provide the relevant services. This was not the outcome and not the programmatic focus either and therefore it can only be recommended that one be drafted and workshopped with key departments and result in memorandums of understanding and contractual arrangements for delivery.
These are other lessons issuing from either modelling successful practice and from gaps in practice:
ï‚·
Have a retention plan in place for Social Workers since turnover is not good for the programme (examine all policies to ensure that salaries, benefits, training opportunities,
© NICRO 2013 166
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013 succession planning, mentoring and promotion policies are in place) and ensure that the office ambience and environment is friendly so as to promote happy social workers (see
Annexure 7.3 question 2)
ï‚·
One SW per site on NCS is not adequate to provide the client or SW with the most appropriate conditions for therapy (support is needed at least in terms of social auxiliary workers)
ï‚·
Try to avoid, whenever possible, SWs multitasking between service streams as an SW devoted to NCS services is more effective and multitasking SWs reported stress, which can also lead to a reduction in client services (see Annexure 7, Section 7.3, questions 7,
8, 9, 10/11,12, 13 and 19
—all indicating too much stress)
ï‚·
For an indigenous model of the services (either diversion or NCS) Include traditional leaders on the Local Stakeholder Committees for the court services and combine this with pro-active community engagements to prevent crime (owing to distances and other problems this can be more effective than working with the traditional courts)
ï‚· Don’t underestimate the time required for trust-building with magistrates and have a plan to get referrals going at a site as this start-up can be very slow; it might take a year of meetings, processes, and the magistrate might offer an initial referral or two to test the waters, and with feedback (they will quit referring if he or she gets no feedback on outcomes) magistrate trust will increase (rushing this process to meet targets can be counterproductive)
ï‚·
Clearly establish for all stakeholders the time frames for an assessment explaining under what circumstances an assessment might be done in one day and under what circumstances it might take two weeks or more (allowing a range is better than stipulating a precise period because ‘an edict’ on time periods scares busy magistrates away when in fact 79% of SWs said that assessments could be done in under 7 days given the suitable conditions (see Annexure 7, Section 7.3., question 7)
© NICRO 2013 167
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
SUMMARY OF RECOMMENDATIONS
The recommendations mainly hinge upon and integrate around programme design features.
NICRO has d emonstrated the viability of NCS services and that it can manage such services as a NICRO offering and plans to as one of its service streams. So the need to focus on programme management is not as great as it was at the time of the Mid-term Review although there are four items requiring attention (further below).
The overarching recommendation and the one to which all recommendations refer is that
NICRO engage the national departments and stakeholders in a joint strategic plan to sustain the services more widely. Annexures 17-25 show that all the main national stakeholders express willingness on that point and while it is known that NICRO struggled to consolidate partnerships (not always seeing the same degree of willingness as reported in this study), the evaluator found that efforts to consolidate a plan had not been fully tested (i.e. it is too early to say that all approaches were tested and failed). This might be a tough path but it must be understood that if the NCS services can be implemented at a national level it will produce reforms needed throughout the entire CJS (i.e., these services can be a catalyst for addressing many dysfunctions —and major ones—amongst and between the CJS departments). NICRO was very consumed in deliverables at 45 sites (understandably) and it is being recommended that NICRO refocus again (try a new angle) on integrating all the departments, bodies and agencies mandated to deliver justice into a plan so as to sustain the services as a national offering and replicate it.
RECOMMENDATIONS RELATING TO PROGRAMME DESIGN
The recommendations relating to programme design are discussed under sub-headings A-I below and apply to NICRO specifically in terms of taking the lead in developing the partnerships to sustain and replicate the services nationally. These are to:
A.
Reduce the number of sites to ‘sites of excellence’ so as to design a service that can be presented as a model for national roll-out
© NICRO 2013 168
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
B. Design a long-term DRAFT strategy for providing sustainable and replicable services nationwide as a discussion document and distribute it to key stakeholders, national departments and especially the JCPS Cluster Body
C. Follow up on the above discussion document by meeting with the JCPS
Cluster and each department at Deputy Director level with the objective of integrating stakeholders into a long-term strategic plan for the delivery of NCS services
D. Enter into a partnership agreement with SAPS and the DCS (Community
Corrections) that includes control groups for measuring the impact of the NCS services on recidivism
E. Design with other stakeholders a magistrate-to-magistrate programme of sharing information about NCS services and a programme to educate new magistrates at the Justice College
F. Design with other stakeholders a joint strategy for monitoring offenders, providing aftercare and tracking them consistently at all sites
G. Design a comprehensive strategy for providing sufficient SWs for NCS services in view of the SW shortage in South Africa
H. Design a community-based model for working in indigenous areas and drop the existing indigenous based model of working with traditional courts
I. Design a time-based model for various kinds of assessments so that magistrates can understand a specific time range for completing different kinds assessments
REDUCE THE 45 SITES TO FEWER SITES OF EXCELLENCE SO AS TO MODEL A
SERVICE THAT CAN BE PRESENTED FOR NATIONAL ROLL-OUT
One of the main MTR recommendations (see Section 2.4.5b and 2.4.5c in the MTR) was to reduce the focus on achieving a large number of new sites and referrals (this was too much the focus) and rather concentrate on gaining the support of national role-players and bodies in order to sustain the services in the long-term and over a much wider area of the country. In other words, the evaluator viewed modelling best practice as a method of advocacy which might integrate all national stakeholders into a national plan to sustain the services and not
© NICRO 2013 169
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013 demonstrating that NICRO could manage 45 sites
—not all of them sites of excellence--that would later be reduced to 20-30 anyway once the funding cycle was completed.
By the time of this writing (early 2013), NICRO was already choosing the sites where NCS services could be retained. On that point, these should be the ones that can be modelled as sites of excellence in terms of some of main recommendations of the report such as:
:
ï‚·
Operate with 2 NCS SWs or some combination of SWs and professionals who can address all components of service delivery
ï‚·
Offer highly professional services (dedicated, professional, well-paid, and senior SWs in the job since high SW turnover and competition for the very limited human resources that are available is wasting the time of the experts in the organisation who spend their time familiarising SWs with the job requirements
ï‚·
Showcase the best partnership practices (e.g., consider the need for a good partnership with DCS and SAPS for the monitoring, aftercare and tracking of offenders)
ï‚·
Locate services where NICRO can operate most economically (e.g., from
NICRO offices instead of court houses as feasible; avoid rural areas that involve too much petrol and time)
In other words, work in fewer areas of the country, from NICRO offices when possible, with a sufficient number of experienced SWs to offer ‘top-notch’ services. Otherwise, NICRO is in the position of overstretching delivery beyond its resource base and being dependent on young SWs sometimes working alone in courthouses and looking at NICRO as a stepping stone to better salaries and opportunities elsewhere. The recommendation is to move the organisation toward working with experienced professionals who can provide ‘top of the class’ services and demonstrate precisely how to reduce the costs of high imprisonment and realise social benefits at the same time.
The sites selected should not be a unilateral NICRO decision but one made jointly with NCS partners because certain partnerships must be functional at those sites too such as places where there are partnerships for tracking and aftercare since these are indispensable practices.
© NICRO 2013 170
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
DESIGN A LONG-TERM STRATEGY FOR PROVIDING SUSTAINABLE AND REPLICABLE
SERVICES NATIONWIDE AS A DISCUSSION DOCUMENT AND DISTRIBUTE IT TO KEY
STAKEHOLDERS, NATIONAL DEPARTMENTS AND ESPECIALLY THE JCPS CLUSTER
BODY
This recommendation is to NICRO (initially) and is contingent upon them having the will, budget and time to generate a DRAFT plan that can be workshopped with key departments and the JCPS cluster. NICRO expressed to the evaluator willingness to sustain the NCS services as a NICRO offering but the evaluation recommends a multi-disciplinary, interdepartmental effort to sustain NCS services as a nationwide practice (this involves replication). NICRO cannot offer the complete package of services required in all parts of the nation without a consolidated national partnership effort. The data showed that there is a shortage of SWs nationally; NICRO SWs multitasked between service streams (not ideal); funding cycles disrupted services, monitoring the offender was less than required, and NICRO simply had too few sites to be termed a national system.
Recommended is a long-term (10-20 year) plan to develop and sustain the services as a national offering and such a recommendation is contingent upon willingness but it is argued here that civil society cannot offer permanent services to replace government service delivery and has no mandate. Civil Society is not government although some civil society organisations in South Africa are choosing to move toward a business model and if they become businesses without donations and public funding, then matters change. However, publically funded entities have an appropriate role in advocacy (to champion a public cause) and now that
NICRO has successfully modelled a system, it is recommended that NICRO develop a draft discussion document outlining and illustrating how the programme might be rolled out, which stakeholders are needed, and the role of NICRO is willing to play in support of a nationwide delivery plan.
In terms of a plan, some degree of specificity is needed but the purpose is to launch a national dialogue on the delivery of the NCS services because many key issues still need to be addressed in consolidating a partnerships (e.g., Legal Aid and the NPA are major sources for referrals since it is they who plea and not the magistrate but this was not well developed from national). Another issue might be changes in the Criminal Procedures Act (some want to see
© NICRO 2013 171
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013 these and some do not) and the appropriate application of that act. Such details have to be discussed amongst key stakeholders and these stakeholder roles need to be identified in a draft consolidated strategy to initiate a process that can result in delivering the services more widely. Every involved department and sector also has an area of expertise that can contribute to and improve upon the delivery strategy in as yet unknown ways.
The draft plan should suggest strategic objectives, targets, areas of responsibility, success indicators, timelines and a monitoring and evaluation plan. This was originally suggested in the MTR but never materialised in the course of the intervention and when the evaluator interviewed DCS in Pretoria (Annexure 17) it was found that all the Deputy Directors were most willing to partner with NICRO but said they had not been formally approached. This provides evidence that the essential and willing partners must be formally approached with a plan. It was also identified during the evaluation study that the Justice, Crime Prevention and Security
Cluster at national (a formal body that meets) should be a key focus of this kind of effort since this is the group that examines and develops ways for the entire cluster to work together. This body had not been targeted, which may help to explain why a consolidated approach never materialised. Approaching this one body would not be enough either as each department would have to be lobbied at the level of Deputy Director as well (NICRO says this was done but the national stakeholders said this was not done in accordance with protocol —see
Annexures 17-25).
FOLLOW UP ON THE DISCUSSION DOCUMENT BY MEETING WITH THE JCPS
CLUSTER AND EACH DEPARTMENT AT DEPUTY DIRECTOR LEVEL WITH THE
OBJECTIVE OF INTEGRATING STAKEHOLDERS INTO A COMPREHENSIVE AND LONG-
TERM (10-20 YEARS) STRATEGIC PLAN FOR THE DELIVERY OF NCS SERVICES
The DRAFT plan as described above must be distributed as a discussion document first and then later discussions must follow (do not present the document the same day as discussion) with the deputy ministers of key departments and the JCPS cluster around the draft strategy.
Do they buy in? Do they want to make changes? If it does not go anywhere out of lack of will, then one can say definitively that the nation is not ready for NCS services and it should
© NICRO 2013 172
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013 continue as a NICRO offering only. However, the data in the evaluation said the will was there but not the way. Hence, this set of recommendations.
More talk shops and big conferences are not being recommended and no educational campaigns either but rather working at department level (deputy ministers, especially at DCS to start) and then the JCPS cluster level (referring to a specific body by this name at national) to develop a way to resolve together all the challenges that prevent NCS services from functioning optimally (doing so will solve many CJS problems other NCS services too). Below are examples of what might be on the agenda (but all the recommendations in this chapter belong on that agenda too):
ï‚·
Rationalising the performance indicators amongst SAPS, magistrates, prosecutors and key departments because these are not the same in relationship to offenders to (e.g., SAPS chases after arrest rates while overwhelmed magistrates chase after a reduced court roll) and this is a major challenge that requires major reform and a strategy to reform it
ï‚·
Training for involved departments and stakeholders (NICRO might be paid to provide such training but this component cannot be pre-determined but must develop from an integrated strategy for delivering services in the manner recommended in this study as discussed in Section 5.1B above).
ï‚·
Change management because a major transition in organisational culture is required within each department and amongst them to provide such services-and NICRO might also want to consider forming partnerships to provide some of the change management required.
ï‚·
Addressing the national shortage of SWs to provide a service. NICRO could not compete with the salaries and benefits offered by the DSD contributing to
SWs leaving NICRO and a solution is needed but was not found (see Chapter
3, section 2.3.3 and 2..3.4 which helps to describe the severity of the challenges or Annexure 7 and Annexure 8 which offers the data)
ï‚·
How to enable magistrates to refer more easily (the vast majority did not want to delay very long for an assessment owing less to learning, thinking and attitudes than to structural conditions (i.e., pressures at the court; incomplete partnership building; no directives from national) and an integrated plan with departmental involvement from national to local is needed is part of the requirement
© NICRO 2013 173
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
NICRO TO ENTER INTO A PARTNERSHIP AGREEMENT WITH SAPS AND DCS
(COMMUNITY CORRECTIONS) FOR RESEARCH THAT INCLUDES CONTROL GROUPS
FOR MEASURING THE IMPACT OF THE NCS SERVICES ON RECIDIVISM
A partnership with the DCS and SAPS for research should have been identified and developed
( before implementation) so that recidivism rates could be measured (NICRO clients could have been compared with a control group of offenders under DCS supervision with no therapy). A focus group in Pretoria with DCS national identified this:
[The] Problem is that NICRO needs DCS and cannot monitor appropriately without partners like SAPS and DCS. People who commit petty crimes need the programme but DCS was never involved from the beginning. This is the gap and now when coming to roll out it out NICRO approaches DCS at national level
(question 1 Annexure 17)
The gap did not prevent the evaluator from establishing the value of the service but in terms of demonstrating impact on recidivism rates or sustaining it as a national offering, the gap is felt.
NICRO TO WORK WITH JUSTICE COLLEGE TO EDUCATE NEW MAGISTRATES AND TO
USE PEER EDUCATION AT SITES WHERE NCS SERVICES IS OFFERED
Justice College where magistrates are trained offers NICRO a targeted place for explaining the
NCS services for long term impact on practice (advocacy through education) and NICRO achieved this to a certain degree by working with Senior Magistrate Basil King, who testified to providing some 11 hours of education in this (Annexure 25). However, Basil King is no longer with Justice College and NICRO will have to reassess this activity to ensure that it continues.
Part of the long-term strategic plan should include a recommendation that was in the MTR (see
Section 2.4.5) to use magistrates convinced of the worth of the NCS services to visit active sites to better explain the programme and the reasons for supporting it. It is well known that magistrates learn best through peer education and these ‘magistrate-to-magistrate’
© NICRO 2013 174
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013 conversations might: (a) increase referrals at the sites where NICRO continues the services;
(b) provide the support for developing the services further. The magistrates could also make presentations to prosecutors and Legal Aid Practitioners.
NICRO TO WORK WITH OTHER STAKEHOLDERS ON THE DESIGN OF AN INNOVATIVE
AND JOINT STRATEGY FOR MONITORING OFFENDERS, PROVIDING AFTERCARE AND
TRACKING THEM
It is not acceptable to have any programme in which offenders are released into the public and not monitored, provided with aftercare and tracked as was the case with DCS monitoring of its own offenders (explaining why NCS failed in the 1990s). NICRO then encountered the same challenges in 2009-2013.
NICRO has policies and procedures for aftercare and tracking ( the SW should make regular contact with them from the time of the initial assessment until a year after coursework is complete) but simply could not deliver on this completely. This policy is common practice internationally because in general the longer the follow-up, the lower the recidivism rate. That this took place at very few sites was indicated in this report as one of the biggest gaps in strategy:
ï‚·
Of 936 social workers surveyed on this question, 383 (41%) said they had tracked clients and this was for periods of a year or less: 93 clients for 1-3 months (10%); 164 clients for 4-6 months (17%); and 126 clients
(13%) for 7-12 months.
ï‚·
At the time of the MTR, more than half of all clients (57%) who completed programmes were NOT reached with aftercare
—so this situation of sites failing to establish the aftercare and tracking persisted throughout the intervention
ï‚·
One magistrate informed the evaluator at the time of the MTR and others at the time of the evaluation that it is not appropriate for NICRO to be held responsible for monitoring since: (a) this is a DCS role; and (b) NICRO is not a government agency.
59
59 Van Zyl, Judge D H (2009) Interview with Judicial Inspectorate for Correctional Services, 22
July 2010, Cape Town
© NICRO 2013 175
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
ï‚·
A no- tolerance policy to perpetrators who fail to abide by the rules needs be discussed and agencies need to cooperate to ensure swift responses to clients that cannot be contacted
ï‚·
The main reason NCS failed in the past was poor DCS monitoring
These challenges persist because NICRO alone is not adequate to a task that requires not only partners but national agreements amongst many stakeholders. For example, monitoring and tracking is really a DCS role but the actual sentence passed by the magistrate has bearing on whether or not the offender is appropriately monitored. Therefore monitoring the offender should be part of a national plan (A-F above) that comprehensively examines and resolves all issues relating to the delivery of the services.
NICRO TO DESIGN WITH DSD, DCS, AND THE JCPS CLUSTER A COMPREHENSIVE
STRATEGY FOR PROVIDING SUFFICIENT SWS FOR NCS SERVICES IN VIEW OF THE
SW SHORTAGE IN SOUTH AFRICA
From the start of the project to the end of the funding cycle, the shortage of social workers and their high turnover was a precarious situation and having only one SW per site meant that the entire service could shut down while waiting for a new SW. This needs to be addressed as the second of the two big challenges referred to above. NICRO can address some of this with better retention packages, more benefits in terms of training and some kind of reward system but this would not be sufficient if salaries and benefits at the DSD remain higher. NICRO also needs highly qualified SWs for this kind of work and cannot be in a position of training younger more inexperienced SWs until they feel ready to leave for the DSD or DCS.
It would appear to be better strategy for NICRO to reduce sites to those where seasoned, wellpaid SW staff can deliver exemplary services rather than training SWS for delivery to the DSD or DCS.
© NICRO 2013 176
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
DROP THE EXISTING INDIGENOUS BASED MODEL OF WORKING WITH TRADITIONAL
COURTS BUT EMPLOY THE LEARNING TO DEFINE A COMMUNITY-BASED MODEL FOR
WORKING IN INDIGENOUS AREAS
A separate model of diversion focusing on the traditional courts did not result in sufficient referrals and was very costly in terms of delivery. However, success was achieved in terms of:
ï‚·
Creating more representation for traditional leaders and their representatives on the LSC and case-flow management committees
ï‚·
A developmental approach to such areas to identify cases and social problems (such as in schools) before they reach the traditional court
This not only improved referrals but offered the kind of help that these communities actually wanted and that the court wanted, which offered evidence that fewer learners were entering the court system on the testimony of both the control prosecutor and the magistrate.
DESIGN A SIMPLE COMMUNICATION STRATEGY FOR CLEARLY AND EASILY
IDENTIFYING FOR MAGISTRATES (AND SWS PERHAPS) THE TIME PERIOD FOR AN
ASSESSMENT IN TERMS OF A RANGE PER TYPE OF CASE SO THAT THE
MAGISTRATE HAS SOME GUIDELINE FOR ESTIMATING WHEN AN ASSESSMENT CAN
BE PROVIDED
Magistrates who want the assessment process shortened are the same ones who are convinced of the value of the NCS services. For many magistrates the assessment is viewed as slowing the speed of case-handling and congesting the courts and this causes them to lose interest or not refer. They wanted NICRO to review the time it is taking to complete an assessment and reduce this without compromising quality. While NICRO cannot reduce it standards, it can develop a better communications strategy. Enough comparative data is available at this point to design a way to explain the time period for assessments in terms of a range of scenarios. From this study (Annexure 7), it can be noted that 79% of the SWs had concluded that assessments could be done within seven days under suitable conditions. Only
20% thought that 14 days were typically required (the SWs were very aware that sometimes up to 21 days might be required). This suggests that a magistrate demand for an assessment
© NICRO 2013 177
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013 within 14 days can be accommodated much of the time (but not always). The suitable conditions need to be described to them and if this can be done, referrals are likely to increase because it is the magistrates who want to refer that are put off by the way the time period has been addressed so far. This recommendation is also made in the context of all the others (a bigger problem is situated a national with consolidating stakeholders into a single plan), such that this is not a suitable strategy on its own but might help with increasing referrals.
FOUR RECOMMENDATIONS RELATING TO PROGRAMME MANAGEMENT
The four main recommendations on programme management are:
WHEN DEVELOPING A NEW SERVICE, DEVELOP AND IMPLEMENT A CHANGE
MANAGEMENT STRATEGY SO AS TO MANAGE AND MONITOR THE TRANSITION IN
ORGANISATIONAL CULTURE
A new service requires a change in organisational thinking, practice and culture because change inevitably meets with resistance by personnel at ground level. Furthermore, this is a large organisation and national may not be learning all it needs to know from ground level about the challenges at the sites. Not enough processes or time were devoted to the organisational shift required to embrace the new thinking behind the service and this was identified at the time of the MTR when it was found that:
ï‚·
Multi-tasking supervisors and managers saw the DFID project as one
‘owned’ by the PM (the evaluator’s observation owing to interviews from local to national)
ï‚· Project staff and supervisors perceived they were spending ‘too much’ time being paper-accountable to the PM (the PM was also in a position of having to re-request data that was not submitted properly because the line functions were not working)
Progress was made under the ‘supervision model’ based on project management data on the
NCS services and site visits, which showed that the services were more integrated into operations at the time of the evaluation than they had been at the time of the MTR. For instance, supervisors and SWs rated themselves in surveys as more capacitated to drive the
© NICRO 2013 178
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013 project (see Figure 3.18, Subsection 4.1 in Chapter 3). However, the PM was still frustrated as late as 2012 in terms of some provinces not implementing the supervision model as designed. Only gradually was the supervision model fully implemented in the Eastern Cape,
Free State, Limpopo, Northern Cape and Western Cape but not adequately in Kwa Zulu-Natal,
Gauteng and Mpumalanga even at the time or writing this (see Annexure 27).
In analysis, it appeared that the supervision model is useful but must be balanced with more opportunities for the SWs, staff, supervisors and provincial managers to meet nationally or regionally and really understand and work out the challenges of managing the cultural change within the organisation together . It will make it much easier for NICRO to shift thinking outside the organisation if it is also shifting within and toward more horizontal, participatory management. There is too much of a perceived gap from national to site and feelings that the national office makes decisions without consulting the organisation as a team (and little evidence in budget, strategy or activities of national conferencing on NCS). Just distributing supervision guidelines and enforcing the supervision model is not team-building especially if supervisors and SWs are already multitasking with other service streams (and it was clearly indicated that nearly all of them multitasked between service streams). It is important to get social workers away from the office environment and together in workshops where they can develop skills and talk about new ways of working and make their input into describing and resolving all the challenges. Most SWs interviewed and surveyed identified this as a big gap.
BUDGET SUFFICIENTLY
The project did not pay enough attention to line items in the budget during the project design phase such as allocating enough funds for:
ï‚·
Social workers, who were too few per site (only one no matter the number of referrals); underpaid; had no benefits and few training opportunities outside
‘the supervision model’ in the workplace such that turnover affected more than half the sites and some had turnover 3 times
© NICRO 2013 179
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
ï‚·
Aftercare, and tracking (non-delivery compromises the model and required are partnerships, landlines, cars, petrol, and social worker time--more than were budgeted--although there was also some under-expenditure on this)
ï‚·
Research can be a major cost but is a necessary one to budget for.
Documenting achievement and monitoring changes is ideally a year activity rather than a before-an-after one but there no funds were even budgeted for a pre-intervention baseline study, for obtaining recidivism data and even the project manager worked overtime to analyse data when she had no funds for contracting researchers
ï‚·
Well-resourced sites
On the first two bullets, NCS services really required two dedicated SWs and not one SW multi-tasking between service streams. For instance, monitoring the client on a suspended sentence and aftercare and tracking are not arbitrary but must be delivered 100% of the time but the evidence showed this was not the case. Furthermore, 77% of supervisors thought the
SWs were insufficiently trained —see Annexure 8, question 7). The salaries are not high and there are no benefits and this contributed to the constant turnover of SWs seeking better jobs or placements. This turnover also compromises delivery including the tasks just described.
On the last item, only the basic resources for delivery were supplied at sites and for some sites there were insufficient cars, petrol and personnel for tracking, no funds to assist clients with transport and much of the time no privacy for client interviews (the offices at the court were usually small and cramped). Aftercare is not dispensable but there were insufficient funds to supply this to all clients (and NICRO requires partnerships for this). One solution to the underbudgeting would have been fewer, better resourced sites where NICRO offices (with personnel, supplies, phones, fax machines, etc.) are in very close proximity to courts (there was not enough initial work on site selection and its impact on the budget either). The data also revealed that referrals increase if there is an SW at the court and so it may be best to work at courts near NICRO offices or have two dedicated SWs rotating between court and office.
The budget has to account for all services and over a length of time that is usually beyond one civil society organisation on funding cycles (what if a person is given a 10 year suspended
© NICRO 2013 180
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013 sentence and sent to NICRO and then two years down the line he cannot be monitored?).
This redounds to calculating the costs accurately and also to the theme of consolidating partnerships because one civil society organisation cannot carry the costs. This is also about learning and NICRO knows much more about the costs involved.
ENSURE THE MAGISTRATES RECEIVE FEEDBACK ON THE TREATMENT PLAN AND
OUTCOMES
Magistrates in surveys and interviews complained about lack of feedback on the progress of the offender. For instance one magistrate at Tembisa quit referring owing to no feedback on the treatment plan. Then, on admission of the SWs in surveys (Annexure 7, Section 7.3, question 20), it was documented that at least 40% of the magistrates were not getting feedback on the treatment plan — a serious gap.
This was also confirmed by supervisors
(Annexure 8, Section 8.2, and Question 12). There was also widespread confusion amongst
SWs, as to when feedback is required. After assessment and after treatment are correct and only 9 of 31 (29%) SWs could offer such a reply.
ALL EVALUATIONS SHOULD BEGIN WITH A PRE-INTERVENTION BASELINE STUDY
All evaluations (by definition) should start with a baseline study so that the evaluator (who should be external--the appropriate way to ensure objectivity) can study the project objectives and indicators and then design a study in ample time to take pre-intervention measurements that can test the programme theory. At the time of project design (2008), a baseline study for taking pre-intervention measurements was not part of the design. There had been some studies but a baseline study does not refer to a general study but to quite precise preintervention measurements related to the programme theory and its objectives. That there was no budget/time/plan for the kind of baseline study described was identified in the MTR but such a gap will have ramifications for the course of the intervention. A baseline study can:
ï‚·
Help to generate and consolidate partnerships prior to the start of the project
(a good evaluation by a seasoned specialist has this impact)
ï‚·
Help with site selection (e.g., NICRO set up sites to discover only later that resistant magistrates were unlikely to be referring but data could have been
© NICRO 2013 181
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013 generated on magistrate attitudes at proposed site as part of either the scoping study or the baseline).
ï‚·
Test the programme theory at an early stage and allow for modifications in design (in terms of the Indigenous Justice Project, funding, time and resources could have been saved with a proper baseline study as it is highly likely that the original programme theory would have been questioned with early data but this particular project was not even part of the MTR).
On the last point, it was fairly evident that distances and challenges with the traditional courts as examined in this report could have been picked up in the baseline/scoping study. In fact, it was actually covered in the mid-term review in terms of the NCS sites that rural areas have particular challenges for the efficient and effective delivery of services.
© NICRO 2013 182
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
ANNEXURES AND REFERENCES
19
20
21
22
23
ANNEXURE CONTENTS TO ANNEXURES
1 Achievement Rating Scales
2
3
Terms Of Reference
Research Timetable
4
5
6.1
6.2
List Of Research Participants
Documents Consulted
Client And SW Surveys (Excel
Spreadsheet)
SUMMARIZED CLIENT SURVEY DATA
[Removed For Confidentiality Purposes]
7
8
9
10
11.1-11.4
11.5
12
SW Surveys (31 Sites)
Supervisor Surveys
Provincial Manager Surveys
Magistrate Survey Data
Public Outreach 11.1-11.4
NICRO Media States In Excel
CPA
Reader)
Expenditure Report (Adobe
13
14
15
16
17
18
Client Interviews
NICRO Programme Offerings
National Manager Focus Group
Summarized Data On Eerstehoek
Indigenous Justice Pilot Project
Cape Town Focus Group
Pretoria Based Focus Groups And
Interviews
DSD Focus Group Pretoria
NPA Focus Group Pretoria
DoJ Focus Group Pretoria
SAPS Major General Geldenhuys
Magistrate Peet Van Vuuren (Serves On
Magistrate Commission)
24
25
26
27
28
29
30
31
32
Restorative Justice Centre
Senior Magistrate Basil King (Justice
College)
Value For Money Calculation (Excel
Spreadsheet)
NICRO Analysis Of Supervision Model
Community Engagement Workshop
‘Before’ Surveys By NICRO (Excel)
Community Engagement Workshop
Surveys (Third Phase In Excel)
NICRO Integrated Stats Sheet
Targets And Intake
Status Of The LSCs 31 July 2012
STATUS
Included in this report
Available on request
Included in this report
Included in this report
Included in this report
Available on request
Available on request
Available on request
Available on request
Available on request
Available on request
Available on request
Available on request
Available on request
Available on request
Available on request
Available on request
Available on request
Available on request
Available on request
Available on request
Available on request
Available on request
Available on request
Available on request
Available on request
Available on request
Available on request
Available on request
Available on request
Available on request
Available on request
Available on request
Available on request
The above annexures are available on request. They are excluded from this document because of space and size reasons.
© NICRO 2013 183
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
ANNEXURE 1: FOUR ACHIEVEMENT RATING SCALES
ANNEXURE 1.1 Achievement Rating In Terms Of Broad Purpose = 2
Objective
Statement
Achievement on broad purposes
Rating for whole programme period
Log frame
Indicators
Baseline and Target for
Indicators
Actual achievements demonstrated by end of programme
2
Comments on final results, including unintended Impacts
The purpose of the programme is to introduce and entrench noncustodial responses to crime within the
South African criminal justice system in order to offer more appropriate, empowering therapeutic interventions to address high crime levels and to prevent reoffending. This included:
ï‚·
Create a safer
South
Africa by contributin g to penal reform
ï‚·
Change offender behaviour and reduce crime
ï‚·
Develop, implement and promote the use of noncustodial sentencing in magistrate
Local
Steering
Committees
(LSC) established at 45 magistrate courts by the year
2013.
6000 offenders on NCS services by project end
There were four LSCs at magistrate courts in the country at the time the GTF project began; no offenders on NCS
Departments and service providers were found working together in an integrated fashion on Local Steering
Committees (or otherwise NICRO serving on the Case
Flow Management
Committees at the courts) to provide NCS services to 3,960 offenders 0-5 referrals (0 if the DoJ pilot sites from 2008-
2009 are discounted)
LSCs functional at 37 of
45 sites around the country as of 30 July
2012
3,960 offenders on NCS programmes or 66% of target attainment
(making it unlikely that
6000 target will be achieved)
The programme was preventing reoffending by both addressing the behaviour and by not exposing and petty offenders to hardened prisoners who have a strong influence on fellow inmates in terms of increasing the skills and propensity for committing crimes
NICRO aberrant first-time has introduced the NCS services and will sustain them in about
20-25 sites such that it cannot be said that the services were entrenched in terms of a national offering
936 clients (a 24% sample) were given a diagnostic test and their
SWs completed a survey on each of those clients to find that the risk factors for reoffending had been reduced (only a 1-3% variance on a scorecard for reoffending between
SW and client) indicating that the core behaviours that led to the offense were diagnosed addressed and through
Participation levels on
LSCs were less than desired that reviewed sentencing and promoted NCS services)
(See
Annexure 32 and/or
Annexure 7, question
21 that details the participation levels of the departments and stakeholders
There is still much work to be done in terms of consolidating partnerships to sustain the effort
© NICRO 2013 184
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
ANNEXURE 1.1 Achievement Rating In Terms Of Broad Purpose = 2
Objective
Statement
Achievement on
Rating for whole programme
Log frame
Indicators
Baseline and Target for
Actual achievements demonstrated by end of programme broad purposes courts period Indicators cognitive therapy behaviour
ï‚·
Develop an indigenous justice model based on referrals from traditional courts
(diversion)
Magistrates at 45 sites were presented with a more credible range of sentencing options so that they can implement existing appropriately 60 policy
Public and official awareness that there were viable and better alternatives to imprisonment was indicated to by a R10million-value media campaign and outreach activities at 45 sites involving magistrates and public figures
Comments on final results, including unintended Impacts although it will continue as a NICRO offering. See the detailed log frames in terms of outputs below
Target not attained as to offenders on programmes
Targets not attained for indigenous justice and hypothesis failed
The DoJCD was making space and facilities available for various aspects of the NCS services magistrate at 45 courts around the countries
(they were also a partner in the original pilot —some 5 years of partnering with NICRO)
In an August 2012 focus group with evaluator, the
5 top Deputy Directors of DCS expressed
Community engagement workshops from July
2010 with before and after surveying at 44 sites identified a shift in thinking from retributive justice-punishing offender--toward the restorative justice-repairing the harm caused by crime.
Client tracking and aftercare persists as the weakest areas of delivery and yet these areas are critical components of a fully-fledged NCS service
60 There were no therapeutic programmes for NCS until NICRO provided them and this enabled magistrates to refer to it knowing that part of that benefit, is increased respect for human rights and the rule of law. Otherwise, NCS had been available in South Africa but fell into disuse because: (a) the alternatives were not therapeutic
—fines, suspended sentences, or reduced sentences; (b) the public did not support these; and (c) monitoring by Correctional
Services of these cases was so weak that many magistrates simply quit supporting it.
© NICRO 2013 185
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
ANNEXURE 1.1 Achievement Rating In Terms Of Broad Purpose = 2
Objective
Statement
Achievement on broad purposes
Rating for whole programme period
Log frame
Indicators
Baseline and Target for
Indicators
Actual achievements demonstrated by end of programme willingness to work with
NICRO from 2013 to try to sustain the services at sites throughout the country (see also
Annexure 18)
NICRO measured a rise in public support in targeted areas from a
2009 baseline indicator of only 44% in favour of rehabilitation programmes and 54% in support of more prisons, to 70% agreeing that certain kinds offenders can of be rehabilitated without going to prison (see
Annexures 28 and 29 or discussion in Chapter 4 on Innovation)
Comments on final results, including unintended Impacts
To entrench a programme to change offender behaviour and reduce crime on a large scale is necessarily a multidisciplinary and multi-stakeholder effort that requires a national plan to integrate all stakeholders with memorandums of understanding and other components critical of strategic planning and this was not found.
ANNEXURE 1.2: DFID ACHIEVEMENT RATINGS APPLIED TO 7 INTENDED OUTPUTS
FOR AN OVERALL PROJECT RATING
See Chapter
3
Subsection:
Objective/Output Rating
2.3.1 A fully-fledged referral system developed for alternative sentencing at 45 magistrate’s courts
2
2.3.2 Therapeutic programmes available for alternative sentencing 1.5
2.3.3
2.3.4
Professional social worker expertise available at 45 courts
Expert alternative sentencing programme evaluations
3
2
© NICRO 2013 186
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
ANNEXURE 1.2: DFID ACHIEVEMENT RATINGS APPLIED TO 7 INTENDED OUTPUTS
FOR AN OVERALL PROJECT RATING
See Chapter
3
Subsection:
Objective/Output Rating
2.3.5 A national media and marketing campaign that increases public awareness of AS sanctions and restorative practices
2
2.3.6
2.3.7
A newly developed indigenous justice diversion project available
A developed knowledge base of non-custodial sentencing systems and practices to further public and sector understanding and promote awareness
Overall Project Rating = largely achieved despite a few shortcomings See details below
2
1
2
ANNEXURE 1.3 DFID LOG FRAMES ON OUTPUTS WITH DETAILED COMMENTS (SEE
ALSO CHAPTER 3)
Output 1: A fullyfledged referral system developed for alternative sentencing at 45 magistrate’s courts
Overall rating (explanation given in two indicators below) = 2
1-5 rating
Log frame
Indicators
Baseline and Target for
Indicators
Actual achievements by 31
July 2012
Comments on final results including unintended impacts
2
Partially achieved
45
Magistrate
Courts referring offenders to
AS interventions no less than
10X per month by project end
Baseline:
0-5 referrals
(0 if the
DoJ pilot sites from
2008-2009 are discounted)
© NICRO 2013
 A referral system for NCS services was developed, established and functional at 37 of 45 courts-
82% (Annexure
32)
 NCS services were provided to 3,960 offenders by 31
Jul 2012 (66%
The reasons for low referrals:


A tension exists between the magistrate’s desire to complete cases quickly and reduce backlogs (owing to high court rolls) and NICRO ’s professional requirement to conduct thorough assessments requiring up to two weeks or more, which are perceived to cause court delays (see
Annexure 10, question 6)
A small percentage of magistrates are making the referrals-- (sometime
187
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
Output 1: A fullyfledged referral system developed for alternative sentencing at 45 magistrate’s courts
Overall rating (explanation given in two indicators below) = 2
1-5 rating
Log frame
Indicators
Baseline and Target for
Indicators
Actual achievements by 31
July 2012
Comments on final results including unintended impacts of target) one or two per court), which produces insufficient referrals to achieve targets
 2 sites (4%) achieved the monthly target of 10 referrals per month by
31 July 2012
(see Annexure
31)
 50% of magistrates surveyed said speedier assessments would improve the referral rate (see
Annexure 10; question 6)
 Some magistrates see NCS as an unproven risk and want more evidence of success such as this evaluation presents
 It appeared that 7 sites
(16%) at most might meet the 10X a month target magistrate for referrals by project end
(see
Annexures
3.1 and 3.2 for additional details on target attainment)


Some magistrates reported not getting feedback on cases such as at
Tembisa, East London and Welkom
(the ones at Welkom stated on 29 Aug that they were not referring owing to this) —see question 1 to magistrates,
Annexure 10, Section 10.3)
That some magistrates did not receive feedback was confirmed by SWs
(Annexure 7, question 20) and by supervisors (Annexure 8, Section 8.2, and Question 12). There was also widespread confusion amongst SWs, as to when feedback is required. After assessment and after treatment are correct and only 9 of 31 (29%) SWs could offer such a reply.
 Some magistrates lack confidence in a particular SW (this occurs when magistrates are not given feedback on cases or when there has been SW turnover)
 SW turnover in terms of reducing magistrate confidence is a big challenge owing to SW shortages and
NICRO ’s inability to offer better
© NICRO 2013 188
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
Output 1: A fullyfledged referral system developed for alternative sentencing at 45 magistrate’s courts
Overall rating (explanation given in two indicators below) = 2
1-5 rating
Log frame
Indicators
Baseline and Target for
Indicators
Actual achievements by 31
July 2012
Comments on final results including unintended impacts salaries and benefits while some other employers competing for the same
SWs offer more pay and benefits
2
Partially achieved
45 LSCs established and functional
(as either
LSCs or as
NICRO participating in case flow management committees)
Baseline: 0  As of 30 July
2012: 37 of 45
(82%) sites were meeting either as
23 LSCs or as
NICRO serving on 14 caseflow management committees
(Annexure 2)
 As of 30 Jul
2012: 8 LSCs
(18%) had not yet met —one
Year 5 site
(Pinetown); five
Year 4 sites —
Attridgeville,
Katlehong,
Boksburg,
Meadowlands, and
Kanyamazane; and two Year 3 sites (Vaal and
Mamelodi) —see
Annexure 2
 The LSCs were originally designed as therapeutic case management forums but this did not occur owing to time constraints and skills deficits; instead members cooperated on ensuring the smooth delivery of the services
(although this did not match the original function, it proved to be important)
 Certain LSC partners reported in focus groups and interviews at a sample sites visited and on surveys that they were not receiving the feedback on client outcomes via the
NICRO SWs (it has improved since the MTR but these reports were confirmed by the SWs themselves,
40% of whom indicated on question
20 Annexure 7 –they were not offering appropriate feedback on client outcomes)
 All protocols, tools, processes and guidelines in place
 The Department of Justice was making space and facilities available for NCS services at selected courts around the country
 A ‘fully fledged’ system requires monitoring, aftercare and tracking and this this was not always occurring; a partnership with DCS and the
Judiciary on appropriate systems and sentencing for NCS might help to address this (see also Annexures18,
23, 25)
© NICRO 2013 189
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
Output 2: Therapeutic programmes available for utilization in alternative sentencing
Overall rating (given in three indicators below) = 1.5
1-5 rating
Log frame
Indicators
Baseline and
Target for
Indicators
Actual achievements by 30 July 2012
Comments on final results including unintended impacts
1 Programme s developed, reviewed and available
NICRO designs, reviews updates and therapeutic programmes continuously
Programmes developed and reviewed during 2009-
2012:
 Adapt
 Adult Life Skills
 Community
Service
 Matrix
 PIPV facilitator and assessment guide (resource manual participant and workshops still outstanding)
 Parenting
Development
 RGC
 Yes Programme
 Professional
Skills
Development
Initiative —this includes onsite coaching, video library,
Knowledge
NING
Forum,
Information telecons
 Critical Thinking
Manual
 An introduction to bullying: what educators need to know
 Diversion systems training
 NCS training systems
(90%
 NICRO demonstrates a high level of professionalism in terms of programme design and development
 No significant shortage of programme offerings were detected while a more significant problem was detected in terms of SW capacity to deliver certain programmes —see Appendices
7 and 8; especially question
8.7
 54% of the supervisors thought that all the available intervention offerings were adequate. A little under half thought that some additional offerings were ‘needed’.
However, the definition of what is most needed varied greatly from site to site. In analysis, intervention tools that help to address anger issues, poverty, and substance abuse were viewed as needed by those who said no. However, based on all data, training in the existing offerings is more important than developing new offerings.
 Matrix was only utilised for about 10 clients —it is potentially the best of the programmes but it is regarded as too lengthy and not all SWs have the skills required
 Still in development on 30 July
2012 were Journey, Adult
Lifeskills (content development underway); Community Service
(90% complete); Critical
Thinking Manual (was very close to completion —mainly formatting)
© NICRO 2013 190
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
Output 2: Therapeutic programmes available for utilization in alternative sentencing
Overall rating (given in three indicators below) = 1.5
1-5 rating
Log frame
Indicators
Baseline
Target and for
Actual achievements by 30 July 2012
Comments on final results including unintended impacts
Indicators complete)
2 6,000 offenders enrolled in programme s as part of alternative sentences by project’s end
Baseline = 0 to
3 enrolments per month (the latter if the
2008-2009 pilot is considered as the baseline)
 3,960 offenders
 enrolled level of 6000) in programmes (66% of target or 2/3rds of the anticipated
The project will come closer to target by project end but is likely to fall just short
 SW turnover disrupted services at many sites
 Magistrates convinced of the worth of the services refer but too few magistrates who want to refer were happy with the time period for the assessments
 6000 was also a revision downward from original targets
@ 14,640 offenders enrolled in programmes by March 2013 — these targets were unrealistic for a variety of reasons but also for the above bulleted point
1 70% of recommen dations for alternative sentencing accepted by magistrates in 45 courts by end of funding
---  72% of SW recommendations for AS accepted by magistrates
(yearly average to
31 July 2012 for years 1-4)
 The sentence recommendation acceptance rate was generally high but dropped to
45% during the
2012-2013 budgetary while evaluation year the was underway (see
Figure 3.5 below)
The drop in Year 5 owed to


SW turnover
 NICRO restructuring Nov 2011-
April 2012, which included a moratorium on filling SW vacancies;
 Retrenchments at NICRO from
November 2011; and
Relocating sites with low referrals to new sites in the hopes of improved referrals (this was a calculated risk because the start-up time at a new site can be very slow and this changeover can also dampen impact on the improving referrals)
© NICRO 2013 191
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
YEARLY RATE OF ACCEPTING NICRO ’S SENTENCING RECOMMENDATIONS 2008-2013 61
Sentence
Acceptance Rate
NCS →
Indigenous Justice →
Recommendation 2008-
2009
100%
NA
2009-
2010
85%
NA
2010-
2011
85%
100%
2011-
2012
73%
94%
2012-
2013
45%
100%
TOTAL
72%
98%
Finding: if a magistrate requests an assessment, there is a high acceptance rate. The challenge is that not enough magistrates refer for an assessment per court (often just one or two referring since most magistrates are too consumed with reducing the court roll to want to delay for an assessment as indicated in interviews and
Magistrate Survey Data (see Appendix 10; see also Annexure 7, 7.3 question 7 wherein most SWs thought they could reduce the assessment time).
Output 3: Professional SW expertise available at 45 sites
Overall rating (given in two indicators below) = 3
1-5 rating
Log frame
Indicators
Baseline and Target for
Actual achievements by
30 July 2012
Indicators
Comments on final results including unintended impacts
2
Partiall y
Achiev ed
All 45 sites staffed by one fully trained SW by project end —one per site
0  43 SW posts filled by 40 social workers (4 sites share posts)
“All 45 sites staffed by one fully trained SW by project end” is a perfunctory measure of output and cannot be rated as fully achieved because there are strong indications that one SW is not sufficient since of sites surveyed (31 in terms of
SW surveys and 34 in terms of supervisor surveys) this was found:
 2 vacant sites


96% capacity
High turnover see
—
SW comments
 Annexure 7, Section 7.3, question 6 shows that at only 10 of the 31 sites (32%) was there a
SW devoting more than 20 hours to NCS. This demonstrates the high degree to which SWs were multitasking and that the time available to them for developing referrals to NCS services was compromised by other obligations.
 44% of SWs on surveys say they do not have all the training required
(Annexure 7, question 18)
 Annexure 7, Section 7.3, questions 5 and 6 show that most SWs were multitasking between service streams-- diversion, NCS, offender reintegration, schools and community work) and 61% of SWs found this difficult or very difficult
 Annexure 8, question 4 on the
Supervisor Survey shows that
62% of supervisors thought the
SWs should NOT be multitasking
61 Source: NICRO (July 2012) National Statistics (NCS statistics extracted by Regan Jules-
Macquet, project manager)
© NICRO 2013 192
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
Output 3: Professional SW expertise available at 45 sites
Overall rating (given in two indicators below) = 3
1-5 rating
Log frame
Indicators
Baseline and Target
Actual achievements by for
Indicators
30 July 2012
Comments on final results including unintended impacts between service streams, rather the SWs on NCS should be fulltime NCS specialists
 There were times when SW turnover was a very serious threat to the project —in March 2012 there were 16 vacancies (36% of sites) —and while it can be stated that by August 2012 there were only 2 vacancies it remains that turnover is a significant programme threat
 Of the 34 sites surveyed (supervisors), more than half (18 or 53%) had challenges of SW turnover. Eight of those 18 sites or 44% were on the 3 rd or
4 th replacement which makes it very difficult to develop magistrate trust and to provide the continuity of service required to achieve the targeted referrals. See Appendix 8, Figure 2 for example.
 There is a national SW shortage and turnover of SW at sites can slow the pace of referrals as NICRO looks for a new SW. This partly explains the failure to achieve minimal referral targets (the supervisor must slow down or suspend referrals while training a new SW; magistrate confidence is weakened; the LSC is weakened)
 NICRO cannot compete with the DSD in terms of salaries, benefits such as bursaries and this caused frustration amongst certain NICRO supervisors and provincial managers who sometimes described themselves as
‘training SWs for the DSD’
 For the reasons above and more, the availability of SWs must be considered in a broader context than filled posts at a given point in time. This is but one of many lines of evidence that NICRO will have to develop a joint strategy with the
DSD, DCS and other stakeholders to
© NICRO 2013 193
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
Output 3: Professional SW expertise available at 45 sites
Overall rating (given in two indicators below) = 3
1-5 rating
Log frame
Indicators
Baseline and Target
Actual achievements by for
Indicators
30 July 2012
Comments on final results including unintended impacts respond to such challenges as these.
4 Aftercare and tracking
© NICRO 2013
0  Survey data on 936 clients
(Appendix
5) shows that only
13% of clients were tracked more than 7 months
 Excluding
145 blank and spoilt responses
, there is survey data on tracking
791 clients and of these more than half (52%) were not tracked at all. Of the remaining
48%, 12% were tracked up to months,
3
21% for up to six months, and 15% in the range of
7-12 months.
This shows that tracking
 Aftercare is essential to describing the delivery a professional service because it creates a support structure for the client to address further needs after completion of the initial programme but it was not regularly provided because it requires social auxiliary workers in support of the
SW and that was seldom available (this is why 2 SWs per site are needed as stated above)
 Client tracking refers to follow up for monitoring purposes and only 13% of clients were tracked more than 7 months
 Client tracking and aftercare persists as the weakest areas of delivery and yet these areas are critical components of a fully-fledged NCS service (see Figure below)
 In interviews and surveys SWs and supervisors admitted aftercare was not occurring and explained the reasons--see
Figure below for reasons
 The challenge of insufficient Aftercare and
Tracking persisted throughout the intervention, was identified at the time of the MTR, and might be described as the
Achilles Heel’ of the project
 In the 1990s when DSD ran a Pretoriabased diversion programme for youthful offenders, the programme collapsed because there was no monitoring and follow-up with some serious consequences
(see Basil King Interview, Annexure 25).
 DCS must be a partner in the delivery of
NCS services (a vital recommendation of this report) but they cannot cope with their policing/correctional services/monitoring role. The old system described collapsed this way and little has changed —basically,
Correctional Services has an insufficient budget for a monitoring and follow-up role.
Correctional services cannot simply be
194
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
Output 3: Professional SW expertise available at 45 sites
Overall rating (given in two indicators below) = 3
1-5 rating
Log frame
Indicators
Baseline and Target
Actual achievements by for
Indicators
30 July 2012
Comments on final results including unintended impacts seldom occurred and when it did it was for a short period only. See
Figure 3.7 in chap 3
 relieved of their duty and are ultimately responsible for monitoring and yet have proven over decades that they cannot deliver without support
A DCS/NICRO partnership from national level is essential to NCS services but this was not consolidated even though there has been dialogue making this a critical focus for NICRO (see Annexure 18)
SW And Supervisor Data: Why Tracking And Aftercare Not Occurring 62
Reason aftercare was weak Explanation
Communications Most perpetrators are poor and have pay-as-you-go phones with changing numbers and not much airtime; landlines are also pay-as-you-go and
Locating the client
Insufficient time and funds
SW turnover
Transport
No assistance to SW frequently disconnected
No permanent address given for tracking
The SW has little or no time and an insufficient budget for visiting clients in the community
The SW resigns and there is no follow-up with the client
The client does not have transport money to visit the office; the NICRO SW does not have funds to visit the client; the NICRO SW does not have a driver’s license
A shortage of social auxiliary workers for this support function
Output 4: Expert alternative sentencing programme interventions
Overall rating (given in two indicators below) = 2
1-5 rating
Log frame
Indicators
Baseline and Target for
Actual achievements by 30 July 2012
Indicators
Comments on final results including unintended impacts
2
Largely achieved with a few shortcomings
All SWs trained and supervised according to the NICRO
PSP manual requirements and able to deliver a broad range of NICRO
0
No suitably qualified
SWs per court who had been oriented, trained and supervised
NICRO training was indicated to be to the standards of the social work profession:
 There is a strong supervision
 The log frame indicator is not sufficient for exposing the challenges including gaps in delivery owing SW turnover
(they do not stay in positions long) and posts not filled or a new SW starting up —all of
62 Extracted from interviews, surveys and focus groups with supervisors, social workers
© NICRO 2013 195
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
Output 4: Expert alternative sentencing programme interventions
Overall rating (given in two indicators below) = 2
1-5 rating
Log frame
Indicators
Baseline and Target
Actual achievements by 30 July 2012 for
Indicators therapeutic programmes according to the supervision model model that is part of the
Policies,
Standards and
Procedures
Manual
Comments on final results including unintended impacts which compromise the quality of the intervention. Those that were trained and became familiar with the intervention offerings departed with sufficient frequency that some
NICRO sites were being drained of capacity constantly
 A Professional
Skills
Development
Initiative (PSDI) was launched in 2010 with many building skills opportunities including intensive onsite coaching, video libraries, the
NING knowledge forum, materials, reviews and information, 20 telecons to date reaching 299 people
 Staff provided with
NCS toolkits and trained on site supervisors in
NCS were by
(mentoring, training weekly to monthly) involving assessments,




The supervision model and the superbly developed PSDI can still leave gaps as identified by both supervisors and SWs in their surveys
Annexure 8 (see Figures 6 and 7) shows that 77% of supervisors did not think the
SWs had all the requisite skills and training to deliver in all needed examples)
Annexure 7 shows that the SWs themselves wanted formal training opportunities to areas
(Perpetrators of Intimate
Partner Violence, Sexual
Offences and Matrix are 3
Annexure 8, question 2.5 shows
13 supervisors, identifying a challenge with either: (a) new recruits lacking sufficient training (having missed formal opportunities) or (b) SWs who were skilled but could not advance to provide certain needed programme through self-training only (sexual offences, matrix, perpetrator programmes were cited)
© NICRO 2013 196
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
Output 4: Expert alternative sentencing programme interventions
Overall rating (given in two indicators below) = 2
1-5 rating
Log frame
Indicators
Baseline and Target
Actual achievements by 30 July 2012 for
Indicators review of programme content, facilitation, skills development, critical thinking, restorative justice principles, legislation and lobbying and advocacy
Comments on final results including unintended impacts advance their skills so that they could address all criminogenic risk factors amongst clients (see questions 2, 4, 7 8 and
18) identified
The NICRO SWs demonstrated expert capacity since the risk factors that produce criminal behaviour were reduced therapy:
 Of 936 surveyed (24% of the 3,960 on programmes) on particular through clients risk factors most will not reoffend
 A second survey of SWs on the same 936 clients found agreement
(within a small variance) that the risks were reduced
 While professional the NICRO skills development initiative is an example of a good management effort in the face of very limited resources, it is not sufficient to replace all formal training opportunities away from the workplace. These interactive opportunities are needed to produce a cadre of well skilled SWs who can be retained to provide a large range of needed services (an essential strategic component when it comes to marketing the services)
 Rather than formal training as an NICRO internal offering, it would be better to provide training with all involved departments and stakeholders as part of an integrated strategic effort to deliver services as recommended in this study
(see recommendations)
 A random sample of 237 clients (25% of those surveyed;
6% of all clients) showed that only
9 reoffended between 2010-
© NICRO 2013 197
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
Output 4: Expert alternative sentencing programme interventions
Overall rating (given in two indicators below) = 2
1-5 rating
Log frame
Indicators
Baseline and Target
Actual achievements by 30 July 2012 for
Indicators
2012 (2% per year) —another indicator expert care of
Comments on final results including unintended impacts
 The reduction in risk factors
(Annex 6) for
96.3% of all
NICRO clients suggests that most of those exposed to the programme received expert care
The biggest gap in terms of SW expertise was supervision of which these are indicators:
 Only 39% of the
SWs (or 40% if the response discounted) blank is really understood the clauses in the criminal procedure act relevant to their work. See
Annexure 7.3 question 4.
 On admission of the SWs in surveys
(Annexure 7,
Section 7.3,
© NICRO 2013 198
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
Output 4: Expert alternative sentencing programme interventions
Overall rating (given in two indicators below) = 2
1-5 rating
Log frame
Indicators
Baseline and Target
Actual achievements by 30 July 2012 for
Indicators question 20),
40% of the courts are not getting feedback on the treatment plan — a serious gap.
There was also widespread confusion amongst SWs, as to when feedback is required. After assessment and after treatment are the closest to correct and only
9 of 31 (29%) respondents offered such replies.
Comments on final results including unintended impacts
2 All sites with resources and equipment
© NICRO 2013
 All data and evidence corroborate including surveys, site inspections and project reports that the sites were resourced by 30 July 2012
 Not all sites received equipment and furnishings in a timely way but by the time of the evaluation they had what was required
 Wherever NICRO utilised facilities provided by the courts, there was less of a therapeutic atmosphere than might be desired (no waiting areas, poor design, and minimum space) —sometimes shipping containers were used
(turned into offices).
 The biggest gap was the lack of landlines in several offices.
199
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
Output 4: Expert alternative sentencing programme interventions
Overall rating (given in two indicators below) = 2
1-5 rating
Log frame
Indicators
Baseline and Target
Actual achievements by 30 July 2012 for
Indicators
Comments on final results including unintended impacts
The SWs tried to use cell phones but these are a more difficult way to communicate and more expensive and compromised some components of delivery including Aftercare and
Tracking.
Output 5: A national media and marketing campaign: increase public awareness of NCS and restorative practices
Overall rating (rating given in 5 components below) = 2
1-5 rating
Log frame
Indicators
Baseline and
Target
Indicators for
Actual achievements by
30 July 2012
Comments on final results including unintended impacts
2 1. A national media and marketing campaign that measures public attitude and opinion toward AS
Baseline was estimated at
44% public support for
NCS and target was
55% or more
First Phase
NICRO Public
Attitudes
Survey of 253 people: 54% of the public support NCS and rehabilitation
Note: workshops figures fluctuate the and surveys were continuous and the
 The public attitude surveys showed that public support for NCS exceeded targets and rose as high as 70% agreeing that ‘nonviolent’ offenders can rehabilitated. be
ï‚·
Second Phase:
740 surveys show
60% support
ï‚·
Third Phase surveys show 59-
67% support
ï‚·
Support ratings exceeded targets and were in the range of 59--70% public support
 There was never a point in the intervention in which the community resisted or protested against NCS services such that it can be said that services were welcomed by communities
 The support ratings are coming mainly form poor communities as do most
NICRO beneficiaries (mainly young, male and poor) and this suggest that amongst family members in these poor communities there is little overt resistance to NCS services (at least less than anticipated design). in project
 Those being surveyed are mainly responsible citizens interested in programmes that can alleviate crime and
© NICRO 2013 200
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
Output 5: A national media and marketing campaign: increase public awareness of NCS and restorative practices
Overall rating (rating given in 5 components below) = 2
1-5 rating
Log frame
Indicators
Baseline and
Target for
Indicators
Actual achievements by
30 July 2012
Comments on final results including unintended impacts therefore there can be a bias in the figures
 Owing to the above it would be beneficial to identify in more precise terms the characteristics or traits of those opposed to restorative justice.
 It could be hypothesised based on the general data in support of NCS in poor communities that resistance is found in greater numbers amongst the middle class and wealthy (often the decision makers and those allocating budgets) than amongst the poor (the segment most subject to arrest)
1 2. A national media and marketing campaign that
(b) promotes a culture of human rights within a criminal context, educates primary stakeholders and assists the public to understand the benefits of restorative justice
Target:
Entrench restorative justice concepts in the national psyche and educate public
NICRO arranged for the
Newsclip Media
Monitoring Service to provide a review of NCS coverage (see Appendix
11) which identified a nearly R10 million value in radio, TV and newspaper coverage 2008-2013 as follows :



R4,210,150.82 broadcast
R2, 902,050.17 in print
R2, online
663,059.44
 Total value:
R9 775 260.43
© NICRO 2013
 See Annexures 11 (11.1. and
11.2) and Annexure 11.3 for more details
 The NICRO media and marketing work was comprehensive (broadcast, print, and online) and of such a quality that it had impact (as evaluated by Newsclip) and assisted both primary stakeholders and the public to better services. understand the
 NICRO first commissioned some professional support for marketing and then gradually assumed responsibility for it.
Total Media initiated a
201
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
Output 5: A national media and marketing campaign: increase public awareness of NCS and restorative practices
Overall rating (rating given in 5 components below) = 2
1-5 rating
Log frame
Indicators
Baseline and
Target for
Indicators
Actual achievements by
30 July 2012
Comments on final results including unintended impacts
 Reach = campaign Apr 2009-Feb 2010
2,118,793 starting with media roundtables that introduced
 Print coverage
(alone) could also be valued selected media to the NCS concept and generated a in terms of advertising cost media conversation, and then
NICRO worked on a pro-rata and the
Institute
PR of basis in Years Two and Three with Redline with NICRO
South Africa estimated this at assuming marketing responsibility in the final years. more than R4.6 million in
NICRO demonstrated the capacity to run a good advertising value with a campaign and the stakeholder and public videos produced by consumer reach of 2, 118,793 the project were of a very high standard —high enough to air people on national television. This suggests that had NICRO
 NICRO used its own networks (45 sites and the national office) to distribute booklets explaining NCS services —
Moments of
Growth, Voices of
Change
(sentencing stories) assumed responsibility earlier
(rather than contracting out) some expenses might have been saved.
 Brochures
NCS and cards explaining the service distributed from both national and all 45 sites
2 3. Community
Engagement
Activities
No indicators other than to engage the community through events and
© NICRO 2013
 8,800 community members reached by 31
 At some sites the media published stories about the services
(sometimes
NICRO community renders following engagement
202
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
Output 5: A national media and marketing campaign: increase public awareness of NCS and restorative practices
Overall rating (rating given in 5 components below) = 2
1-5 rating
Log frame
Indicators
Baseline and
Target for
Indicators
Actual achievements by
30 July 2012
Comments on final results including unintended impacts activities
(baseline = 0)
July 2012 activities) and supervisors reported receiving positive
 The Supervisor comments from the public survey showed following these publications that community engagement  Considering that nearly all sites workshops were conducted at all had but one SW who normally multi-tasked between projects
 sites except for
Durban (44 of
45 sites)
Community outreach events commonly
(NCS and others) and one supervisor multi-tasking between sites, the human resources are insufficient at each site for marketing activities to the strategic level required to occurred schools,
CPF during holiday at malls, meetings, public events where brochures, move the national from punitive thinking about justice to thinking of justice in more restorative terms —NICRO needs either more staff at each site for this engagement and more partners
(see the bullet below as well)

NCS cards and other materials were distributed
Annexure 8;
Section 8.2.9 shows that 11 of
 Scaling back from 45 NCS sites to about half as many or even fewer; and mainstreaming NCS into a regular NICRO offerings; should release some energies
13 supervisors
(85%) convinced or highly were for marketing NCS as part of the range of NICRO offerings and building community level understanding and participation convinced that the described events and activities were promoting a culture of human rights and restorative justice because there would be an increase in
© NICRO 2013 203
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
Output 5: A national media and marketing campaign: increase public awareness of NCS and restorative practices
Overall rating (rating given in 5 components below) = 2
1-5 rating
Log frame
Indicators
Baseline and
Target for
Indicators
Actual achievements by
30 July 2012
Comments on final results including unintended impacts referrals after such presentations or members of the public would come to visit the
NICRO office but the only hard evidence of impact was given above in terms of Public
Attitude Surveys where the majority of the public indicated support for NCS services
 Annexure 7, question 16: 31
SWs identified the top two indicators of improved public attitudes: (a) increased referrals with
34% of SWs saying this; and
(b) positive verbal feedback from either the community or the LSC with
30% saying this
1 4. Develop and implement education, information and outreach programmes
No previous materials
 Magistrate developed
2010) toolkit
(2009-
 4,175 court stakeholders trained —
 As early as the MTR and persisting into the evaluation, magistrates stated that no more educational opportunities, workshops or publications for
© NICRO 2013 204
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
Output 5: A national media and marketing campaign: increase public awareness of NCS and restorative practices
Overall rating (rating given in 5 components below) = 2
1-5 rating
Log frame
Indicators
Baseline and
Target for
Indicators
Actual achievements by
30 July 2012
Comments on final results including unintended impacts for magistrates (sensitising court stakeholders supports the referrals magistrate magistrates are required and such activities are unlikely to change referral levels:
 The rate of magistrate referrals cannot be improved with educational or outreach activities because the challenge that needs to be addressed is systemic: magistrates are under pressure to reduce the court roll and this requires that they speed up case processing times and therefore they want
NICRO to speed up the assessment time owing demands on them (79% of magistrates surveyed said this;
See Annexure 10, question 7;
See also the Basil King interview Annexure question one )
25,
 Most magistrates want the assessment back in 2 weeks or less and in one day in some cases —see question 6 on the
Magistrate Survey, Annexure
10
 Magistrates and court stakeholders are so overworked and pressurized that most of these project materials end up in a drawer
3 5. Develop and implement a
Lobbying
Programme aimed at the
Parliamentary
Portfolio and
Standing
No previous materials on
NCS
Seven submissions made to parliamentary portfolio committees (mainly for
Correctional Services):
 2009-2010 —four submissions
© NICRO 2013
 Submission were not so specific to NCS
 These were made to parliamentary portfolio committees and framed within an overall advocacy effort to
205
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
Output 5: A national media and marketing campaign: increase public awareness of NCS and restorative practices
Overall rating (rating given in 5 components below) = 2
1-5 rating
Log frame
Indicators
Baseline and
Target for
Indicators
Actual achievements by
30 July 2012
Comments on final results including unintended impacts
Subcommittees  2010-2011-Three submissions see more of the budget allocated to prevention as opposed to police, courts and jails
 2011-2012--
Lobbying and advocacy (written or oral) to portfolio committees promote NCS to
 Findings from the project were not integrated into an advocacy effort but advocacy was more general (as described above) and appeared to have had little impact on sustaining the project or increasing referrals
 The particular log frame indicator appeared incomplete. NCS services need to be sustained more in terms of partnerships with government departments such that a strategy to lobby the
Parliamentary Portfolio committees (mainly Correctional
Services) did not go to the issue of sustainability.
 Ultimately perpetrators on NCS become a DCS responsibility and it is also possible that in terms of a sustainability plan, they might fund NICRO.
Achieving this outcome requires advocacy and lobbying national level departments but this was not done sufficiently as clearly indicated in the focus group with
DCS National (see Annexure 18)
 See Annexures 18, 19, 20 and
21 –interviews at national--which clearly informed the evaluator that by late 2012 a partnership had not been consolidated with the most important decisionmakers. Only DSD national
© NICRO 2013 206
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
Output 5: A national media and marketing campaign: increase public awareness of NCS and restorative practices
Overall rating (rating given in 5 components below) = 2
1-5 rating
Log frame
Indicators
Baseline and
Target for
Indicators
Actual achievements by
30 July 2012
Comments on final results including unintended impacts appeared to have no interest in
NCS for ‘adults’ (short-sighted thinking) but DOJCD, DCS,
JCPS, and DSD Provincial stated they were very interested in NICRO approaching them with some formal partnership proposals.
Output 6: A newly developed indigenous-based diversion programme available (Eerstehoek)
Overall rating (given in 2 components below) = 2
1-5 rating Log frame
Indicators
Baseline and
Target
Indicators for
Actual achievements by 30 July 2012
Comments on final results including unintended impacts
4
Very limited achievemen t, extensive shortcomin gs
330 offenders diverted through the indigenous based diversion programm e by projects end
0
No indigenous justice diversion programme had been available before the
NICRO initiative
 23% (N=76) of target attained by end July 2012 as follows:
 Traditional courts:
(13%)
10
 Target not attained
 Most of the target attainment was
NOT via the traditional courts but owed to school, community and magistrate referrals
 Magistrate courts: the DSD
38
(50%) including 2 via
 Most referrals started in the third year of the programme (very slow start up)


Schools:
(22%)
Community:
11 (14%)
17
 NICRO dropped work in the traditional courts that did not refer including eNikwakuyengwa (1 referral) and concentrated on the ones that did refer —Eerstehoek
Magistrate Court and the tribal courts of eMbhuleni,(3 referrals)
Mandlamakhula (1 referral) and
Mpisikazi (5 referrals)
 The school referrals owed to a crime prevention programme in two schools 63 which then referred at-risk youth to NICRO and then
63 The High Schools: Bantfwabetfu and Dlomodlomo
© NICRO 2013 207
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
Output 6: A newly developed indigenous-based diversion programme available (Eerstehoek)
Overall rating (given in 2 components below) = 2
1-5 rating Log frame
Indicators
Baseline and
Target for
Actual achievements by 30 July 2012
Comments on final results including unintended impacts
Indicators these youth became mentors to other at-risk youth
 NICRO ’s original hypothesis that traditional practices of restorative justice could be incorporated into diversion proved false —most of the traditional leaders and courts had little knowledge of restorative justice principles
 The failure of the indigenous justice model does not mean that
NICRO failed since upon identifying the challenges in 2011, the organization then focused on schools and community work, which was effective in increasing referrals to NICRO programmes
(this locates an appropriate model for working in such areas — community based work).
2 70% of recommen dations for indigenous based diversion accepted by the courts/site s by end of funding
0
No indigenous justice diversion programme had been available before the
NICRO initiative
 The sentence recommendation rate of acceptance was
98% (see Figure
3.5 further above)
 Despite a high acceptance rate, this is a perfunctory measure of performance that involves only 10 referrals such that a 2 rating does not capture the challenges
 The area was too big in both population and distances for
NICRO to provide an effective intervention either in terms of
NCS or diversion from traditional courts. NICRO served 3 traditional courts but there are 12.
School and community interventions provide a better way in rural communities to make impact on more people.
© NICRO 2013
 The LSC thought NICRO should rather intervene in schools and communities to prevent crime and
208
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
Output 6: A newly developed indigenous-based diversion programme available (Eerstehoek)
Overall rating (given in 2 components below) = 2
1-5 rating Log frame
Indicators
Baseline and
Target for
Actual achievements by 30 July 2012
Comments on final results including unintended impacts
Indicators divert people out of the system and to advocate that national and provincial departments provide training in the Traditional Affairs
Act (see Annexure 16).
 The magistrate and the case flow manager (NPA) thought that court cases had been reduced owing to the schools work and strongly recommended that this be the focus (see Annexure 16)
Output 7: Develop a knowledge base of NCS systems and practices to further public and sector understanding and to promote awareness
Overall rating (given in 5 components below) = 1
1-5 rating
Log frame
Indicators
Baseline and
Target for
Indicators
Actual achievements by 30 July 2012
Comments on final results including unintended impacts
1
1
Develop accessible online forums for the collection and dissemination of research on sentencing and offender reintegration
0
No online forums regarding NCS
Research into NCS systems and related practices
0
No such research in SA specifically
© NICRO 2013
A Facebook Group
(278 members)
Ning
Justice
Criminal
Online
Forum(@153 members; active)
30+
 The traffic is was relatively low in terms of number but the forums attract researchers and media interest such that there are
‘spinoffs’ for disseminating findings
Newsletter: reach of 390 people
 Indigenous Justice
Research (Apr
2009-2010)
 Stakeholder
Seminar on NCS and conference reports (Apr 2009-
 A large number of research products were generated that enable other stakeholders and organizations to learn about the practice
 The data from the NCS services offers very good value
209
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
Output 7: Develop a knowledge base of NCS systems and practices to further public and sector understanding and to promote awareness
Overall rating (given in 5 components below) = 1
1-5 rating
Log frame
Indicators
Baseline and
Target for
Indicators
Actual achievements by 30 July 2012
Comments on final results including unintended impacts
Mar 2010)
 GTF DFID reports
(annual and quarterly
2012
2009-
 3 phases of Public
Attitude Research-
One-July 2010;
Two--July 2011-
2012 Three--2011-
2012
 Mid-Term Review
(Sep 2010)
 Telecon
Knowledge
Sessions Report
(Oct 2010)
 NCS Sentencing
Stories Booklet
(Dec 2011)
 SW attitudes and values research
(Feb 2011)
 NCS Colloquium
Transcripts (Mar
2011)
 Traditional Leaders
Research
2011)
(Aug
 NCS compliance
Nonresearch (2011)
 Case
Research
Studies
(Jan internationally and within South
Africa
© NICRO 2013 210
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
Output 7: Develop a knowledge base of NCS systems and practices to further public and sector understanding and to promote awareness
Overall rating (given in 5 components below) = 1
1-5 rating
Log frame
Indicators
Baseline and
Target for
Indicators
Actual achievements by 30 July 2012
Comments on final results including unintended impacts
2012)
 Magistrate
Attitudes and
Values Toward
Sentencing (Jun
2011-July 2012)
1 Materials produced
No previous history of this
Baseline = 0
 NCS Toolkit (Apr
2009-March 2010)
 NCS
Redesign
Brochure
(Aug
2010)
 NCS Videos (Nov
2010-May 2011)
 NICRO promotional materials---posters, bracelets and bags
 Moments
Growth of booklet
(Feb 2012)---letters and poems from
NCS offenders
 SiSwati Indigenous
Justice Brochure
(Jan 2012)
 All materials were of a very high standard
 It cannot be said that all promotional materials
(especially bracelets and bags) offered high value but in general most materials were of high value and effectively utilized —see Figure 3.8
Below)
1 Print
Media
© NICRO 2013 and This was covered and rated as ‘1’ in
2.3.5 above and Annexures
11.1 - 11.3 offer a detailed listing of the achievements in print and
 This was covered in 2.3.5 above and
Annexures 11.1 -
11.5 offer a detailed listing of the achievements in print and media
 This was covered in 2.3.5 above and need not be repeated but should enter into the ratings
 Annexures 11.1 - 11.5 offer a detailed listing of the achievements in print and media
211
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
Output 7: Develop a knowledge base of NCS systems and practices to further public and sector understanding and to promote awareness
Overall rating (given in 5 components below) = 1
1-5 rating
Log frame
Indicators
Baseline and
Target for
Indicators media
Actual achievements by 30 July 2012
Comments on final results including unintended impacts
© NICRO 2013 212
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
ANNEXURE 1.4: Activities To Achieve All Outputs (At
£4 916 108)
(NO RATING; CONTRIBUTES TO ASSESSMENT OF OVERALL ACHIVEMENTS AND TO
EXPENDITURE ANALYSIS)
ACTIVITIES EXPENDITURE
OR EVIDENCE
SHORT NARRATIVE
PERFORMANCE,
ACHIEVEMENT
SUMMARISING
ASSESSING
Output 1: A Fully-Fledged Referral System Developed For Non-Custodial Interventions At Magistrates
Courts (Total Cost Of Output One In Pounds Sterling = = £378 493
1.1 Stakeholder engagement and consultation
1.2 Confirm commitment from local stakeholders and practitioners & draw up working agreements / Memoranda of
Understanding
1.3 Develop intervention procedures, implementation protocols, guidelines and indicators
£195 674
EXPENDITURE
LSCs functional at 37 of 45 sites around the country as of 30
July 2012
£11 982
EXPENDITURE
LSCs functional at 37 of 45 sites around the country as of 30
July 2012
£6 170
These were normal and necessary activities at sites that achieved the outputs in terms of stakeholder committees. This involves with administrative costs such as technical support from national at 4 visits per site including flight costs to provinces and accommodation, road and travel costs to set up and monitor these.
These were normal and necessary activities and expenditures. These efforts led to working agreements Memoranda of Understanding with court sector and other service providers.
Includes travel and accommodation expenditure to affect this.
Normal development costs calculated @ £18 per hour. Development time - 2 months (21 days per month x 8 hours per day x 2 months = 336 hours x £18)
Normal development costs. Includes flight, accommodation, meals for visiting sites for two staff members
1.4 Establish intersectoral project committees at project sites for programme governance and to monitor implementation and progress, identify and address challenges
£51 336
LSCs functional at 37 of 45 sites around the country as of 30
July 2012
£22 500 1.5 Implement programme procedures and systems, including assessment, referral mechanisms, monitoring, tracking, feedback and evaluation
1.6. Secure project resources £90 831
Assessment tools developed and incorporated into NICRO client management system. Pre- and post-testing systems which were critical to identifying impact. A good value.
This was about 25% underspent during the time of the evaluation. The under expenditure owed to communications from site to national. The budget was specific to computers, key boards, books, DVDs, games, toys and other kinds of specific project items but the requests were sometime inappropriate (e.g., a refrigerator) or not made at all (communications and efficiency issues)
© NICRO 2013 213
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
ACTIVITIES EXPENDITURE
OR EVIDENCE
SHORT NARRATIVE
PERFORMANCE,
ACHIEVEMENT
SUMMARISING
ASSESSING
Output 2: Therapeutic programmes available for alternative sentencing (£2 841 742 in total costs)
2.1 Research, Design and Development of new non-custodial therapeutic programmes sentencing
£111 793 expendi ture
Programmes developed and
This was expertly done but applied to all NICRO programmes (not just NCS). Programme design and development and piloting involved reviews and travel. reviewed during 5 new programmes in Year 1;
2009-2012:
ï‚·
Adapt
ï‚·
Adult Life
Skills
ï‚·
Community
Service
ï‚·
Matrix
ï‚·
PIPV facilitator and assessment
1 new programme in Year 2,
2 new programmes in Year 3,
1 new programme in Years 4 & 5
Reviewing and finalising models
This has long-term value for NICRO which copyrights the materials.
guide
(resource manual and participant workshops still outstanding)
ï‚·
Parenting
Development
ï‚·
RGC
ï‚·
Yes
Programme
ï‚·
Professional
Skills
Development
Initiative —this includes onsite coaching, video library,
NING
Knowledge
Forum,
Information telecons
ï‚·
Critical
Thinking
Manual
ï‚·
An introduction to bullying: what educators need to know
ï‚·
Diversion
© NICRO 2013 214
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
ACTIVITIES EXPENDITURE
OR EVIDENCE
SUMMARISING
ASSESSING systems training
ï‚·
NCS systems
2.2 Programme implementation training (90% complete)
£2 489 489
37 OF 45 sites fully functional at time of evaluation
Implementing NCS at new sites and courts and operating costs. This is not a high figure for 45 sites and fuel and communication costs appeared underestimated as many sites said that aftercare and tracking was either not taking place or to a very limited degree owing to these kinds of shortages and time shortages.
Internal monitoring of progress and programmes 2.3 Programme Monitoring £240 460
15 hours per programme per quarter.
Qualitative and quantitative evaluation reports.
Includes technical support, local references groups (monthly local steering committees), programme governance. Facilitation.
Output 3: Professional Social Worker Expertise Available At 45 Courts (Total Cost £154 612)
3.1 Recruitment of Staff
3.2 Professional
Development of Staff
Management and
£8 929
£145 683
Advertising posts; personal profile analysis and other relevant testing: £150 per new staff member (Year 1). Turnover was fairly high and this may have been under-budgeted.
This was not entirely effective based on surveys.
Weekly supervision and consultation rested almost entirely upon supervisors with many indications of weak supervision in this evaluation. Recommended was supplementing this model with some more training opportunities and workshops and with a greater number of attendees including partners in order to improve relationships and understanding of partner roles.
Social Workers also complained there was not enough formal training away from the workplace.
Most SWs were incapable of offering the full range of programme offerings.
Output 4: Expert Alternative Sentencing Programme Evaluations (£325 099)
4.1 Development of Training Materials £24 143
SHORT NARRATIVE
PERFORMANCE,
ACHIEVEMENT
4.2 Training of NICRO Programme Staff £66 337
New programmes were developed for training at about 40 hours per programme and materials were printed. Required may be some training materials that apply to cross-disciplinary teams rather than social workers only for the purpose of developing a partnership delivery system, especially since SWs are in such short supply.
One training session per region per programme offered but there were major gaps in terms of
SW capacity to deliver on all programmes although this was also related to SW turnover
(new SWs missed the training opportunities).
Printing costs, facilitators, accommodation, meals, transport are all required and expenditure
© NICRO 2013 215
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
ACTIVITIES EXPENDITURE
OR EVIDENCE
SUMMARISING
ASSESSING was not excessive. One could rather argue there was under expenditure since SWs were not using all the programme offerings and many tended to funnel clients into a small range of them. Furthermore the cascading model of sharing information from such workshops into the organisation has not tested well as an international model (regular training of all personnel is much more effective, albeit more expensive).
This involves training LSC members and magistrates with training costs, venue costs,
4.3 Training of other Practitioners and
Stakeholders
£95 854
Magistrate toolkit developed (2009-
2010)
4,175 trained — supports court stakeholders
(sensitising court stakeholders the refreshments, printing and all that attend to providing workshops. This might have focused more selectively on developing training according to who needs what training. For instance magistrates need very little training and
SAPS needed much training but the latter was hardly addressed. Another key focus would be prosecutors and Legal AID and the latter was magistrate referrals not well addressed. Some of the marketing materials (gimmicks) were not needed and more value for money could have obtained with very targeted training and proper research from a baseline that identified training needs because it varies between types of stakeholders and some of the key ones were not addressed well.
4.4 Appointment of Project Manager £138 766 The project manager was needed to drive the project and without her, the project would not have gone far. More effort could have gone into change management within the entire culture of the organisation but she was efficient and effective.
Output 5: Independent Impact Evaluation And Publicising Findings
£55 000
SHORT NARRATIVE
PERFORMANCE,
ACHIEVEMENT
5.1 Independent impact evaluation to assess the crime reduction and recidivism impact of intervention at end of funding period
In progress. This refers to evaluation activities including the flights and accommodation for evaluation activities, evaluation processes and professional fee; yet to be determined in terms of value but greater value would have been obtained by contacting the evaluator at the start of the project and enabling the evaluator to design the research from baseline measurements. All evaluations should start with a pre-intervention baseline study but there are few evaluators and few organisations conducting properly designed studies (lack of awareness as to how measurements are taken). NICRO and many South African organisations could benefit from understanding what an evaluation is and how it is conducted as there many myths and it is more often viewed as a performance appraisal of personnel than not, which is precisely what an
© NICRO 2013 216
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
ACTIVITIES EXPENDITURE
OR EVIDENCE
SHORT NARRATIVE
PERFORMANCE,
ACHIEVEMENT evaluation should not be.
Not yet undertaken 5.2 Stakeholder Seminars (To publicise research findings)
£15 860
5.3 National Seminar (To publicise research findings)
£28 985 Not yet undertaken
SUMMARISING
ASSESSING
5.4 Prepare and disseminate final evaluation report with recommendations for wider implementation
£14 600 Not yet undertaken
Output 6: Develop and Implement on-going Public Relations Campaign to Entrench Restorative Justice Concepts in National Psyche and Educate Public (£509 712)
6.1 Public Relations Campaign (£223 470) Started with a public relations agency which
Fully-fledged national marketing and campaign
Media
Conferences
Road shows.
Testimony media was undertaken that resulted in more than R10 million marketing value. of increased referrals proved expensive and less fruitful than NICRO management of a PR campaign. However a sustained, fully-fledged national marketing and media campaign was affected that result in values of more than R10 million for marketing.
One conference per year per province contributed to a successful marketing and media effort with excellent news coverage
Regional managers and staff entered into malls, schools, city halls and public venue to explain the values of NCS services. This had a multiplier effect since it generated media releases . and visits from the community.
Clear indication of improving opinions of NCS through surveying
£178 796 6.2 Develop and Implement Lobbying
Programme: Parliamentary Portfolio and
Standing Sub-Committees
A dedicated lobbyist went to parliament and worked mainly with the portfolio committee on
Community Corrections. The money could have been better spent on national lobbying in
6.3 Develop and Implement Education, £107 446
Pretoria. The education for magistrates programme could have entered into this budget
(see below) since the key to magistrate buy in and the buy-in of other key stakeholders is working in Pretoria with key departments and stakeholders in DCS, NPA, SAPS, the
Judiciary, Justice College the Justice
Department and the cluster that works on
Justice issues from national (JCPS cluster).
These and other key partners have to be integrated into a national strategy to sustain
NCS services and there is little evidence of this kind of work and really no evidence of a budget for it either.
From the time of the MTR it was stated that
© NICRO 2013 217
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
ACTIVITIES
7.1 Secure partnerships with traditional leadership, magistrates courts and
National Prosecuting Authority prosecutors
EXPENDITURE
OR EVIDENCE
SHORT NARRATIVE
PERFORMANCE,
ACHIEVEMENT
SUMMARISING
ASSESSING
Information and Outreach Programme for
Magistrates
Alternative sentencing tool kit produced
Brochures, leaflets
Research publications — handbooks, magistrates do not refer owing much less to attitudinal problems than to structural problems pertaining to pressures to reduce the court roll, which also owes to SAPS which measures its success in terms of arrest rates, which clog the courts. Educating magistrates was not as necessary as lobbying and advocacy from the marketing materials, brochures, leaflets level of national departments to address the kinds of problems just described. The magistrates did not really use the tool kit and mainly put it in a drawer despite the 6 months it took to develop, product and print the handbook.
7. Eerstehoek: A Newly Developed IndigenousBased Diversion Programme Available (£592 005)
£28 547
3 Traditional
Courts/one making referrals
7.2 Obtain formal agreements
7.3 Finalise programme design and development
7.4 Develop intervention procedures, implementation protocols, guidelines and indicators
7.5 Programme governance
7.6 Training and orientation of traditional leaders and professional practitioners
£7 489
£5 460
£9 408
£10 192
£26 863
The expenses on this and overall did not really justify the outputs since there were only 11 referrals by 30 July 2012. Includes travel and accommodation, flights, car hire, meals for at least 2 NICRO staff per meeting, technical support.
Draw up working agreements / Memoranda of
Understanding with court sector and other service providers: 3 hours per meeting x 2
NICRO staff members per meeting @ £35 per person per hour, Refreshments, etc.
Design / development costs calculated @ £28 per hour. Development time - 3 months x 65 hours per month
Development costs calculated @ £28 per hour.
Development time - 2 months (21 days per month x 8 hours per day x 2 months = 336 hours x £28 )
Steering committee meetings and refreshments and other costs to support LSC
Facilitation costs and venues, training kits, meals, workshops
7.7 Implement interventions with young offenders
£277 780
7.8 Programme Monitoring £226 265
This is the school and community outreach with youth participants. There were sports, drama, chorus and youth activities. Most of the budget line got absorbed into staff salaries and programme costs such as the journey etc. The facilities and supplies for sporting programme were limited (few soccer balls, no uniforms, etc.)
Identifying and measuring progress (internal reports); tracking participants. This was necessary as this was a pilot and it was learnt through monitoring that the original programme theory was not working. The costs are almost as high as the programmes, however, and substantial savings could have been realised project wide with a baseline study that looked at
© NICRO 2013 218
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
ACTIVITIES
GRAND TOTAL ALL OUTPUTS
(POUNDS STERLING)
EXPENDITURE
OR EVIDENCE
£4 916 108
SHORT NARRATIVE
PERFORMANCE,
ACHIEVEMENT
SUMMARISING
ASSESSING value (identifying the challenges with the traditional courts as examined in this report could have been picked up in the baseline/scoping study)
ANNEXURE 3: TIME-TABLE FOR RESEARCH ACTIVITIES
A CTIVITY
1 IDENTIFYING
INDICATORS
2 NICRO FORMS
A REFERENCE
GROUP
D ESCRIPTION O RIGINAL
TARGET
May
3
4
WORKSHOP
INDICATORS
SCHEDULE
SITE VISITS
Evaluator reviews and analyses project documentation to identify the indicators for measuring progress.
NICRO PM and possibly other head office managers form a reference group to review indicators and support the site selection process so as to enable research processes to proceed in a timely way.
Work-shopping the indicators and measures —an evaluation matrix developed by the evaluator is refined with input from the PM and/or a reference group; finalise site selection based on the data that needs to be recovered
NICRO PM and evaluator work together to ensure that the site visits can take place with all identified informants available.
Data collection instruments based on the refined indicators are designed (e.g., questionnaires, focus groups and site observation forms). These are also reviewed with the PM
May
Late May
June
5 DESIGN
DATA
OF
COLLECTION
INSTRUMENTS
6 ISSUE LETTERS
OF
INTRODUCTION
7 RETURN OF
SURVEY DATA
7 SITE VISITS
8 INTERVIEWS
9 THE DESIGN OF
SURVEYS
The interview participants at sites are informed of the process with letters and a sample of data collection instruments so that they can think about and prepare for the interview process.
This was delayed: the sites were granted extensions owing to personnel shortages
In-person interviews are conducted at five sites with clients, social workers, supervisors, LSC members, magistrates, prosecutors and key project partners
Interviews with NICRO national and available regional managers and three high court judges follow from site visits
(preceded the design of e-mail questionnaires)
All previous research processes contribute to the design of these surveys which will also be pre-viewed in detail with the project manager
June
June
June/July
July
July 2012
July/Aug
2012
© NICRO 2013
D ATE
ACHIEVED
May
May
Late May
June
June
June
30 August
July/August
Oct-Nov
July/Aug
219
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
A CTIVITY D ESCRIPTION
10 E-MAIL
SURVEYS
E-mail surveys: project supervisors, SWs, clients, magistrates, regional managers (these did not come in at once but slowly from the sites over a period of months)
11 DATA CAPTURE Two surveys required data capture onto spread sheets; the surveys did not return until end October owing to extensions granted (sites were under pressure owing to restructuring)
12 DATA
ASSESSMENT
13 REPORT
WRITING
This is the analysis of all data sets and the generation of tables for analysis using all project materials
Report writing of the data is anticipated as the longest period owing to satisfying the TOR and DFID objectives (a late start owing to the late return of all data for assessment)
14 REVIEW The evaluator meets with NICRO PM on the initial findings to anticipate feedback and expedite processes
15 DRAFT REPORT A draft report is issued for discussion and refinements
O RIGINAL
TARGET
Aug 2012
Sep 2012
Sep 2012
D ATE
ACHIEVED
Sep-Dec
2012
Sep/Oct/Nov Jan/Feb
Nov
The evaluator meets with a NICRO national to present the
MTR findings and solicit comments on the draft evaluation
15
2012
30
2012
Nov
Nov
Nov 2012
Nov 2012
Feb 2013
Feb 2013
Feb/Mar 16 REFERENCE
GROUP
MEETING
17 FINAL
EVALUATION
18 PUBLIC
PRESENTATION
The final report is issued 31 Jan 2013 March
The evaluator presents the results of the impact evaluation Apr 2013 June
ANNEXURE 4: LIST OF RESEARCH PARTICIPANTS
The following tables provide a list of stakeholders who met with the evaluator or completed surveys for the evaluation. To protect their privacy, clients/offenders were omitted from this list. The listed research participants, mainly from government and civil society, are listed alphabetically with contact details (if available) along with the research activity or activities in which they participated.
NAME ORGANISATION RESEARCH ACTIVITIES
Abdulla, Zurina
Abrahams, P. M.
NICRO Provincial Manager, Eastern
Cape
Social Worker, Department of
Correctional Services
Anthony, Anthony CEO Business Against Crime
Batley, Mike
Boshoff, Alida
Director, Restorative Justice Centre
Provincial Manager Survey, 22 Oct 2012
Cape Town Focus Group, 09 Oct 2012
Cape Town Focus Group, 09 Oct 2012
Interview, Pretoria, 17 Aug 2012
NICRO Provincial Manager, Gauteng Focus Group with Tembisa Case Flow
Management Committee 15 Aug 2012’ Survey of
Provincial Managers, Oct 2012
Social Worker, Chatsworth SW Survey, 03 Oct 2012 Buthelezi,
Sibonelo
Chakuwamba,
Anthony
Supervisor, Eastern Cape outside
Port Elizabeth
Supervisor Survey, 01 Oct 2012
© NICRO 2013 220
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
NAME
Chanakira, Patricia NICRO Social Worker, Germiston
Chirwa, R. M.
Davids, Michelle
Dawson, Celia
ORGANISATION
Chief Magistrate, Sub-cluster head at
Eerstehoek (Justice)
NICRO Social Worker, Cape Town
NICRO Deputy CEO
RESEARCH ACTIVITIES
SW Survey, 01 Oct 2012
Focus Group at Eerstehoek 08 Aug 2012;
Magistrate Survey 25 Sep 2012
SW Survey, 08 Oct 2012
Cape Town Focus Group, 09 Oct 2012; Interview
25 Oct 2012; National Manager Focus Group 05
Nov 2012
Interview, 16 Aug 2012 Dube, Methuseli Social Worker, Restorative Justice
Centre, Soshanguve
Dube, Nomagugu Social Worker, Kabokweni
Erasmus, Clive Senior Magistrate, Wynberg, Athlone,
Philippi and Fazeka
Erasmus, Marina NICRO Supervisor
SW Survey, 25 Sep 2012
Magistrate Survey, 09 Oct 2012
Evans, Ivan
Ferreira, Steve
NICRO Supervisor
Acting Senior Magistrate, Welkom
Gaba, M Department of Social Development
Geldenhuys, Thea NICRO supervisor, Soshanguve
Supervisor Survey, 21 Sep 2012; SW Survey 20
Sep 2012
Survey, 18 Sep 2012
Focus Group, LSC Members, Welkom, 29 Aug
2012
Cape Town Focus Group, 09 Oct 2012
Interview, Soshanguve, 16 Aug 2012
Grootboon, Merrel Department of Correctional Services,
Social Worker
Hogewind, Este Social Worker, Kuils River
Ipeleng
Johannes, Taryn
NICRO Social Worker, Kimberley
NICRO Social Worker, Wynberg
Cape Town Focus Group, 09 Oct 2012
SW Survey, 01 Oct 2012
SW Survey, September 2012
SW Survey, 08 Oct 2012
Joubert, Mariette
Jules-Macquet,
Regan
NICRO SW, Batho
NICRO Project Manager
SW Survey, 12 Sep 2012
Cape Town Focus Group, 09 Oct 2012; Interview
25 Oct 2012; National Manager Focus Group 05
Nov 2012
SW Survey, 09 Oct 2012 Khobeni,
Sibongile
Khoza, Gcina
Hilda Social Worker, Johannesburg
Social Worker, Eerstehoek Interview, 07 Aug 2012, Eerstehoek; Survey 01
Oct 2012
Cape Town Focus Group, 09 Oct 2012 Koen, Shawn
Kok, Corlia
Business Against Crime, WC Project
Manager
Administration, Office of the Director
General, Department of Justice and
Constitutional Development
Kruger, Elizabeth Control Prosecutor, Tembisa
Interview, 14 Aug 2012
Kunene,
Jubulile
Sharon Deputy Commissioner
Reintegration, Department
Social of
Correctional Services
SANCO, Tembisa
Interview, 15 Aug 2012, Tembisa
Focus Group, 17 Aug 2012. Pretoria
Langa, Nelson
Lametyi, Melphin SANCO, Tembisa
Focus Group with Tembisa Case Flow
Management Committee, 15 Aug 2012, Tembisa
Focus Group with Tembisa Case Flow
Management Committee, 15 Aug 2012, Tembisa
E-mail survey 26 Sep 2012 Leander,
Tashreequa
Legong, Andries
NICRO Supervisor, Port Elizabeth
Control Prosecutor, Soshanguve
Lekgetho, Jennifer Deputy Director,
Interview with LSC member, 16 Aug 2012
Community Focus Group, 17 Aug 2012, Pretoria
© NICRO 2013 221
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
NAME ORGANISATION
Lotz, Steven
Ludick, Sarah
Lusunzi, V C
Involvement, Department
Correctional Services of
Business Against Crime Project
Manager
Supervisor, Pietermaritzburg
Social Worker, Department of
Correctional Services
Magistrate, Soshanguve Luus, J. E.
Mambumba,
Nomkita
Department of Correctional Services,
Social Worker
Mabamga, Somto Albert Luthuli Human Rights Advice
Center
Mailoka, Moksawa Department of Social Development,
Sekgosese (probation officer)
Mako, Busisiwe
Bernadette
NICRO Supervisor, Vaal
Maphumulo, Regi Senior Prosecutor, NPA, Welkom
RESEARCH ACTIVITIES
Cape Town Focus Group, 09 Oct 2012
Supervisor Survey, 27 Sep 2012
Focus Group, 21 Aug 2012
Interview, 16 Aug 2012
Cape Town Focus Group, 09 Oct 2012
Focus Group at Eerstehoek, 08 Aug 2012
Focus Group with LSC members, Sekgosese
Survey, Sep 2012
Mare, Claudine
Masango,
Johannes
Mashonganyika,
Caroline
Masina, Mr M S
Masipa, Guy
Mbedzi, Tinny
Mbetse, Portia
NICRO,
Mpumalanga
Traditional Leader, Ndebele Ndzusa
Tribal Authority (near Tembisa)
NICRO social worker, Eerstehoek
Traditional
Social Worker, Soweto
NICRO supervisor, Sekgosese and
Mankweng
Provincial
Leader,
Traditional Authority
Court Manager, Sekgosese
Manager,
Mpisikazi
Focus Group, LSC Members, Welkom, 29 Aug
2012
Site visit to Eerstehoek 06-09 August (site-based observations, interviews, arranging research processes) the supervisor survey 01 Oct 2012;, the provincial manager survey
Interview, 15 Aug 2012, Tembisa; Focus Group with Case Flow Management Committee 15 Aug
2012
Interview, 07 Aug 2012, Eerstehoek; Survey 19
Sep 2012
Focus Group, Mpizikazi Traditional Authority, 09
Aug 2012
Focus Group, LSC members, Sekgosese, 21 Aug
2012
SW Survey, 25 Sep 2012
Interview at Sekgosese; 20 Aug 2012; Supervisor
Survey, October 2012’ Focus Group, LSC members, Sekgosese, 21 Aug 2012
Magistrate Survey, 10 Oct 2012 Menyiwe, Patrick
Xolani
Meyer, Ms S
Mhlanga, M. L.
Magistrate, Khayelitsha Court
Senior Public Prosecutor, NPA
Magistrate,
Kanyamazane
Kabokweni
Department of Justice and
Cape Town Focus Group, 09 Oct 2012
Magistrate Survey, 26 Sept 2012
Mimi, Constance
Mkhize,
Thembelihle
Mlambo,
Nkosinathi Mike
Mlambo, Lymon
Lucas
Moeng,
Letihogonolo
Moganedi,
Rethabile Pearl
Social Worker, Katlehong
Social Auxiliary Worker, Eerstehoek
Traditional Leader,
Traditional Authority
Magistrate, Bloemfontein
NICRO Social
Boksburg/Alberton
Embuleni
Worker,
Focus Group with Case Flow Management
Committee, Tembisa
SW Survey, 01 Oct 2012
Interview, 08 Aug 2012, Eerstehoek
Focus Group, Embuleni Traditional Authority, 08
Aug 2012
Magistrate Survey, 02 Oct 2012
SW Survey 01 Oct 2012
© NICRO 2013 222
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
NAME ORGANISATION RESEARCH ACTIVITIES
Moitaung, T. I. Senior State Prosecutor, National
Prosecuting Authority, Pretoria
Mokhele, Tebogo Legal Aid, Welkom
Focus Group, NPA, Pretoria, 13 Aug 2012
Mokwenanangulel e, Remembrance
Molata, Ben
Molwantwa-
Mogotsi,
Nthabiseng
Mothupi, L. M.
Senior Prosecutor, NPA
NICRO Provincial Manager. Limpopo
Prosecutor, National Prosecuting
Authority
Motloung, Mr B. J. Deputy Director, Social Profiling and
Networking, Department
Correctional Services of
Motlora, Lusani Magistrate, Sekgosese
Focus Group, LSC Members, Welkom, 29 Aug
2012
Interview, 20 Aug 2012, Sekgosese; Survey of
Provincial Managers, 08 Oct 2012
Focus Group, LSC members, Sekgosese, 21 Aug
2012
Focus Group, 17 Aug 2012, Pretoria
Focus Group, LSC members, Sekgosese, 21 Aug
2012
Focus Group, 17 Aug 2012, Pretoria Mpambani,
Nomanwethu
Rosebella
Deputy Director, Department of
Correctional Services, Offender
Reintegration
Mpelane, Xolani
Mr
Mphalele, W K K
Mpho, Lillian
NICRO Supervisor, South Gauteng
Department of Social Development,
Gauteng
Senior Prosecutor, National
Prosecuting Authority, Pretoria
Prosecutor, Sekgosese
Focus Group, LSC Members, Welkom, 29 Aug
2012
Survey, September 2012; Focus Group with
Case Flow Management Committee, 15 Aug
2012
Focus Group with DSD, 13 Aug 2012
Focus Group, 13 Aug 2012, Pretoria
Interview, 21 Aug 2012, Sekgosese
Mpilo, Suzan NICRO Social Worker, Tembisa Interview 15 Aug 2012, Tembisa
Mtebele,
Nomanethu
Muelau, L. M.
Legal Aid, Tembisa
Social Worker, Department of Social
Development
Mukaronda, Job M NICRO Social Worker, Mamelodi
Muleya, Emmison NICRO Social Worker, East London
Focus Group with Case Flow Management
Committee, Tembisa
Focus Group, LSC Members, Sekgosese, 21 Aug
2012
SW Survey, 27 Sep 2012
SW Survey, 01 Oct 2012
Murenga, Margaret NICRO Social Worker, Port Elizabeth SW Survey, 27 Sep 2012
Mzandolo,
Busiswe
Naidoo, T
Netshimbupfe,
Mulatedzi
Ngcobo,
Nokuthula
Mrs.
Ngcobo,
Mrs.Nommiselo
Social Worker, Benoni
Senior Public Prosecutor, NPA
Wynberg
District Court Prosecutor, NPA
Probation Officer, Department of
Social Development, Tembisa
NICRO Social Worker, Durban
Ngulele,
Remembrance
Ngwenya,
Duduzile
NICRO Supervisor, Tembisa
NICRO supervisor, Durban
Ngewenya, Lungile NICRO SW, Queenstown
SW Survey, 01 Oct 2012
Cape Town Focus Group, 09 Oct 2012
Cape Town Focus Group, 09 Oct 2012
Focus Group with Tembisa
SW Survey, October 2012
Interview, 16 Aug 3012, Tembisa
Supervisor Survey, 08 Oct 2012
SW Survey, October 2012
Case
Management Committee, 15 Aug 2012
Flow
Nhalapo, Mr M A Traditional Leader, Mpisikazi Focus Group, Mpizikazi Traditional Authority, 09
© NICRO 2013 223
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
NAME
Magistrate, Sekgosese
Magistrate, Tembisa
RESEARCH ACTIVITIES
Nicholas, Barbara NICRO Provincial Manager
Nkosi,
Mfanasirili
Nkosi,
Mfana
Nkosi, Mbali
Alex-
Bheki
Administration Officer, Embuleni
Traditional Authority
Traditional Leader,
Traditional Authority
Embuleni
Probation Officer, Eerstehoek
Nkosi,
Johan
Salibona
Nkosi, Thembani
Managing Director, Crime Combating
Response Group, Eerstehoek
NICRO Social
Daveyton/Benoni
Worker,
Nkuna, Nkhensani NICRO Social Worker, Tembisa
Supervisor Survey 03 Oct 2012; Cape Town
Focus Group, 09 Oct 2012; Survey of Provincial
Managers 22 Oct 2012
Focus Group, Embuleni Traditional Authority, 08
Aug 2012
Focus Group, Embuleni Traditional Authority, 08
Aug 2012
Interview, Eerstehoek, 07 Aug 2012
Interview, 07 Aug 2012
SW Survey, 01 Oct 2012
Nomiselo NICRO Social Worker, Pinetown
Nthupi, Nthabiseng NICRO SW, Vaal
Interview 15 Aug 2012, Tembisa; Focus Group with Case Flow Management Committee 15 Aug
2012; SW Survey 12 Sep 2012
SW Survey, 02 Oct 2012
SW Survey, Oct 2012
Ntuli, Mr Social Profiling at Department of
Community Corrections
Focus Group, 17 Aug 2012, Pretoria
Oosthuizen,
Barend
Magistrate, White River
Opperman, Lanie Magistrate, Bloemfontein
Padayachee,
Venessa
Peck, Ibtisaam
Magistrate Survey, 12 Oct 2012
Interview, 28 Aug 2012; Magistrate Survey 08
Oct 2012
Advocacy Manager (national), NICRO Focus group, NICRO National, 05 Nov 2012.
Interview 09 Nov 2012
NICRO Social Worker, Simonstown SW Survey, 01 Dec 2012
Peer, Celia Magistrate, Durban
Phofa, Kuomotso Director, Department of Social
Development, Gauteng
Pillay, Mrs. A. M. Magistrate, Bloemfontein
Radebe, J Correctional Services, Welkom
NICRO Social Worker, Mankweng
Aug 2012
Magistrate Survey, 04 Oct 2012
Focus Group with DSD, 13 Aug 2012
Interview, 28 Aug 2012
Focus Group, LSC Members, Welkom, 29 Aug
2012
SW Survey, 09 Oct 2012 Ramakgwakgwa,
Florina
Scheepers,
Annelize
Schnetler, Elsa
ORGANISATION
Traditional Authority
Schutte, C. E.
Sello, Thakane
Senoko, M C
Sethumele, Gani
Shezi, Siboniso
Sihawy,
Veronica
Ms
Senior Magistrate, Port Elizabeth
NICRO Social Worker, Welkom
Chief Magistrate, Soshanguve
NICRO Social Worker, Sekgosese
NICRO Social
Pietermaritzburg
Worker,
Department of Social Development,
Gauteng
Interview, 21 Aug 2012; Focus Group with LSC
21 Aug 2012; Magistrate Survey 02 Oct 2012
Focus Group with Case Flow Management
Committee, Tembisa
Magistrate Survey, 26 Sep 2012
Interview with Social Worker, Welkom, 27 Aug
2012; SW Survey 21 Sep 2012
Interview, Soshanguve, 16 Aug 2012
Interview, Sekgosese, 21 Aug 2012; Focus
Group, LSC members, Sekgosese, 21 Aug 2012;
SW Survey, 08 Oct 2012
SW Survey, 28 Sep 2012
Focus Group with DSD, 13 Aug 2012
© NICRO 2013 224
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
NAME ORGANISATION
Skhonde,
Stephanie
National Prosecuting Authority
Snitcher, Norman Cape Law Society
Sololia, Pearl
Solomon, Sonia
Tobeka
Department of Social Development,
Sekgosese (probation officer)
Provincial Manager, KwaZulu-Natal
NICRO SW Khayelitsha
Van der Merwe, A
Van Kraayenberg,
Marita
Magistrate, Nelspruit
NICRO Provincial Manager, Free
State and Northern Cape
Van Staden, Rene NICRO Supervisor
RESEARCH ACTIVITIES
Focus Group at Eerstehoek, 08 Aug 2012
Cape Town Focus Group, 09 Oct 2012
Focus Group with LSC members, Sekgosese
Survey of Provincial Managers, 12 Oct 2012
SW Survey, Khayelitsha
Magistrate Survey, 01 Oct 2012
Interview, 28 Aug 2012, Welkom; Provincial
Manager Survey, 08 Oct 2012
Supervisor Survey 29 Oct 2012; Cape Town
Focus Group, 09 Oct 2012
Interview, 14 Aug 2012, Pretoria Van Vuuren, Peet Senior Magistrate, Judiciary and
Magistrate Commission
Van Zyl, Miriam NICRO Supervisor
Vilikazi, Samuel
Von Bratt, K. J.
Traditional Leader,
Traditional Authority
Magistrate, East London
Mpisikazi
Williams, J Senior Prosecutor,
Prosecuting Authority, Paarl
National
Yennkosi, Ntando NICRO Social Worker, Soshanguve
Site Based Observations, 29 Aug 2012;
Interview, Bloemfontein 29 Aug 2012; Supervisor
Survey, 26 Sep 2012
Focus Group, Mpizikazi Traditional Authority, 09
Aug 2012
Magistrate Survey, 27 Sep 2012
Cape Town Focus Group, 09 Oct 2012
Interview, Soshanguve, 16 Aug 2012
ANNEXURE 5: DOCUMENTS CONSULTED
1. Clark, Patrick (2012) Preventing Future Crime with Cognitive Behavioural Therapy.
National Institute of Justice, US Department of State, Washington, D C
2. Department of Correctional Services (February 2005), White Paper on Corrections in
South Africa (revised), Republic of South Africa
3. Department of Correctional Services (10 June 2008) White Paper on Community
Corrections, Republic of South Africa
4. Dawson, Celia, Pierces Betzi and Regan Jules-Macquet (26 February 2010) Meeting
Notes on the DFID Indigenous Justice Pilot Project
5. Department of Justice and Constitutional Development (April 2012) Restorative Justice
National Policy Framework, Draft Revision, 19 pages
6. Hoffman, Sandy (2005) ‘Rehabilitation of Prisoners in a transforming South Africa, www.csvr.org.za/confpaps/hoffman.htm
7. Jules-Macquet, Regan (2011) Exploring Non-Custodial Sentencing: Four Offender Case
Studies, NICRO, Cape Town
8. Mare, Claudine (2012) Intakes and Sources of Referral for the Indigenous Justice Pilot
Project (to end July 2012)
9. Mare, Claudine (2012) Community Projects in Eerstehoek to end July 2012
10. Minister of Justice and Constitutional Development (2008) Act No. 75 of 2008, Child
Justice Act
© NICRO 2013 225
© NICRO 2013
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013
11. NICRO
(22 September 2009), ‘Minutes of Meeting Held at Nelspruit Magistrate’s Court’,
Indigenous Justice Project, Nelspruit
12. NICRO (undated, internal document) Community-based Marketing Guide for Noncustodial Sentencing, Cape Town
13. NICRO (20 April 2008) NICRO Corporate Plan 2008-2010, Cape Town
14. NICRO (2008-2012) Rate of Magistrates Accepting NICRO Sentencing Options
15. NICRO (undated internal document) NICRO SW Policy and Procedures Manual, Cape
Town
16. NICRO (2009) Review of the Non-custodial sentencing project funded by DFID, Cape
Town
17. NICRO
(2009) ‘Full report on the 2009 Public Attitudes toward Crime, Punishment and
Rehabilitation Surve y,’ Cape Town
18. NICRO (June 2010-June 2012) Annual Reports to DFID on GTF112
19. NICRO (2010) Toward Indigenous Practice with Offenders Linked to Indigenous
Knowledge Systems, (a compilation of papers, field observations, analysis and presentation material)
20. NICRO (2010) Developing and Indigenous Justice Pilot Project: A Research Report,
Cape Town.
21. NICRO (2011) ‘Research into the Role of Traditional Leaders and the Possibility of
Integrating Siswati Practices into NICRO Services’, Traditional Leaders Research
Paper, Cape Town
22. NICRO (2012) Indigenous Justice Pilot Reporting Form (internal document)
23. NICRO (2012) Integrated Statistics 2008-2012
24. Open Society Foundation of South Africa (November 2010) Report on the Open Society for South Africa Conference on Recidivism and Reoffending in South Africa, Sandton
Sun Hotel, Johannesburg
25. Restorative Justice Centre (2012) Executive Summary of the Report on Traditional
Leaders (Inputs into the RJ National Policy Framework)
26. Padayachee, Venessa (30 November 2009) NICRO Submission: DCS Awaiting Trail
Detainees Discussion Document , pp. 1-4
27.
Padayachee, Venessa (18 March 2010) ’NICRO’s contribution to the reintegration of offenders in SA-2009/10 and planned targets 2010/11, Submission to Department of
Correctional Services, pp. 1-12
28. Padayachee, Venessa (10 March 2010) NICRO submission: DCS Budget 2010/11 and
Strategic Plan 2010/11-2014/15 , pp. 1-10
29. Padayachee, Venessa (16 March 2011) NICRO Submission: Department of
Correctional Services Budget Vote 21 , Cape Town
30. Padayachee, Venessa (24 August 2011) NICRO Submission to the Portfolio Committee on Corrections: The Jali Commission Report-Progress, Cape Town.
31. Padayachee, Venessa (16 Oct 2012) NICRO Submission on Judicial Inspectorate
Annual Report 2011/2012, Cape Town
32. Padayachee, Venessa (16 Apr 2012) Submission by NICRO to the Portfolio Committee on Correctional Services on the DCS Budget Vote, 2012/13, Vote 21 and the Strategic
Plan, Cape Town
33.
Rossouw, Henry (2010) ‘Alternatives to traditional sentencing methods-the efficacy and constitutionality of periodic imprisonment in South Africa,’
226
GTF 112: NICRO Final Impact Evaluation Report 2008 - 2013 http://www.uclawsociety.com/colr/edutions/2010/alternatives%20totraditional%20senten cing%methods.pdf
34. Schroeder, Fatima (25 November 2012) ‘Crisis as SA jails bursting at the seams’
Sunday Argus, p.4
35. Seria, Nasreen (29 Jul 2010) ‘Crime in South Africa is Businesses Biggest Constraint;
Bloomberg, http://www.bloomberg.com/news/print/2010-07-29/crime-in-south-africa
36. Total Media (February 20
10) ‘Campaign Analysis for NICRO’ unpublished document
37. United Nations (14 September 1990) Standard Minimum Rules for Non-custodial
Sentencing (Tokyo Rules) http://www.un.org/documents/ga/res/45/a45r110.htm
38. Van der Westhuizen, AEM and A Lombard (2005) A Model for Community Corrections
Residential Centres in South Africa from A Social Work Perspective, Acta Criminologica
18(3) pp 101-109
Please note that the remainder of the Annexures have been submitted to DFID as attachments, and are available on request from regan@nicro.co.za
.
© NICRO 2013 227
