A Review of the use of Performance Information in the ...

A Review of the use of Performance Information

in the Crown Prosecution Service

September 2005

CCOONNTTEENNTTSSCCOONNTTEENNTTSS

PPAAGGEEPPAAGGEE

11..11.. IINNTTRROODDUUCCTTIIOONNIINNTTRROODDUUCCTTIIOONN 1111The purpose of the review 1Performance information in the CPS 1Performance reporting in the CPS 2The staffing structure of the CPS 3

22..22.. SSUUMMMMAARRYY OOFF FFIINNDDIINNGGSS AANNDD RREECCOOMMMMEENNDDAATTIIOONNSSSSUUMMMMAARRYY OOFF FFIINNDDIINNGGSS AANNDD RREECCOOMMMMEENNDDAATTIIOONNSS 5555Good practice 7Recommendations 7

33..33.. MMEETTHHOODDOOLLOOGGYYMMEETTHHOODDOOLLOOGGYY 9999The focus of the review 9Our approach 9The Issues Analysis and Dinner PartyTM approach 11External assistance 11General points about the review 11

44..44.. SSTTRRUUCCTTUURREE OOFF FFIINNDDIINNGGSSSSTTRRUUCCTTUURREE OOFF FFIINNDDIINNGGSS 11331133

55..55.. HHAAVVEE AARREEAASS IIDDEENNTTIIFFIIEEDD TTHHEE PPRRIINNCCIIPPAALL PPEERRFFOORRMMAANNCCEEHHAAVVEE AARREEAASS IIDDEENNTTIIFFIIEEDD TTHHEE PPRRIINNCCIIPPAALL PPEERRFFOORRMMAANNCCEE 1155 1155

MMEEAASSUURREESS TTHHAATT RREEQQUUIIRREE TTRRAACCKKIINNGG??MMEEAASSUURREESS TTHHAATT RREEQQUUIIRREE TTRRAACCKKIINNGG??Overview 15Area agreement with purpose of performance measures 15Area performance reporting systems 16Data range 16Inter-agency data sharing 18Ownership of performance measurement systems 18

66..66.. HHAAVVEE TTRRAAIINNEEDD RREESSOOUURRCCEESS BBEEEENN AALLLLOOCCAATTEEDD TTOO IINNPPUUTT HHAAVVEE TTRRAAIINNEEDD RREESSOOUURRCCEESS BBEEEENN AALLLLOOCCAATTEEDD TTOO IINNPPUUTT 22112211

AANNDD CCOOLLLLAATTEE PPEERRFFOORRMMAANNCCEE IINNFFOORRMMAATTIIOONN??AANNDD CCOOLLLLAATTEE PPEERRFFOORRMMAANNCCEE IINNFFOORRMMAATTIIOONN??Overview 21Data input 21Data collection and extraction 22Data transfer 23Training 24

Promoting Improvement in Criminal Justice

The use of PerformanceInformation in the CPS

77..77.. DDOO AARREEAASS PPRREESSEENNTT IINNFFOORRMMAATTIIOONN TTHHAATT HHAASS BBEEEENN DDOO AARREEAASS PPRREESSEENNTT IINNFFOORRMMAATTIIOONN TTHHAATT HHAASS BBEEEENN 22552255

VVAALLIIDDAATTEEDD EEFFFFEECCTTIIVVEELLYY??VVAALLIIDDAATTEEDD EEFFFFEECCTTIIVVEELLYY??Overview 25Area perception of data accuracy 25Inspectors’ findings on data accuracy 26Dismissed - no case to answer 27Judge directed acquittals 28Convictions after trial 28Guilty plea - specified proceedings 29Administrative finalisation 30Quality control 31Quality assurance 32

8888 CCAANN AARREEAASS DDEEMMOONNSSTTRRAATTEE IINNCCIISSIIVVEE DDEECCIISSIIOONN--MMAAKKIINNGGCCAANN AARREEAASS DDEEMMOONNSSTTRRAATTEE IINNCCIISSIIVVEE DDEECCIISSIIOONN--MMAAKKIINNGG 33333333

TTOO IIMMPPRROOVVEE PPEERRFFOORRMMAANNCCEE??TTOO IIMMPPRROOVVEE PPEERRFFOORRMMAANNCCEE??Overview 33Performance management at different tiers of management 34Benchmarking with other Areas/units to identify better practices 35Aligning resources based on performance information 37Making changes to improve performance 38Understanding/skills of numerical analysis for performance management 38Use of performance information to obtain value for money 40Incisive decision-making to improve performance 41Focus of performance management 41Trends analysis 41Depth of analysis and cause and effect 42The relationship between performance management systems 42and delivery results

AANNNNEEXX 11AANNNNEEXX 11 Question tree

AANNNNEEXXAANNNNEEXX 2222 Questionnaire responses

AANNNNEEXXAANNNNEEXX 3333 Glossary

HM Crown Prosecution Service Inspectorate


11..11.. IINNTTRROODDUUCCTTIIOONNIINNTTRROODDUUCCTTIIOONN

TThhee ppuurrppoossee ooff tthhee rreevviieeww

1.1 This is the report on Her Majesty’s Crown Prosecution Service Inspectorate’s(HMCPSI) thematic review of the use of performance information in the CrownProsecution Service (CPS).

1.2 The aim of this review is to consider whether the CPS is making effective use ofperformance information to manage and deliver its business, with particularreference to the delivery of effective prosecutions within the 42 Areas. It seeks toascertain what data is being collected by the CPS, the accuracy of that data, andthe use of data to manage performance.

PPeerrffoorrmmaannccee iinnffoorrmmaattiioonn iinn tthhee CCPPSS

1.3 The CPS collects a wide range of information about its performance. First, it records the number of cases received and finalised, and their outcomes. This type of information is commonly termed “performance indicators” (PIs) in the CPS.Apart from giving a general indication of an Area’s performance, it also informsActivity Based Costing (ABC) - the tool with which CPS allocates funding to Areas.ABC funding considers the relative types and numbers of cases handled byindividual Areas and provides data that informs Headquarters’ Business DevelopmentDirectorate as to how the CPS budget should be allotted. ABC does not provideinformation about the actual unit costs of cases handled or activities undertaken bythe CPS, but is used to inform broad average costs for the purposes of informingapplications for costs against convicted defendants, and costing the impact of newlegislation. Secondly, the CPS collects information of a more qualitative nature,such as the reasons for certain case outcomes, together with other service deliverymeasurements including timeliness and compliance with national prosecutionpolicies. Thirdly, like all other organisations of comparable size, information iscollected on human resources and financial performance. Finally, the CPS usesinformation which help partners in the criminal justice system to assess andmanage joint performance, for example, cracked and ineffective trial rates.

1.4 Much of the information is inputted routinely into the Compass Case ManagementSystem (CMS), the CPS’s case tracking and management system. The systemtracks a case from registration (the file arriving at the CPS from the police) tofinalisation. Entries should be made after a hearing has taken place or work hasbeen done on the file, for example, after a trial review. The system thereforecontains information on what has been done and when the actions occur; whatneeds to be done and by when; and court outcome and a brief note of the reasonsfor it. CMS also contains a set of standard reports to assist managers in monitoringand managing performance.

1



1.5 Additionally, Areas have access to the associated Management Information System(MIS), a more sophisticated reporting tool. Using data downloaded from CMS,MIS produces a suite of reports on caseload, case type and outcome. These reportsform a basic but important management tool for CPS managers. MIS is extremelyflexible, allowing Areas to develop local ad-hoc reports as required. This represents asignificant opportunity for the CPS to take a more dynamic, yet focused, approachto performance management.

1.6 National data on caseload and outcomes is stored in the Corporate InformationSystem (CIS), which also contains data on deployment and finance. CIS is fed bydata manually inputted into local systems and sent electronically to Headquarters.An automatic interface from MIS to CIS was implemented as planned in July 2005and the ability to make manual adjustments to CIS information ceased. As aconsequence, Areas must ensure that data for any given month is complete andaccurate by the 11th day of the following month.

1.7 Access to both MIS and CIS is only available to a limited number of authorisedlicence holders.

PPeerrffoorrmmaannccee rreeppoorrttiinngg iinn tthhee CCPPSS

1.8 Historically, most Areas have viewed the quarterly reports produced by InternalResource and Performance Branch (IRPB) at CPS Headquarters as the primarynational CPS performance management system. This is supplemented in mostcases by inter-agency data produced by the Local Criminal Justice Boards (LCJBs).Areas needed to submit quantitative data on PIs and a small number of corporateperformance measures (mainly timeliness). There was no requirement for Areas toprovide any qualitative data other than an annual self-assessment against somekey processes via a Certificate of Assurance scheme.

1.9 Over the past 18 months the CPS has been implementing and testing a morecomprehensive performance management reporting system which takes advantageof the new reporting functionality of improved technology. The process started with13 “priority” Areas and has gradually been rolled-out more widely. Changes havebeen made to incorporate new initiatives and revised objectives and priorities. FromApril 2005, all Areas participate in the scheme which involves monitoring,on a quarterly basis, progress in 15 key aspects of work. The new measures aremore relevant and outcome-focused than those previously used. Headquarters staffpopulate the performance report and Areas may be called upon to explain performancelevels, either in writing or at a meeting. The CPS produces a high-level “performancedashboard” that indicates how each Area, and the organisation as a whole, areperforming against key targets and objectives. Assessments are based on thelikelihood of reaching targets based on the most recent quarter’s performance.They also consider performance against similar sized Areas and whetherperformance is improving or deteriorating. Areas need to have systems in place toenable them to understand and comment on performance levels, and havefreedom as to how they achieve this. They will also need the necessary insight toenable them to interpret the numbers and identify appropriate remedial actions.


2The use of PerformanceInformation in the CPS

1.10 This review covers a period of development and reflects both the reporting andanalysis of performance as it stood, and how it needs to adapt to support the newperformance measures.

TThhee ssttaaffffiinngg ssttrruuccttuurree ooff tthhee CCPPSS

1.11 Throughout this report, we refer to various job functions and grades within the CPSstaffing structure and some of the tasks carried out by staff in those levels. To assistthe reader we outline briefly the levels, and main tasks associated with each.

1.12 Each Area has an Area Business Manager (ABM) who is responsible for establishingand maintaining effective business systems and processes. They also take the leadon planning and finance in most Areas.

1.13 Lawyers within the CPS, outside the Senior Civil Service and Chief CrownProsecutor (CCP) cadre, are banded in four levels - C1, C2, D and E - with level Ebeing the most senior. Level Es typically manage operational units in an Area inaddition to having a personal caseload and court commitments. In some Areas,their job title is Unit Head, while in others they are known as Branch CrownProsecutors or District Crown Prosecutors. Levels Ds tend also be managers,but of smaller units or teams. In some Areas, level Ds report directly to the CCP,while in others they are managed by level Es.

1.14 Caseworkers are at levels A and B. Level B staff subdivide to B1, B2 and B3.The higher the number, the more senior the post. Some level B2 staff aredesignated caseworkers (DCWs) and their main - sometimes sole - role is to presentnon-contentious cases in the magistrates’ courts. Many B1 staff are Crown Courtcaseworkers, some of whom have line management responsibilities. Other level Bstaff are mainly involved in process and/or personnel management. In this report,we have referred to job titles such as Area Performance Officers, Unit BusinessManagers and Unit Performance Officers. They are invariably level B staff.

1.15 Level A sub-divides into A1 and A2, with the latter mainly responsible for data input.A number of Areas operate magistrates’ courts’ units (sometimes known asCriminal Justice Units) that are co-located with the police. In some cases, CPSadministrative tasks such as file registration and case tracking have been takenover by the police. We shall comment on this further in Chapter 6.

3



22..22.. SSUUMMMMAARRYY OOFF FFIINNDDIINNGGSS AANNDD RREECCOOMMMMEENNDDAATTIIOONNSSSSUUMMMMAARRYY OOFF FFIINNDDIINNGGSS AANNDD RREECCOOMMMMEENNDDAATTIIOONNSS

2.1 We found good progress has been made over the last 12 months in raising theprofile of performance management as an essential, underpinning aspect of CPSAreas’ work.

2.2 There is a greater recognition of the need for effective performance managementregimes and the concept of local accountability for performance is now more widelyaccepted within the CPS, when compared with findings in previous single agencyinspections.

2.3 There has been some improvement in the sharing of data between agencies,particularly where either the CPS or the LCJB has a dedicated Performance Officer.

2.4 Investment in technology has led to more meaningful, relevant and outcome-focusedperformance data than had previously been available. It is unlikely that some of theimprovements noted would have happened without clear targets, supported bytransparent measures. The introduction of CMS and the associated ManagementInformation System has provided easy access to a wide range of information.Effective performance management has therefore become a realistic goal.

2.5 The use of this new technology is improving, but there is still some way to go inensuring that the full benefits of the investment are realised. Further education ofstaff is required to maximise the opportunities available.

2.6 All the Areas visited had developed a performance pack that was considered bytheir management team on a regular basis. There is considerable variation as tothe quality and utilisation of the packs, but there is a solid foundation now in placeon which Areas can build where necessary. Improvements to the presentation ofperformance data will improve the understanding and ‘buy-in’ of staff. A greaterfocus on trends in data and the use of more pictorial representation of informationwould be of benefit.

2.7 The new national performance management reporting system is leading to greaterconsistency of data collection and has narrowed the gap between those Areas withwell-established systems and those where performance measures are less developed.Most of the Areas visited were well placed to adopt or adapt to the requirements ofthe new national system.

2.8 Though it is as yet difficult to achieve precise comparisons between Areas because data systems are still evolving, those Areas with well-establishedperformance management systems did appear to be performing better in keyrespects. This underlines the need for the CPS to ensure that the new systems are used as effectively as possible across all Areas to drive up performance.

5



2.9 The challenges faced by large Areas are significantly more complex than forsmaller ones in regard to getting buy-in from staff, communicating new practicesand generally improving staff skills.

2.10 We found considerable differences in the extent to which staff at all levelsunderstood performance management techniques. This applied to the wholeprocess of data extraction, analysis and using data to drive decision-making.There is a clear need for more effective training and guidance to ensure high andconsistent standards throughout the CPS. Area analysis of data did not alwaysidentify the root cause of weak performance, or highlight what remedial actionneeded to be taken to bring about improvements.

2.11 With the increased importance of performance management, the CPS shoulddevelop a more consistent approach to the responsibilities and competenciesrequired to fulfil a performance role effectively.

2.12 Whilst the accuracy of data has often been questionable, in the Areas visited wenoted an improvement in validation since the beginning of 2005. Our own testsnevertheless identified high error rates in some PIs. The impact of most of theseerrors on overall data integrity is comparatively low, because the great majority ofcases are finalised as guilty pleas or proofs in absence, where accuracy is muchbetter. Although substantial additional investigation would be needed to assess theoverall accuracy of data, its integrity could be improved by some comparativelysimple changes within CMS that would reduce the keying errors which represent asignificant proportion of the mis-recording.

2.13 Further work is required in ensuring that advice given through the pre-charge schemecan be matched to a corresponding charge file when the decision is to proceedwith the case. Improved police co-operation in providing the appropriate uniquereference number (URN) will be important in realising this aim. Without someimprovements, the data in respect of both benefits realisation - and for PI purposes -is unlikely to be reliable. Staff will also devote unnecessary time in trying to identifymatches through various systems-driven, or in some cases manual, searches.

2.14 If data is to be used to drive performance it is essential not only that it is of a highquality, but that it is seen to be so. At present, the extent to which CPS staff haveconfidence in the integrity of the data varies widely. We recommend that Areasroutinely audit their data to an agreed standard so that all concerned can beassured of its quality and will therefore feel confident about relying on it in theirdecision-making.

2.15 As people’s confidence grows in respect of data accuracy, so the likelihood increasesthat they will be able to engage more actively and effectively in benchmarkingperformance between units and Areas. Benchmarking as an improvement techniquewas not widely practiced. Where any activity was underway, it tended to be a high-level comparison of Area performance figures as distributed by Headquarters,rather than a developed system of comparing processes and outcomes betweenAreas using a defined method.



2.16 In conclusion, the CPS is moving in the right direction with a much more positiveapproach to performance management. The infrastructure is now in place and thereis a solid foundation on which to build. Optimising the use of technology and training,particularly on analytical techniques, are the keys to moving to the next level.

GGoooodd pprraaccttiiccee

2.17 We have identified two aspects of good practice that all Areas should consider.These are:

� Adoption of data specifications, similar to those introduced in West Yorkshire,which set out what is to be measured, by whom, when, why and how. Thiswould provide Areas with a common performance management language.

� Adoption of systematic quarterly data audits as conducted by Cheshire,where errors are identified and results are disseminated so that any lessonscan be quickly learned.

RReeccoommmmeennddaattiioonnss

We have made 11 recommendations to help improve the collection, understandingand use of performance information by the CPS. We recommend:

Performance management

1111 That Headquarters and Areas need to establish data integrity regimes which provide for an appropriate level of accuracy and an accompanyingconfidence factor (Paragraph 7.44).

2222 To promote more focussed discussion on causes and remedial actions,Areas should develop their presentational and data consolidation methods,so that performance information is presented pictorially with underlyingtrends readily discernable (Paragraph 8.43).

3333 That Areas should do more by way of carrying out objective comparisonsbetween Areas and units with different performance attainment levels.Consideration should be given by Headquarters to a more formal andorganised approach to benchmarking (Paragraph 8.17).

Information technology

4444 That Headquarters considers, with its IT partners, the need for additional‘mistake proofing’ in the CMS system by:

� revising the wording of outcome codes in CMS to avoid mis-codingdue to confusion over their meanings;

7



� reviewing the order of outcome codes in drop-down/pop-up menus toavoid mis-selection of similar codes; and

� the insertion of additional error messages/prompts to detect andhighlight illogical/unlikely entries (Paragraph 7.12).

5555 That Headquarters and Areas improve the understanding and effectivenessof use of the reporting capabilities of MIS and CMS, by a programme ofeducation and knowledge sharing (Paragraph 6.11).

6666 That Headquarters keeps under review the access of Area staff to MIS -particularly those charged with analysing performance - and increaseslicence availability as necessary (Paragraph 6.11).

7777 Headquarters considers extending the functionality of the CMS software toensure automatic flagging of racial incident and Direct Communication withVictims cases (Paragraph 6.11).

Training

8888 That Headquarters and Areas look at training in performance analysistechniques used in other types of organisation, including volumemanufacturing, and draw on this wider experience to establish trainingappropriate to CPS/criminal justice system staff (Paragraph 8.34).

9999 That the CPS develops structured training to improve the understanding andaccuracy of outcome codes. Where applicable this should include police staffwho input data into CPS systems (Paragraph 6.5).

Business systems and human resources

11110000 That CPS managers, in conjunction with police senior managers, should takestrong measures to improve the correct use of unique reference numbers inpre-charge advice cases (Paragraph 7.33).

11111111 That Headquarters and Areas should consider and document the nature ofperformance management duties to be undertaken by the various roles, anddefine detailed performance management competencies criteria, whichshould be clearly set out in published guidance (Paragraph 8.34).



33..33.. MMEETTHHOODDOOLLOOGGYYMMEETTHHOODDOOLLOOGGYY

TThhee ffooccuuss ooff tthhee rreevviieeww

3.1 This review has looked at the information gathering and performance managementsystems of different types of CPS Areas. We have identified two aspects of goodpractice and been able to compare and contrast the effectiveness of differentapproaches. However, the review did not seek to look at capture and usage for thefull range of data collected by the CPS. The emphasis is on service delivery andthe review has focused on data capture and information usage with reference tocaseload, case types and outcomes. Since much of the data is collected byCompass CMS and MIS, the review has also considered the use of these systemsas management tools.

OOuurr aapppprrooaacchh

3.2 The purpose of a thematic review is to paint a national picture about how the CPSdeals with a particular subject throughout England and Wales, based on evidencedrawn from a number of Areas and, where appropriate, CPS Headquarters.

3.3 For this review, we surveyed the views of CCPs, ABMs and Area PerformanceOfficers (APOs), Unit Heads and Unit Business Managers throughout England andWales, by means of questionnaires, as indicated in the following table.

9



QQuueessttiioonnnnaaiirreess ffoorrCCCCPPss//AABBMMss,, AAPPOOss,,

UUnniitt HHeeaaddss aanndd UUnniitt BBuussiinneessss MMaannaaggeerrss;;

aanndd oonn--ssiittee iinntteerrvviieewwss

QQuueessttiioonnnnaaiirreess ffoorr

CCCCPPss//AABBMMss,, AAPPOOss,,

UUnniitt HHeeaaddss

QQuueessttiioonnnnaaiirreess

ffoorr CCCCPPss//AABBMMss

aanndd AAPPOOss

Cheshire Dorset All other CPS Areas

Suffolk Gloucestershire

Sussex Lincolnshire

Thames Valley London North Sector

West Midlands North Wales

West Yorkshire Northumbria

South Yorkshire

Surrey

3.4 The response rate to the questionnaire was 83.4% (141 responses from 169questionnaires sent).

3.5 We visited six CPS Areas, choosing as far as possible ones that had not beensubject to many inspection visits in the recent past. The chosen Areas provided us with a mix of urban and rural environments, with a range of annual caseloads,thereby representing a cross-section of the entire CPS. During the site visits wespoke to managers who use and/or who are responsible for checking andcompiling performance information, as well as level A staff who undertake datainput.

3.6 Prior to the site visits we examined a selection of case outcome records drawnfrom Compass CMS, as shown in the following table. The selection of theseoutcomes was based on their impact on performance results, perceived out-of-lineperformance trends in the Areas at the time of the review, and our experience ofthe levels of data accuracy in Area inspections.



CCaassee oouuttccoommee WWhhaatt wwee wweerree llooookkiinngg ffoorr WWhhaatt iitt aaffffeeccttss

Dismissed no case toanswer in themagistrates’ courts

Accuracy of outcome code and reasonAnalysis of why acase failed

Judge directedacquittals

Accuracy of outcome code and reasonAnalysis of why acase failed

Convictions after trial Accuracy of outcome code

Contested caseoutcomes;Guilty plea rates;Budgetary allocation

Guilty pleas

Accuracy of outcome code in what isby far the biggest category of finalisation

As a secondary measure we also lookedat whether or not specified proceedings(cases which are prosecuted solely bythe police) are counted as CPS cases

Overall performanceprofile; Budgetaryallocation

Overall performanceprofile; Budgetaryallocation

Administrativefinalisation of casesafter the CPS hasadvised the police tocharge under the pre-charge advice(PCA) scheme.

Whether cases that have been thesubject of an advice to charge have infact been charged and, if so, whathappened to them

Outcome and costbenefits of the PCAscheme; Budgetaryallocation

TThhee IIssssuueess AAnnaallyyssiiss aanndd DDiinnnneerr PPaarrttyyTTMM aapppprrooaacchh

3.7 The review has benefited from the use of the Issues Analysis Dinner PartyTM

(IADPTM) technique. This is a tool developed by the National Audit Office for twopurposes: first, it aids the scoping of questions at the planning stage of a review/inspection. Secondly, it assists in report writing by identifying the issues that are ofinterest to the organisation which is the subject of the review. The process washelpful in identifying the key questions and lines of investigation. As part of theprocess, assistance was provided by four members of CPS Headquarters whoparticipated at various stages. The Chief Inspector is grateful for the assistancefrom the National Audit Office, whose staff facilitated the process, and the participationby CPS representatives.

EExxtteerrnnaall aassssiissttaannccee

3.8 In addition to the involvement of Headquarters in the planning and evaluationprocesses, Adam Parkin, Area Performance Officer at CPS Nottinghamshire,participated as a member of the inspection team and was involved in all stages of the review. Similarly, the team benefited from the inclusion of David Moxon,formerly the Head of the Home Office unit responsible for research and statisticson crime and justice. Again, the Chief Inspector is grateful for these valuablecontributions.

GGeenneerraall ppooiinnttss aabboouutt tthhee rreevviieeww

3.9 The Inspectorate report Performance Indicator Compliance and Case Outcomes(Thematic Report 3/00) was published in July 2000. Since then there have beensignificant changes in policy, with the introduction of charging and working practicesincluding extensive uptake of technology. The CPS also moved to more uniformperformance reporting arrangements from 1 April 2005.

3.10 This review does not examine the new performance measurement reporting systemdirectly. However, since some of the national performance measures rely on datacollected routinely by CMS, data capture and data integrity issues discussed in thisreview are of equal relevance to the new regime.

3.11 Paper files were not examined unless it was absolutely necessary. Instead, inspectorscarried out data integrity checks on electronic files in CMS, concluding that fileentries were correct if there was some corroboration supporting the entry.Inspectors concluded that a file entry, such as a finalisation code, was incorrect ifthere were contradictory entries in the same case. Examples included a judgedirected acquittal at a plea and direction hearing, or a no case to answer whenthere had been a discontinuance notice.

11





3.12 Inspectors also concluded that file entries are incorrect if events recorded on CMSare most unlikely and there are other indications that the entries are incorrect.For example, when the case is recorded as a conviction after trial and yet therehas only been one hearing close to the data of charge and no indication of anynecessary witness warning or disclosure. We invited Areas to look at some of thecorresponding paper files for these cases and have not been asked to amend ourconclusions.

3.13 During this process, we noticed gaps in record keeping on CMS. Common examplesof lapses were:

� outcomes had not been recorded properly against each charge;

� sentences had not been fully recorded, if at all; and

� the use of “other” as a reason or outcome code, without specifying what itactually meant.

3.14 Paper files are currently the preferred working medium and staff should be able toresolve any queries from them. However, the CPS intends to switch to electronicfiles in due course and there needs to be a recognition that current standards ofelectronic file endorsements must improve to enable that to occur.

44..44.. SSTTRRUUCCTTUURREE OOFF FFIINNDDIINNGGSSSSTTRRUUCCTTUURREE OOFF FFIINNDDIINNGGSS

4.1 To assist in structuring the review the inspection team used the Issues Analysis and Dinner PartyTM technique. In doing so a ‘question tree’ was developed, witheach underlying ‘branch’ listing enquiry questions. This is included at Annex 1.

4.2 The findings of the report follows the structure of the question tree taking thereader through issues related to:

� identification of measures requiring tracking;

� use of trained resources for data input and information extraction;

� presentation of validated information; and

� incisive decision-making to improve performance.

4.3 Each chapter begins with the leading question of each branch and discusses the issues and implications as revealed by the inspection activity. Annex 2summarises the responses to questionnaires and Annex 3 contains a glossary ofterms used in this report.

13



55..55.. HHAAVVEE AARREEAASS IIDDEENNTTIIFFIIEEDD TTHHEE PPRRIINNCCIIPPAALL PPEERRFFOORRMMAANNCCEE MMEEAASSUURREESSHHAAVVEE AARREEAASS IIDDEENNTTIIFFIIEEDD TTHHEE PPRRIINNCCIIPPAALL PPEERRFFOORRMMAANNCCEE MMEEAASSUURREESS

TTHHAATT RREEQQUUIIRREE TTRRAACCKKIINNGG??TTHHAATT RREEQQUUIIRREE TTRRAACCKKIINNGG??

OOvveerrvviieeww

5.1 Overall, most of the Areas visited have data available to them on the most importantissues, but do not always have the information soon enough or use it to best effectin identifying problems, causes and solutions. The three Areas with well-establishedsystems tend to be more pro-active and identify problems more quickly.

5.2 The new national performance management reporting system has provided a freshfocus on performance measurement. The new measures have greater relevance tothose previously in place and are more outcome-focused. There is, nevertheless,limited agreement between staff as to the nature and purpose of tracking. We founda good practice in West Yorkshire which has drafted a data specification to promoteunderstanding of performance data.

5.3 The Areas visited have all improved their performance management systems tosome degree and continue to develop them. Greater understanding of the technologywill play an important part in ensuring that they are able to get the right informationin the most efficient way in the future.

AArreeaa aaggrreeeemmeenntt wwiitthh ppuurrppoossee ooff ppeerrffoorrmmaannccee mmeeaassuurreess

5.4 The CPS has recently implemented a national performance management reportingframework based around 15 key aspects of work. This has generally been wellreceived by Areas and there has been a significant change in how performancemanagement is viewed in the CPS. The overwhelming majority of senior managersnow accept it as an essential part of day-to-day business, rather than a datagathering exercise.

5.5 With the increased importance attached to effective performance managementcomes an acceptance that Areas must identify the key aspects of performance forwhich systematic measurement is required. Most of the sites visited were workingon aligning their local systems to the national arrangements, or had already doneso. This new system represents a major opportunity to bring a more focused andconsistent approach, particularly to those Areas who have traditionally struggledwith performance management processes. The recently published Manual ofGuidance should help staff understanding of the aims and requirements of thesystem.

15



AArreeaa ppeerrffoorrmmaannccee rreeppoorrttiinngg ssyysstteemmss

5.6 Of the six Areas visited, three had well-established performance reporting systems,two had systems that were evolving, and one was still developing its framework.

5.7 Most Areas have developed their own systems based on a process of monthlyreporting. While there is some consistency in a small number of measures,there is wide variance overall in the type and amount of data gathered andanalysed. In some cases, units within the same Area have had a differing focus,although they believed they were operating a common system. The volume ofperformance data collected in each Area varies tremendously and it does notnecessarily follow that a wide range of data leads to an effective performancemanagement system. It is more important that Areas look at the right data and useit well, to drive improvements where necessary.

DDaattaa rraannggee

5.8 Most Areas concentrate on casework outcomes and volumes, with lawyer deploymentbeing the most common non-casework measure. The table below indicates the keylocal measures identified as important by specific roles (ordered by the number oftimes they were identified in questionnaires).

5.9 There is a clear consensus as to the three most important/frequently used measures.Thereafter the roles seem to have a different focus, as shown in the following table,demonstrating the number of responses against selected specific measures.This is partly to be expected, as Areas will have different priorities based on existingperformance levels and to what degree they are involved in implementing nationalinitiatives.



CCCCPPss//AABBMMss UUnniitt HHeeaaddss PPeerrffoorrmmaannccee OOffffiicceerrss

Unsuccessful outcomes Unsuccessful outcomes Trial effectiveness

Trial effectiveness Trial effectiveness Unsuccessful outcomes

Receipts/finalisations Receipts/finalisations Receipts/finalisations

Charging benefits Deployment/workload Proceeds of Crime Act

Budget Charging Court sessions

5.10 Unit Heads tend to focus heavily on operational casework delivery measures,whereas CCPs and ABMs have stronger links to national and strategic issues.This may help explain the big difference in importance attached to some of thewider Government Public Sector Agreement (PSA) targets such as OffencesBrought to Justice (OBTJ) and Public Confidence.

5.11 Even when Areas have identified the same topic as an important measure, there canbe a significant difference as to the depth of information examined - for example thereason codes for unsuccessful outcomes are examined extensively in one Area.They are broken down by unit and specific types of cases, which has enabledmanagers to target discrete aspects of performance.

5.12 Some managers are concerned that the MIS functionality enables very wideranging data collection and that there is some risk of “measuring things becausewe can and not because we should”. Clearly a balanced approach needs to beachieved, with a mix of standard and ad hoc reporting aligned to the key objectivesand targets of the Area itself, Local Criminal Justice Board, and the CPS nationally.There is little point in expending significant effort measuring things week-in week-outunless there is some return on the investment. Some Areas are already developinga more risk-based approach, with better targeting of measures. Conversely, systemsmust be sufficiently robust that they identify quickly any performance that is out ofline with expectations.

5.13 While a significant number of questionnaires and interviewees identified lack oftime and volume of measures as inhibitors to effective performance management,there was not a great deal of consensus as to any unnecessary performance measures.The majority of the Areas visited carry out periodic reviews of the measures andcould identify those which had been dropped or added. Some Areas continue withthe old style corporate performance measures, whereas others have discontinuedthis activity believing it has little value. The use of the cross-agency J-Track systemwas singled out by many of the staff we interviewed as inefficient and unproductive,as it often required duplication of data entry already conducted by other agencies.

17



MMeeaassuurree UUnniitt HHeeaaddssPPeerrffoorrmmaannccee

OOffffiicceerrss

Charging benefits 27 8

CCCCPPss//AABBMMss

16

Offences Brought toJustice 24 14

Public Confidence 10 01

Sick absence 5 60

Budget 25 96

5.14 While there was a lot of comment as to whether the Casework Quality Assurance(CQA) scheme was an effective use of time, particularly in the early days, mostinterviewees believed that it had improved and could serve a useful purpose ifproperly targeted. A number of Areas have abandoned the TQ1 system as part ofCPS/police joint performance management (JPM) as it was considered to be anineffective process by many.

IInntteerr--aaggeennccyy ddaattaa sshhaarriinngg

5.15 Areas are making better use generally of data from other agencies. This may meanCPS staff reducing efforts in some measures and accepting data supplied byothers, or in some cases reconciling data held by different agencies to assure integrity.The most common examples are comparing data with police on Anti-Social BehaviourOrders (ASBOs) and cases involving confiscation under the Proceeds of Crime Act(POCA). In one Area the criminal justice system agencies rely heavily on CPS-produceddata.

5.16 While understanding the importance attached to ineffective trial rates, it is disappointingthat some Areas feel the need to monitor separately from the national system, asthey do not have confidence in court supplied data. This should not be necessary ifthe national system is implemented properly.

5.17 In Area inspections CPS managers have cited progress against the OffencesBrought to Justice target as a significant success. The findings of this inspectionsuggest that Areas have limited understanding of the underlying data, other thanhigh-level results as produced by the Local Criminal Justice Board. Some Areasuse unsuccessful outcomes as the most relevant proxy measure for gauging CPSimpact. Our analysis suggests there is little correlation between unsuccessfuloutcomes and OBTJ and, in reality, the CPS has limited influence over the successor otherwise against this target. However, the potential impact to the OBTJ targetof not progressing languishing pre-charge advice cases was not readily apparent tosome Areas.

OOwwnneerrsshhiipp ooff ppeerrffoorrmmaannccee mmeeaassuurreemmeenntt ssyysstteemmss

5.18 Overall, the majority of staff are aware of what they are expected to ‘measure’ orproduce data about. However, understanding of the performance managementsystem as a whole was significantly less widespread among most of the Areasvisited.

5.19 There was a very consistent approach and understanding of performancemanagement requirements in one of the Areas visited. Of medium size within theCPS, all staff seemed to be very aware of not only their own role, but also of thesystem as a whole, which has contributed to an embedded culture of continuousimprovement.



5.20 In the larger Areas, it has proved more difficult to get the same level of understandingof performance measures and management to the varying levels of staff, which isoften linked to more general challenges in communicating effectively. Too manylevel A - and to a lesser extent level B - staff were unaware of the purpose or theresults of the tracking they were undertaking. As the primary users of IT they maybe able to identify better ways of extracting data if they have greater understandingand involvement.

5.21 In order to ensure greater consistency of understanding, West Yorkshire was closeto finalising a data requirement specification that outlined what was to be measuredin the Area. In addition to the ‘what’, it also covers the who, when, why and how.The document is very useful and we consider this to be good practice.

19



66..66.. HHAAVVEE TTRRAAIINNEEDD RREESSOOUURRCCEESS BBEEEENN AALLLLOOCCAATTEEDD TTOO IINNPPUUTT AANNDD HHAAVVEE TTRRAAIINNEEDD RREESSOOUURRCCEESS BBEEEENN AALLLLOOCCAATTEEDD TTOO IINNPPUUTT AANNDD

CCOOLLLLAATTEE PPEERRFFOORRMMAANNCCEE IINNFFOORRMMAATTIIOONN??CCOOLLLLAATTEE PPEERRFFOORRMMAANNCCEE IINNFFOORRMMAATTIIOONN??

OOvveerrvviieeww

6.1 Generally, trained resources have been allocated to input and collate performanceinformation. However, the level and source of errors suggest that understanding ofCMS/MIS needs to be significantly improved, and there is a need to enable thosewho undertake analytical functions to have greater access to MIS.

DDaattaa iinnppuutt

6.2 Data input is mainly done by level A staff using CMS. Some Areas have staffdedicated to file tracking and finalisation, although the size of the operational unittends to determine how staff are deployed. There are advantages and disadvantages tousing dedicated staff: they tend to be more knowledgeable about their task and keepa closer eye on preventing or managing backlogs. On the other hand, finalisationrequires a level of experience and skill to ensure that the data input is accurate.If this key skill is based on one or two members of staff, Areas will need to ensurethat there is sufficient skilled cover should the dedicated staff become unavailable.Turnover of staff at level A has been traditionally high and Areas have found it difficultto maintain the levels of understanding of CMS among new joiners. Staff were oftennot fully aware of the impact of their entries on performance data or finance.

6.3 Administrative staff completing data entry often rely on hand written notes or fileendorsements to inform them of what is required. It is not uncommon for thenotation to be unclear, making the job of accurately inputting data more difficultthan necessary. The nature of data input errors are discussed further in the nextchapter. They fall into several broad categories. The first is an error of judgement,which occurs when the data inputter responsible misunderstands what should beinputted. A common example is the finalisation of a case which resulted in adismissal after full trial as “dismissed no case to answer”.

6.4 The second type of error is one where the judgement is correct but the wrong codehas been selected. This is usually caused by the selection of the wrong option in adrop-down or pop-up list. We discuss the desirability for CMS to have a higherlevel of validation in the next chapter, but we think that better trained staff will bemore able to identify errors. We say this because some errors that resulted inobviously inconsistent entries had not been picked up during checks by level B staff.

6.5 Some CPS units that are co-located rely on police staff to register, track andfinalise files. This can cause difficulties when police staff are faced with conflictingpriorities and the accuracy of CPS data and associated training are not necessarilyat the top of the agenda. This occurred even in Areas we found to have soundinformation systems. To their credit, it is their robust systems that enabled them toidentify inaccuracies in their data and the source of the errors, and they have takensteps to address them.

21



We recommend that the CPS develops structured training to improvethe understanding and accuracy of outcome codes. Where applicablethis should include police staff who input data into CPS systems.

DDaattaa ccoolllleeccttiioonn aanndd eexxttrraaccttiioonn

6.6 The frequency of reporting varies between Areas and, in a few, reporting at unitlevel occurs more frequently than at Area level. In response to our questionnaire,Area Performance Officers across the country indicated that their reports clearlyshow the relative performance of each operational unit within their own Area.All Areas we visited collate performance information based on organisational units.

6.7 Generally, although there have been some queries whether all the data that iscurrently collected in their Area serves a useful purpose, data collection does notseem to be an issue with staff. The majority of managers questioned indicated thatperformance management tasks take between one and two days per month at level D,with level Es devoting more time.

6.8 We are satisfied that all the Areas visited have gathering arrangements in place,although some are significantly more robust and effective than others. Some data,such as financial or HR information, is collected by dedicated IT systems, whileothers make use of spreadsheets.

6.9 Overall, Areas now have access to a far wider range of casework performance datathrough the introduction of CMS and MIS. This has presented a significant opportunityto the CPS to improve the effectiveness of its performance management regimes.Feedback via questionnaires and interviews indicates that many Areas have yet torealise the benefits that the systems offer. This is mainly attributed to insufficientawareness of, and training on, the capability of the technology, allied to restrictionson licenses. In the Unit Head questionnaires, lack of understanding of CMS/MIS wasthe most common reply to the question “What aspects of performance managementdo you have most difficulty with?” Areas with greater confidence in the use of theavailable technology were monitoring a wider range of performance measures and were doing so in a targeted and more efficient way.

6.10 There was some agreement among more experienced CMS/MIS users that thesystem could replace and improve on some existing manual measures - for examplein the management of racist incidents and the domestic violence snapshot measures.It is CPS policy for certain types of crime, including racist incidents and domesticviolence cases, to be marked to facilitate monitoring and ensure appropriate handling.This is done once the charge has been determined, and staff are also required toset the appropriate flag in CMS. The Inspectorate has found that responsibilitiesbetween lawyers and administrative staff for identification of cases to be monitoredand flagged is not always clear. With racial incidents and cases involving a victimthat come under the Direct Communication with Victims (DCV) scheme, it usuallyobvious from the nature of the charge that flagging is required, which could bedetected in most instances by CMS software and thus flagging could be achievedby automatic means. This would mitigate the impact of staff lapses and improvethe quality of reporting and action taking.



6.11 In all the Areas visited there was a desire amongst most staff interviewed to get thebest out of CMS and MIS. When allied to growing confidence in data integrity inrecent months, and a new phase of training for license holders, there is room forsome optimism that Areas will get greater benefits from the technology in 2005-06.However, it is unlikely that full advantage will be gained unless a more broad-basededucation of staff on IT capability and analytical skills is achieved.

We recommend that Headquarters and Areas improve the understandingand effectiveness of use of the reporting capabilities of MIS and CMS,by a programme of education and knowledge sharing.

We recommend that Headquarters keeps under review the access ofArea staff to MIS - particularly those charged with analysing performance -and increases licence availability as necessary.

We recommend that Headquarters considers extending the functionalityof the CMS software to ensure automatic flagging of racial incidentand Direct Communication with Victims cases.

DDaattaa ttrraannssffeerr

6.12 Historically, the CPS has used multiple front-ended systems with a variety of meansof feeding other reporting and management systems. In some cases manual recordingplayed a significant part, particularly for Crown Court cases. The situation hasimproved greatly with the implementation of CMS and MIS in all Areas.

6.13 While MIS is the primary system for extracting performance indicator data, at thepresent time it is not the system that is used to record official national data or toinform allocation of funds. Official data, including historical information over thepast ten+ years, is held in the CIS system.

6.14 There is an automated interface between the two systems with data downloadedfrom CMS into MIS overnight. This gives CPS staff access to much more up-to-dateinformation and enables a more dynamic approach to data validation. Area staff areable to correct any errors detected without impacting final data, provided all amendmentsare completed before the ‘freeze’ date (the 11th of the following month). Until recentlythere was a safety net which enabled late manual adjustments to be made, but thisfacility has recently been withdrawn. Whilst we agree with the philosophy of ‘rightfirst time’ embodied by this approach, there is some risk in that the level of accuracyin some Areas may not be adequate to underpin data quality into the future.

6.15 The outcome of having the two systems does mean that Areas need to exercisecaution when evaluating data. This is particularly true when looking at trends, as forsome Areas there may be a significant difference between the historical data recordedin each system (MIS data does not change when manual adjustments to CIS havebeen made). It would be quite easy for an Area to interpret better outcomes as animprovement in performance, when it may simply be attributable to more accuraterecording. This should be a comparatively short term issue if data accuracy improves.

23



TTrraaiinniinngg

6.16 In our questionnaire survey staff at all levels expressed satisfaction that their datais accurate and that they have received sufficient training on input and collation.This view is also held generally among staff we interviewed, although as inspectorsdiscussed their findings on data accuracy, there was some acknowledgementthat further training should be actively considered.

6.17 Training should also help to foster better appreciation of why data is being collectedand hence why accuracy is important. One of the Areas we visited had a strongculture of data accuracy and data use. A member of staff said: “Data accuracy wasdrummed into me the first day I walked into the office. I am interested in getting mydata right because I am proud of my unit’s performance.” This Area had the lowestdata error rates across the categories examined.

6.18 Training need not always be classroom type teaching, which is often costly.Indeed, all staff who have access to CMS should already have attended a three-dayCMS course. The emphasis of the CMS course is on the IT side, for example,to teach users how to navigate the system. Administrative staff are generally fairlyskilled technically. They know which CMS screens need to be filled in and how toget to those sections when tracking and finalising cases. Where they need furtherhelp is in the selection of the proper outcome code/reason when completing therelevant sections; in other words, a better understanding of the meaning of the variouscodes and their significance in terms of performance management. Presently, theassistance is unstructured and relies mainly on members of staff seeking assistancefrom those thought to be more experienced. There needs to be a more structurededucation regime using desk-top instructions, mentoring and error analysis.

6.19 Most CMS and MIS training has been outsourced to technical trainers. This centrallyprovided training would have benefited from a more integrated approach, wherebydelegates had access to tutors with technical knowledge and people who understandCPS business issues. While it was envisaged that this would be supported bylocally provided desk-side training and coaching, feedback suggests that the levelof support available has varied widely within Areas. Some staff with advanced MISskills have provided assistance to other Areas, which has proved very useful and isto be encouraged. As detailed in the previous section, there is a need to improveunderstanding of the IT systems’ reporting capabilities.

6.20 Further training should not be restricted to level A staff. The Areas we visited allhave varying levels of data integrity checks, but the fact that errors remainuncorrected suggests that the understanding of the meaning of outcome andreasons codes among level B staff can also be improved.



77..77.. DDOO AARREEAASS PPRREESSEENNTT IINNFFOORRMMAATTIIOONN TTHHAATT HHAASS BBEEEENN VVAALLIIDDAATTEEDD DDOO AARREEAASS PPRREESSEENNTT IINNFFOORRMMAATTIIOONN TTHHAATT HHAASS BBEEEENN VVAALLIIDDAATTEEDD

EEFFFFEECCTTIIVVEELLYY??EEFFFFEECCTTIIVVEELLYY??

OOvveerrvviieeww

7.1 Most Areas believe that they enjoy a high degree of data accuracy, but they are notable to give validated assurance. Interviewees estimated data accuracy ranging from80% - 95%. The majority of Areas visited had some form of data validation checks,although many of the systems were not particularly effective during 2004. We werepleased to note an improved approach in 2005 with more errors being prevented,or detected and corrected, before the freeze date. However some errors remainundetected despite checks and controls.

7.2 There is limited quality assurance. Most Areas rely on management checks and controlsas a form of assurance, but the effectiveness of these processes is assumed. Only oneof the Areas visited conducts a systematic data integrity audit which is documented.

7.3 Our own checks indicated high error rates in the adverse outcome categoriesinvestigated and significant over-recording of convictions after trials in the magistrates’courts, although it should be borne in mind that these categories account for lessthan 10% of the total CPS caseload. Guilty pleas are far and away the most commonoutcome and here we found a much higher degree of accurate recording.

AArreeaa ppeerrcceeppttiioonn ooff ddaattaa aaccccuurraaccyy

7.4 We asked managers and staff at various grades whether they believed that theirperformance figures and performance indicators were accurate:

25



SSttrroonnggllyy aaggrreeee TTeenndd ttoo aaggrreeee NNeeuuttrraall TTeenndd ttoo ddiissaaggrreeee SSttrroonnggllyy ddiissaaggrreeee

DDoo CCCCPPss//AABBMMss,, PPeerrffoorrmmaannccee OOffffiicceerrss aanndd UUnniitt HHeeaaddss bbeelliieevvee tthhaatt

tthheeiirr AArreeaa’’ss ppeerrffoorrmmaannccee ffiigguurreess aanndd ppeerrffoorrmmaannccee iinnddiiccaattoorrss aarree aaccccuurraattee??

18.2% (8) 77.3% (34) 0 4.5% (2) 0CCPs/ABMs

55.6% (20) 41.7% (15) 2.7% (1) 0 0POs

31.7% (13) 51.2% (21) 17.1% (7) 0 0UHs

33.9% (41) 57.9% (70) 6.6% (8) 1.6% (2) 0Total

DDoo UUnniitt HHeeaaddss aanndd UUnniitt BBuussiinneessss MMaannaaggeerrss bbeelliieevvee tthhaatt

tthheeiirr uunniittss’’ ppeerrffoorrmmaannccee ffiigguurreess aanndd ppeerrffoorrmmaannccee iinnddiiccaattoorrss aarree aaccccuurraattee??

41.5% (17) 46.3% (19) 9.8% (4) 2.4% (1) 0UHs

43.5% (10) 47.8% (11) 8.7% (2) 0 0UBMs

42.2% (27) 46.9% (30) 9.4% (6) 1.5% (1) 0Total

DDoo CCCCPPss//AABBMMss bbeelliieevvee tthhaatt nnaattiioonnaall ppeerrffoorrmmaannccee ffiigguurreess aanndd ppeerrffoorrmmaannccee iinnddiiccaattoorrss

aarree aaccccuurraattee aanndd vvaalluuaabbllee aass aa bbeenncchhmmaarrkk??

6.8% (3) 70.5% (31) 6.8% (3) 13.6% (6) 0CCPs/ABMs

7.5 The position can thus be summarised as: the majority of staff believe that theirperformance data is generally accurate. There is a tendency for Areas to believethat their data is more accurate than that of other Areas, and unit managers tend to have a better opinion of their own unit’s data compared to their view of dataaccuracy in the Area as a whole.

7.6 With the exception of Cheshire and a few staff in West Yorkshire, interviewees wereunsure as to the level of accuracy of their data. Estimates ranged from 80% up to95% and tended to be based on instinct rather than any systematic monitoring.

IInnssppeeccttoorrss’’ ffiinnddiinnggss oonn ddaattaa aaccccuurraaccyy

7.7 We looked at several data streams: dismissed no case to answer, judge directedacquittals, convictions after trial, and guilty pleas. The first two categories of casescan indicate critical review or preparation failures, while the latter relate to caseloadand case type and are therefore indirectly linked to budgetary allocation (thenotional earning for a summary trial can be almost four times that of a guilty plea).The selection was based on the potential impact of inaccuracy and took account ofthe fact that Areas indicated in questionnaires that unsuccessful outcomes are oneof the most important measures to the CPS. The resultant number of cases wastherefore small compared to the entire caseload of an Area, and should notnecessarily be taken as an indicator of overall inaccuracy. However, they are validqualitative indicators of issues regarding data accuracy, quality control and qualityassurance. In the case of conviction after trial, we selected files randomly in threeAreas and on a targeted basis in the other three, based on the level of trials recorded.Therefore the overall error rate should not be taken as representative of a nationalpicture, albeit there is undoubtedly a widespread problem in respect of such cases.

7.8 We comment later in the report on aspects of pre-charge advice finalisations.

7.9 This table summarises our findings on error rates.

** WWhhiillee wwee oonnllyy ffoouunndd oonnee ccaassee tthhaatt wwaass nnoott aa gguuiillttyy pplleeaa,, tthheerree wweerree ffiivvee ccaasseess tthhaatt aappppeeaarreedd ttoo bbee ssppeecciiffiieedd ooffffeenncceess aanndd sshhoouulldd nnoott tthheerreeffoorree hhaavvee bbeeeenn iinn tthhee AArreeaass’’ ffiigguurreess aatt aallll..



PPeerrcceennttaaggee eerrrroorr ((ccaasseess//ssaammppllee ssiizzee))

GGuuiillttyy pplleeaassCCoonnvviiccttiioonn aafftteerr ttrriiaallJJuuddggee ddiirreecctteedd aaccqq..NNoo ccaassee ttoo aannsswweerrAArreeaa

1/209.1 (1/11)16.7 (1/6)27.3 (6/22)A

0/2076.0 (19/25)16.7 (1/6)7.7 (1/13)B

0/1031.6 (18/57)18.2 (2/11)30.0 (3/10)C

0/1071.4 (25/35)70.0 (7/10) 83.3 (10/12)D

0/2040.0 (8/20)27.3 (3/11)32.0 (8/25)E

0/1015.9 (7/44)50.0 (7/14)32.4 (12/37)F

1.1* (1/90)40.6 (78/192)36.3 (21/58)33.6 (40/119)Total

7.10 The errors can be due to misreading of the court outcome as endorsed by the advocate,misunderstanding of the meaning of certain outcome codes, ignorance of countingrules or, indeed, simple input mistakes.

7.11 The problem with lack of clarity of file endorsements is exacerbated by the descriptionfor more than one code commencing with the word “dismissed” or “guilty”. We havealready mentioned that the level of understanding of outcomes codes among somestaff needs to be improved. Having several similarly worded options together cancreate further confusion. It might be better, for example, if the word “dismissed”was to be removed altogether. It can encourage staff to select the wrong code,simply because they pick on the word rather the reason for the dismissal.

7.12 As the contents of drop-down boxes are organised alphabetically, all codes beginningwith the same word, for example, “dismissed no case to answer” and “dismissedafter full trial” are next to one another. This can also cause wrong entries to bemade even when level A staff have made a correct decision on what finalisationcode should be used.

We recommend that Headquarters considers, with its IT partners,the need for additional ‘mistake proofing’ in the CMS system by:

� revising the wording of outcome codes in CMS to avoid mis-coding due to confusion over their meanings;

� reviewing the order of outcome codes in drop-down/pop-upmenus to avoid mis-selection of similar codes; and

� the insertion of additional error messages/prompts to detectand highlight illogical/unlikely entries.

DDiissmmiisssseedd -- nnoo ccaassee ttoo aannsswweerr

7.13 Dismissed no case to answer (NCTA) cases are ones that proceeded to trial butwere dismissed because magistrates ruled that the prosecution case, taken at itshighest, did not disclose evidence upon which they could properly or safely convict.In some cases, this was caused by key witnesses giving evidence that was notconsistent with their statements or not appearing at all. In other cases, it is that one or more of the essential elements are missing from the prosecution case.It can therefore be an indicator of a mis-application of the evidential test in theCode for Crown Prosecutors. The national NCTA rate in 2002 was 0.18% of the total caseload (1,722 out of 980,214 cases) and had been lower in previous years,but in 2004 this has risen to 0.39% (3,901 cases out of 1,008,202) a rise of nearly120% in absolute terms. The number of cases is still small, but the increaserequires explanation.

27



7.14 We found that 33.6% of the cases recorded as NCTAs were mis-classified.Inspectors were satisfied that some of them were not trials at all. In others, the outcomewas inconsistent with the history of the case, for example, a discontinuance noticeand DCV letter had been issued for the only charge. Reason codes quoted wereoften dubious, particularly when the reason given was “conflict of evidence”,when only one witness gave evidence.

7.15 We consider that an improvement in the clarity of file endorsements would reducethe need for administrative staff to guess what had happened at trial. Some lawyerstend to describe NCTA as discharged, dismissed or dismissed half time. Theirassumption or expectation that level A staff will know what finalisation code to usefrom these descriptions, or from an analysis of what other endorsements they putdown on the file, is unrealistic.

JJuuddggee ddiirreecctteedd aaccqquuiittttaallss

7.16 Judge directed acquittal (JDA) cases are trials in the Crown Court where the judgehas ruled after commencement of the trial that the prosecution has not adducedsufficient evidence to enable a properly directed jury safely to convict.The judge therefore directs the jury to acquit without the defence having to giveevidence. The reasons that give rise to a JDA are therefore similar to those thatresult in an NCTA.

7.17 We found an error rate that is similar to that for NCTAs. Here again, we found casesfinalised as JDA when the evidence was that no trial had taken place. In one of theAreas which had a low error rate, finalisation is undertaken by level B caseworkersimmediately after the conclusion of the case, as they have access to CMS at theirCrown Court. This supports our finding that the quality of file endorsement and theunderstanding of what the endorsements mean are factors in JDA mis-recording.

CCoonnvviiccttiioonnss aafftteerr ttrriiaall

7.18 When a case is finalised as a conviction after trial, it indicates that a trial has taken place and that the Area has progressed the case to a successful conclusion.The number of trials is a factor in determining the budget for an Area because trials attract significantly higher notional earnings than non-contested outcomes.

7.19 This category was of interest as inspectors had noted fluctuation over time innational performance. For example, in the calendar year 2002 there were 39,516cases nationally, which in 2004 rose to 59,630 - an increase of over 50%.This occurred during a period where the total national caseload of completed cases increased by just 2.8%. In the Areas visited we found there was significantover-counting of convictions after trial in our sample, the most common reasonbeing the counting of guilty pleas as guilty verdicts. The generic problems ofendorsements and drop-down boxes apply to these cases, but there is anadditional factor surrounding late guilty pleas - i.e. cases that have been preparedas a contested basis, but do not result in a trial.



7.20 Guidance from CPS Headquarters is clear: these cases should be classified as guilty pleas. We found, however, that not all relevant staff are aware of,or understand, this guidance. Some staff are aware that this indicator relates toworkload, and therefore count late guilty pleas as guilty verdicts, to reflect the workthat has been done in preparation for the trial. This erroneous practice needs to bestopped.

7.21 We believe the extent of these mis-recordings warrant further remedial action byAreas.

7.22 As guilty pleas are by far the biggest category of case outcome, they have a very significant impact on determining the overall performance results of an Area.Our examination of the CMS file sample demonstrated that error rates werevirtually zero, albeit using a very small sample in relation to total cases. One ofthe Areas conducted a comprehensive test using one month’s data covering two units and found an error rate which was approximately 5%.

GGuuiillttyy pplleeaa -- ssppeecciiffiieedd pprroocceeeeddiinnggss

7.23 Specified proceedings can be used for certain (mainly minor road traffic) offenceswhen a guilty plea is acknowledged by the defendant. With a few exceptions (e.g. careless driving) police prosecute these cases unless, and until, the CPStakes them over after not guilty pleas are entered. The inclusion of a significantnumber of inappropriate specified proceedings distorts Area performance resultsand inflates caseload, which will lead to a larger budget allocation than is justified.

7.24 Historically, inclusion of specified offences in CPS figures has been a problem andHeadquarters have required Areas to ensure these cases do not enter CPS figures.Two of the Areas visited had already been identified during our normal inspectionprogramme as having inappropriately included specified cases. We were satisfiedthat both had made good progress in reducing the number of inappropriatefinalisations.

7.25 However, our CMS file sample of guilty pleas indicated that a few specified caseshad been incorrectly included in PIs. Our on-site interviews also highlighted thatthere is still confusion among staff as to what can be legitimately ‘counted’ andAreas need to clarify understanding amongst staff.

7.26 One Area had recently made great effort in excluding specified offences -particularly those withdrawn because of the production of driving documents - and eliminating these cases has reduced the Area’s apparent unsuccessfuloutcome rate. In two of the other Areas visited we still found sizable numbers ofthis type of specified offences being included in their caseload.

29



AAddmmiinniissttrraattiivvee ffiinnaalliissaattiioonn

7.27 Pre-charge advice cases are recorded on CMS. We looked at the number of casesthat have been administratively finalised after the CPS has advised police to charge.When the advice is to charge a defendant, CMS effectively leaves the electronicadvice file open, so that a prosecution file can be added. Administrative finalisationoccurs when the latter has not been added after six months, which CMS doesautomatically without the need for user input.

7.28 We found a significant number of these in five of the six Areas visited. Severalexplanations can account for this. First, the prospective defendant might havefailed to surrender to bail and had not been re-arrested. Secondly, the police mayhave decided not to charge for whatever reason and, thirdly, the prosecution couldhave commenced but not been linked to the advice file on CMS.

7.29 The Areas are not able to quantify or even estimate how many cases fall into each of the three categories. The most probable explanation, according to them,is the failure to link the prosecution file to the advice file, while our researchshowed examples of all three reasons.

7.30 National police and CPS guidance recommends that advice files should beidentified by a unique reference number (URN) that should be used by CMS totrack the case from advice to disposal after charge. None of the six Areas wereable to do so consistently, with police IT systems being an issue in some of them.While it is possible to do manual and system driven checks, the process ofmatching charge files to the original pre-charge advice can be time consuming and inefficient. Cases where advice is to charge do not necessarily show on themain CMS-driven report covering ongoing advice cases, unless a future action datehas been inputted. Again this makes managing cases more difficult.

7.31 The fact that the failure to link files contributes significantly to administrativefinalisation is to some extent supported by our finding in one of the six Areas,where great efforts have been made - including the use of police case trackingsystem by CPS staff - to link advice files to prosecution files. Administrativefinalisation in that Area was very small indeed, and cannot simply be explained bythe fact that the Area was comparatively new to pre-charge advice. Our concernswith administrative finalisations were:

� the inability of Areas to match pre-charge files with charge files;

� the potential impact on budget, due to some double counting; and

� the risk of cases simply not being charged through unmanaged drift.

7.32 We are advised that system changes have been made for the next release of CMSthat will remove the automatic administrative finalisation function. While this willhelp by putting greater onus on Areas to manage such cases, it does not overcomeall the process inefficiencies that currently exist.



7.33 Recent central guidance on setting up pre-charge advice schemes is that Areasshould use URNs. The fact remains that none of the six Areas we visited were ableto do so, which suggests that the problem should be tackled on a national basis, inaddition to local negotiations.

We recommend that CPS managers, in conjunction with police seniormanagers, should take strong measures to improve the correct useof unique reference numbers in pre-charge advice cases.

7.34 CPS Areas tend to see the management of ongoing pre-charge cases as a policefunction, whereas the national Charging Programme Office clearly sees this as ajoint responsibility. There is scope for greater co-operation in the management ofpre-charge advice cases, which will become easier if staff are more familiar withthe range of reports available from CMS and MIS.

QQuuaalliittyy ccoonnttrrooll

7.35 All Areas pointed to some management check on data accuracy. In some, all entriesof certain adverse finalisation codes such as “dismissed no case to answer” and“judge directed acquittals” are checked. In others these codes cannot be enteredwithout prior approval. The most stringent system we came across was the policyin one Area where level A staff are not permitted to decide on the finalisation codeat all. Advocates are expected to endorse clearly on the file what the code shouldbe and, failing that, the files must be sent to managers for a decision.

7.36 Managers are involved in information validation in other ways. The most commonexample is that, in all six Areas, adverse outcomes trigger a report to a Unit Heador Team Leader. In one of these Areas this was extended to all unsuccessfuloutcomes. The majority of Areas indicated that this is an opportunity for managersto check that the finalisation codes are correct, but the fact that errors still appearcalls into question the effectiveness of these checks. Managers conducting thesechecks tend to concentrate on whether the legal decisions were of good quality,but somewhere within this process there is a need to reconcile the finalisationcodes recorded on the file and within CMS.

7.37 Of greater concern is that unit/team managers seemed not to be aware that thenumber of NCTAs and JDAs were high enough to require some form of verification.Our experience in Area inspections indicates that some unit managers only querythese numbers when asked to produce the files for inspection purposes. We notedthat one of the Unit Heads we saw expressed concern at a performance meetingthat the number of NCTAs (which was high in relation to the caseload) did not tallywith her lawyers’ perception of their performance, but nobody was able to accountfor the difference. Having examined on CMS a sample of NCTAs from this unit,we were satisfied that the NCTA code has been vastly over-used to record casesdismissed after full trial as well.

31



7.38 We have recommended that staff at all grades should receive further training ondata accuracy, which should improve the accuracy of the quality control checks too.Training on analytical skills, as discussed later in Chapter 8, will also assist inidentifying potential data faults.

7.39 It is important to record results of data checks so that managers can have someidea of how staff are performing in this regard and to track improvements made.We were able to find evidence that the results of data checks are documented andanalysed in only two of the Areas we visited - Cheshire and West Yorkshire.

7.40 We have found some manual adjustments, but they are not significant in proportionto case volume. They tend to reflect the vigour with which an Area checks for dataintegrity.

QQuuaalliittyy aassssuurraannccee

7.41 One of the six Areas, Cheshire, undertakes a periodic data integrity audit to assureitself of the accuracy of performance information. There is a quarterly audit whichis fully documented and involves a 50 file sample from each unit being thoroughlyscrutinised by a manager from another unit for a wide range of aspects, includingcase finalisation code. The total effort for the audit requires three B1 staff days perquarter. The size of the sample enables the Area to conclude that data accuracy ofits case load figures is of the order of 95% and improving. Our examination of theirCMS records supports this finding. We consider documented, periodic data auditsare good practice.

7.42 Data does matter to an organisation and it is worth investing to assure its integrity,as data in itself is an expensive commodity. It is wasteful in resource terms not to usegood data to drive up performance, but it is worse to use defective data to drive decisions.

7.43 Data reflects past performance and so has a major impact on future decision-making.Poor quality data may therefore lead to poor quality decisions and data integrity iscrucial to performance management. At present, data quality is checked with varyingdegrees of rigor. Audits are essential, and the challenge is to ensure that these aredone in rigorous, consistent, but also cost-effective, ways. Using ‘external’ staffcan add significantly to costs and the one Area is trialing an approach which usesthe unit’s own staff.

7.44 Inspectors estimate, based on CPS Headquarters’ financial information, that thecost of employing a B1 staff officer amounts to £230 per day. Therefore the overallcost of the audit function in Cheshire is £2,760 per annum representing 0.07% oftheir £4.2m budget. We believe that the objectivity and level of quantifiedassurance provided by the ‘external’ audit represents good value for money.

We recommend that Headquarters and Areas need to establish dataintegrity regimes which provide for an appropriate level of accuracyand an accompanying confidence factor.



8888 CCAANN AARREEAASS DDEEMMOONNSSTTRRAATTEE IINNCCIISSIIVVEE DDEECCIISSIIOONN--MMAAKKIINNGG TTOO CCAANN AARREEAASS DDEEMMOONNSSTTRRAATTEE IINNCCIISSIIVVEE DDEECCIISSIIOONN--MMAAKKIINNGG TTOO

IIMMPPRROOVVEE PPEERRFFOORRMMAANNCCEE??IIMMPPRROOVVEE PPEERRFFOORRMMAANNCCEE??

OOvveerrvviieeww

8.1 New national performance measures have raised the profile of performance dataand performance management in Areas. Managers feel more accountable and aretherefore keen to make more use of the management tools provided by CMS/MIS.

8.2 Most managers are comfortable with using performance information to deal withtheir own unit performance, but they are less keen to address individual performancein the same way, and there is a distinct reluctance to view this as an integral part ofa supervisory line management function.

8.3 Benchmarking or comparative evaluation between individual units within an Area,and that Area with others, is not advanced. There is a cautious approach tocirculation and comparison of performance information on the basis that it may beseen as ‘naming and shaming’. Some reluctance was also based on reservationsabout data accuracy.

8.4 While all Areas have a performance reporting regime, and there is some evidenceof a relationship between a mature performance management system and deliveryof results, the link between performance management and improvement action andtheir outcomes are not always clearly evidenced. There is evidence that caseloaddata is being used to align resources in most Areas, although finesse of themethods used varies considerably. Some managers can also point to actions beingtaken in response to performance information. In many cases the benefits arequalitative, but we have seen some impressive quantified cross-agencyimprovement measures.

8.5 Managers tend to focus on and react to under-performance and benefits fromanalysing trends, or a more in-depth examination of some data streams, can bemissed.

8.6 We found considerable differences in the extent to which staff at all levelsunderstood performance management techniques. This applied to the wholeprocess of data extraction, analysis and using data to drive decision-making.There is a clear need for more effective training and guidance to ensure high and consistent standards throughout the CPS. Clarity is required to define staffcompetencies for those engaged in duties associated with performance management.

33



PPeerrffoorrmmaannccee mmaannaaggeemmeenntt aatt ddiiffffeerreenntt ttiieerrss ooff mmaannaaggeemmeenntt

8.7 Each of the Areas we visited operated different performance management systemsin terms of performance responsibilities, comprehensiveness of the reporting,degree of integrity checking, and the proportion of analysis performed by thedifferent grades. The maturity of systems also varied considerably; some were verywell established and, in one case, followed principles and practices set down atleast six years ago. In this particular Area, it was clear to inspectors that a strongperformance culture existed, while in the others, the development of the performancemanagement system was recent and in one case yet to be established. In thislatter Area, systems for joint performance management with other agencies werevery well established both in respect of the form of reporting and analysis, but themanagement team were still in the process of defining the requirements for internalperformance management and adjusting the organisational structure.

8.8 In the Areas where the performance management function was not developed,Unit Business Managers did not undertake full performance analysis duties,despite these being in their job descriptions. They undertook some analysis ofnon-casework measures, such as budget and sickness, but passed on caseworkinformation to Unit Heads for analysis. However, Unit Heads felt that they did nothave sufficient time to devote to the activity. In some cases level E managers aretranslating data into tables, graphs etc., whereas we consider that this work couldbe done by less senior staff. We see no reasons why Unit Business Managerscannot go beyond the production of data tables and be involved in trend analysis of casework levels. Unit Heads can then focus on what lies underneath the trends.

8.9 Generally, there were checks and balances at the administrative level for performanceindicators. When checks showed erroneous inputting, the more developed Areastended to require staff who made the errors to correct them, to facilitate the learningprocess. We were given a number of examples of aspects of performance that hadbeen improved through this process, including improved flagging of relevant cases,and proper use of reason codes following a purge on the over-use of the “other”category.

8.10 Unit Heads responded to our survey proposition “I am accountable for my unitsperformance” with 100% of respondents agreeing, whereas, in the CCPs/ABMsquestionnaire, the response to the proposition “My managers are held to accountfor achieving targets” showed 86% of respondents agreed. Evidence of activeperformance management from Performance Officers was also not as clear-cut.Their questionnaire showed a mixed picture, with Officers responding to theproposition “The performance management processes in each unit are equallyeffective” showing that 33% were either neutral or tended to disagree.



8.11 In one large Area senior management had become aware of a complex performanceissue concerning unsuccessful case analysis, with the Area having a persistently highunsuccessful outcome rate in both the magistrates’ and Crown Courts. Unit Headsoriginally undertook failed case analysis, then later the task was given to otherlawyers in an attempt to facilitate learning from mistakes. The Area found thatneither approach revealed the true underlying reasons for case failure. We weretold that there had been reluctance by Unit Heads and peer evaluators to criticisethe work of fellow lawyers, although a special Area investigation showed that therewas poor casework. While some - mainly less experienced - lawyers were keen tohave feedback, others were concerned that this type of professional scrutinyformed part of a line management function.

8.12 This is a cultural issue that managers must grapple with. The point of the processis to identify the reasons for unsuccessful outcomes so that they can be addressed.If this turns out to be review weaknesses, then managers and peers should feelfree to point them out so that everybody can learn from them. Equally, lawyers mustbe prepared to accept justified criticism of decision-making and not be hideboundby their view of professional independence.

BBeenncchhmmaarrkkiinngg wwiitthh ootthheerr AArreeaass//uunniittss ttoo iiddeennttiiffyy bbeetttteerr pprraaccttiicceess

8.13 Generally, Areas do not undertake any formal benchmarking with each other tocompare practices and learn of opportunities for improvements. Most tended toconsult national tables compiled by Internal Resources and Performance Branchor information in HMCPSI reports, but this is not substantive benchmarking.Some Areas occasionally undertook fact-finding visits associated with setting upnew initiatives, for example, one Area visited CPS Surrey to look at how they hadset up a co-located Trial Unit service, and senior managers from another visited a different jurisdiction to see how their public prosecution system functioned.There were also examples of Areas using acknowledged experts in certain fieldsfrom other CPS units.

8.14 In only one of the six Areas visited did the Area Business Manager monitor otherArea’s Headquarters’ performance reports and request relevant Action Plans wherethey were of interest.

8.15 At interview some senior managers indicated that, although they use Headquartersdata for comparisons purposes they had reservations over data accuracy, and thistended to be confirmed in our survey. CCPs and ABMs responded to the proposition“I believe that national performance and PIs are accurate and valuable as abenchmark” with 77% agreement. We were rather disappointed that nearly aquarter of senior staff did not have confidence in centrally produced information.This information has been improving, but it is clear that it still does not commandthe full confidence of CCPs and ABMs. This may be largely because of pastshortcomings and more a problem of perception than current reality. The informationis essential for benchmarking and it is important to ensure not only that the data is

35



of high quality, but that it is seen to be so. This will ensure that it can be routinelyand confidently used for benchmarking and therefore play a full part in performancemanagement. This situation should improve once the new CPS performancereporting system becomes effective across the Service.

8.16 A similar but more marked situation existed at Unit Head level as they tended notto benchmark extensively and only 44% agreed with the proposition “We activelybenchmark our performance with other similar CPS units”. At interview some UnitHeads indicated that comparisons were undertaken, but only on discreet topicssuch as the time taken to deal with persistent young offenders from arrest tosentence.

8.17 Benchmarking is an improvement method whereby an organisation compares its own results and processes with other organisations acknowledged as operating best practices. As CPS Headquarters implements the new performancereporting framework it will be easier for Areas to identify benchmarking partners.For benchmarking to be worthwhile staff need a better understanding of itsmethods. There are many examples in industry and commerce of sector-specificbenchmarking clubs facilitated by academia and we believe that the introduction ofthe CPS performance framework has the possibility to spin off a successful CPSbenchmarking initiative.

We recommend that Areas should do more by way of carrying outobjective comparisons between Areas and units with differentperformance attainment levels. Consideration should be given byHeadquarters to a more formal and organised approach to benchmarking.

8.18 The Areas with more developed performance management systems were veryfirmly committed to the concept of performance measurement and its transparency.Unitised information sets were reported on as a matter of routine, which enabledindividual units to be compared and their contribution to overall Area performanceassessed. These comparisons were recorded in performance report packs for seniormanagement teams and discussed at unit performance meetings. Several Areasalso had specific Area-wide performance forums where staff undertaking similarwork had periodic meetings to exchange knowledge.

8.19 In other Areas, the practices were much less precise, with Unit Heads becomingaware of their relative contribution to Area performance through discussion atsenior management meetings or the Red, Amber or Green “traffic light” systemallocated in reports for unit performance against overall Area outturns. In manyAreas, the degree to which relative unit performance information was available tostaff was unclear. For example, in some cases information had recently beenmade available through a shared computer drive, but often staff were not aware ofthis. There were also many instances where staff at levels B1 and below did notknow of their unit’s performance against similar Area units and we believe theabsence of this direct feedback diminishes opportunities for group learning and theencouragement of self-motivated teams.



8.20 A similar pattern was found in the CCPs/ABMs and Unit Heads questionnaireresponses. To the propositions “We understand and record the relativeperformance of each unit against overall Area performance” 98% of CCPs/ABMsstrongly agreed or tended to agree, whereas to the proposition “Team and Areaperformance is communicated regularly throughout the Area”, the agreement ratereduced to 82%. For a similar proposition to Unit Heads: “My team is aware of ourunit’s performance in relation to other Area units” the agreement rate was 66%.

8.21 There were also some very diverse opinions on comparative evaluation amongstsenior managers in different Areas. For example, one was against publishingrelative unit performance as he felt it amounted to a name and shame culturewhereas what he wanted was a culture which encouraged staff. In another Area,a senior manager had started to publish relative performance information on onetopic because he was keen to improve the Area’s performance and wanted to instilla sense of competition amongst staff to motivate them and assist the learningprocess. This latter manger was stimulated to take this action having seen theArea’s placing in national league tables.

8.22 We do not believe that transparency of relative performance information is namingand shaming provided it is undertaken in a constructive way. Having this informationin a convenient and digestible form completes the feedback loop to staff and bringsa sharp focus to the results of their endeavours, and the benefits outweigh thedisadvantages.

AAlliiggnniinngg rreessoouurrcceess bbaasseedd oonn ppeerrffoorrmmaannccee iinnffoorrmmaattiioonn

8.23 All Areas were able to give examples of actions taken to check the staff balances inunits against caseload. Several used an Activity Based Costing (ABC) approachwith assistance from Headquarters. Our findings in relation to the accuracy of dataindicates that the use of ABC figures may not be overly reliable in some Areas,where high error rates have distorted the model’s outputs. Resource reviewstended to be undertaken every six months, or when a major restructuring exercisewas taking place, in which case the work formed part of a wider programme.

8.24 Re-balancing lawyer resources did not appear to be an issue in most Areas andexamples were given of their re-deployment between functional units. However,some Areas reported that it was more problematic to balance administrative staff,as these grades tended to be non-mobile. In one Area where staff mobility hadbeen a problem the workflow was re-directed.

8.25 Not all apparent imbalances were attributed to work volume ratios, as we foundthat business managers in one Area were undertaking a review between twolocations, because the apparent imbalance was believed to be indicative ofinefficient processes.

37



MMaakkiinngg cchhaannggeess ttoo iimmpprroovvee ppeerrffoorrmmaannccee

8.26 Unit Heads responded to our survey proposition “I regularly seek performanceimprovement in my unit” with 98% agreement and provided 113 examples ofimprovement actions. These could be categorised into actions for: productivitygains, process improvement to improve work quality, and improved outcomeperformance. Of the 113 examples, only ten had stated quantified benefits,two actions were not successful and the remainder were deemed to be successfulwithout quantification.

8.27 During interviews, all staff felt they were empowered to make changes within theirown ambit of responsibility or to initiate discussion if the change impinged on widerpolicies. Examples of such actions included changing Casework Quality Assuranceanalysis to a thematic approach to make it more meaningful, making Graduated FeeScheme payment and processing the responsibility of one person - thus eliminatingbacklogs and CPS-originated delays - and introducing a system for tackling driftingcases, which was very successful and then adopted as an Area-wide system.

8.28 Where quantified improvement was stated the gains were impressive and tendedto be associated with cross-agency issues. Examples included reducing CrownCourt ineffective trials from 32% to 15.9% over a 12-month period, achieved by the Unit Head undertaking trial readiness checks, fortnightly listing meetings withthe Crown Court Liaison Officer and having monthly one-to-one meetings with theResident Judge to establish a common view to focussed improvement actions.We believe these are good examples of focused action to improve outcome results.

8.29 Similarly, examples of improvement actions cited by CCPs and ABMs in theirquestionnaire returns showed that cross-cutting measures represented the largest listof improvement actions, with actions on cracked and ineffective trails being themost prominent.

UUnnddeerrssttaannddiinngg//sskkiillllss ooff nnuummeerriiccaall aannaallyyssiiss ffoorr ppeerrffoorrmmaannccee mmaannaaggeemmeenntt

8.30 In our questionnaires to staff, all groups were asked to comment on the sufficiencyof understanding/numerical analysis skills amongst staff with performance managementduties. Generally, the results show that CCPs/ABMs, at 53%, and Unit BusinessManagers at 46%, were markedly more pessimistic that staff have the necessaryskills to be effective. The figures for Unit Heads and Performance Officers were83% and 75% respectively.



8.31 Inspectors believe that Area staff generally tended to overestimate their skills inrelation to the actual job required. There were numerous examples where staffthought they were undertaking analysis, but were essentially tabulating figures withno supporting analysis. Moreover, in those Areas where the performance managementsystem was not fully developed, the performance analysis duties had not been setout.

8.32 Some senior staff had indicated agreement on the questionnaire - but to a differentquestion - stating that better understanding of analysis would assist in improving Areaperformance. At interview, it was explained that staff were able to do the arithmeticnecessary to perform analysis, but were less able to draw the right conclusions fromthe information. When asked if training was available to develop deeper analytical anddecision-making skills, staff at all levels were very keen for this to be provided if thetechniques taught were set in a CPS/criminal justice system context. One intervieweesaid “Essentially we are all self-taught, but we do not actually know what is expectedof us”.

8.33 In other fields of endeavour - volume manufacturing in particular - there have beensignificant advances in productivity and product quality improvement by the use of recognised analytical techniques. Many of these gains have requiredstaff at all levels to gain broader skills in methods associated with qualitative and quantitative analysis, cause and effect determination, and trend presentation, to enable appropriate decision-making. It has been found that techniques developedin manufacturing and service industries can be adapted to the professional servicessector, provided suitable contextual examples are used. It is interesting to notethat many of the CPS Headquarters’ initiatives embody principles coined elsewhere.For example, the charging initiative builds on the quality cost principle thatinvestments in prevention measures can reduce product failures, which can be veryexpensive in resource terms.

8.34 These concepts can be applied to every day work and help to develop a performanceculture if widely understood by staff.

RReelleevvaanntt ssttaaffff ((oorr II)) hhaavvee ssuuffffiicciieenntt sskkiillllss ttoo bbee eeffffeeccttiivvee iinn aannaallyyssiinngg ppeerrffoorrmmaannccee

SSttrroonnggllyy

aaggrreeee

TTeenndd ttoo

aaggrreeeeNNeeuuttrraall

TTeenndd ttoo

ddiissaaggrreeee

SSttrroonnggllyy

ddiissaaggrreeeeGGrraaddee

17 10 8 2 1ABMs

39



NNoo

aannsswweerr

6

26 5 2 0 0UHs 8

7.5 10 3.5 0 0UBMs 4

18 7 2 0 0POs 9

We recommend that Headquarters and Areas should consider anddocument the nature of performance management duties to beundertaken by the various roles, and define detailed performancemanagement competencies criteria, which should be clearly setout in published guidance.

We recommend that Headquarters and Areas look at training inperformance analysis techniques used in other types of organisation,including volume manufacturing, and draw on this wider experienceto establish training appropriate to CPS/criminal justice systemstaff.

UUssee ooff ppeerrffoorrmmaannccee iinnffoorrmmaattiioonn ttoo oobbttaaiinn vvaalluuee ffoorr mmoonneeyy

8.35 There is little evidence to suggest that performance management information isbeing used to support value for money aspects in decision-making, albeit it isaccepted that some decisions will result in a positive financial impact for theArea/CPS, e.g. increasing Higher Court Advocate (HCA) activity to access furtherfunding. Most changes are designed to improve Area compliance to numericaltargets or objectives. In some cases, the targets can be achieved in ways that donot necessarily represent a value for money approach. For example, an Area canhave a better than average ineffective trial rate and yet still have an inefficientprocess by not fixing a trial until multiple hearings or pre-trial reviews have takenplace.

8.36 It is rare to find that there is any clear indication of the expected cost or, indeed,level of benefit/quality premium of the more significant decisions taken. It is recognisedthat this is not necessary for many of the smaller changes made, and that someactivity is predicted on national policy and strategy.

8.37 Some Business Plans link activities to the overarching Public Service Agreementobjective of “Improving Value for Money by 3%”, however they are often silent inhow and to what degree the activities will contribute.

8.38 Areas cite increased deployment of designated caseworkers (DCWs) as a key part of their value for money approach, particularly in light of their extendedpowers. While there are clear indicators that the ratio of courts covered by DCWshas increased (two of the six Areas visited achieved approximately 16% coveragein 2004), it is not always clear how this translates into better value for money.Most Areas make an assumption that freeing up lawyer time will automatically lead to better value for money. Our experience suggests that this is not so,and that only Areas who actively manage the time created derive the anticipatedbenefits.



IInncciissiivvee ddeecciissiioonn--mmaakkiinngg ttoo iimmpprroovvee ppeerrffoorrmmaannccee

8.39 Incisive decision-making requires an organisation to have established performancemanagement functions and a discriminating reporting system allowing the presentationof accurate information that has been analysed to show performance trendsagainst expected goals. Each arm of the performance management system mustbe functioning effectively to enable timely decision-making based on well-foundedconclusions. From our survey results, the CCPs and ABMs gave 45 reasons aboutthe biggest barriers to effective performance management, of which the largestvolume of responses related to lack of time/too many initiatives and insufficient staff skills at all levels (which comprised a third each of all reasons given).To enable informed decision-making to improve performance we found that Areasneed to have a better focus on performance management, be more able in trendanalysis and have a more complete understanding of how to determine cause andeffect.

FFooccuuss ooff ppeerrffoorrmmaannccee mmaannaaggeemmeenntt

8.40 Very few Areas had complete performance management systems and, often,the senior management team preferred to manage by exception, which essentiallymeant they were dealing with adverse situations where problems were alreadyapparent. Thus the management team were then reacting to adverse situationsrather than managing performance. For example, a senior manager said, “Two yearsago our persistent young offender figures were on target, but both the police andourselves lost the grip for different reasons; the police had many staff changes andthe CPS was pre-occupied by charging”. The Area has failed to achieve the nationaltarget for 13 months on a three-month moving average basis and it is not expectedto improve for at least another six months. In another Area we were told that they “Were concerned at the high sickness levels and have introduced more rigidsystems for recording and monitoring and now have a more interventionist approach.”

8.41 All Areas visited were in the process of adapting their systems to mirror, at unitlevel, the new CPS performance reporting arrangement for Areas. We believe thiswill assist Areas in achieving the performance management focus they require.

TTrreennddss aannaallyyssiiss

8.42 Some performance reports were clear in presentation, showing last year’s outturn,current year’s quarterly performance and the year-to-date performance. This typeof presentation does permit an experienced team to deduce top-level cause andeffect issues, but trends cannot necessarily be confidently established if the data isvolatile and erratic. To detect trends more readily, particularly in three-monthmoving averages, data should be used as it is for most Local Criminal JusticeBoard data. None of the Areas visited regularly include such trending informationin their own internal measures, although some indicated they would be includingpictorial reports with trend lines in future reports.

41



DDeepptthh ooff aannaallyyssiiss aanndd ccaauussee aanndd eeffffeecctt

8.43 We saw some examples where extensive information packs were provided,giving a breakdown of measurement sets for units, but there was no meaningfulanalysis provided to the management team. In some cases summary reports were produced, but they did little more than state the percentage change from last month’s performance figures and, as such, were of little value to the team.This is clearly not performance analysis.

We recommend that to promote more focussed discussion on causesand remedial actions, Areas should develop their presentationaland data consolidation methods, so that performance informationis presented pictorially with underlying trends readily discernable.

8.44 We saw many examples where Areas were able to influence outcomes throughcause and effect action, but these tended to be for simple issues like sickness andlate Graduated Fee Scheme payments. For more complex issues where therewere many variables there was little evidence of the use of cause and effectanalysis techniques. We were shown results of analytical studies to informimprovement actions, but these did not include cause and effect mapping.

TThhee rreellaattiioonnsshhiipp bbeettwweeeenn ppeerrffoorrmmaannccee mmaannaaggeemmeenntt ssyysstteemmss aanndd ddeelliivveerryyrreessuullttss

8.45 Inspectors considered the degree to which delivery results are influenced by thecompleteness of an Area’s performance management system. To test if correlationwas possible, four data sets were used. Data set one is a long data set coveringfour quarters’ total - or the nearest to March 2004 - and embraced persistent youngoffenders (PYOs), unsuccessful outcomes, and ineffective and effective trials forboth the magistrates’ and Crown Courts. Data set two is the same long-runningdata, but for four quarters’ total to March 2005. For both of these data sets eachmeasure was compared to the national average and Areas scored positively if theywere better than this.

8.46 In addition, Areas were scored against national measurements in the traffic lightperformance reports part of the Headquarters’ Area scorecard as at the third andfourth quarters of 2004-05. This information is based on a short-running data set forthe respective quarter. Areas scored positively for each Green or Amber Green light.

8.47 The performance management system of each Area was categorised according tohow well developed it was considered to be in terms of performance organisation,transparency of the reporting system, degree of integrity checking, and competenceof analysis. This was somewhat subjective, but where inspectors thought all elementswere well developed, the performance management system was considered to be“well established”; where the system was well established but had significantweakness in one or more elements, it was categorised as “well established butimpaired”; and where the Area did not have an established internal system it wascategorised as “not developed”. For all other cases, performance managementsystems were categorised as “developing”.



8.48 These tests are also somewhat subjective, but do tend to show that Areas with wellestablished systems which are free from compromising weakness are demonstratingmanaged performance over the two-year test interval for long data sets and for theshort data sets. What is also interesting is that the Area without an establishedinternal management system, which has well developed cross-agency performancemanagement arrangements, is scoring similarly to a peer Area which has anestablished, but impaired, system. Areas B and D show progress by having moremeasures under control over the last two years.

8.49 More detailed analysis across a larger number of Areas - and including a widerrange of variables - would be needed to be fully confident that better performancemanagement necessarily delivers better results. But such evidence as we havebeen able to draw together suggests that putting effort into improving the quality ofdata and then using it to drive performance can deliver real improvements.

43



AARREEAA AA AARREEAA BB AARREEAA CC AARREEAA DD AARREEAA EE

DevelopingWell

establishedDeveloping

Wellestablished,but impaired

Notdeveloped

PMS Status

AARREEAA FF

Wellestablished

3/7 7/7 4/7 2/7 1/7Four quarters to Mar 04 measures

7/7

4/7 7/7 6/7 1/7 2/7Four quarters to Mar 05 measures

7/7

5/11 10/11 7/11 8/11 6/11National measuresquarter 3

10/11

8/12 11/12 9/12 8/12 7/11(No OBTJ fig)

National measuresquarter 4 10/12

45



QQUU

EESS

TTIIOO

NN TT

RREE

EEQQ

UUEE

SSTTIIOO

NN TT

RREE

EE

Are

Are

a m

anag

ers

effe

ctiv

ely

usin

g pe

rfor

man

ce in

form

atio

n to

man

age

busi

ness

?

Has

the

Are

a id

entif

ied

the

prin

cipa

l per

form

ance

mea

sure

s th

at

requ

ire t

rack

ing?

Hav

e tr

aine

d re

sour

ces

been

allo

cate

d to

inpu

t an

d co

llate

perf

orm

ance

info

rma

tion?

Doe

s th

e A

rea

pres

ent

valid

ate

d pe

rfor

man

ce in

form

atio

nsh

owin

g an

ana

lysi

s of

tren

ds?

Can

the

Are

a de

mon

stra

te

inci

sive

dec

isio

n-m

akin

g to

impr

ove

perf

orm

ance

?

Has

the

Are

a se

t/bee

n al

loca

ted

targ

ets

for

each

of

the

prin

cipa

lm

easu

res?

Do

the

Are

a m

anag

ers

agre

e w

ith t

he p

urpo

se

ofth

e m

easu

re?

Do

seni

or s

taff

cons

ider

th

emse

lves

to

be a

ccou

ntab

le

for

allo

cate

d pe

rfor

man

ce m

easu

res?

Do

the

prin

cipa

l per

form

ance

m

easu

res

adeq

uate

ly c

over

C

PS

natio

nal/l

ocal

/CJS

part

ners

’re

quire

men

ts?

Doe

s th

e A

rea

colla

tefig

ures

ba

sed

on

orga

nisa

tiona

lun

its?

Doe

s th

e A

rea

colla

tefig

ures

on

a re

gula

r ba

sis

eg m

onth

ly?

Are

col

late

dfig

ures

com

plet

edw

ithou

t un

due

use

ofbu

cket

cod

eseg

“ot

her”

?

Is t

eam

/uni

tpe

rfor

man

ce

thro

ugho

utth

e A

rea

regu

larl

y?

com

mun

ica

ted

Is t

he A

rea

colla

ting

figur

es

from

the

be

st a

vaila

ble

sour

ce?

Are

all

hate

case

s fla

gged

in

CM

S?

Isof

fend

ers’

ethn

icity

flagg

ed

in C

MS

?

Is t

he A

rea

conf

iden

t th

at

all c

ases

subj

ect

tosp

ecia

lm

onito

ring

are

appr

opria

tely

flagg

ed?

Are

res

ourc

essu

ffic

ient

to

pro

vide

timel

yco

llatio

n?

Are

sta

ffco

nfid

ent

tha

t co

llate

dpe

rfor

man

cefig

ures

are

accu

rate

?

Are

any

perf

orm

ance

mea

sure

sco

nsid

ered

to

be ir

rele

vant

?

Are

the

Are

asa

tisfie

d th

at

the

best

reso

urce

/sou

rce

is u

sed

toco

llate

info

rma

tion?

Is t

here

evid

ence

to

show

tha

t th

e A

rea

suff

ers

from

perio

dic

back

logs

whi

ch a

ffec

tth

eir

figur

es?

Can

the

Are

ade

mon

stra

te

the

use

ofca

sefin

alis

atio

nco

des?

Are

man

ual

adju

stm

ents

sign

ifica

nt in

prop

ortio

n to

case

vol

ume?

Are

sen

ior

staf

fin

volv

edin

info

rma

tion

valid

atio

n?

Are

sen

ior

staf

fin

volv

edin

iden

tifyi

ngre

ason

s fo

rtr

ends

?

Are

rep

orts

prod

uced

for

each

uni

t on

a

mon

thly

basi

s?

Are

sen

ior

staf

fen

gagi

ngw

ith C

JSpa

rtne

rs

to a

naly

sepe

rfor

man

ce?

Are

the

sen

ior

man

agem

ent

team

sen

t a

mon

thly

perf

orm

ance

repo

rt?

Doe

s th

e C

CP

hav

e a

mon

thly

on

e-to

-one

mee

ting

with

Uni

t H

eads

to

rev

iew

uni

tpe

rfor

man

ce?

Do

Are

a st

aff

thin

k th

at

colla

tion

coul

d be

sim

plifi

ed?

Are

the

Are

aaw

are

ofan

you

t-of

-line

tren

ds in

the

irva

lida

ted

perf

orm

ance

figur

es?

Is t

he A

rea

awar

e of

any

erro

neou

s da

ta b

eing

incl

uded

inpe

rfor

man

cefig

ures

?

Can

the

Are

ade

mon

stra

teth

at

itspe

rfor

man

cem

easu

rem

ent

syst

em is

corr

ect?

Do

the

Are

aun

dert

ake

perio

dic

info

rma

tion

audi

ts t

oen

sure

the

inte

grit

y of

perf

orm

ance

info

rma

tion?

Are

Are

a pe

rfor

man

ce

mea

sure

s al

igne

d w

ith

staf

fpe

rson

al o

bjec

tives

?

Doe

s th

e A

rea

use

stan

dard

ised

proc

esse

sac

ross

uni

ts?

Do

the

deci

sion

s m

ade

by t

he A

rea

prov

ide

valu

e fo

r m

oney

?

Do

the

Are

a ev

alua

tere

lativ

e pe

rfor

man

cebe

twee

n un

its to

iden

tify

unde

r-pe

rfor

man

ce?

Can

the

Are

ade

mon

stra

te t

he

re-a

lignm

ent o

fre

sour

ces

base

d on

perf

orm

ance

info

rma

tion?

Do

seni

or s

taff

thin

k th

at

ther

e is

suf

ficie

ntun

ders

tand

ing/

skill

s of

num

eric

al a

naly

sis

for

perf

orm

ance

man

agem

ent

to b

eef

fect

ive

in t

he A

rea?

Is t

here

evi

denc

e th

at

perf

orm

ance

is a

ctiv

ely

man

aged

at

all l

evel

s?

Doe

s th

e A

rea

unde

rtak

e be

nchm

arki

ngw

ith o

ther

Are

as t

oid

entif

y be

tter

prac

tices

?

Are

sta

ffem

pow

ered

to

mak

e ch

ange

s w

ithin

thei

r ow

n re

mit

toim

prov

e pe

rfor

man

ce?

Is t

here

evi

denc

e th

at

man

ager

s ar

e pr

o-ac

tive

in p

ursu

ing

perf

orm

ance

impr

ovem

ent

in t

heir

own

rem

it?

AANNNNEEXXAANNNNEEXX 1111

47



AANNNNEEXX 22AANNNNEEXX 22

QQUUEESSTTIIOONNNNAAIIRREE RREESSPPOONNSSEESSQQUUEESSTTIIOONNNNAAIIRREE RREESSPPOONNSSEESS

This thematic report on the use of performance information in the CPS sought todetermine the degree to which CPS Areas were making effective use of performanceinformation to manage and deliver their business. The review considered what data isbeing collected by the CPS, data accuracy and the use of data to manage performance.

For ease of use and to determine relative strength of opinion, each questionnaire putforward a number of propositions in the form of statements to which respondent wereinvited to respond i.e. Strongly Agree, Tend to Agree, Neutral, Tend to Disagree, StronglyDisagree.

In addition, factual information was requested, for example the five most useful performancemeasures which are actively monitored.

In some instances the same question, making appropriate adjustment to the nature of therole, was included in more than one questionnaire to test perceptions at different levels.

The information gathered through the questionnaires was used to augment the lines of enquiry during on-site interviews and to inform findings. This Annex tabulates theresponses.

CCCCPPSS//AABBMMSS,, UUNNIITT HHEEAADDSS AANNDD AARREEAA PPEERRFFOORRMMAANNCCEE OOFFFFIICCEERRSS RREE:: SSKKIILLLLSS

CCPs/ABMs: Relevant staff are skilled in numerical analysis techniques

UHs: I have sufficient skills to be effective in analysing performance

APOs: Relevant staff have sufficient analytical skills to be effective in analysing performance

100%

80%

60%

40%

20%

0%

PE

RC

EN

TAG

E O

F

RE

SP

ON

DE

NT

S

CCPs/ABMs Unit Heads Area PerformanceOfficers

RESPONDENTS

Strongly agree

Tend to agree

Neutral

Tend to disagree

Strongly disagree

Not applicable/known



CCCCPPSS//AABBMMSS QQUUEESSTTIIOONNNNAAIIRREE

QUESTION NUMBER

Q12Q11Q10Q9Q8Q7Q6Q5Q4Q3Q2Q1

Q1 Performance management is essential to the management of my business

Q2 We understand and record the relative performance of each unit against the overall Area performance

Q3 Firm quantified targets have been set for our main performance measures

Q4 I believe that Area performance and PIs are accurate

Q5 I believe that national performance and PIs are accurate and valuable as a benchmark

Q6 My managers are held to account for achieving targets

Q7 All managers have quantified targets set in their personal objectives

Q8 Team and Area performance is communicated regularly throughout the Area

Q9 We can demonstrate that managers actively seek performance improvement

Q10 CMS is helpful in measuring and managing performance

Q11 MIS is helpful in measuring and managing performance

Q12 The Area produces or has access to reliable data on sickness absences throughout the year

100%

80%

60%

40%

20%

0%

PE

RC

EN

TAG

E O

F

RE

SP

ON

DE

NT

S

CCPs/ABMs provided 33 CPS Area-specific and 38 National/CJS examples of the mostuseful performance measures that they actively monitored, the majority being on a monthlybasis. The five most popular performance measures were:

CPS Area-specific National CJS

� Discontinuance/Unsuccessful rates � Cracked & Ineffective Trial and Attrition

� ABC � Sanction detection OBTJ� Cracked & Ineffective trial � PYO� Resource management � Charging� Court sessions/Staff deployment � Unsuccessful outcomes and

Public Confidence

Strongly agree

Tend to agree

Neutral

Tend to disagree

Strongly disagree


49



CCPs/ABMs gave 18 examples of main changes made in the last year as a result ofperformance information, the most common being:

� Joint action with police and courts to improve performance (16%)

� Resources/Staff deployment and ABC/HCA/DCW (14%)

� Charging (12%)

� CMS (10%)

� Casework (9%)

� Performance reporting (8%)

They listed 117 examples of the reasons and effects. These were mainly to achieve PSA targets, improve the process to enhance work quality, improve outcome performance,manage resources effectively, and to increase public confidence.

The results were in the main deemed to be successful without quantification, with fourexamples given where an action resulted in both an improvement in one aspect ofperformance and deterioration in another, or where success was limited.

BBAARRRRIIEERRSS TTOO EEFFFFEECCTTIIVVEE PPEERRFFOORRMMAANNCCEE MMAANNAAGGEEMMEENNTT

5%HR management14%

MIS training andrestrictions on

licences

34%Lack of skills

30%Time restraints and

high number of initiatives12%

Staff culture

5%Impact of

other agencies



CCHHAANNGGEESS TTHHAATT CCCCPPSS//AABBMMSS WWOOUULLDD LLIIKKEE TTOO MMAAKKEE TTOO IIMMPPRROOVVEE

TTHHEEIIRR PPEERRFFOORRMMAANNCCEE MMAANNAAGGEEMMEENNTT AARRRRAANNGGEEMMEENNTTSS

10%Casework monitoring

20%Appointment of APO

or more time for analysis

22%MIS/CMS

training and licence

28%Performance reports usage

and structure

5%Timeliness of data

2% Team/individual ownershipor accountability

2% Reduction in manual collation

4% Other data required

7%Joined-up performance

data with LCJB or OCJR

Additional performance data that should be collected nationally. This part of the form wasin the main not completed or noted as “None”. Examples of additional data providedwere: HR data, such as internal discipline, salaries, and sickness data; and data for benchmarking, such as charging benefits, and lawyer and caseworker deployment.

UUNNIITT BBUUSSIINNEESSSS MMAANNAAGGEERRSS QQUUEESSTTIIOONNNNAAIIRREE

UBMs provided 46 examples of the data items that they most frequently collate (generallymonthly). The five most common were:

� Lawyer deployment, use of resources and manpower

� Adverse cases/NCTA/Adverse case validation

� Timeliness of briefs, committals and section 51s

� Fees

� Monitoring categories, such as racial incidents

Provision of data and its purpose were as follows:

Provided to Purpose

DCP/UH/Level D managers Identification of trends and assessment of performance issues; Unit/Area reports; Discussion meetings with CCP

Headquarters/Area co-ordinator MonitoringManagers Completion and timeliness of tasks checks/

Finalisations on CMS and quality of data input checksArea Secretariat Fees and manpower

Data is mainly collected using in-house tools. There is apparent minimal use of MIS atthis level, except for PIs and checks of adverse cases on CMS using MIS-generated reports.

Cases with monitoring codes are checked using both in-house tools and CMS.

Q1 Collating data for my team is a regular feature of my job

Q2 I feel confident that the information I provide is accurate

Q3 I have had sufficient training to be effective in managing my unit’s performance data

Q4 Manual adjustments are made on a regular basis after validation checks

Q5 My PAR objectives are linked to the proven performance of the staff that I manage

Q6 I am confident that all my staff have a good understanding of data entry requirements

Q7 I regularly communicate performance issues to my staff

Q8 I am confident that all monitoring codes are actively used

Q9 Backlogs rarely affect our performance indicators

UBMs provided 15 examples of changes that they would like to make to improve theaccuracy of PIs, the most popular of which were:

� Development of MIS usage, access to and training on MIS (19%)

� All users to be fully trained on PI system (15%)

� Clearer file endorsements (12%)

� Safeguards on CMS for correct inputting (6%)

� Removal of “Other” code (6%)

51



QUESTION NUMBER

Q9Q8Q7Q6Q5Q4Q3Q2Q1

100%

80%

60%

40%

20%

0%

PE

RC

EN

TAG

E O

F

RE

SP

ON

DE

NT

S

Strongly agree

Tend to agree

Neutral

Tend to disagree

Strongly disagree


UUNNIITT HHEEAADDSS QQUUEESSTTIIOONNNNAAIIRREE

Q1 Performance analysis is essential to the management of my business

Q2 I have regular one-to-one meetings with the CCP/ABM to review my unit’s performance

Q3 I am accountable for my unit’s performance

Q4 My team is aware of our unit’s performance in relation to other Area units

Q5 We actively benchmark our performance with other similar CPS units

Q6 I am confident my unit’s performance figures and PIs can be trusted

Q7 I am confident my Area’s performance figures and PIs can be trusted

Q8 I regularly seek performance improvements in my unit

Unit Heads provided 38 examples of CPS Area-specific and 23 National/CJS main performance measures that they find personally useful, of which the the five main were:

CPS Area-specific National CJS

� Cracked and ineffective trial data � PYO figures

� Receipts and finalisations/Caseload/ � Cracked and ineffective trial dataCasework Levels/Workload (cases received/finalised/dropped compared with OBJT target)

� Adverse case outcomes/Dismissed � CQA/PCA dataNCTA

� Lawyer caseloads � Charging benefits model

� Discontinuance/Prosecution dropped � PI/Compass usage/OBTJ figures



QUESTION NUMBER

Q8Q7Q6Q5Q4Q3Q2Q1

100%

80%

60%

40%

20%

0%

PE

RC

EN

TAG

E O

F

RE

SP

ON

DE

NT

S

Strongly agree

Tend to agree

Neutral

Tend to disagree

Strongly disagree


CCPPSS AARREEAA//CCJJSS PPEERRFFOORRMMAANNCCEE GGRROOUUPPSS TTHHAATT UUNNIITT HHEEAADDSS PPAARRTTIICCIIPPAATTEE IINN

Unit Heads provided 113 examples of actions taken in the last year to improve performance.These can be categorised into actions for productivity gains; process improvement toimprove work quality; and to improve outcome performance.

Of the 113 examples, only ten had stated quantified benefits, two actions were notsuccessful and the remainder were deemed to be successful without qualification.

TTIIMMEE SSPPEENNTT OONN AANNAALLYYSSIINNGG PPEERRFFOORRMMAANNCCEE IINNFFOORRMMAATTIIOONN PPEERR MMOONNTTHH

53



17%CJB and sub groups

11%Court user groups

7%Area/team performance

17%Unit/Area management team

6%Charging

6% Victims andwitnesses (inc NWNJ)

9% PYO, Race, Domestic violence etc

12% Cracked and ineffectivetrials (inc ETMP)

15%Police liaison/JPM

54%7 - 15 hours

34% More than 15 hours

12%Less than 7 hours

AASSPPEECCTTSS OOFF PPEERRFFOORRMMAANNCCEE MMAANNAAGGEEMMEENNTT WWHHIICCHH UUNNIITT HHEEAADDSS

HHAADD TTHHEE MMOOSSTT DDIIFFFFIICCUULLTTYY WWIITTHH

Q1 Are any aspects of performance management a poor use of your time?

Please specify. Where the answer was “yes”, CQA was their main concern.

Q2 Are there any other aspects of performance management missing

or incomplete in your Area?

Q3 Do you consider any of the data you collected to be redundant?

Please specify. There the answer was “yes”, TQ1 was the most mentioned,

although these were being phased out in the Area.



20%Time pressures

32%Lack of skills/knowledge/

guidance

4%Accuracy of

CMS/MIS data7%

None

4%Staff relations

9%Volume 7%

Financial management

4% CQA

13%Reliance on others

25

20

15

10

5

0

NU

MB

ER

OF

R

ES

PO

ND

EN

TS

Q1 Q2 Q3

QUESTION NUMBER

RESPONDENTS

Yes No

LLOOCCAALL AARREEAA PPEERRFFOORRMMAANNCCEE OOFFFFIICCEERRSS QQUUEESSTTIIOONNNNAAIIRREE

AARREEAA PPEERRFFOORRMMAANNCCEE OOFFFFIICCEERR’’SS FFUUNNCCTTIIOONN

RREECCIIPPIIEENNTTSS OOFF RREEPPOORRTTSS

FFRREEQQUUEENNCCYY OOFF RREEPPOORRTTSS

55



24%Analyse

41% Recommend actions

35%Collate

20%CCP/SD

12%UBM

4%PTLs

19%AMT

3%NA

17% ABM/SBM

16% UHs

9%All

65%Monthly

30% Quarterly

5%Weekly



APOs provided 47 examples of performance measures at local level and 30 at Nationallevel which they considered useful and which they included in their reports.

TTHHEE FFIIVVEE MMOOSSTT CCOOMMMMOONNLLYY UUSSEEDD PPEERRFFOORRMMAANNCCEE MMEEAASSUURREESS

AANNDD TTHHEEIIRR FFRREEQQUUEENNCCYY OOFF UUSSEE

NNAATTIIOONNAALL//CCJJSS PPEERRFFOORRMMAANNCCEE MMEEAASSUURREESS

100%

80%

60%

40%

20%

0%

PE

RC

EN

TAG

E O

F

RE

SP

ON

DE

NT

S

MONTHLY WEEKLY QUARTERLY

FREQUENCY

Trial effectiveness/Cracked andineffective trials

Unsuccessful outcomes

Court sessions

POCA

Receipts, timeliness,quality of police files

100%

80%

60%

40%

20%

0%

PE

RC

EN

TAG

E O

F

RE

SP

ON

DE

NT

S

MONTHLY WEEKLY QUARTERLY

PYOs Unsuccessful outcomes

CQA

PIs

MIS charging reports/benefits

OTHER

Cracked andineffective trials

Discontinuances

FREQUENCY

CMS usage

AREA PERFORMANCE OFFICERS QUESTIONNAIRE

Q1 The report clearly shows the relative performance of each Area unit

Q2 The performance report is timely, complete and accurate for decision-making purposes

Q3 Our performance management system is driven by internal needs

Q4 The Area regularly reviews unit resources based on performance information

Q5 I am confident our performance figures are free from erroneous data

Q6 Our data is thoroughly validated each month

Q7 The performance management processes in each unit are equally effective

Q8 I think performance management is taken seriously in all our Area units

Q9 I am confident we can successfully diagnose issues raised by performance information

PERFORMANCE ACHIEVEMENT IS FORMALLY CONSIDERED

BY AREA SENIOR MANAGEMENT

57



QUESTION NUMBER

Q9Q8Q7Q6Q5Q4Q3Q2Q1

100%

80%

60%

40%

20%

0%

PE

RC

EN

TAG

E O

F

RE

SP

ON

DE

NT

S

Strongly agree

Tend to agree

Neutral

Tend to disagree

Strongly disagree

60%Monthly

31% Quarterly

9%Annually at business planning



Area Performance Officers provided 26 examples of the main changes introduced as aresult of performance information. These could be categorised into changes to: helpachieve PSA targets; manage resources effectively; and enhance the process to improvework quality. Only one of the 26 examples had stated a quantified benefit.

APOs gave 27 examples of changes that they would like to make to Area performance. These included increasing the numbers of MIS license holders and the provision oftraining and support with MIS (25% of respondents) and amending performance reports tomirror the Headquarters Marwick 15 (13% of respondents).

AANNNNEEXX 33AANNNNEEXX 33

GGLLOOSSSSAARRYYGGLLOOSSSSAARRYY

59



AAGGEENNTTSolicitor or barrister not directly employed by the CPS who is instructedby them, usually on a sessional basis, to represent the prosecution inthe magistrates’ courts

AARREEAA MMAANNAAGGEEMMEENNTT

TTEEAAMM ((AAMMTT))The senior managers (both legal and non-legal) of an Area

CCMMSS -- CCOOMMPPAASSSSThe IT system for case tracking used by all 42 CPS Areas. The systemshould be used to record key events in a case, including the recordingof the final outcome

CCHHAARRGGIINNGG

SSCCHHEEMMEE

The Criminal Justice Act 2003 took forward the recommendations ofLord Justice Auld in his Review of the Criminal Courts, so that the CPSwill determine the decision to charge offenders in more serious cases.“Shadow” charging arrangements are already in place; the statutoryscheme will have a phased roll-out across priority Areas and subsequentlyall 42

CCHHIIEEFF CCRROOWWNN

PPRROOSSEECCUUTTOORR

((CCCCPP))

One of 42 chief officers (all barristers or solicitors) heading the localCPS in each Area. Has a degree of autonomy, but is accountable toDirector of Public Prosecutions for the performance of the Area

CCOO--LLOOCCAATTIIOONNCPS and police staff working together in a single operational unit (TU orCJU), whether in CPS or police premises - one of the recommendationsof the Glidewell report

CCOOUURRTT SSEESSSSIIOONNThere are two sessions each day in the magistrates’ courts, morningand afternoon

CCRRAACCKKEEDD TTRRIIAALLA case listed for a contested trial which does not proceed, eitherbecause the defendant changes their plea to guilty - or pleads to analternative charge - or the prosecution offer no evidence

CCRRIIMMIINNAALL JJUUSSTTIICCEE

UUNNIITT ((CCJJUU))

Operational unit of the CPS that handles the preparation and presentationof magistrates’ courts’ prosecutions. The Glidewell report recommendedthat police and CPS staff should be located together and work closely togain efficiency and higher standards of communication and case preparation.(In some Areas the police administration support unit is called a CJU.)

DDEESSIIGGNNAATTEEDD

CCAASSEEWWOORRKKEERR

((DDCCWW))

A senior caseworker who is trained to present straightforward cases onpleas of guilty, or to prove them where the defendant does not attendthe magistrates’ courts

DDIISSCCOONNTTIINNUUAANNCCEEThe dropping of a case by the CPS in the magistrates’ courts, whetherby written notice, withdrawal, or offer of no evidence at court

GGOOOODD PPRRAACCTTIICCEE

An aspect of performance upon which the Inspectorate not onlycomments favourably, but considers that it reflects in manner ofhandling work developed by an Area which, with appropriateadaptations to local needs, might warrant being commended asnational practice

HHIIGGHHEERR CCOOUURRTT

AADDVVOOCCAATTEE ((HHCCAA))In this context, a lawyer employed by the CPS who has a right of audience in the Crown Court



JJOOIINNTT PPEERRFFOORRMMAANNCCEE

MMOONNIITTOORRIINNGG ((JJPPMM))

A management system which collects and analyses information aboutaspects of activity undertaken by the police and/or the CPS, aimed atsecuring improvements in performance

IINNEEFFFFEECCTTIIVVEE TTRRIIAALLA case listed for a contested trial that is unable to proceed when it was scheduled to start, for a variety of possible reasons, and isadjourned to a later date

CPS grades below the Senior Civil Service, from A (administrative staff)to E (senior lawyers or administrators)

MMAANNAAGGEEMMEENNTT

IINNFFOORRMMAATTIIOONN SSYYSSTTEEMM

-- MMIISS

The core CPS system for obtaining performance reports andinformation. Areas are able to draw on a suite of standard reports ordesign their own. The system receives automatic downloads from CMSovernight and is therefore populated with up-to-date information

““NNOO WWIITTNNEESSSS

NNOO JJUUSSTTIICCEE”” ((NNWWNNJJ))::

VVIICCTTIIMM AANNDD WWIITTNNEESSSS

CCAARREE PPRROOJJEECCTT

This is a project to improve witness care: to give them the support andinformation that they need from the inception of an incident through tothe conclusion of a criminal prosecution. It is a partnership of the CPS and the Association of Chief PoliceOfficers (ACPO), and also involves Victim Support and the WitnessService. Jointly staffed Witness Care Units will be introduced into allAreas by December 2005

PPEERRFFOORRMMAANNCCEE

IINNDDIICCAATTOORRSS ((PPIIss))

A generic term used within the CPS to denote the volume, type andoutcome of the cases they handle. They are not necessarily truemeasures of performance

PPEERRSSIISSTTEENNTT YYOOUUNNGG

OOFFFFEENNDDEERR ((PPYYOO))A youth previously sentenced on at least three occasions

LLEEVVEELL AA,, BB,, CC,, DD,, EE

SSTTAAFFFF

LLOOCCAALL CCRRIIMMIINNAALL

JJUUSSTTIICCEE BBOOAARRDD

The Chief Officers of police, probation, courts, CPS and YouthOffending Team in each criminal justice area who are accountable tothe National Criminal Justice Board for the delivery of PSA targets

PPRROOCCEEEEDDSS OOFF CCRRIIMMEE

AACCTT 22000022 ((PPOOCCAA))This Act contains forfeiture and confiscation provisions and moneylaundering offences, which facilitate the recovery of assets from criminals

PPUUBBLLIICC SSEERRVVIICCEE

AAGGRREEEEMMEENNTT ((PPSSAA))

TTAARRGGEETTSS

Targets set by the Government for the criminal justice system (CJS),relating to bringing offenders to justice and raising public confidence inthe CJS

RREECCOOMMMMEENNDDAATTIIOONN

This is normally directed towards an individual or body and sets outsteps necessary to address a significant weakness relevant to animportant aspect of performance (i.e. an aspect for improvement) that,in the view of the Inspectorate, should attract highest priority

SSPPEECCIIFFIIEEDD

PPRROOCCEEEEDDIINNGGSS

Minor offences which are dealt with by the police and the magistrates’courts and do not require review or prosecution by the CPS, unless anot guilty plea is entered

TTQQ11A monitoring form on which both the police and the CPS assess thetimeliness and quality of the police file as part of joint performance monitoring

Operational unit of the CPS which prepares cases for the Crown CourtTTRRIIAALL UUNNIITT ((TTUU))

UUNNIIQQUUEE RREEFFEERREENNCCEE

NNUUMMBBEERR -- UURRNN

The number used by police to identify an individual case file. This reference number is also entered into the CPS CMS system and isused in management reports to identify cases

61



A Review of the use of Performance Information in the ...

Documents

Transcript of A Review of the use of Performance Information in the ...