Findings and Recommendations - Staff Report
Committee Accepted
December 16, 1999

Performance Measurement
Findings and Recommendations

Section I

Background

Purpose of the Study

The purpose of the study is to identify ways to strengthen and systematize the availability and use of performance measurement information for the Connecticut General Assembly.

Basic Elements of a Performance Measurement System

There is no single structure for a performance measurement system that meets every organization's needs. However, there are basic elements that should be present in any good system. These include:

• a plan identifying a set of objectives to be achieved and the
• activities and resources needed to accomplish those
• objectives;
• a process for identifying, collecting, and analyzing information
• on how well the plan's objectives are being met; and
  • a process for making decisions concerning activities undertaken
  • in pursuit of the stated objectives and the corresponding
  • allocation of resources.

Figure I-1 shows a simple performance measurement system that illustrates how the elements identified above are connected in a sequence of steps. The process begins with a plan, moves through a data collection and analysis phase, and ends with decisions being made based on an assessment of performance data.

 Performance Measurement Models

The committee staff briefing report (October 20, 1999) identified two basic performance measurement models. One was labeled the special review model. Under this model, data on government operations are gathered and analyzed by staff reporting to a central authority -- typically a special commission empowered to make recommendations to the executive and legislative branches. The other model, commonly referred to as performance based budgeting, involves the systematic identification and collection of performance data, which are then incorporated into the regular budget process. It should be noted neither model is rigidly structured.

Performance Measurement in Connecticut

Over time the Connecticut General Assembly has introduced, modified, and attempted to strengthen various aspects of financial and programmatic oversight. The legislature’s attention has been on creating a process wherein performance measurement data will be identified and forwarded to decision-makers. As a result, Connecticut has statutes requiring the basic elements of a performance measurement system such as: the development of a long-range vision for the state; agency strategic plans; agency performance measures; and progress reports. Relevant statutory sections are summarized below.

• C.G.S. Sec. 4-67r -- mandates the Connecticut Progress Council to develop a long-range vision for the state and establish benchmarks to measure progress toward achieving this vision (P.A. 93-387). Biennially, the council submits the benchmarks to the Office of Policy and Management (OPM) and the General Assembly for use in developing and reviewing the state budget.
• C.G.S. Sec. 4-67m -- requires OPM in consultation with state agencies to develop goals and objectives and quantifiable outcome measures for every agency program and annually submit a report concerning such matters to the General Assembly (P.A. 92-8, May Special Session). OPM must include in its report an evaluation of the progress of budgeted agencies in achieving benchmarks established by the progress council (P.A. 93-387).
• C.G.S. Sec. 4-73 -- requires the governor's biennial budget request to include, among other items, information on state agency programs, resources, and performance measures (P.A. 81-466).

State Entities Responsible for Performance Measurement

Table I-1 identifies entities having a role in the operation of the state's existing performance measurement system. Each entity's responsibilities with respect to the process outlined in Figure I-1 on page 2 (i.e., step 1 -- planning, step 2 -- identification of measures, step 3 -- collection of data, and step 4 -- analyzing data) are classified in the column labeled "Roles." Excluded from the table is step 5 -- decision-making.

Section II

Findings and Recommendations

The General Assembly has taken numerous actions over the past several years to develop a performance measurement system. These efforts have produced a clear statutory record of legislative intent, but not a functioning performance measurement system. Committee staff believes this result is caused by an absence of commitment and direction from the executive branch and the lack of effort on the part of the legislature to identify wayward state agencies and hold them accountable.

This section lays out the current status of the state's performance review system through reports produced by the Office of Policy and Management. The system is critiqued in analyses conducted by state contracted consultants, a legislative task force, and the program review committee staff. Options outlining structures for producing performance measure data are presented and assessed. Section II concludes with recommendations proposed by the committee staff.

Compliance with State Statutes

In a July 1994 report to the General Assembly, OPM outlined efforts undertaken by the executive branch to comply with the statutory requirements for agency performance measures (P.A. 92-8, May Special Session). The report indicated it would take several years to develop a meaningful performance measurement system, which, according to OPM, meant the measures would be used in making decisions about program activities, policies, and budget priorities. The report included preliminary measures for 21 state agencies.

In a March 1996 memo to the chairs of the General Assembly’s Appropriations and Planning and Development Committees, OPM indicated it was working with state agencies to institute a business planning concept as a means of linking the state's benchmark project with "specific state agency goals, objectives and their related performance measures." OPM noted the Department of Revenue Services and the Department of Children and Families were already involved in a pilot test of this approach. (The memo was in response to P.A. 95-232, which required OPM to submit a plan for the use of benchmarks in the development of the budget.)

In September 1998, OPM issued a set of guidelines for developing an agency strategic plan and performance indicators. In the preface to the guidelines, it was noted eight agencies had already developed their first business plan. The OPM web site has links to guidelines used by selected other states to help their agencies develop and implement performance measurement systems.

In September 1999, OPM reported the results of its survey of state agencies. The purpose of the survey was to determine the current status of strategic planning and performance measurement in all 65 state budgeted agencies. Eighty-three percent of the agencies responded. OPM found 36 of the 54 responding agencies claimed to have a strategic planning process. However, the responses indicated only 29 of the 54 responding agencies had developed performance measures for all their budgeted programs.

Based on the survey, OPM drew the following conclusions with respect to strategic planning and performance measurement.

These findings led OPM to make the following five recommendations regarding the office's role in developing the state's performance measurement system. OPM should:

Compliance summary. Overall, OPM has taken steps to provide state agencies with information and guidance on the actions necessary to develop a performance measurement system. Unfortunately, so far it appears much of the effort has been aimed at starting and restarting the process at the agency level. As the results of OPM's agency survey show, there is a long way to go before the statutory mandates contained in sections 4-67m, 4-67r and 4-73 can be met.

Analysis of Existing Performance Measurement System

Two studies of Connecticut's existing performance measurement system are worth noting. One was done for the State Comptroller's Office by a private consulting firm. The other study was undertaken by a special legislative task force.

Consultant's report. In January 1998, Deloitte & Touche LLP submitted a report to the State Comptroller's Office containing recommendations for improving the way the state collects management information and makes the data available for analysis. The report indicated Connecticut's performance measurement system contributed little of value to:

Deloitte & Touche LLP attributed this in part to a budget process that allocates and accounts for funds on a line-item basis. The report indicated this process contributes to a commonly held view among state agency managers that the collection and reporting of performance information is a "bother," and the results are widely ignored by executive and legislative branch budget analysts.

The report notes most of the performance measures developed in response to state statutory requirements are simple indicators of workload and output. The consultants expressed an opinion that such measures are of little value in assessing efficiency and effectiveness.

All four of the consultant's key recommendations addressed ways to improve the state's management information, and one dealt specifically with performance measurement. This recommendation calls for Connecticut to:

• • develop and utilize appropriate performance measurements;
  • align defined programmatic categories with those utilized in financial reporting to capture a complete picture of program, financial, and operational performance;
  • dedicate resources in order to modify performance measures to changing information needs;
  • develop procedures to integrate program performance data into planning, budgeting, and management processes of the state; and
  • refine existing performance measures and develop new indicators useful to stakeholders at the state agency and program levels.
• Legislative task force. A legislative task force studied performance measurement between the 1997 and 1998 sessions of the General Assembly. It took testimony from the heads of relevant Connecticut agencies and staff from key performance review agencies in Texas and Florida. In addition, the task force examined documents describing the oversight and monitoring efforts being made in Connecticut and other states. In February 1998, the task force issued its report, which included the following findings and recommendations.
• Findings:
  • Under OPM's guidance, performance measures are in place for 20 percent of state agencies, but the program is voluntary, and OPM has no plan to require the remaining 80 percent to participate.
  • Legislative auditors have a performance review division that conducts a small number of in-depth reviews of select programs. However, given the intensive nature of the reviews, extending this effort to all state agencies and programs does not appear feasible.
  • LPR&IC annually evaluates the effectiveness and efficiency of five to seven programs, but rather than being focused on state expenditures many of the committee's recommendations are directed at how a program is run and policies carried out or how its resources are allocated.
  • Texas and Florida have budget-based performance review processes that are more comprehensive and focused on savings than Connecticut's efforts.

Recommendations:

As a result of the work of the task force a bill to implement the recommendations was introduced into the 1999 session of the General Assembly. A substitution version of the bill (File 463) made it to the House calendar but was not acted upon.

Staff analysis. The program review committee staff conducted an analysis of the state's performance measurement system focused on comparing the status of the state's activities with respect to the requirements of a good performance measurement system referenced in the steps shown in Figure I-1. The results of the comparison are outlined in Table II-1.

The row in Table II-1 dealing with the current status of the state's performance measurement system reflects the committee staff’s opinion that existing state statutes meet the basic requirements of a good system -- agency planning and the identification of performance measures. The table highlights the staff's belief that, if a performance measurement system is to be successful, there is a need for:

Alternative Performance Measurement Mechanisms

As pointed out in the staff briefing paper, a number of different models could be used to address the needs identified in Table II-1. Based on an assessment of current and previous efforts in Connecticut and activities in other states, the committee staff explored two general models the state could pursue to systematize how it evaluates performance.

One model dealt with creating a special mechanism to review the performance of governmental operations on a continuing basis. The other involved establishing a performance measurement process within the existing structure of state government.

Special performance review model. This model has been in existence for many years and is frequently employed in times of budgetary crisis. Under this approach, responsibility for reviewing state government operations would be assigned to a central authority whose sole function would be to measure the performance of government operations and recommend ways to improve efficiency and effectiveness. Committee staff included three variations or options under this model. Two of the options involve blue-ribbon commissions composed of individuals representing the executive and legislative branches and given a mandate to make recommendations to the governor and legislature to improve the efficiency and effectiveness of state government operations.

Examples of operations similar, though not identical, to these two options are the Thomas (S. A. 89-40) and Harper-Hull (P.A. 91-3 June Special Session) Commissions, noted in the staff briefing report. The difference is that the Thomas and Harper-Hull Commissions were one-time-only entities, while the options put forth under the special review model involve on-going commissions.

The activities of these commission-type options would be directed by a small core staff of three individuals. Key tasks beyond those performed by the core administrative staff such as problem identification, data collection, data analysis, and commission briefings would be performed by either:

The third option under the special performance review model would be to establish a permanently staffed professional office to systematically review all state government operations and report its findings and recommendations to the governor and legislature. This would be similar to the Florida legislature's Office of Program Policy Analysis and Government Accountability, which was discussed in the staff briefing report.

Modify the existing governmental structure model. The second model that could be employed to address the needs identified in Table II-1 focuses on the systematic identification and collection of performance data, and is similar to the performance-based budgeting model described in Section I. The only option described by committee staff under this model involves assigning to distinct entities responsibility for carrying out key elements of the performance measurement system. Specifically: 1) OPM would be charged with overseeing the development of strategic plans and performance measures by executive branch agencies; 2) the state auditors office would serve as an independent monitor of agency compliance; and 3) program review committee would review reports and data produced by state agencies and forward it to the legislature's Office of Fiscal Analysis and the appropriate committees within the General Assembly.

Cost. Cost is an important factor to be weighed when evaluating any of the performance review models and related options. Table II-2 contains some cost estimates for the four options developed by committee staff. In the first column of the table are the four basic options identified above. The second column shows the estimated cost of administrative operations. Columns three, four, and five show the costs associated with various methods of producing agency reviews -- permanent staff, loaned staff, or consultants.

The first three options listed in Table II-2 correspond to the approaches identified under the special performance review model. The last option relates to structuring a performance measurement system using existing governmental units.

Prior to reviewing the information presented in the table some factors must be considered. First, before the committee staff could develop cost estimates for the basic performance measurement options a determination had to be made as to how often each of the 65 executive branch agencies would be reviewed. The staff chose four-year cycles as the frequency rate for assessing state agencies.

A four-year cycle means approximately 16 agencies per year will be reviewed -- a quarter of the state’s agencies. Further, the committee staff believes a four-year review cycle allows the changes resulting from one review to be put into practice, any associated implementation problems resolved, and operations to be normalized before a second study is undertaken. Other benefits from this approach include the need for fewer resources to conduct the reviews than would be required by an annual or two-year cycle and a corresponding reduction in the funds needed to pay for those resources.

The four-year agency review cycle does not apply to the option calling for the modification of the existing governmental structure. This option is geared toward providing a steady flow of performance data from agencies, through executive and legislative analysts, and on to decision-makers.

Each option listed in the Table II-2 has a small core administrative unit consisting of a director, professional assistant, and one support staff. Regardless of the structure of the performance measurement system, committee staff believe such units are necessary to manage overall operations and handle daily problems. The estimated first-year cost of the core administrative unit is $225,000.

The cost variation among the options is found in the area dealing with the source of the staff producing the performance reviews. As Table II-2 illustrates, the first option relies on contracted consultants, the second on staff loaned or reassigned from existing state operations, the third on permanent staff, and the fourth on a combination of new and loaned or reassigned staff.

A key factor in comparing the options is how the costs associated with the loaned or reassigned staff are treated. If it is assumed the work previously performed by the loaned or reassigned staff would be foregone, then there is no increase in costs to the state for staff services and none should be attributed to the corresponding option. On the other hand, if replacement staff are hired to produce work loaned or reassigned staff can no longer provide, then the cost of the replacement staff represents an additional expense and should be added to the cost of funding the related option.

Table II-3 presents a comparison of the cost estimates for each option when the expense of producing the work previously provided by the loaned or reassigned staff is included in the cost of the option (column two) and when it is not (column three). Under the first condition, the option dealing with the modification of existing governmental structures is the least expensive, while the option associated with a commission using loaned or reassigned staff is the cheapest when the cost of producing work previously provided by the loaned or reassigned staff is not counted.

It should noted the data in Table II-3 do not represent the full cost of each option due to the absence of data on the costs incurred by state agencies in developing strategic plans; identifying, collecting, analyzing, and reporting data; and responding to other demands imposed by performance measurement systems. However, committee staff believes it is reasonable to assume state agency costs would be similar across all four options and as a result comparisons based on the data in Table II-3 are useful.

Regarding the issue of state agency costs associated with performance measurement requirements, attention should be given to a fiscal note prepared by OFA on a bill in the 1999 session of the General Assembly. File 436, An Act Reducing Inefficiency and Waste in State Government Operations, required state agencies to perform similar activities to those called for in the options outlined above -- develop strategic plans and performance measures. OFA’s analysis of the proposed law indicated state agency compliance costs would be minimal.

Benefits. In addition to the costs associated with operating a performance measurement system, there are benefits produced as a result of the system. Unfortunately, objective data detailing such benefits are difficult to obtain. What follows is a brief discussion of benefits, which are typically divided into two categories -- cost savings and improved services.

Cost savings. Cost savings resulting from performance review mechanisms are widely reported. Texas indicated its first-year performance review effort resulted in proposals totaling $4.5 billion in savings and revenue enhancements. Florida claimed its recommendations produced $233 million in savings and increased revenue between 1994 and 1997. One of Connecticut's own performance review efforts, the Thomas Commission, put the value of its recommendations at $333 million spread over four years. The Harper-Hull Commission did not put a cost-savings figure on its recommendations.

A couple of points regarding claimed savings and revenue enhancements need to be made. First, the proposed savings are rarely realized in full. For example, Texas claimed about 85 percent of its proposed savings and revenue enhancements made it into law; Florida put its passage rate at 40 percent. Second, the claimed savings might have occurred without a specially created performance review mechanism. Specific instances of this are pointed out in an analysis of the Texas and Florida data conducted by the Connecticut legislature's Office of Fiscal Analysis at the request of the General Assembly’s 1998 Performance Review Task Force. The analysis identified numerous instances where savings reported by Texas and Florida were also realized by Connecticut through its regular budgetary review process. For example, Florida's Office of Program Policy Analysis and Government Accountability claimed $74.3 million in savings from a recommendation dealing with Medicaid prepaid health plans. The OFA analysis notes Connecticut saved $56.5 million in the same area through its normal budget review processes.

Improved services. Concerns similar to those involving cost savings can be raised about the relationship between proposals to improve the effectiveness of state services and the performance review mechanisms credited with producing those proposals. As with cost savings, there undoubtedly will be instances where valid improvements will not be implemented because they do not meet the need of a constituency group. Also, there is no guarantee the service improvements would not be proposed and implemented if the performance review mechanism credited with the proposal did not exist. For example, in Connecticut, existing mechanisms including the program review committee, state auditors, organized interest groups, and private citizens are constantly proposing new and better ways of delivering services.

Summary. Summing up the discussion of benefits, the program review committee staff concludes there is evidence to show the cost savings and service improvements attributed to various performance review mechanisms are overstated. Further, analyzes have identified instances where similar results have been produced without having to create new comprehensive on-going performance review mechanisms.

Staff Recommendations

The fact that benefits may be overstated and can be produced through other mechanisms does not mean performance review systems should never be employed. Committee staff believes performance measurement systems produce at least an incremental increase in the efficiency and effectiveness of government operations and Connecticut could gain through changes aimed at supplementing and enhancing the state’s existing performance review mechanisms.

Assuming state agency compliance costs and the service benefits related to each of the four options are equal, the option involving a commission staffed by loaned or reassigned personnel offers the best cost-to-benefit ratio when the expense of replacing work produced by the loaned or reassigned staff is excluded. When such costs are included, modifying the existing government structure is the option with the best cost-to-benefit ratio.

Although Table II-3 indicates that a commission served by consultants is not a leader in the area of cost-to-benefit ratio, a case can be made for this option. The appeal of this option is it does not set up a permanent staff presence, which could become entrenched and outlive its usefulness. After all, reducing or eliminating funds appropriated for hiring consultants is a much easier task than changing the amount budgeted for permanent staff.

The argument favoring the remaining option -- a new permanent staffed state office -- must be based on the notion a permanent office could develop expertise and credibility over time. The state auditors office and the program review committee serve as models supporting this idea.

After reviewing the alternatives, committee staff concluded -- with the benefits and agency compliance costs assumed equal across all options and the differences in the estimated costs associated with implementing each option being relatively minor within the scope of the state budget -- picking the best performance review mechanism is more a matter of preference than sorting among the facts. Based on these understandings, the committee staff initially narrowed its preferences to two options: 1) a commission served by consultants; or 2) modifying the existing state government structure.

In the final analysis, the latter option emerged as the most attractive. In the opinion of the committee staff, this option can be implemented faster and with less disruption to the existing system than the commission served by consultants option. It is also the cheaper of the two options regardless of the cost assumptions used. Specifically, the program review committee staff recommends:

1. The Office of Policy and Management shall be responsible for:
   1. assuring all budgeted state agencies develop strategic plans that identify the relevant benchmarks established by the Connecticut Progress Council and include goals, objectives, and performance measures for each program provided by such state agency;
   2. assuring the goals, objectives, and performance measures included in each budgeted state agency's strategic plan address performance information needs identified by the joint standing committees of the General Assembly having cognizance of matters relating to appropriations, government administration, or the subject matter addressed by the agency;
   3. assuring each budgeted state agency collects data on the performance measures and benchmarks included in the agency's strategic plan;
   4. assuring an annual report is prepared on each budgeted state agency and its programs based on an analysis of the benchmark and performance measurement data included in the agencies strategic plan and submitting such reports to the Program Review and Investigations Committee; and
   5. assuring access to all performance and benchmark data to the Program Review and Investigations Committee and to the joint standing committees of the General Assembly having cognizance of matters relating to appropriations, government administration, or the subject matter addressed by the agency.
2. The Program Review and Investigations Committee shall be responsible for:
   1. analyzing and commenting on the agency reports submitted by OPM;
   2. analyzing and commenting on the performance measurement and benchmark data made available by OPM including the relevance of the performance data related to each program's objectives; and
   3. distributing the reports submitted by OPM along with the committee's comments on the reports and data to the joint standing committees having cognizance of matters relating to appropriations, government administration, and the specific agency's operations.
3. The Auditors of Public Accounts when conducting an agency or program audit shall be responsible for determining if:
   1. the agency's or program's strategic plan is current;
   2. the strategic plan contains all the required elements;
   3. the data pertaining to the performance measures and benchmarks are being collected; and
   4. the data being collected are reliable and valid.
4. The Office of the State Comptroller shall pursue the development, funding, and implementation of a new state automated accounting system capable of providing performance data at the program level.
5. Not later than 90 days after the effective date of the act, the Secretary of the Office of Policy and Management shall establish a schedule whereby each budgeted state agency shall biennially develop a strategic business plan. The schedule established by the Secretary of the Office of Policy and Management shall require at least 20 percent of the budgeted state agencies to develop such a plan by July 1, 2001, and no later than July 1, 2004, all budgeted state agencies shall be developing strategic plans biennially.
6. Beginning September 1, 2002, and annually thereafter, the Secretary of the Office of Policy and Management shall submit to the Program Review and Investigations Committee a report on each budgeted state agency based on an analysis of the data associated with the benchmarks and performance measures included in the agency's strategic plan.
7. Beginning January 1, 2003, and annually thereafter, the Program Review and Investigations Committee shall distribute the reports and data submitted to it by Secretary of the Office of Policy and Management along with the program review committee's comments to the joint standing committees of the General Assembly having cognizance of matters relating to appropriations, government administration, and the subject matter addressed by the agency.
8. Funding for five additional staff positions in the Office of Policy and Management and two additional staff positions in the Office of the Auditors of Public Accounts shall be phased in along with the implementation of the proposed performance measurement system.
9. Repeal C.G.S. Title 2c "Connecticut Sunset Law."

Further Explanation

As pointed out in the briefing paper and again above, state statutes require a more advanced performance measurement system than is currently in place. Committee staff believes major reasons for the widespread noncompliance by state agencies are the absence of consistent and forceful direction from the executive branch and the lack of effort on the part of the legislature to identify nonconforming agencies and hold them accountable.

The first three staff recommendations are aimed directly at these problems. The first gives OPM the responsibility and authority to direct state agencies to perform the activities necessary for Connecticut to have a fully functioning performance measurement system. The second and third staff recommendations put the state auditors and the program review committee -- two independent legislative branch entities -- in a position to monitor and evaluate state agency compliance with the state's statutory performance measurement requirements. The committee staff believes assigning OPM, the program review committee, and the state auditors independent oversight roles will help assure non-compliant or inadequate responses on the part of state agencies are highlighted at every point of occurrence. Further, the interrelated reporting requirements of the three entities will force this information to be brought to the attention of legislative and executive leaders and the public.

In addition, the first two recommendations provide legislative committees the opportunity to engage OPM and the program review committee to help assure state agency efforts to collect and report performance data formally requested by committees. For example, if a budget subcommittee requested an agency report on the future use of a new piece of equipment, adoption of the staff recommendations would result in a mechanism being put in place that could be used to monitor the agency's compliance.

The intent of the fourth recommendation -- a performance data base linked to the state's accounting system -- is to encourage the development of a central repository of performance data. The committee staff strongly believes this is the best long-term solution to the performance measurement needs of all legislative and executive branch decision-makers. It provides hard data at the program level to supplement other information used in making decisions.

The fifth, sixth, seventh, and eighth staff recommendations merely spell out a schedule for implementing the performance measurement proposals and identify the additional staff resources needed. The recommendations call for the system to be phased in over a three-year period.

The ninth recommendation calls for the repeal of the state's sunset review law. The committee staff believes if the sunset law is not repealed, the demands it places on the program review committee coupled with the requirements of the above recommendations will force the committee to either severely reduce the number of studies it undertakes annually or add staff. In the opinion of the committee staff, the expense of adding staff could be avoided and the legislature better served by repealing the sunset law. This law consumes an enormous amount of staff and legislative time and, as currently written, is disproportionally focused on relatively small and narrow programs.

Return to Year 1999 Studies