Legislative Program Review

& Investigations Committee

Department of Children and Families

Monitoring and Evaluation

December 2007

DEPARTMENT OF CHILDREN AND FAMILIES MONITORING AND EVALUATION

Executive Summary i

Introduction 1

Purpose 1

Study Approach 2

Methodology 6

Report Organization 8

I. Background: Overview of DCF. 9

Mission, Guiding Principles, and Goals 9

Major Duties and Responsibilities 13

State Mandate Areas and Programs 15

Organization and Budget 22

II. DCF Internal Monitoring and Evaluation Activities 29

Bureau of Continuous Quality Improvement 29

Other Internal Monitoring and Evaluation Efforts 41

Contracted Monitoring and Evaluation 43

DCF Information Systems 48

III. DCF External Monitoring and Evaluation Activities 51

U.S. Department of Health and Human Services 51

Federally-Required Child Abuse Prevention and Treatment Plan 65

DCF Federal Grant Funding 68

Judicial Oversight 70

Legislative Oversight 83

Accrediting Body Oversight 88

Other Regulatory Monitoring and Evaluation Reporting 94

IV. Outside Investigations and Reviews 97

Office of the Child Advocate 97

Child Fatality Review Panel 102

State Attorney General 105

V. Advisory Group Monitoring and Evaluation Activities 107

Overview 107

Agencywide and Area Advisory Groups 109

DCF Facility Advisory Groups 109

Federally Mandated Advisory Groups 111

Issue-Specific Advisory Groups 111

Inactive Advisory Groups 113

Advisory Groups Requiring DCF Participation 113

VI. Results from Monitoring and Evaluation Efforts 117

Overview 117

Detailed Monitoring and Evaluation Results Within Child Protective Services 119

Detailed Monitoring and Evaluation Results Within Behavioral Health Services 132

Detailed Monitoring and Evaluation Results Within Juvenile Services 138

Detailed Monitoring and Evaluation Results Within Prevention Services 139

Detailed Agencywide Results 141

Summary of Results from Key Monitoring and Evaluation Efforts 142

VII. Findings and Recommendations 145

Agency Goals 146

Overall System Assessment 150

Internal Efforts 159

External Efforts 171

Outside Investigations and Reviews 178

Advisory Group Efforts 181

Executive Summary

Department of Children and Families Monitoring and Evaluation

Formed in 1974 as a consolidated children's agency, the Connecticut Department of Children and Families (DCF) has broad authority and primary responsibility for state mandates concerning child protection, children's behavioral health, juvenile delinquency, and prevention services related to children and families. The department has been studied, audited, reviewed, and subject to legal action almost continuously since it was created due to ongoing concerns about its ability to carry out its challenging mission.

Numerous internal quality improvement efforts, as well as oversight by multiple outside entities including federal and other state agencies, various advisory groups, the courts, and the legislature, have focused on how to achieve better outcomes for the children and families DCF serves. The Legislative Program Review and Investigations Committee (PRI) alone had completed seven different reports on the department prior to undertaking a review of the overall DCF accountability system in April 2007. Unlike previous PRI reviews, this study evaluated a critical function -- monitoring and evaluation of agency results -- rather than a particular mandate.

An effective results-based monitoring and evaluating system is important for three main reasons: 1) it provides an agency with productive feedback on actual outcomes and progress toward goals; 2) it allows agency staff, policymakers, and stakeholders to know where the agency is successful, where it is not, and how to make improvements; and 3) ultimately, it helps the agency provide services that meet clients' needs and make cost-effective use of taxpayer resources. The purpose of the 2007 program review committee study was to determine areas of strength and weakness, as well as gaps and redundancies, in the existing DCF accountability system, and to identify needed improvements.

Study approach and methods. The committee study employed two primary research methods: interviews with key stakeholders; and analysis of monitoring and evaluation reports and other documents produced through DCF quality assurance, performance evaluation, and oversight efforts. There were four main sources of efforts:

To put into context all the information gathered about the process, sources, and results of DCF monitoring and evaluation, the core components of the current system were also compared with a national model for child welfare agency quality improvement.

The program review committee's final report contains an assessment of the overall DCF monitoring and evaluation system, details the system's positive features as well as deficiencies, and recommends nearly 40 administrative and legislative changes to improve its effectiveness. The report also summarizes data on agency accomplishments that were compiled by program review staff from more than 100 different monitoring and evaluation documents analyzed during the study.

Main findings. The program review committee found little attention has been given to examining DCF as a whole or assessing how well the agency is achieving its broad goals of safety, permanency, and well-being for all children and families. Further, while the department is responsible for carrying out four major mandates, monitoring and evaluation is focused primarily on the child protective services mandate, due largely to the ongoing impact of the federal Juan F. lawsuit consent decree and requirements of federal agencies.

The PRI study showed there is greater emphasis on tracking how services for children and families are delivered rather than on assessing their end results. While high quality service delivery is important, the crucial indicator of effectiveness is whether programs are making a difference and achieving stated goals. In general, more attention to outcome information is needed throughout the DCF accountability system.

The committee's review also identified pockets of strength within the system, such as the Juan F. exit plan process and related DCF area office quality improvement processes, the department's licensing procedures, the agency's recently revised special review process, and the activities of on-site facility monitors.

Some major weaknesses were revealed as well. In particular, the agency's contracting process provides little accountability, consequences for poor performance are rare, and working relationships with private providers need improvement. The committee also found ineffective use of some important sources of feedback on services and programs, such as child fatality reviews, OCA investigations, and even the department's own program review reports and contracted evaluations.

In part, these deficiencies are due to both fragmentation of quality improvement efforts within the agency and the fact that results data are not regularly integrated and analyzed. Both problems are related to the department's information systems, which are themselves fragmented and in some cases inadequate. Another challenge is a lack of department staff with the analytic skills and research experience needed to use results data and information. Further, there is no centralized place – like an agencywide strategic plan – where all DCF goals and information about service delivery and outcomes are brought together.

Duplication of external monitoring efforts also was revealed by the program review committee's examination of statutorily required DCF plans and reports. The committee determined several mandates could be eliminated without a loss of accountability, as certain documents have become obsolete or been replaced by newer sources of similar information. In addition, reducing the number and clarifying the purpose of reporting mandates could improve the quality of information on department results available to the legislature and the public.

Committee recommendations. Ultimately, the point of all monitoring and evaluation efforts, whether internal, external, investigatory, or advisory, is to ensure programs and services are having desired results. Taken together, the PRI committee recommendations listed below are aimed at making the current DCF accountability system more effective by:

Recommendations

Introduction

Department of Children and Families Monitoring and Evaluation

The Connecticut Department of Children and Families (DCF) has broad authority and primary responsibility for the state's main child welfare mandates -- protecting children from abuse and neglect, providing children's behavioral health and juvenile justice services, and carrying out prevention efforts for children and families at risk of abuse, neglect, mental illness, substance abuse, and juvenile delinquency. To carry out its mission, the department has an annual budget exceeding $800 million and a full-time staff of nearly 3,500 employees.

Since its formation as a consolidated children's agency in 1974, the department has been studied, audited, reviewed, and subject to legal action almost continuously. Multiple entities outside the agency, such as the state Office of the Child Advocate (OCA), legislative committees, the Judicial Branch, national accrediting bodies, federal agencies that provide funding for children's services, and the federal courts, track aspects of DCF performance and provide oversight of various program outcomes. Concerns continue to be raised about the efficiency, effectiveness, and advocacy capabilities of DCF through these many oversight efforts.

At the same time, there has been a number of initiatives to improve department operations and services in recent years. Many changes have been made in response to the ongoing federal Juan F. consent decree concerning Connecticut's child welfare system.

Reports from the federal court monitor show caseloads for the agency's social workers comply with national standards, community-based treatment and support services have been expanded, and there is more collaboration with other agencies involved with children and families such as the Departments of Social Services, Mental Health and Addiction Services, Developmental Services (DSS, DMHAS, DDS) and the Court Support Services Division (CSSD) of the Judicial Branch. The Department of Children and Families also has instituted various internal monitoring and evaluation efforts, such as those carried out by its continuous quality improvement bureau, as ways to strengthen management and policy decision making.

An effective process for tracking and assessing results is the cornerstone of accountability and improved performance of state agencies. In April 2007, the Legislative Program Review and Investigations Committee (PRI) voted to undertake a comprehensive assessment of efforts to monitor and evaluate DCF.

Purpose

The purpose of the program review committee study of DCF monitoring and evaluation was to determine areas of strength and weakness, as well as any gaps and redundancies, within the existing agency accountability system. The main goal was to identify improvements to internal and external oversight efforts that would lead to better agency performance and, ultimately, to better outcomes for children and families.

Specifically, the study centered on: 1) describing how goals set for and by the agency are measured and tracked; 2) evaluating the department's progress in attaining its goals; 3) examining the extent to which the results of monitoring and evaluation efforts are used by DCF to improve the services it provides to children and families; and 4) identifying ways to increase the overall effectiveness of the DCF accountability system. Unlike all previous PRI studies of the Department of Children and Families, the focus was on a critical function -- monitoring and evaluation -- rather than a particular agency mandate.

Previous committee DCF studies. It is not uncommon for the program review committee to conduct multiple studies of a state agency over time, especially a complex one like the Department of Children and Families. Six prior PRI studies evaluated how the agency carried out its various mandates and identified ways to make specific programs more efficient and effective.² The committee's last report on DCF, issued in December 1999, examined implementation of its overall consolidated children's services mission. PRI proposed a significant restructuring of department duties that was intended to achieve higher quality services, better coordination and integration, and stronger leadership and oversight.

While there has not been a major reorganization of child protection, behavioral health, and juvenile justice programs in Connecticut since the 1999 PRI study, a number of changes have occurred within DCF and in the general environment of children's services. Since 1999, DCF's capacity for self-evaluation and corrective action has increased and there are more external mechanisms for providing productive feedback and accountability for results. Major developments in effective oversight within and outside DCF are highlighted in Chapter I, and described more fully in Appendix C.

In reviewing these developments, it appeared to the program review committee that better monitoring and evaluation efforts might attain the improvements in agency performance sought previously through restructuring proposals. This study, therefore, centered on the effectiveness of the system for tracking, assessing, and using information on DCF outcomes to reach agency goals for children and families.

Study Approach

The committee study scope was limited to monitoring and evaluation of DCF that has occurred both within and outside of the agency over the past three to five years. For the purposes of the study, the following definitions were used:

If the DCF monitoring and evaluation system is working well, the quality of agency programs and services should continually improve, benefiting the clients, and justifying the public's investment. When information on actual results is produced, and then used by the agency to guide decisions on policies, operations, and resources, more efficient and effective services for children and families should result. The key research question is: do existing efforts to track agency accomplishments and assess client outcomes result in better services for children and families?

The committee's overall approach to the study, illustrated in Figure 1, had five main components. These components, described in detail in Appendix B, were:

As Figure 1 shows, four main sources of DCF monitoring and evaluation were identified and analyzed: internal efforts; external efforts; outside investigations and reviews; and advisory groups established under federal or state law. Each source is summarized in Table 1.

Methodology

The committee, through its staff, employed two main research methods to study the DCF monitoring and evaluation system. The methods were: key stakeholder interviews; and analysis of monitoring and evaluation reports and related documents produced through all four sources of efforts -- internal, external, investigative, and advisory.

Stakeholder interviews. Committee staff conducted approximately 100 interviews with: division and unit personnel within the DCF bureaus; court monitor staff; Office of the Child Advocate staff; advisory group chairs; federal agency officials; external evaluators; and representatives of providers and advocacy groups. Given the time and resource constraints of the study, not every area office or DCF facility could be visited; however, efforts were made to visit or interview staff from a full range of locations and types of operations.

Document analysis. Committee staff reviewed 126 reports and materials pertaining to monitoring and evaluation of DCF. The study focused on reports and other materials documenting monitoring and evaluation efforts that occurred within the past three to five years (through September 2007). In order to assess the efforts, PRI staff evaluated each document using an internally developed standardized rating system to answer the questions shown in Table 2. Ratings required agreement between two PRI staff who had independently reviewed the documents and then met to discuss their ratings.

Study limitations. Several limitations were encountered during the committee study. First, PRI staff was unable, within the study time frame, to completely assess every effort to monitor and evaluate the Department of Children and Families. For example, while work force development and employee performance evaluation procedures have an important role in supporting quality improvement, the DCF human resources division, the department's Training Academy, and the agency's use of the Performance Assessment and Recognition System (PARS) were not examined.

Additionally, a key department program, foster care, was undergoing a major restructuring at the time of the committee review. Nearly all aspects of foster care monitoring and evaluation were being revamped, so little about efforts in that area could be assessed as it was too soon to know the impact of the new procedures.

Further, not every activity or product within a given type of monitoring and evaluation effort (e.g., licensing visit report, quality improvement plan, advisory group meeting, evaluation report, etc.) could be examined. Due in part to the fragmentation of the DCF monitoring and evaluation system, it is likely that other documents that could have been included in the analysis were not identified. However, the committee believes that the statistical reports, studies, and other documented information about major programs within the department reviewed by PRI staff are a representative sample of monitoring and evaluation currently underway.

There were also situations where multiple monitoring and evaluation efforts were overlapping or occurring simultaneously. This made it difficult to discern which effort led to changes in programs or facilities, particularly when recommendations were similar. Another challenge was attributing outcomes to particular programs, especially when children and families were receiving a variety of services and supports at the same time.

Lastly, it is important to note the rating system employed for the study does not capture the magnitude of a particular monitoring and evaluation activity. The committee staff did not try to assess the significance of each effort or any resulting recommendation. Staff calculated only the percent of recommendations implemented, and did not attempt to rate their impact or importance.

Report Organization

This report consists of seven chapters and a number of appendices. Chapter I provides background information on the Department of Children and Families including descriptions of its mission, goals, major mandates and activities, current organization, and operating budget. The agency's internal efforts to monitor and evaluate its programs and services through various quality assurance and improvement functions such as licensing, case reviews, performance-based contracting, program reviews, risk management analysis, and contracted evaluations are described in Chapter II.

External monitoring and evaluation of DCF activities that are carried out by federal agencies, the courts, the legislature, other state agencies, and national accrediting organizations are described in Chapter III. Outside investigations and reviews, which are conducted by the Office of the Child Advocate, the Child Fatality Review Panel, and the state attorney general's office are discussed in Chapter IV. The many advisory groups responsible under state or federal law for assisting the department in carrying out its mission and assessing its performance are highlighted in Chapter V.

Information about the results of DCF programs and services that was compiled from the monitoring and evaluation reports reviewed for this study is provided in Chapter VI. The program review committee's findings concerning the strengths and deficiencies of current internal, external, investigatory, and advisory monitoring and evaluation efforts are presented in Chapter VII, along with recommendations for improving the DCF accountability system through administrative and legislative changes.

Agency response. It is the policy of the Legislative Program Review and Investigations Committee to provide agencies subject to a study with an opportunity to review and comment formally on committee recommendations prior to publication of the final report. A written response to this study was submitted by the Department of Children and Families and is presented in Appendix A.

Chapter I

Background: Overview of DCF

Connecticut established its consolidated children's agency, the Department of Children and Families, in the 1970s. The legislature combined the state's primary child welfare programs in one organization with the intent of achieving a comprehensive, coordinated statewide system of services for children and families who are at risk because of abuse or neglect, delinquency, mental illness, emotional disturbance, or substance abuse problems.

Since its formation, the department has undergone numerous internal reorganizations, shifts in policy and practice, and almost continuous critical review as it seeks to carry out its complex mission. Background information on DCF is presented in this chapter and includes: an overview of the agency's mission and operating principles; descriptions of its major mandates and associated programs and activities; and a summary of the department's current organization and budget. A brief history of the agency and children's services in Connecticut is provided in Appendix D.

Mission, Guiding Principles and Goals

The purpose and goals of the Department of Children and Families are implied in many of its legislative mandates, although there is no single statutory policy statement about the agency's role. Over time, the department has adopted various mission statements that reflect its broad scope as well as the general evolution of child welfare policy and practice. The current mission of DCF, as stated on the agency's website, is: to protect children, improve child and family well-being and support and preserve families.

DCF management officially adopted six guiding principles for all agency activities based on its mission statement. They include the following overarching principle encompassing the core agency mission and five specific principles intended to guide department practice:

Descriptions of each principle were developed by the department and are provided to all employees and contracted providers, and made available to the general public. A copy of the agency's mission and guiding principles document is presented in Appendix E.

Many goals have been established internally and externally for the Department of Children and Families. At this time, the department does not have a single document containing all goals for the overall agency, its mandate areas, or its specific programs. The information about DCF goals presented below was compiled from a variety of sources, including state

statutes, agency plans and budget documents, mission and goal statements included on the agency's webpage, and interviews with agency staff. A summary is in Appendix F.

Agencywide goals. As its overarching principle indicates, the Department of Children and Families has three main goals for children: 1) safety; 2) permanency; and 3) well-being. These goals, like the agency mission statement and its guiding principles, are not specified in state statute. However, they are implied in many of the laws directing DCF operations.

Safety and permanency as goals for children in the department's care and custody do have a statutory basis. Since 1998, DCF is required by law to prepare a written plan for each child and youth under agency supervision that includes, but is not limited to: “… a goal for permanent placement … which may include reunification with the parent, long-term foster care, independent living, transfer of guardianship or adoption. The child's or youth's health and safety shall be the paramount concern in formulating the plan.” Under another state statute, it is the policy of Connecticut to protect children from abuse, strengthen the family and make homes safe for children, and provide a temporary or permanent nurturing and safe environment for children when necessary.

Other agencywide goals are the department's Positive Outcomes for Children. These 22 positive outcomes mirror the exit plan outcome measure established under the federal Juan F. child welfare consent decree, which are described in detail in Chapter III and summarized in Appendix F. All of the positive outcomes/exit plan outcome measures are focused on safety, permanency, and the well-being of children and families. The agency mission, guiding principles, and positive outcomes are posted throughout the agency and the department has developed and revised an action plan for meeting the performance goals set under the Juan F. consent decree exit plan.

Child protection mandate goals. The department's goals related to its children's protective services mandate are based on state statutory policy directives to protect children from abuse and neglect, plan for permanent placement, and provide comprehensive services to meet the needs of at risk children and their families. They parallel the child welfare goals set for states under federal legislation. Like its agencywide goals, DCF's main child protection goals are: safety; permanency; and well-being.

Specific child protection goals include the 22 outcome measures for the Juan F. exit plan and the closely-related federal outcomes standards for state child welfare agencies. These standards are summarized, with all other major agency goals, in Appendix F.

As noted above, the department has an action plan, with specific strategies and time frames, for achieving compliance with the Juan F. consent decree goals. Progress in implementing the plan is regularly assessed by department management as well as the court monitor. The Juan F. action plan also is incorporated in DCF's Child and Family Services Plan, developed in accordance with federal requirements to outline the agency's child welfare goals and strategies for achieving them. Another document containing department child protection goals is its Performance Improvement Plan that must be prepared and implemented in response to federal Child and Family Services Review (CFSR) findings. (CFSR and other federal requirements are discussed in more detail in Chapter III.)

Behavioral health mandate goals. The goals of the DCF's behavioral health mandate, as defined in the agency's FY2008-2009 biennium governor's budget document, are:

Goals for the department's overall behavioral health system are not clearly set out in statute. However, expected outcomes for the state's major behavioral health reform initiative, the Connecticut Behavioral Health Partnership, and for KidCare, the children's services component overseen by DCF, are described in state law. The statutory goals for KidCare are included in Appendix F.

DCF participates in the statewide mental health planning process the Department of Mental Health and Addiction Services carries out to meet federal mental health block grant funding requirements. DCF prepares the section of the federal plan on children's services, which must describe how the state will implement an organized, community-based system for improving mental health services for children with serious emotional disturbances.

In addition to describing the current state service system, the federal mental health plan must: identify and analyze system strengths, needs, and priorities; and discuss performance goals and action plans for improvement. Although goals and measures are outlined in the children's services section, the document does not appear to be used by DCF or its behavioral health bureau as a strategic guide for providing services.

A two-year strategic plan that sets goals for Riverview, the children's psychiatric hospital operated by DCF, was developed by facility staff with the help of the DCF Bureau of Continuous Quality Improvement in the spring of 2007. A multidisciplinary hospital staff workgroup is responsible for implementation, and progress is reviewed quarterly by facility management, a BCQI representative, and an on-site monitor from the Office of the Child Advocate.

Juvenile justice mandate goals. DCF's juvenile justice goals, as outlined on the agency's Juvenile Services Bureau website, are:

Further, there are specific statutory goals for the state juvenile justice system, which apply to the courts as well as DCF. These are also listed in Appendix F and are generally reflected in the juvenile services bureau goal statement.

A statewide juvenile justice strategic plan was prepared by the DCF Juvenile Services Bureau and the Court Support Services Division of the Judicial Branch with input from many public and private stakeholders.⁵ Issued in August 2006, it sets a vision, mission, 10 guiding principles, and 12 broad system goals in four areas (resource development; coordination, collaboration, and information sharing; data analysis; and work force development).

A workgroup of staff from the DCF Juvenile Services Bureau and the Court Support Services Division, advocates, and parents, with the help of a consultant, operationalized the statewide plan into a results-based accountability format. In addition, DCF and CSSD have jointly developed a plan that both carries out the goals and meets the required service outcomes under the final settlement agreement for the Emily J. juvenile justice class action lawsuit.

Staff at the DCF Connecticut Juvenile Training School (CJTS) developed a strategic action plan for that secure juvenile justice facility in the summer of 2005. In addition to setting six main goals for improving programming and accountability, the plan: defined objectives and outcomes for each goal; included specific action steps for each one; and outlined implementation time frames and responsibilities. Progress was monitored and strategies were revised as needed on a monthly basis until the end of 2006. Strategic planning for CJTS has been put on hold pending a final decision about the facility's future.

Prevention mandate goals. State statute specifically includes prevention services as a DCF responsibility in providing comprehensive services to children and families at risk for abuse, neglect, delinquency, and behavioral health problems. The department's goals for its prevention mandate are set out in detail on the agency webpage and budget document. In brief, they are to:

The DCF prevention office also has adopted seven guiding principles that reflect and expand on the agencywide guiding principles (see Appendix E). Further, the department developed a five-year child welfare prevention plan in 2006 that outlines four goals related to primary prevention and early intervention efforts carried out by the agency. Progress is monitored by the prevention office director, who provides status reports as needed or on request to agency top management.

Major programs. Goals of each of the major department programs within each of the four mandate areas are also listed in Appendix F. The main source for program-specific goals is the agency's budget document. All of the more than 70 specific budgeted programs reviewed have stated goals, although they do vary in specificity, measurability, and relevance.

Many of the program goals are related to outcomes for children and families, usually in very broad terms (e.g., “foster positive youth development”), but a significant number primarily relate to how services are to be delivered (e.g., “receive appropriate services in the least restrictive setting”). Few of the program goals identified by PRI staff incorporate the agency's guiding principles concerning family-centered practice, partnerships, and cultural competence. For the most part, however, they are consistent with the agency's overall and mandate area goals.

Major Duties and Responsibilities

The Department of Children and Families has broad authority and responsibility for protecting and supporting children and families by carrying out state and federal child welfare, juvenile justice, and children's mental health and substance abuse programs. Current state statutes require the department to:

At present, the agency contracts with nearly 200 different private providers for more than 100 types of services for its clients. The Department of Children and Families, as specified in state statute, also operates the state's only public psychiatric hospital for children and youth, two residential treatment facilities, and a secure correctional facility for delinquent boys. The department runs a therapeutic program for troubled youth through its Wilderness School, another facility named in statute. Table I-1 provides a brief description of each DCF facility.

Monitoring and evaluation. The agency has a number of specific statutory charges to monitor, assess, and evaluate its activities. It is required to:

DCF is also required by law to award funding to community service programs in proportion to their effectiveness. Furthermore, it must: evaluate the programs based on analysis of their outcomes and an assessment of service needs; and collect, maintain, and analyze data used for evaluation on an ongoing basis. As noted below in the discussion of the current agency organization, a grants development and contracts division within the Bureau of Finance has responsibility for the DCF performance-based contracting process. The agency's contract monitoring procedures, including how contractor performance information is used for decision making, is described in Chapter II.

Under state statute, DCF must report each year to the governor and legislature on the status of all children committed to the department. It also must establish and maintain a central registry of all children with permanency plans that recommend adoption and, under legislation enacted in 1999, have a system in place to monitor progress in implementing such plans. Information on the status of the various reports, plans, and reviews the department is required by state or federal law to produce, or to receive from service providers and advisory groups, is provided in more detail in Chapter III.

Legislation enacted in 2005 requires the department to seek accreditation from the national accrediting body for public child welfare agencies, the Council on Accreditation (COA). The COA accreditation process and standards and DCF efforts to comply with this requirement are also discussed in Chapter III.

Federal mandates. DCF is the state agency responsible for carrying out a number of federal mandates in areas of child welfare, children's behavioral health, and juvenile delinquency. Currently, the department is subject to oversight by: the U.S. Department of Health and Human Services, Administration for Children and Families; the Substance Abuse and Mental Health Services Administration of HHS; and the U.S. Department of Justice, Office of Juvenile Justice and Delinquency Prevention. It must prepare any required state plans, grant applications, and reports for these federal agencies.

Federal monitoring and evaluation activities related to DCF, such as the Child and Family Services Reviews carried out for all state child welfare agencies, were examined in depth by committee staff. Major federal oversight activities on DCF services and programs for children and families are described in Chapter III.

Advisory groups. More than a dozen councils, committees, commissions, and boards established in accordance with state and federal law have responsibility for advising and assisting DCF or generally providing input to the governor and/or legislature on matters within the department's purview. These groups include:

Program review staff reviewed the roles of these advisory groups in tracking program outcomes, assessing performance, and making recommendations to DCF for service improvements. Descriptive information on the advisory groups is presented in Chapter V.

State Mandate Areas and Programs

The department's many programs and activities are generally organized by its four main statutory mandate areas: child protective services; children and youth behavioral health services; juvenile justice services for adjudicated delinquents; and prevention services. DCF also categorizes its treatment services within each area on a continuum ranging from community-based and in-home services to increasingly intensive out-of-home placements. Like federal and other state children's agencies, providing appropriate care in the least restrictive, most family-like environment possible is the underlying goal of most of the department's efforts.

Each DCF mandate area and the main programs and activities it includes are described briefly below. Figure I-1 summarizes, by area, the many types of services carried out or funded by the department at the time of the committee's review. (The most recent annual data available for DCF activities were for FY 06 while funding information reflects FY 07 appropriations).

Child protection. Efforts to protect children from abuse or injury are the core work of DCF in its role as the state's primary child welfare agency. If children cannot remain safely at home, the department must arrange temporary placements with relatives, in foster homes, or in other residential settings. When reunification with their families is not possible, DCF is required to seek permanent homes for children through other means, such as adoption and subsidized relative care.

Services in the child protection area usually start with the Child Abuse and Neglect Hotline, which is the state's single point of contact for reporting suspected child abuse and neglect. It is operated 24 hours a day, seven days a week by DCF. Reports accepted for investigation are forwarded to trained professional social work staff in the department's area offices. If abuse or neglect is substantiated, the case is assigned to an area office treatment social worker for ongoing services to help ensure the child is safe and the family is supported. DCF received 43,500 abuse and neglect reports, investigated 28,790, and substantiated 7,568 during FY 06.

The treatment social worker is responsible for providing appropriate services to the child and family. If the child's safety can be assured without removal, services may include in-home supports, such as a parent aide or substance abuse screening. If removal is required, out-of-home care is provided. In accordance with federal and state requirements, DCF must develop an initial written treatment plan for every child under its supervision within a specific time frame and treatment plans must be reviewed every six months.

In most cases, children who are removed from their homes are placed in foster homes, all of which are licensed by the department. On average during FY 06, about 3,200 children were living in foster care. If the department determines reunification with the child's own family is not possible, the social worker will try to achieve permanency through other options such as adoption, a subsidized guardianship with a relative, or sometimes, in the case of older children, independent living arrangements. In FY 06, over 1,200 children were living with licensed relative caregivers and over 700 youths were in independent living situations. Also that year, the department finalized 498 adoptions and granted 308 subsidized guardianships.

Behavioral health. DCF is responsible for addressing the behavioral health needs of Connecticut's children by planning, developing, and providing appropriate mental health and substance abuse assessment, treatment, and aftercare services. The agency provides behavioral health services to: children committed to DCF because of abuse and/or neglect; delinquents committed to its custody; and to children and youth with behavioral health needs and no involvement with DCF. State law allows families to apply on a voluntary basis to the department for state funded mental health and substance abuse services for children under 18.

The department operates three behavioral health facilities for persons under age 18 -- Riverview Hospital, High Meadows Center, and Connecticut Children's Place, which were described earlier in Table I-1. It also contracts for residential treatment services as well as a variety of behavioral health treatment programs of lesser intensity, such as partial hospitalization, extended day treatment, child guidance (outpatient) clinics, and emergency mobile psychiatric services.

In FY 06, DCF had 874 children in behavioral health residential placements and the capacity to serve about 2,000 children per year with intensive in-home programs. Riverview Hospital had an average daily census of about 80 children and the department's two other residential behavioral health facilities together served about 260 children during the year.

It is DCF's objective to develop a system of community-based services that allows children with mental health and substance abuse problems to be served in their homes and communities to the greatest extent possible. In collaboration with the Department of Social Services, DCF is implementing the Connecticut Behavioral Health Partnership (BHP), a system for coordinating, financing, and delivering family-focused, community-based behavioral health services and supports mandated by the legislature in 2005 (P.A. 05-280). The children's services component of this effort is called Connecticut Community KidCare.

KidCare. During the 1980s, through federal research projects and pilot programs, states began developing “system of care” models intended to eliminate gaps and barriers in mental health and related services for children with emotional disturbances. Connecticut Community KidCare grew out of efforts made over the past two decades by children's advocacy groups and parents to establish local systems of care in the state.

Under the model, state agencies, local entities including schools, community-based organizations, public and private service providers, and families collaborate at the local level to deliver an array of services to meet children's needs through a coordinated network. The principles underlying the system of care concept are:

Legislation enacted in 1997 mandated a system of care planning process for certain mentally ill or emotionally disturbed children, but required DCF to develop and implement services within available appropriations. Limited resources prevented development of comprehensive local systems of care statewide. However, collaborative service networks did begin to operate in some areas of the state in the late 1990s.

In 2000, DCF, in consultation with DSS, was mandated to develop, jointly fund, and evaluate the integrated, community-based behavioral health service delivery system called KidCare for children who: are in DCF custody; receive DCF voluntary services; or are eligible for the state HUSKY medical care program. The subsequent Behavioral Health Partnership enabling legislation incorporated the KidCare program. The BHP law also established an oversight council responsible for monitoring and evaluating implementation and administration of the new partnership, including its KidCare services.

At present, 25 KidCare community collaboratives have been established with DCF assistance and cover all communities in the state. The collaboratives are local systems of care networks composed of behavioral health and community service providers, parents, and advocates. Available services and operations vary, but the following services are in place statewide: inpatient; outpatient; home-based and emergency mobile psychiatric services; partial hospitalization; and crisis stabilization beds. Children with complex behavioral health needs are eligible for enhanced services that may include: care coordination; comprehensive assessment; intensive home-based services; respite services; extended day treatment; residential treatment; individualized support services; and behavioral management and consultation services.

DCF currently funds about 60 care coordinator positions. These employees work with the community collaboratives to provide assistance to families who need help identifying and procuring appropriate services. In partnership with the families, the care coordinators, who largely act as “service brokers,” are responsible for ensuring individual service plans are developed and implemented to meet children's needs.

In accordance with statutory provisions, the Behavioral Health Partnership contracts with an Administrative Services Organization (ASO) for utilization management services that include clinical oversight, authorizing the correct level of care, and monitoring the types of services used. The current ASO contractor, Value Options, which began operating in January 2006, manages and supports a number of services provided through KidCare. It also generates data for DCF on child-specific service outcomes and service needs by type and area of the state.

Juvenile justice. Primary responsibility for carrying out the state's juvenile justice policies rests with the Judicial Branch. The Juvenile Court and the Court Support Services Division conduct intake and assessment of all juveniles charged with a crime and operate the state's juvenile probation and detention programs. The Judicial Branch also contracts for a variety of community-based services for delinquent youths.

DCF's juvenile justice mandate is limited to the system's most challenging children --adjudicated delinquents committed by the courts to the agency for care and treatment. Of the approximately 14,000 youths under age 16 referred to the Juvenile Court each year, about 1,200 adjudicated delinquents are committed to DCF for secure out-of-home care.

By law, the department runs the state's only secure residential facility for committed delinquents, the Connecticut Juvenile Training School. DCF also contracts with licensed, private providers for various types of residential treatment needed by juveniles committed to its care. In addition, the agency is responsible for:

The Connecticut Juvenile Training School, which opened in 2001 with a 240-bed capacity, now serves an average daily census of about 100 boys. It replaced the Long Lane School, the department's co-educational facility for delinquent boys and girls. Although planned to be a “state of the art” secure juvenile correction facility, CJTS has been the subject of much criticism since it opened.

Citing serious operating problems, the governor announced in August 2005 a plan to close the facility during 2008 and replace it with several small, regional treatment facilities developed specifically for the CJTS population. That plan is currently under review, in part because no funding has been provided for any of the proposed residential facilities for delinquent boys. Another consideration is what facilities and services will be needed when the new law that raises the age of juvenile jurisdiction to under 18 years old goes into effect in three years (P.A. 07-4, June SS).

The agency does not operate any secure facility for delinquent girls at this time. Instead, DCF sends most of the adjudicated females in its care to private residential treatment programs or Riverview Hospital. In some cases, they are placed at the adult prison for women in Niantic.

A study conducted by an outside consultant for DCF in 2005 outlined a plan for new services for girls in the Connecticut juvenile justice system.⁷ The department currently is working on implementation of that proposed service system for girls as well as initiatives to address the strategic plan for juvenile justice services developed in August 2006. As noted earlier, the plan was prepared by DCF, CSSD, and a group of stakeholders convened by DCF, through a process facilitated by the Child Welfare League of America (CWLA).

DCF also is working with the Court Support Services Division, in response to the settlement agreement for the Emily J. lawsuit, to develop and implement the previously mentioned corrective action plan for services that can divert children involved with juvenile court from CJTS and other congregate care placements to community-based services. At present, these services include but are not limited to, special foster care, therapeutic group homes, mentoring, and family-based substance abuse treatment.

Families with Service Needs (FWSN). Connecticut, like many states, enacted legislation a number of years ago to remove status offenses from the definition of delinquency. Status offenses are behaviors considered unlawful only when committed by individuals under a certain age (usually 16), such as failing to go to school, running away from home, and being beyond parental control. The intent of the law was to remove children who have not committed crimes from the juvenile justice system and provide an alternative, treatment-oriented approach for handling status offenses that can promote positive development and reduce recidivism.

Under P.A. 79-567, which was later amended and went into effect in 1981, the state established separate law enforcement and judicial procedures, and a Families with Service Needs program, for juveniles through age 15 committing status offenses. A parallel program called Youth In Crisis (YIC) that extends a similar process and court services to 16 and 17 year olds acting out in non-criminal ways was established under legislation enacted in 2000.

The FSWN and YIC programs allow families and certain other parties to request and receive services from the juvenile court, ranging from counseling and community-based supervision to evaluations and residential treatment, without going through delinquency proceedings. Children found eligible for the programs are subject to court order and can be held in custody for violating such orders at this time.

However, legislation enacted in 2005, which became effective on October 1, 2007, prohibits children adjudicated as FWSNs from being held in a juvenile detention facility or being found delinquent solely for violating a FWSN court order. In addition, before ordering an out-of-home placement or commitment to DCF for a FWSN child, a judge must find there is no less restrictive alternative appropriate to the child's and the community's needs.

In 2006, an advisory group was created by statute (P.A. 06-188) to monitor and make recommendations concerning implementation of the requirements of the FSWN program amendments by DCF and the Judicial Department. Legislation requiring the state to establish a network of family support centers to meet the service needs of juvenile status offenders, a key recommendation from the FSWN advisory group, was passed during the June 2007 special session (P.A. 07-4, June SS).

Prevention. The department's broad prevention mandate is to promote positive development in children, youth, families, and communities. To achieve this mandate, the department funds or directly provides: child abuse prevention services; parent education and support; positive youth development programs; early childhood services; juvenile criminal diversion projects and juvenile review boards; mentoring programs; and public awareness campaigns. Specific DCF prevention programs operating in FY 07 are listed in Figure I-1.

Children's Trust Fund. Preventing child abuse and neglect is the sole mission of the Connecticut Children's Trust Fund, which provides more direct resources for primary prevention efforts related to children and families than the department. The Children's Trust Fund was established by statute in 1983 in response to a national movement to create mechanisms in every state to coordinate and fund community-based child abuse and neglect prevention efforts (P.A. 83-20, June SS).

The fund was administered originally by DCF with input from the Children's Trust Fund Council. In 1997, the legislature made the council an independent agency with the authority to use the resources of the Children's Trust Fund to develop, operate, and fund services and initiatives to strengthen families and prevent child abuse and neglect. The council also administers the Parent Trust Fund, which was created in 2001 to fund programs aimed at improving the health, safety, and education of children by teaching parents leadership skills. Each year, the council must report to the legislative committees on human services, public health, and education concerning the sources and amounts of funds received by both trust funds and how they were administered and disbursed.

The Children's Trust Fund Council is composed of 16 members, including the commissioners of the Departments of Children and Families, Education, Public Health, and Social Services, or their designees and various community representatives appointed by the legislative leadership. Its total estimated budget for FY 07 was nearly $12.1 million, about 94 percent of which was state General Fund money appropriated to the Children's Trust Fund. Other sources were federal grant monies and private donations. Including the executive director, the Children's Trust Fund Council is presently staffed by 18 full-time employees.

Among the prevention programs currently funded by the Children's Trust Fund are: The Nurturing Families Network; Family Empowerment Initiatives; The Help Me Grow Program; Kinship and Grandparents Respite grants; and three initiatives supported by federal child abuse prevention grant funding -- shaken baby syndrome prevention, childhood sexual abuse prevention, and family development skill training for human services agency staff. Responsibility for the Nurturing Families Network, a statewide system of preventive services aimed at high-risk infants originally known as Healthy Families, was transferred from DCF to the Children's Trust Fund Council in 2005.

By law, the council must: develop training, standards, and protocols for Nurturing Families Network providers; develop and implement a request for proposal process to procure required services; establish a data system that provides a variety of standardized provider information; and report to the legislature every six months on progress made by the network. The network is also monitored by a 13-member statutory commission that is, among other duties, responsible for advising the legislature on program outcomes and recommending necessary modifications.

Organization and Budget

At present, the Department of Children and Families organization is made up of a central office with eight main bureaus and 14 service areas statewide. Figure I-2 shows the structure of the agency as of July 2007.

The department is staffed by approximately 3,500 permanent full-time employees. As the figure indicates, the department's Bureau of Child Welfare Services employs the largest number of staff (over 2,100), with almost 90 percent of those positions assigned to the DCF area offices.

The agency's eight functional bureaus are shown in detail in Figure I-3. That figure also shows the four facilities (Riverview Hospital, High Meadows, Connecticut Children's Place, and the Connecticut Juvenile Training School) and the therapeutic camp (The Wilderness School) operated by the department.

Five of the eight DCF bureaus have responsibility for carrying out programs and services related to the agency's mandate areas. The Child Welfare Bureau carries out all child protection functions of the agency from intake through the DCF Hotline to investigation of reports of abuse or neglect, to in-home services and out-of-home placements. Substantiated cases are assigned to treatment social workers in one of the department's 14 area offices. They provide on-going services to support children and families.

The Bureau of Behavioral Health and Medicine has jurisdiction over the department's mental health and substance abuse services, both community-based and those provided at DCF behavioral health facilities – Riverview Hospital, High Meadows, and Connecticut Children's Place. Similarly, the Juvenile Services Bureau oversees the Connecticut Juvenile Training School and all community-based services the department provides for adjudicated delinquents committed to its care.

Two other bureaus, Adoption and Adolescent and Transitional Services, as their names imply, are focused on those particular aspects of the department's broader child welfare, behavioral health, and juvenile services mandate areas. Programs of the adolescent services bureau, which include the Wilderness School program, are aimed at providing DCF youth with the skills, supports, and resources they need to succeed as adults.

Responsibility for the fourth DCF mandate area is centered in the Prevention Division of the agency's Prevention and External Affairs Bureau. There are three central office prevention staff, and prevention liaisons have been appointed within each DCF area office and facility. The prevention staff in the community assist in shaping area prevention plans through monthly meetings.

In addition to the Prevention Division, the department's External Affairs Bureau includes the recently reorganized Office of Ombudsman that is responsible for receiving and investigating inquiries and complaints about DCF services and facilitating a resolution that is in the best interests of children. The bureau's research unit primarily focuses on conducting independent reviews of all critical incidents and child fatalities, and developing findings and recommendations to improve agency practice, policy, and management based on those reviews.

The Bureau of Continuous Quality Improvement encompasses all agency divisions and units involved in monitoring, evaluating, and correcting and improving department performance. Much of the program outcome and management information currently available for the department is produced by BCQI. The bureau's licensing and other compliance functions as well as its review and reporting efforts, all of which are central to this study's focus, are described in detail in Chapter II.

The bureau also encompasses the department's Training Academy. In accordance with the Juan F. consent decree, the department established a training academy to identify and provide training needs for DCF staff in 1997. The academy, which is operated by the agency, has 19 full-time staff including a training director. A 22-member advisory group consisting of representatives of the agency, educational institutions, service providers, and foster and adoptive parents consults with the DCF training director and reviews the department's annual statewide training plan and reports.

The Finance Bureau of the department handles all accounting, auditing, central business operations, and other fiscal functions and has responsibility for DCF's automated statewide child welfare information system (LINK) and all other agency computerized databases and information systems. The bureau's Grants Development and Contracts Division oversees all external contracting for services and is responsible for the agency's performance-based contracting process.

Operating budget. For FY 07, the DCF budget totaled more than $820 million, most of which came from the state General Fund. Federal funding accounted for less than 3 percent of the total budget, about $22.3 million. The agency also received an estimated $999,000 in private funds for the current fiscal year.

The allocation of funding among the department's four mandate areas and for overall agency management for the current fiscal year is shown in Table I-2. Child protective services, which include the 14 area office operations and the majority of DCF staff, account for about half of the agency budget. About one-third of DCF funding is allocated to the behavioral health area, which encompasses three of the department's residential facilities. Another 8 percent is spent on the juvenile justice area including CJTS operations, and less than 1 percent goes for the department's prevention programs and services.

Management services, which account for less than 5 percent of the total DCF budget, include all the administrative infrastructure functions that support the agency's programs and facilities for children and families. In addition to all fiscal, human resources, legal, and contracting activities, agency management consists of policy setting, ombudsman, and other external affairs functions, as well as the planning, evaluation, and quality assurance efforts that were the focus of the program review committee study.

Figure I-4 compares the portion of the department budget expended on each major category -- child protective services (CPS), behavioral health (BH), juvenile justice (JJ), prevention, and agency management -- in FY 07 with those for FY 99, the time of the committee's last program review of the agency. The information provided in the figure is only an initial look at agency spending patterns since the items included in the various categories may not be completely comparable. For example, in some years, certain funding for the agency's automated information systems was included as an agency management cost while at other times it was included with child protective services expenditures. Consistent definitions of the spending categories in the DCF budget have not been developed.

However, based on available data, shifts in the overall allocation of DCF resources have occurred during this time period. Funding for the CPS mandate still makes up the largest portion of the agency budget, and prevention spending remains 1 percent or less of total expenditures. The percentage of the DCF budget allocated to the behavioral health and, to a lesser extent, the juvenile justice mandates, has increased while the percentage of spending on agency management has dropped.

Chapter II

DCF Internal Monitoring and Evaluation Activities

Internal efforts by the Department of Children and Families to monitor and evaluate progress in achieving the goals of its programs, mandate areas, and agency overall occur primarily within its Bureau of Continuous Quality Improvement. Monitoring and evaluation activities also are carried out by the Bureaus of Prevention and External Affairs, Finance, and Child Welfare, as indicated in Figure II-1. Especially within the Bureau of Child Welfare, DCF's 14 area offices play a key role in the agency's efforts. Additionally, the department contracts with outside organizations to supplement its internal analytical and evaluation resources. This chapter describes major components of the department's internal monitoring and evaluation activities.

Bureau of Continuous Quality Improvement

Key monitoring and evaluation responsibilities for DCF are carried out by the Bureau of Continuous Quality Improvement (BCQI). The previous Bureau of Quality Management, first created in the late 1980s, had similar responsibilities. The current BCQI, established in 2003, is staffed by a bureau chief and 104 staff, some of whom are assigned to the 14 area offices to conduct Administrative Case Reviews.

The bureau is in charge of various department initiatives to assess and improve performance and the services received by children and families. Efforts range from specific case reviews to analysis of multiple cases. The bureau includes five units and divisions: 1) Program Review and Evaluation; 2) Internal Quality Improvement; 3) Training Academy; 4) Planning, Policy and Program Development; and 5) Licensing (see Figure II-2).

Program Review and Evaluation Unit. By law, the Department of Children and Families has been required since 1975 to “conduct studies of any program, service or facility developed, operated, contracted for, or supported by the department in order to evaluate its effectiveness.” (C.G.S. Sec. 17a-3(a)(6)). The Program Review and Evaluation Unit (PREU) carries out this mandate with eight staff. Until 1995, there was a focus primarily on paper reviews (i.e., checking that policies were in place). Since then, the emphasis has shifted to outcomes, and the quality and effectiveness of programs. The following are highlights of some of the PREU activities shown in Figure II-2.

Ad hoc studies. The Program Review and Evaluation Unit conducts ad hoc studies of any program, service, or facility to evaluate its effectiveness. Ad hoc studies may originate from: problems indicated by patterns found among agency data (e.g., Hotline reports); or concerns raised by DCF workers, the Office of the Child Advocate, the Court Monitor, or parents. Recent ad hoc study topics have included Riverview Hospital, therapeutic group homes, and residential programs.

Restraint and seclusion data. Restraint and seclusion data are submitted on a monthly basis to PREU by all programs, agencies, or institutions currently licensed, contracted, funded, or operated by the department. The unit aggregates the data by program type and reports on the number and duration of physical restraints, non-serious and serious injuries, and other relevant information.

Extensive in-state facility monitoring. On occasion, a program within Connecticut is found to have significant issues that require extensive monitoring. One program, for example, is currently slated for closure and, until the program closes, PREU staff will visit the facility on a daily basis. Children and staff are observed, formal monitoring procedures followed, and levels of responsibility and actions for PREU monitoring staff outlined.

Private Non-Medical Institution Initiative (PNMI). Non-medical facilities that are licensed by DCF to provide behavioral health services for children, such as therapeutic group homes and residential treatment centers, participate in the Connecticut Medicaid Private Non-Medical Institution program (PNMI). Enrollment as a PNMI provider occurs through the execution of a Medicaid Provider and Billing Agreement by the Connecticut Department of Social Services, DCF, and the performing provider of PNMI services for children. This enrollment allows reimbursement from the federal government of 25 percent of the allowable cost of therapeutic group homes and residential treatment centers.

Three staff from PREU monitor and evaluate the PNMI requirements of the therapeutic group homes and residential treatment centers. The PNMI review is a paper review, examining such areas as whether the appropriate person signed the proper documents and whether an activity occurred within a given time frame. The Program Review and Evaluation Unit expanded these PNMI reviews to include qualitative areas. Program Review and Evaluation Unit staff also conduct site visits to residential and therapeutic group homes for PNMI compliance.

Recent focus has been on the therapeutic group homes. There are 33 items that reviewers examine in the case records that fall into the categories of: general; need for services; treatment planning; clinical service delivery; residential service delivery; and DCF reporting. PNMI requirements, for example, include facility development of an individualized treatment plan within 30 days of admission; treatment plans that are developed in conjunction with DCF, the child, and the child's family if possible; and specific behavioral health goals and objectives within every treatment plan.

After the record review, PREU staff provides the group home or treatment center with verbal feedback as well as a form documenting any corrections required. The intent of the review is to ensure that required structures and procedures are in place. In the instance of therapeutic group homes, every record is examined during visits that occur every 1-2 months until corrections have been completed.

Out-of-state facility evaluations. The Program Review and Evaluation Unit also authorizes and evaluates out-of-state programs. In spring 2007, there were approximately 280 children in out-of-state programs. A priority of DCF several years ago was to reduce the more than 500 children in out-of-state programs at that time. The steps that typically occur during the out-of-state facility evaluation process are:

During the PREU site visit, more than 70 items are reviewed, covering such areas as:

Internal Quality Improvement Division. The Internal Quality Improvement Division is staffed by 39.5 FTE positions. Some staff are located in the DCF central office, and others are in the 14 area offices. The role of the division is to encourage and support area office and facility quality improvement efforts. The functions of the Internal Quality Improvement Division are carried out by two units: Administrative Case Review, and FOCUS Review. Each is now described.

Administrative Case Review Unit. The Administrative Case Review unit, or ACR unit, has 28.5 FTE staff located in area offices. Federal regulations require that independent case reviews occur every six months, assessing such areas as the appropriateness of placement, safety, permanence, and well-being. Specifically, their responsibilities include a review of treatment plans, examining such areas as the way in which treatment goals are defined, determining who is responsible for implementing the treatment plan within a given time frame, and producing quarterly reports for the Office of the Court Monitor. Case reviews may occur more frequently when circumstances require a new treatment plan to be prepared.

Each ACR takes approximately 1.5 hours. They are conducted in the area offices and mandatory participants include the administrative case reviewer, the DCF social worker whose case is being reviewed, and his/her supervisor. Any member of the Area Resource Group, a community consultant, support-staff worker, and/or community service provider who has participated in any aspect of the case in the seven months prior to the review are also required to participate in the ACR, as well as an adoption specialist as needed. The parents of children without terminated parental rights, foster parents, and foster children themselves who are age 12 or older are also invited to the ACR.

At least two weeks prior to the ACR, a written case summary and copy of the current and previous treatment plans are submitted to the reviewer by the DCF social worker. This information is then shared with the remainder of the review participants at least one week prior to the review. The case record is also reviewed prior to the ACR and available at the ACR itself.

Specifically, the process goals of the Administrative Case Review are to:

The outcome goals of the Administrative Case Review are:

At the ACR, each case plan component is discussed fully. Any obstacles to achieving treatment plan goals, and recommendations for eliminating obstacles are identified and formally contained in a written report called an “FYI.” An FYI, or “For Your Information,” is an email that is sent to all that are involved in the case, including the social worker, social worker's supervisor, and area office manager. The FYI identifies areas of strength as well as areas for improvement related to safety, permanency, child and family well-being, placement, treatment planning, case work practice, and/or child welfare system performance. Common examples of reasons to send FYIs include safety issues, inappropriate placement, or no discharge plan.

A member of the ACR staff will then review the child's case record within 45-60 days of the ACR to assess whether the issues contained in the FYI were resolved. If not resolved, the program supervisor is notified and must respond within 30 days to the Director of the Division of Internal Quality Improvement and appropriate Division program director.

In addition to the case-specific ACR process, there is also a systemic review process. Systemic problems may relate to a particular program or facility. All reports related to significant events, sentinel events, critical incidents, Hotline, etc., are reviewed bi-weekly by the Service Evaluation and Enhancement Committee (SEEC), a cross-section of agency staff from the Bureau of Child Welfare, Division of Internal Quality Improvement, Risk Management, and others. Sentinel events are one-time occurrences that can be either critical (i.e., related to child abuse or neglect) or significant (a non-child abuse or neglect related concern). For example, treatment planning was identified as a systemic problem. It was found that there was difficulty obtaining consistency on the treatment goals and objectives. The solution was to conduct relevant training and increase time devoted to reviewing automated data, reading hard copy material on cases, developing more qualitative information, and promoting strengths-based language in the treatment plan.

FOCUS Review Unit. The Focus Review Unit has 9 full-time and 2 part-time staff, and a durational program director assigned to area offices to assist with quality improvement plans and initiatives, setting priorities, and developing strategies. The unit also conducts statewide studies, such as a recent study of exit planning for youth 18 years and older transitioning from DCF to DMHAS and DDS. During 2007, there were three FOCUS staff temporarily assigned to assist the Juan F. court monitor with the monitor's comprehensive case review (see Chapter III).

Area Office Quality Improvement Teams. Each area office has an Area Office Quality Improvement (QI) Team, formed in 2004 in response to the Juan F. consent decree exit plan requirements. The makeup of the QI teams varies by DCF area office, with some including the area office director. There is no department policy on the membership of QI Teams.

A QI team's main duty is to develop and implement a Quality Improvement (QI) plan for its area office. Plan goals and activities may focus directly on the Juan F. exit plan outcome measures or indirectly, through topics such as safety, adolescent issues, social worker support, case practice improvements, and diversity sensitivity. It is up to the QI team, the Bureau of Child Welfare (under which the area offices operate), and other program bureau chiefs to look at why goals are not being reached, and the changes needed to address deficiencies.

When the Internal Quality Improvement Division staff identifies a problem at an area office, they work with the office to develop corrective actions. For example, the Internal Quality Improvement Division helps with Juan F. exit plan measures that may be especially challenging for a particular area office, such as repeat maltreatment or re-entry. Area offices differ on performance, having different needs and resources.

Training Academy. The Training Academy has 20 staff and conducts all major statewide training initiatives, but may hire consultants to provide additional training. The Academy anticipates as well as responds to skill and knowledge needs identified during monitoring and evaluation activities. For example, the QI Teams, discussed earlier, may identify staff training needs based on the results of their monitoring of the 22 Juan F. exit plan outcome measures.

The Training Academy is currently working with the Child Welfare League of America⁸ to help identify what areas are needed for child case worker training based on the National Child Welfare Competencies, a nationally recognized curriculum.⁹

The department requires newly hired social workers, about 300-400 annually, to participate in training that takes approximately one year to complete. Pre-test and post-test measures on participants are collected and changes to the training offerings modified accordingly.

Division of Planning, Policy and Program Development (PPPD). This newly formed division was created from a combination of several previously existing functions in Spring 2007. Approximately 15 staff work within the division's three units: Risk Management; Decision Support Unit; and Policy and Accreditation Unit. The idea behind the division is that through analytically monitoring and evaluating risk, and making decisions with the best possible data, services to children and families can be improved. According to the division director, staffing for the analytical requirements of the new division is a challenge to meet given the limitations of the relevant current job classifications.

Risk Management Unit. The Risk Management Unit has four full-time staff. The unit acts as an agencywide conduit for information including critical incidents, flagging areas of concern, notifying the appropriate DCF unit for action, and subsequently monitoring the department's response. Upon request, the unit compiles ad hoc reports (e.g., 9-1-1 calls made by providers).

Critical incidents are those occurrences related to suspected abuse and/or neglect involving:

All critical incidents are contained in a Critical Incident Database with information obtained from the Incident Report, most often completed by DCF Hotline staff. There is additional information included in the Critical Incident Database that is taken from the Critical Incident Update and the department's statewide automated child welfare system. Critical incident reports are distributed within DCF and externally to the Office of the Child Advocate and Office of the Court Monitor for the purpose of information sharing and possible subsequent changes in department policies and procedures.

Critical Response Teams. In some instances, a Critical Response Team may be formed to respond to an issue, and the Risk Management Unit is responsible for convening the team. The Critical Response Team, composed of different DCF staff relevant to the issue at hand, takes an in-depth look at the reasons behind an event and, working with the provider or facility, develops recommendations.

Trend analyses. The Risk Management Unit also conducts trend analyses on topics such as type of incident and area of concern. The unit identifies incidents that rise above an acceptable threshold (e.g., number of restraints in a particular program). These “rate-based incidents” may be shared with the department's Service Evaluation and Enhancement Committee and/or another area of DCF for further action.

Service Evaluation and Enhancement Committee. The SEEC comprises staff from Policy, Licensing, Program Review and Evaluation, Risk Management, all bureau chiefs, representatives from the commissioner's office, ombudsman, contracts, fiscal, and Hotline. It is the central mechanism for tracking available results information to “red flag” patterns of poor performance or undesirable outcomes. Another SEEC role is to identify issues that require proactive intervention from areas throughout the agency.

SEEC meets every two weeks to look at aggregated critical incident data, significant events, and other program information (e.g., ombudsman complaints, staff turnover rates, or providers with financial trouble) compiled by Risk Management Unit staff. With the help of that staff, it looks at trends, identifies problem programs or providers, and develops ways to take corrective actions as early as possible.

Decision Support Unit. The purpose of the Decision Support Unit is to bring together DCF staff with an interest and expertise in outcome measurement and performance data. The unit consists of five full-time staff and is currently led by the Division Director of Planning, Policy and Program Development. Responsibilities of the Decision Support Unit include:

A key responsibility of the Decision Support Unit is to identify and develop requests for data needed to make management and quality control/improvement decisions. The work includes shaping inquiries, determining report specifications, understanding data concepts, and assuring consistency. The unit works with Information Systems staff, Division of Grants and Contracts Management staff, and other CQI staff.

Finally, the unit oversees the Results-Oriented Management system (ROM). Managed by the University of Kansas, the system takes some of the LINK data and “scrubs” it for subsequent use in outcome reports. DCF ROM reports have been available since FY 06. Managers are able to look at office-specific information, monitoring frequency of supervision, number of children in unlicensed foster homes, etc. The ROM is also used to track performance goals and Juan F. exit plan outcomes. The data for the majority of Juan F. exit plan outcomes are now produced by ROM.

Policy and Accreditation Unit. The Policy and Accreditation Unit, with six staff, is responsible for all DCF policy manual revisions and additions. The unit also has duties related to accreditation, ROM, the Juan F. exit plan, and several federal requirements such as the Adoption and Foster Care Analysis and Reporting System (AFCARS) and the Child and Family Services Reviews (CFSRs).

Other responsibilities of the policy unit include: playing a key role in the Council on Accreditation accrediting process; and managing the Closed Records Unit, the unit responsible for physically getting files to area offices when a closed case is reopened (has one full-time and three part-time staff).

Licensing Unit. The Licensing Unit has nine licensing inspectors and is responsible for assessing compliance with federal, state, and local regulations, laws, and ordinances. The Licensing Unit processes licensing applications (new and renewal), makes site inspections (scheduled and unscheduled), approves and monitors correction plans, and makes recommendations related to licenses, including temporarily closing admissions, reducing capacity, suspension of license, and revocation of license. The licensing function is considered a quality assurance effort as it is an assessment of regulatory compliance. There are five types of in-state licenses for which the unit is responsible. These are:

Out-of-state agencies. Through the Interstate Compact Unit, the Licensing Unit approves out-of-state agencies that place children with Connecticut families. Approval of out-of-state child placing agencies requires receipt of current license, program description, and contact with DCF's Licensing Unit. Two-year approvals are received; no site visits are required. There are more than 100 out-of-state agencies that are currently approved.

New in-state provider. When a provider is selected from a pool of applicants responding to a request for proposals (RFP), the Licensing Unit will send the new program requesting licensure an application packet with all the requirements. The licensing requirements are based on the regulations.

When the application packet is returned to the Licensing Unit, it is assigned to a licensing inspector. The unit examines each specific requirement and reviews the facility to ensure compliance with the requirement. Requirements pertain to staff, physical plant, and policies such as the use of restraints. A program cannot become licensed until all of the requirements are in place, usually taking a program three to six months to complete.

A provisional license, good for 60 days, is then granted for a new program. Up to six consecutive provisional licenses can be received. During the provisional licensing period, the Licensing Unit is monitoring the program, assessing, for example, whether the program is fully staffed, treatment plans are being written for all children in the program's care, fire drills are taking place, and administration of medication is being done according to regulation.

Site visit schedule. Once all systems are fully operational in a new program, then a regular license is issued. The regular license is good for two years from the date of the first provisional license and must be renewed every two years thereafter (except for permanent family residences). The license renewal involves two to three licensing inspectors visiting the program for two to three days. Permanent family residences, which are being phased out by DCF, receive licenses for one year. There are currently four permanent family residences, which are a hybrid between a foster home and group home, with as many as 13-16 children in a single home. Two of the permanent family residences serve significantly handicapped children, one is a respite shelter, and one is for temporary shelter.

Additionally, child care facilities receive site visits from the Licensing Unit every three months, usually one licensing inspector for a half-day visit. During the brief site visits, the inspector typically focuses on one or two regulation areas. By the end of the two-year cycle, the quarterly visits will have reviewed all of the regulation areas.

Non-compliance. Any time a site visit occurs and a program is found to be out of compliance with a regulation, the facility is required to submit a corrective action plan to the Licensing Unit within 30 days. Corrective action plans are monitored with the assistance of an ACCESS database that tracks license expiration dates and site visits. A license will not be issued until the areas of noncompliance covered by the corrective action plan have been addressed. Adherence to the corrective action plan is monitored through quarterly site visits.

Site visits may also be triggered by a complaint coming into the hotline. While the hotline team will investigate initially, the Licensing Unit will also make a site visit if there is a problem with regulation compliance.

Licensure requirement differences. There are specific requirements in regulation regarding DCF licensure of particular programs or facilities. For example, outpatient psychiatric clinics for children are required by regulation to have a comprehensive and well-designed plan for measuring and improving performance (R.C.S.A. 17a-20-59).

Other Internal Monitoring and Evaluation Efforts

Performance-based contracts. Within the Bureau of Finance is the Grants Development and Contracts Division, responsible for approximately 300 contracts DCF has with outside program providers. There are 19 staff in this division, who manage over 100 different types of services procured through these 300 contracts. It is possible for one provider to have just one contract, with that one contract covering multiple types of services.

Within the standard contract template is a section pertaining to expected performance from the provider. Periodically, the providers are required to report progress on meeting these service expectation goals. Fiscal performance is monitored by the Grants Development and Contracts Division, and program implementation is monitored by program leads within each of the bureaus. The Program Review and Evaluation Unit, with the assistance of a vendor (Advanced Behavioral Health), aggregates provider-supplied information about the clients served, including demographics, length of service, and reasons for service discontinuation, into various quarterly reports. Use of performance-based contracting can be both a means to monitor purchased services as well as to evaluate overall provider performance, and serve as a consideration in future contract awards.

Special Investigations Unit. The Special Investigations Unit (SIU), within the Bureau of Child Welfare Services, is staffed by a Program Supervisor and eight social workers. The SIU is responsible for investigating child abuse or neglect allegations regarding a child in congregate care, foster care, or of a DCF employee.

Structured decision making. Another endeavor of the Bureau of Child Welfare Services is called structured decision making. This initiative is intended to provide the tools and reports necessary to help guide the decision making of child protective services workers relative to assessing risk and safety. Structured decision making is also intended to reduce the variability in decision making during the child protective services process.

Best Practice Unit. Approximately two years ago, a Best Practice Unit was planned but never got off the ground. According to the Chief of the Bureau of Behavioral Health and Medicine, the unit's intent was to focus on residential care. One aspect of that focus was to address fragmentation between the Bureau of Behavioral Health and Medicine and the Bureau of Continuous Quality Improvement, a problem because staff from both bureaus needed to come together quickly to respond to critical incidents at congregate care facilities. However, the behavioral health bureau chief reported that there were challenges hiring staff for the unit as well as clearly defining the purpose of the unit.

There is now a new Residential Treatment Unit that will provide services to the approximately 20 contracted residential treatment centers that will include some of the original ideas behind the Best Practice Unit. Each Residential Treatment Unit staff member is expected to have two residential facilities for which to act as a best practice “guru.” Half their time will be spent in the field at residential facilities, getting to know the staff, children, and programs.

Administrative Service Organization (ASO). The KidCare ASO, Value Options, has the capability to track services received by children in the KidCare system of care. There are 3.5 staff within the Behavioral Health Bureau assigned to manage the ASO function. Examples of the more than 200 reports produced include: length of time to answer the telephone; length of time for providers to get questions answered; list of children residing in a residential facility; hospital discharge delays; daily census reports; and aggregate reports to identify trends.

CJTS performance-based standards. For the past two years, the Connecticut Juvenile Training School has monitored and evaluated its stability through the performance-based standards (PbS), a self-improvement and accountability system for youth correction and detention facilities that is used by more than half the states in the U.S.

The PbS sets national standards for the safety, education, health/mental health services, security, justice, and order within facilities. No more than four facility improvement plans can be developed at one time based on the results of PbS.

Riverview Hospital. Internally, Riverview Hospital has established multiple committees that monitor and evaluate the hospital's services, which are overseen by a Quality Assurance manager. Established committees include: Environment of Care Committee, Staff Development Committee, Infection Control Committee, Medical Records Committee, Pharmacy and Therapeutic Committee, and Patient and Family Education Committee. Each year the committees set goals for the coming year and then monitor progress throughout the year. Additionally, each committee issues quarterly reports that track progress in reaching the goals. These reports are presented to the central office as well as the facility advisory committee for feedback.

As a result of the joint ad hoc program review conducted in 2006, Riverview Hospital developed a strategic plan containing both long and short term goals. An Implementation Committee made up of 25 members of the Riverview staff representing all disciplines and units of the hospital was established to help meet the goals in the plan. In addition, an independent monitor who reports to the Office of the Child Advocate, and the Director of the BCQI Division of Planning, Policy and Program Development review activities related to the Riverview Hospital Strategic Plan.

Research and Development Unit. The Research and Development Unit within the Bureau of Prevention and External Affairs consists of one full-time director. Due to the small staff size, the Child Welfare League of America (CWLA) is used extensively to assist with its efforts.

Internal child fatality reviews are a way to evaluate the causes of such tragedies. Under the Research and Development Unit, they are conducted with up to three key CWLA team members and include a case analysis of the facts (who, what, when, where, and how). The Research and Development Unit also examines what happened as it relates to practice, whether, for example, staff worked together as a team. In recent years, the Office of the Child Advocate has also been invited to participate in this internal child fatality review.

In addition to individual child fatality reviews, the Research and Development Unit also aggregates information from several reviews, identifying patterns and making recommendations. In 2005, such an aggregate review was done for 13 child fatalities.

Office of the Ombudsman. Approximately three years ago, the consolidated Office of the Ombudsman was formed to serve children, foster and adoptive parents, providers, and citizens. Protocols were established, staff size expanded, and an information system developed to track inquiries. There are currently eight staff, some of whom are part-time and assigned to various facilities.

The office receives and investigates inquiries and complaints relating to DCF, including those submitted through grievance boxes located at each of the DCF facilities. The office monitors and evaluates these complaints and tries to resolve issues in the best interest of the children involved. The Ombudsman's Office received 3,788 inquiries in 2006, with 1,000 of the inquiries coming from York Correctional Institution and Manson Youth Institution, both correctional facilities.

The Ombudsman also makes site visits to residential treatment centers and group homes when time permits. The Ombudsman also solicits feedback via letters to residential treatment centers and group homes.

Contracted Monitoring and Evaluation

The Department of Children and Families periodically uses outside organizations to supplement its internal evaluation resources and to obtain special expertise that cannot be found within the agency. Contracting out for evaluation services also can lend credibility to the results by providing an independent assessment of a program's strengths and weaknesses.

Some of the outside evaluations commissioned by DCF have been required as a condition of federal funding or as part of the agreement for using a proprietary service model. Independent reviews of agency programs also have been directed by the legislature. For example, an outside evaluation of the agency's implementation of the KidCare program, which was carried out by the Child Health and Development Institute of Connecticut (CHDI), was a statutory mandate.

In addition to program-specific evaluation projects, the department also contracts for a variety of ongoing monitoring and evaluation services. These services range from conducting child fatality reviews to managing parts of the agency's child welfare data. With the Department of Social Services, DCF also has contracted with a private firm (Value Options) to serve as the ASO for the state's Behavioral Health Partnership. Monitoring and reporting on utilization of, and need for, mental health and substance abuse services by children and families are among the duties of the ASO.

Both types of contracted monitoring and evaluation services are described in more detail below. Information on project-specific contracts for the past five years was developed by the department at the request of the program review committee staff. Efforts by some of the commissioner's staff to start tracking contracted studies began around FY 03. However, as there is no central control over the products resulting from outside monitoring and evaluation efforts, the list provided for this study is not considered exhaustive.

Through interviews with agency managers, advisory groups, and private providers, program review committee staff became aware of several external reviews of DCF programs that were not included on the department's list of contracted evaluations. In addition, some monitoring and evaluation efforts may be carried out as part of other, broader contracts that bureau chiefs, facility heads, or other agency managers develop for the programs they administer.

One example is the foster care division's contract with the Connecticut Association of Foster and Adoptive Parents (CAFAP) for foster parent training and support services. That contract includes a provision for CAFAP to carry out exit interviews with caregivers leaving the system to obtain their feedback about the agency's administration of the program. The foster care division also has an agreement with the University of Connecticut to conduct opinion surveys of the general public and providers regarding strengths and weaknesses of state foster care.

At this time, decisions about contracted evaluations are not coordinated throughout the agency and there are no standard criteria for determining when outside services are needed. Like all agency contracted services, however, authorization of an external evaluation or monitoring project is subject to the approval of top management and procurement is overseen by the central office contract, fiscal, and legal staff.

Recent contracted evaluations. Over the past five years, DCF has contracted for at least 15 different evaluation projects. Information about each one is summarized in Table II-1. On average, the department contracted for three to four external evaluations per year during this period. The cost of the evaluations included in the table ranged from $8,000 to over $1 million each, depending on the scope and time frame of services. Overall, the total value of the external evaluation services provided through these contracts was more than $2 million.

The majority of the contracted services were for studies related to behavioral health issues. This is due to two main factors. First, as part of its ongoing KidCare initiative, and through its participation in implementing the Connecticut Behavioral Health Partnership, the department has developed and expanded a number of new community-based mental health and substance abuse programs for children and families. Second, many of the new behavioral health intensive in-home services mandate provisions for outside evaluations of their effectiveness.

Most of the evaluations shown in Table II-1 extend over a period of several years, although a few short-term reviews (about one year) have been conducted. A variety of entities are involved in performing evaluations for the department including: non profit providers, such as the Village for Families and Children; academic institutions and research centers, like Yale University and the Connecticut Center for Effective Practice (CCEP) of the Child Health and Development Institute of Connecticut (CHDI); and national consultants and research organizations like Matrix and the Casey Foundation.

In three cases, a report was not produced as part of the contract. Instead, training and other workforce development or technical assistance was provided to department staff as a result of the evaluation. Also, copies of reports regarding two other evaluations (regarding flex funds and mentoring) could not be provided by the department. At the time of the committee's study, several evaluations were still in progress or had just released final reports.

The department's arrangement with CHDI and its affiliated research entity, CCEP, differs from the other contracted evaluation services. In many ways, CHDI and the Center serve as an independent research resource for the department on children's health and mental health care matters.

Under a competitively awarded, five-year personal service agreement, the institute provides DCF with broadly defined evaluation and training services related to the state's KidCare behavioral health reform initiative. The institute designed the multi-year evaluation to be done in phases, focusing first on implementation and baseline measures, then system capacity and responsiveness issues, and finally on changes in children's outcomes.

From June 2003 through January 2007, CHDI issued six evaluation reports related to KidCare as part of this agreement and two subsequent amendments made to it. These studies examined the Emergency Mobile Psychiatric Services and Care Coordination components of the KidCare system and measured family satisfaction with services received. Currently, the institute is completing a first-year evaluation of the Behavioral Health Partnership, which will include a set of performance indicators to be used as the system “report card.” CHDI also has organized and funded on-going training in wraparound service delivery for KidCare local providers and care coordinators.

CHDI. The Child Health and Development Institute is the operating arm of the Children's Fund of Connecticut, a public charitable foundation established in 1992 to improve the healthy development of Connecticut's children. CHDI carries out the fund's mission by combining direct funding for research, policy analysis, advocacy, and technical assistance that emphasizes family-centered, comprehensive physical and mental health care.

The institute works in partnership with Connecticut hospitals, universities, state agencies including DCF, and other organizations on a variety of initiatives intended to improve the quality of care for all children in the state. These range from strategic planning for early childhood programs, to evaluations of the effectiveness of juvenile offender treatment therapies and various DCF KidCare services.

In 2002, the institute created CCEP, a partnership of two state agencies, DCF and the Court Support Services Division of the Judicial Branch, and two higher education institutions, the Psychiatry Department of the University of Connecticut Health Center and the Yale University Child Study Center. The center's overall mission is focused on developing, training, disseminating, evaluating, and expanding effective practice models for children with serious emotional, behavioral, and addictive disorders. Core funding for CCEP's work comes from the Connecticut Health Foundation. Additional support has been provided from the Children's Fund of Connecticut, the Tow Foundation, and DCF.

One of CCEP's primary activities is working with DCF to identify and implement cost-effective, evidence-based, behavioral health treatment services for children and youth. Most recently, the center just completed a study with recommendations for the redesign of children's Emergency Mobile Psychiatric Services as a way of addressing the inappropriate use of hospital emergency departments.

Other contracted services. DCF also contracts with outside organizations for ongoing monitoring and evaluation services in several areas. Examples of these types of contracted services are summarized in Table II-2.

Two of the five contracts shown in the table, the Chapin Hall longitudinal foster care data analysis project and the ROM services provided by the University of Kansas, are indirectly related to monitoring and evaluation efforts. They primarily provide DCF with data management services, technical assistance, and advice regarding analysis and performance measurement. Both, however, are critical to the department's ability to assess compliance with Juan F. exit plan outcome measures and federal child welfare performance indicators, as well as to develop related corrective actions and program improvements plans.

The Center for the Study of Social Policy carries out the Technical Advisory Committee (TAC) function required as part of the Juan F. consent decree exit plan. The committee's responsibilities include providing expert advice and technical assistance on methodologies for outcome measures, best practices, and the latest child welfare research. In addition, the TAC occasionally evaluates agency operations. Only one written TAC evaluation report, a 2002 assessment of DCF's quality assurance system, has been issued. Feedback is more often given informally, through memos or meetings. Most recently, the committee arranged for a consultant to help DCF staff develop an agencywide practice model and work on the results-based management system the department calls its Accountability framework. ¹⁰

About three years ago, the Child Welfare League of America was hired to assist the department with its internal child fatality review process. Fatality reviews can be viewed as case-specific evaluations of agency policies and practices. To date, CWLA has conducted over 30 in-depth reviews of deaths and other critical incidents involving children and youth in DCF care.

Value Options was awarded the contract to serve as the ASO for the state's Behavioral Health Partnership in January 2006. Its main roles are authorization and utilization review. However, the ASO also has responsibilities for evaluating the existing behavioral health service network and identifying need for new or expanded programs as well as for assessing the efficiency and effectiveness of clinical work.

DCF Information Systems

Currently DCF has multiple automated data systems that serve its different bureaus, programs, and facilities. The system that supports social workers' case management practice, LINK, became operational in 1996. LINK serves as the federal statewide automated child welfare information system (SACWIS). By law, LINK is required to support the reporting of data to the Adoption and Foster Care Analysis Reporting System (AFCARS) and the National Child Abuse and Neglect Data System (NCANDS). LINK also serves as the primary source of data for the Juan F. outcome measures. It does not, however, provide useable data to supervisors or the necessary analytic capability to improve outcomes. Instead, DCF uses the ROM system from the University of Kansas for reporting capabilities using LINK data. ROM extracts data from LINK and provides management reports on outcome measures with the capability to generate reports by office, unit, and worker.

In addition, there are multiple systems throughout the agency that support day-to-day operations as well as multiple data collection systems. Since LINK only serves as a child protection case management system, each DCF run facility has developed its own database to provide automated client information and support daily operations.

The Bureau of Behavioral Health and Medicine has various data collection systems. For example, the ASO has a database that tracks the services received by children in the KidCare system of care and produces approximately 200 reports. The reports include such information as: length of time to answer the telephone; list of children residing in a residential facility; hospital discharge delays; and trend analyses. In addition, behavioral health providers submit data that is contained in the Behavioral Health Data System. However, there are concerns over the quality of the data, the quantity of required data elements, and the department's intended purpose for collecting all the information.

Risk Management, Licensure, and the Bureau of Adoption each have standalone access databases for their respective areas. In addition, the Office of the Ombudsmen has an ACT system for tracking complaints and the hotline utilizes ACD Navigator for entering and recording calls. Finally, Juvenile Services bureau has CONDOIT for keeping records on their client population. All of these systems and databases act independently and are not connected under one single technology infrastructure.

Chapter III

DCF External Monitoring and Evaluation Activities

External monitoring and evaluation activities of the Department of Children and Families are carried out by a number of entities -- federal agencies that fund and review various state children's services and programs, such as the Children's Bureau of the U.S. Department of Health and Human Services, state and federal courts, and the state legislature. Certain department facilities and functions are also assessed periodically through independent, national accrediting processes and federal or state regulatory activities. This chapter describes the major external efforts to monitor and evaluate DCF during the past three to five years. Chapter VI contains the results of these efforts.

U.S. Department of Health and Human Services Children's Bureau

The U.S. Department of Health and Human Services Administration for Children and Families has a Children's Bureau that monitors state child welfare services in part to assist Connecticut and other states in achieving positive outcomes for children and families. Figure III-1 on page 69 shows the relationships between the reporting systems, reviews, and annual federal reports described in this section.

Children's Bureau Reporting Systems. There are three federal and state reporting systems administered by the Children's Bureau:

The Adoption and Foster Care Analysis and Reporting System. AFCARS is a federally mandated system that collects case level information on every child in foster care for whom state child welfare agencies have responsibility for placement, care, or supervision and on every child adopted under the auspices of the state's public child welfare agency. AFCARS also contains information about foster and adoptive parents. Descriptive foster care information, for example, includes:

The AFCARS data are used in a number of federally required child welfare service assessments and reports, discussed later. During each federal fiscal year, states are required to submit adoption and foster care data twice, once for the period October 1 through March 31, and again for the period April 1 through September 30. The AFCARS reporting requirement is over a decade old.

AFCARS Assessment Review. The purpose of the AFCARS assessment review, which is a case file review conducted by the federal Children's Bureau, is to verify that the electronic data submitted to AFCARS matches the data that is in the state child welfare agency's paper files. Connecticut's most recent review occurred in July 2001.

Because adoption records are sealed in Connecticut, only foster care case files were included in the review. (The federal review team did not require Connecticut to unseal the adoption records due to time constraints in scheduling the review.)

The reporting period reviewed in July 2001 covered April 1, 2000 through September 30, 2000. As would happen in any other state, the minimum tasks required to correct deficiencies found in Connecticut's AFCARS data were included in an AFCARS Improvement Plan. Once DCF makes all of the required changes to the information system, the system will be tested again. The AFCARS Improvement Plan is considered to be completed once the federal government and the state agree that the quality of the data is acceptable. No additional on-site reviews will occur unless DHHS hears of concerns about the quality of Connecticut's data.

The National Child Abuse and Neglect Data System. NCANDS is a voluntary national data collection and analysis system developed to meet the requirements of the federal Child Abuse Prevention and Treatment Act (CAPTA) (P.L. 93-247) as amended by the Keeping Children and Families Safe Act of 2003. CAPTA directed the DHHS secretary to establish a national data collection and analysis program for available state child abuse and neglect reporting information. The information is gathered once a year, with the first system report based on 1990 data. For FFY 2005, a total of 49 states submitted case-level data to NCANDS. Specifically, CAPTA requires each state to report¹¹:

NCANDS data are also used in a number of other federal reports, discussed later. In addition, the data are used for an annual report on child maltreatment, which is published each spring, and for the Program Assessment Rating Tool (PART), which is “a systematic method of assessing the performance of program activities across the Federal government^.”12

Additionally, Children's Bureau programs funded from the CAPTA Basic State Grant use NCANDS data for two program assessment ratings: improve states' average response time between maltreatment report and investigation; and reduce the percentage of children who are repeat victims of maltreatment within six months. Children's Bureau programs provided by funds from the Community-Based Child Abuse Prevention (CBCAP) State Grants use NCANDS data for one measurement for their program assessment rating: decrease the rate of first-time victims per 1,000 children.

NCANDS data consist of aggregated data and more detailed case-level data. The aggregated data include information on:

The more detailed, case-level data contains the following categories of information:

NCANDS also includes the:

The Statewide Automated Child Welfare Information System. SACWIS refers to any of a variety of electronic case management systems designed for adoption and foster care social workers to process child protective services and child welfare information on a statewide basis. As a federally supported project, the primary goals of SACWIS are:

Although information in SACWIS is used to produce AFCARS reports, not all states have fully operational automated information systems. Federal funding may be available to develop a SACWIS, and those states with SACWIS are required to use the system to collect the data required by AFCARS. All but seven states are participating in SACWIS and approximately 30 are fully operational. Connecticut has an operational system called LINK, which is not yet SACWIS-compliant.

The Department of Children and Families and the state's Department of Information Technology (DOIT) have shared responsibility for the LINK system, which became operational more than a decade ago in July 1996. The LINK system replaced the earlier Case Management System (CMS) that had been in use since the early 1980s.

LINK contains several core elements:

LINK has four primary functional areas: service management; provider management; financial management; and common application functions.

Service management. This function gives workers and supervisors the tools to better manage service delivery including child protective services reporting, investigations, risk assessment, voluntary services referrals, case maintenance, and case closing. The management of legal actions, placement, case participant information, medical information and adoption are also included within service management.

Provider management. This function has tools to manage service providers, licensing, contract, and foster homes. Support of the licensing and certification processes, and documentation of home providers is included within provider management.

Financial management. This function contains the business aspects of the department including the processing of payments and voucher requests. The function also supports the “Random Moment Time Study” (RMTS), which documents and gathers costs associated with administering and operating child welfare programs. The information gives the department information about the amount of effort workers spend on various activities associated with child welfare case maintenance. The RMTS study includes observing employees activities on an individual basis during random time intervals.

Common application functions. These functions are required by more than one of the LINK subsystems and cover areas such as person management, worker assignment, approvals, checklists, ticklers, and security. LINK system help is also contained within the common application functions.

Additionally, LINK enables DCF to produce key management reports, including the number of children in different types of placement at a particular point in time, caseload trends, and performance statistics that are submitted to the Juan F. court monitor.

The LINK system does not include information about participants in programs of the Bureaus of Behavioral Health and Medicine as well as Juvenile Services, unless the participants are dually committed. (i.e., children involved both with Child Protective Services or Child Welfare Services as well as Behavioral Health and Medicine and/or Juvenile Services.)

SACWIS Assessment Review (SAR). The federal Children's Bureau conducts an assessment of how well a state's SACWIS is functioning approximately one year after it becomes operational. This assessment review includes a one-week, on-site review. Approximately six weeks prior to the review, states provide the Children's Bureau with background information through completion of a SACWIS Assessment Review Guide. The on-site review includes a system walk-through and interviews with users of the system.

Following the SAR site-visit, a detailed exception report is generated that gives the state a comprehensive description of the review team's findings. Only after the state has either modified its version of SACWIS, or developed an acceptable corrective action plan, is the review process considered finalized.

The first SAR for Connecticut occurred in 1998, and the most recent occurred in September 2006, with the purpose to evaluate progress toward completing the LINK system (the Connecticut version of SACWIS). As part of the visit, the team assessed areas covered in the Connecticut SACWIS Assessment Review Report (SARR). Specifically, the monitoring visit was intended to:

Federal monitoring systems. In addition to the reporting systems just described, the federal government also has two monitoring systems: 1) Child and Family Services Reviews; and 2) Title IV-E Foster Care Eligibility Review. DHHS is also required to annually report to Congress on child welfare outcomes based on national standards.

Child and Family Services Reviews. The CFSR is a results-oriented, comprehensive monitoring system that was first implemented in fiscal year 2001. DHHS developed this review to fulfill a mandate in the Social Security Amendments of 1994 (see section 1123A of the Social Security Act) to promulgate regulations for reviews of state child and family services programs that operate under Titles IV-B and IV-E of the Social Security Act.

The Child and Family Services Reviews assess state performance in the areas of safety, permanency, and child and family well-being. Performance is compared against national standards developed from state adoption/foster care data (AFCARS) and abuse and neglect data (NCANDS). The Child and Family Services Reviews also assess seven systemic factors that affect the agency's ability to achieve the national standards.

The CFSRs are based on six central principles and concepts:

CFSR Statewide Assessment. The CFSR process occurs in two phases. The first phase is the statewide assessment, during which the state analyzes its child welfare data and practice. It involves external partners or stakeholders and the Children's Bureau staff. The process is guided by completion of the Statewide Assessment Instrument. The Statewide Assessment Instrument has five sections:

Data to complete the Statewide Assessment Instrument come from information from state adoption/foster care data (AFCARS) and state abuse and neglect date (NCANDS).

The CFSR Statewide Assessment information is used to:

CFSR On-Site Reviews. After the CFSR Statewide Assessment, the second phase of the CFSR is an on-site review, during which federal and state teams examine outcomes for children and families by assessing child welfare practices, and assessing systemic issues through stakeholder interviews.

The CFSR On-Site Review includes: 1) a random review of foster care and in-home case records; 2) interviews with children and families receiving services; and 3) interviews with community stakeholders (e.g., courts, community agencies, foster families, caseworkers, service providers). The purpose of the on-site review is to evaluate progress in achieving the qualitative CFSR outcomes. The site visit lasts for one week.

The CFSR on-site review is conducted by a team of federal and state representatives (including external partners). Connecticut's team included court personnel, youth, parents, and staff from provider agencies. Members may serve as reviewers of case records or assist in the development of a possible subsequent Program Improvement Plan.

CFSR Outcomes. There are seven CFSR outcomes used to assess state performance, covering the areas of safety, permanency, and child and family well-being. (Two of the seven CFSR outcomes (Safety Outcome #1 and Permanency Outcome #1) are derived from aggregated AFCARS and NCANDS data, and have national standards associated with them.) The seven CFSR outcomes are:

In addition to these seven CFSR outcomes, there are seven operational, systemic factors identified that may affect an agency's ability to achieve the outcomes. The seven systemic factors examined are:

States are rated on a scale from 1 to 4 for each of these systemic factors, with criteria for rating each factor found in the CFSR Procedures Manual. Ratings of “3” or “4” indicate “substantial conformity” and ratings of “1” or “2” indicate “not in substantial conformity” with the factor. The assessment on these seven systemic factors is based on ratings on 22 indicators. The state is rated on each indicator as having either a “strength” or an “area needing improvement.” According to the Children's Bureau website, states are rated on:

National standards. The first round of CFSR reviews of every state, the District of Columbia, and Puerto Rico was conducted between FY 2001 and FY 2004. The national standards for the first round of state reviews were based on relative, rather than absolute, performance across states for each of the six CFSR data measures related to safety and permanency goals. The standard was set at the 75^th percentile based on state NCANDS and AFCARS data from earlier reporting periods (see Table III-1 CFSR Round One column for those national standards).

The second round of reviews is scheduled to occur between FFY 2007 and FFY 2010. The national standards for the second round are higher than those for the first round, and are based on 2004 state performance levels. Connecticut is scheduled for its second review in FFY 2008, on September 22-26, 2008.

In regard to all states, the Children's Bureau reported¹³ that:

Child Welfare Outcomes Annual Report to Congress. The DHHS is required by federal law to produce an annual Report to Congress on child welfare outcomes. (The DHHS is behind in producing these reports; as of October 25, 2007, the 2004 report information still had not been published). These reports provide information about state performance on the seven national child welfare outcomes as well as population characteristics to provide a context for the information. The population characteristics include:

While both the Report to Congress and CFSR contain information on the national child welfare outcomes, CFSR is considered more of a monitoring system, providing more comprehensive information about state performance. The Report to Congress is limited to automated data contained in AFCARS and NCANDS.

Both the Report to Congress and CFSR, however, share similar goals of informing Congress, DHHS, the states, and the public about performance in achieving desired outcomes for children in the public child welfare systems, and identifying areas needing improvement. The DHHS, therefore, connected the Report to Congress and CFSR by establishing national performance standards for six of the measures contained in the Report to Congress:

These national performance standards have been modified somewhat, and the changes to the CFSR highlight the differences.

Changes to CFSR. Following the first round of CFSR reviews, DHHS contracted with a consultant to study the process and make recommendations. One adopted recommendation describes all data performance measures from a positive perspective. Another adopted recommendation replaced the six existing CFSR single data measures (used to set national standards) with four data composites and two single measures. The composite scores were scaled from 50 to 150, with higher scores indicating better performance.

The composite scores combine related measures of permanency already contained in AFCARS, and have the following advantages:

In order to be considered in substantial compliance during the first round of CFSR, states were required to substantially achieve the outcome standard in 90 percent of reviewed cases. For the second round, the percent that must substantially achieve the outcome increased to 95 percent.

Table III-1 shows the changes in the two national child welfare standards and outcomes that occurred between round one and round two. Note that the two national standards are based on state performance in FY 2003 and FY 2004.

Title IV-E Foster Care Eligibility Review. Federally, the Foster Care Program was authorized in 1980 under Title IV-E of the Social Security Act, with the intent of assuring proper care for children requiring placement outside their homes, in a foster family home or institution. The Foster Care Program provides funds to states to help them with foster care maintenance for eligible children, administrative costs, training for staff, foster parents, and staff of child care institutions providing foster care services. In SFY 2007, Connecticut received $106 million for reimbursement for foster care and adoption expenses.

A child is eligible for this financial benefit based on a federal requirement that the child was removed from a family that qualified for, or would have qualified for, cash assistance. The Title IV-E Foster Care Eligibility Reviews also determine whether the state had a valid basis for ensuring that appropriate payments were made on behalf of eligible children, homes and institutions, as specified in regulations to the Social Security Act (45 CFR §1356.71 and §472).

As with the Child and Family Services Reviews, the Title IV-E Foster Care Eligibility Review team consists of federal and state representatives. A minimum size sample of 80 cases is randomly drawn from a state's AFCARS data submission. Using the Title IV-E Onsite Review Instrument, the cases are examined for specific federal eligibility requirements, such as:

Federally-Required Child Abuse Prevention and Treatment Plan

CAPTA State Plan. Another source of external monitoring involving the federal government is the five-year Child Abuse Prevention and Treatment Act (CAPTA) State Plan DCF is required to submit to DHHS. This plan is integrated by DCF with other required federal plans, i.e., the Child and Family Services Plan, which any state receiving federal child welfare funds must file and have approved, and the Independent Living Plan, related to adolescents in foster care.

Under CAPTA, nine potential areas through which a state's child protective services system may be improved are identified. In its last two CAPTA plans, covering 1995-2004, DCF focused on implementing four of those areas:

For its current CAPTA plan for 2005-2009, DCF chose to focus on three of the four areas, dropping the training opportunities area of focus due to funding limitations.

Annual Progress and Services Report. State child welfare agencies are required to submit annual progress and services reports (APSRs) to DHHS for programs and efforts that receive CAPTA and other funds (i.e., Title IV-B (foster care), Chafee Foster Care Independence (CFCIP) and Education and Training Voucher (ETV) programs). Each annual progress and services report requires documentation of progress made since the last progress report, including efforts related to the Child and Family Services Reviews Program Improvement Plans.

Examples of programs/activities funded by CAPTA in 2007-2008 include:

Examples of programs/activities funded by Chafee Foster Care Independent Living Services (for youth in secondary programs) in 2007-2008 include:

Examples of programs/activities funded by the Education and Training Voucher (ETV) (for youth in post-secondary programs) in 2007-2008 include:

The following must be included for each of the programs in the Annual Progress and Services Report:

Other aspects described include collaboration, program support, tribal consultation, monthly caseworker visit data, state plan requirements, and financial and statistical information reporting.

The Annual Progress and Services Report is reviewed by DHHS, and the department responds to any clarifying questions. Examples of recent clarifications required by DCF were to:

DCF Federal Grant Funding

Children's Bureau Funded Specific Programs. Another source of external monitoring are the requirements of specific DHHS Children's Bureau federal grants to submit progress and data on a quarterly/annual/periodic basis. One DCF program funded by a grant from the Children's Bureau under the Adoption Opportunities category is the “Helping to Achieve Permanent Placements for Youth (HAPPY) Program.” As with other grants funded by the Children's Bureau, DCF is required to submit progress reports every six months to the Children's Bureau.

SAMHSA Funded Programs and Block Grants. The Substance Abuse and Mental Health Services Administration, also part of the U.S. Department of Health and Human Services, funds four DCF programs in part or fully by SAMHSA. They are:

There are annual reporting requirements for each of these SAMHSA grants that include plans and accomplishments. Additionally, progress reports and fiscal reports are due every six months. The progress reports require an update on project goals, barriers, and evaluation efforts. SAMHSA site visits occur every two years. As an example, a description of the Building Blocks for Bright Beginnings grant and its monitoring requirements is provided.

Building Blocks for Bright Beginnings. The Building Blocks for Bright Beginnings SAMHSA grant is entering its third year of funding and progress is evaluated by the Yale Consultation Center. The Building Blocks program was established in cooperation with DCF and the Southeast Mental Health System of Care in partnership with Families United for Children's Mental Health.

The purpose of the grant is to enhance the existing coordinated network of mental health and human service providers, community members, and families by providing comprehensive mental health and other services for children birth through five with social emotional challenges and their families from Southeast Connecticut, supported by evidence-based practices. Building Blocks is also expected to expand the existing system of care in an effort to increase the capacity and expertise around early childhood mental health with science-based information on screening, assessment, referral, and early intervention.

SAMHSA provides program funding through the Community Mental Health Services Block Grant Program. This funding has the goal of improving mental health services through the support of existing public services and encourages the development of community-based care for individuals with serious mental disorders. The funding supports grassroots initiatives that are creative and cost-effective.

Progress reports and fiscal reports are required every six months, and reapplication for

the award occurs every March. Additionally, Building Blocks team members are required to attend two national meetings/conferences per year, and site visits occur every two years.

Mental Health Block Grant. Additionally, there is the Mental Health Block Grant from SAMHSA to DMHAS. Approximately $1.3 million of the block grant goes to DCF to supplement respite, FAVOR training, suicide prevention, and maintenance and expansion of the mental health system of care.

There are data reporting requirements for the Mental Health Block Grant, including an annual Youth Services Survey for Families. This 10-15 minute telephone survey conducted by the University of Connecticut Department of Public Policy is given to caregivers of children who have received services from the behavioral health system. The survey collects information in the following seven areas: cultural sensitivity; access to care; participation in treatment planning; outcomes; functioning; social connectedness; and general satisfaction.

Juvenile Justice Grant Programs. Several DCF programs are funded by the Office of Juvenile Justice and Delinquency Prevention (OJJDP) within the U.S. Department of Justice Office of Justice Programs. Funding for the programs comes from the Juvenile Accountability Block Grants (JABG), which are administered by the State Relations and Assistance Division of OJJDP. The goal of the JABG program is to reduce juvenile offending through the use of accountability-based programs that focus on both the offender and the juvenile justice system. Connecticut's JABG grant focuses on programs that have the goal of reducing drug-related and violent crime, and also improving the functioning of the criminal justice system.

Judicial Oversight

The state Judicial Branch, through its various child protection and juvenile delinquency responsibilities, has a broad role in overseeing children in the care and custody of the Department of Children and Families. The federal courts also have been given a direct role in monitoring and evaluating various aspects of DCF performance under agreements resulting from settlements of class action lawsuits brought against the state concerning children's services.

Three recent federal class action lawsuits that have influenced the department and the services it provides are: Juan F., Emily J., and W.R.¹⁴ Federal court monitoring efforts related to each are discussed below.

Juan F.

The federal class action lawsuit filed in 1989 on behalf of nine children in DCF care, including a 10-year boy named Juan F., has had a major impact on DCF policies, programs, and resources. Settlement of the lawsuit was reached by the parties and approved by the federal district court for Connecticut in January 1991.¹⁵ It resulted in a 120-page consent decree and an accompanying set of 12 policy manuals. Together, these documents contained approximately 1,200 mandates for the agency to meet in order to be found in compliance and to end court supervision.

The majority of the original consent decree provisions were process-oriented requirements related to key agency functions carried out for the Juan F. population of children, such as intake, treatment, health management, family training and support, staff training, contracting, and quality assurance. The Juan F. class includes: a) all children in the care, custody, or supervision of DCF as a result of being abused, neglected, or abandoned or being found at risk of such maltreatment; and b) all children about whom the department should know are or will be abused, neglected, or abandoned, or are or will be at serious risk of such maltreatment.

The focus of the Juan F. consent decree and related compliance monitoring, therefore, is on children and families involved in DCF's protective services system and the programs and child welfare services they need (i.e., investigations and assessment, case management, family preservation and support, foster care, and adoption, as well as related therapy and behavioral health treatment, medical care, and education). Requirements of the Juan F. settlement do not apply to children committed to DCF solely for delinquency reasons, or children and families receiving services voluntarily from the agency.

Monitoring history. At first, agency compliance with the Juan F. consent decree was monitored by the same three-judge panel that mediated the settlement. In December 1992, an independent, full-time court monitor was appointed to replace the mediation panel as overseer of consent decree implementation.

The Juan F. court monitor, who reports directly to the trial judge, must “… work actively with the parties to ensure timely and effective compliance of the provisions of the Consent Decree….”¹⁶ Major responsibilities include: submitting periodic compliance reports to the court and the parties, hearing requests from the parties for modifications of the settlement agreement, and trying to resolve disputes without the need for court intervention. Under the court monitoring order and its subsequent revisions, the Juan F. monitor must have timely access to DCF data, documents, staff, and other information, and may retain staff and consultants necessary to perform all duties required under the consent decree.

Between 1995 and 2001, a number of revisions to both the consent decree content and the monitoring process were negotiated. By 1999, the monitor and the parties began discussions concerning an exit plan that would: a) shift the focus of consent decree compliance from procedural requirements to positive outcomes for children and families; and b) lead to termination of court oversight of DCF. In February 2002, the court approved a transition and exit plan that contained an 18-month time frame and 38 areas, including 28 outcomes with specific performance standards, for measuring agency compliance with the provisions of the Juan F. consent decree.¹⁷

A year later, the court found noncompliance in fundamental exit plan areas of caseload reduction and staffing improvements and only modest progress in improving other performance outcomes. The court monitor was ordered by the trial judge in October 2003 to prepare a revised exit plan.¹⁸ The order also established a three-member task force, comprising the Juan F. court monitor, the secretary of the state Office of Policy and Management, and the DCF commissioner, which was given management authority over the entire department.

Initially, the parties identified over 100 possible goals and measures for a new exit plan process. Over about a nine month period, through discussions conducted under the direction of the court monitor, the parties reached agreement on 22 required areas of compliance, as well as definitions of outcomes and methods for measuring them. A group of experts (e.g., judges, child welfare professionals, and foster parents) assembled by the court monitor served as an advisory group during this process.

As with all court monitoring matters, final approval over the plan's outcome measures and methodologies, which are described in more detail below, rested with the monitor and, ultimately, the trial judge. A revised exit plan containing the 22 outcome measures currently used for determining Juan F. compliance was drafted by the court monitor in December 2003 and approved by the court in July 2004.¹⁹ Table III-2 lists each measure with its full definition.

The revised Juan F. exit plan measures are aimed at improving child welfare practice and the quality of department decision making to ensure better outcomes for children. Many parallel the federal Child and Family Services Reviews and related child welfare outcome goals used to assess state child protection agencies, discussed earlier in the chapter. They were intentionally selected by the parties and the court to promote consistency among the dual monitoring efforts and avoid duplicative reporting.

In response to the revised exit plan, DCF prepared an agency action plan called Positive Outcomes for Children (POC) that was finalized in May 2004.²⁰ The POC plan identified: the key steps DCF needed to take to reach the goal set for each of the 22 outcome measures; the agency staff person responsible for coordinating implementation of each step; and the expected time frame for implementation.

In October 2005, a revised monitoring order for the Juan F. consent decree was approved, which incorporated the appointment of a new court monitor and formation of an expert Technical Advisory Committee (TAC) to assist the monitor's office.²¹ About the same time, the plaintiffs, asserting noncompliance with the July 2004 plan, initiated negotiations through the new court monitor concerning what they considered to be the two fundamental indicators of how well children and families are being served by DCF -- effective treatment planning and meeting service needs.

Based on the parties' discussions, with advice from the TAC and the court monitor, changes to the exit plan case review methodology were proposed to assess better agency compliance with the Juan F. goals related to treatment plans and needs met (Outcome Measures 3 and 15). A modification of the exit plan containing a new methodology for reporting on these two measures was approved by the court in July 2006.²² The department and the court monitor

first implemented the revised case review method for the exit plan quarterly report for the third quarter (July 1 - September 30, 2006) results.

The 2006 revised Juan F. Exit Plan also reflects an agreement reached by the parties concerning: 1) a department action plan to address key components of case practice related to meeting children's needs (Outcome Measure 15); and 2) new monthly reporting of point-in-time and longitudinal data on placement and permanency issues. On March 12, 2007, the department submitted the required plan for improving Outcome Measure 15 performance, with action steps, strategies, and implementation time frames.

The department's first monthly “point-in-time” report was issued on March 21, 2007. It covered all Juan F. children in out-of-home placement as of that date (almost 3,400) and included a number of statistics on their characteristics (e.g., age when they entered and when they exited care, or permanency goal) and their permanency status (e.g., legally free, termination of parental rights not filed and why, in care for more than 15 months, no permanency goal after 2, 6, or 15 months in care). As required, the April 2007 monthly report contained a variety of new information on foster family recruitment and retention, such as data on inquiries made, applications filed, licenses issued and revoked, and children on waiting lists.

Under the revised exit plan, sustained compliance -- defined as compliance for at least two consecutive quarters (a six-month period) -- with all 22 outcome measures is required before the court will consider asserting the state to be in compliance. In addition, total compliance must be maintained throughout the decision making process concerning termination of court jurisdiction over DCF. The court monitor must present findings and recommendations about ending supervision to the federal district court, based on a review of a statistically significant sample of case files and other necessary measurements. The parties must have an opportunity to be heard by the monitor before those findings and recommendations are presented.

Court monitor activities. The main activities carried out by the Court Monitor's Office to track DCF compliance with the Juan F. consent decree include:

Additional monitoring procedures for the March 2007 Juan F. Action Plan were developed recently by the court monitor with the assistance of the TAC. They were finalized after review by the parties in June 2007. The new process incorporates provisions for: analysis and presentation of data extracted from the new monthly reports; monitoring implementation of the strategies and initiatives contained in the plan; and targeted case reviews of specific issues related to certain populations of children, for example, those age 12 and under who are living in congregate care, or children with long-term foster care as their permanency goal.

Data analysis. An appendix to the department's May 2004 Positive Outcomes for Children corrective action plan specifies the sources and methods for collecting data related to the 22 Juan F. exit plan outcome measures. Approval by the court monitor is required before the department can make any changes to the methodologies or information systems used to report on Juan F. outcome measures.

The court monitor currently measures compliance with all but two exit plan outcomes based on an analysis of quantitative data submitted in quarterly reports prepared by the DCF exit planning staff within the Bureau of Continuous Quality Improvement. Initially, automated data were available for only a few measures and accuracy was a serious issue. The monitor required DCF to supply quarterly outcome information compiled both from its central computerized child welfare case management system (LINK) and from original paper records until data reliability could be verified for each measure. Furthermore, modifications of the LINK system by an outside consultant have been required in order to produce data in an automated report format (called Results Oriented Management or ROM reporting) for more than half of the exit plan measures.

At present, the department reports on four measures using information produced directly by LINK and on 12 more through the supplemental ROM reports that are based on LINK system data. The DCF exit planning staff develops the quarterly data necessary for Exit Plan Outcome Measures 20 and 21 by reviewing the case records for all youth discharged from agency care each quarter. Automated reporting for these measures is planned for the future. Data related to foster parent training, Outcome Measure 13, also are manually compiled from the department contractor responsible for providing that training (the Connecticut Association for Foster and Adoptive Parents).

The court monitor's office conducts its own case reviews to develop the information necessary for assessing compliance with two outcome measures that require a qualitative approach -- treatment plans (#3) and needs met (#15). The data gathered through these case reviews related to other outcome measures is also compared with the quarterly outcome data submitted by the department as another check on the reliability and validity of the agency's automated information.

Case review process. As noted earlier, revisions to the methodology for conducting case reviews for these measures were adopted in July 2006. The modifications added additional review elements and a provision for the reviewers to attend DCF meetings concerning treatment planning (e.g., Administrative Case Reviews, Treatment Planning Conferences, or Family Case Conferences) held for each case reviewed.

Under the revised methodology, which went into effect for the third quarter of 2006, a random sample of approximately 70 DCF child welfare cases, stratified to reflect the caseload distribution across area offices, is selected each quarter for in-depth review. The reviews are conducted by 10 to 12 experienced social work professionals assigned to two-person teams. Individuals from the court monitor's review staff are paired with current DCF case workers as review teams. Each team member separately completes an individual assessment of treatment planning and needs met for their assigned cases, according to an agreed upon protocol. Teammates meet to jointly arrive at a final score for the case.

When agreement cannot be reached, teams request review by a supervisor and assistance in developing an overall score. If consensus still cannot be reached, the case is submitted to the court monitor for review and final determination of the scoring.

Each case review involves examination of all automated case record (LINK) documentation, concentrating on the most recent six-month period. Narratives prepared by DCF case managers, treatment planning documentation, investigations information, and any narratives prepared by foster care providers are reviewed and scored based on set criteria. Reviewers are trained and provided with definitions and standards for evaluating treatment plans to help ensure consistency and validity. The full process, including attendance at any DCF meetings on the case, typically takes between seven to 12 hours to complete.

Quarterly reporting and follow up. Information developed from the analysis of data submitted by the department and gathered through the case review process is compiled in a quarterly reporting document by the court monitor and assistant court monitor. The report submitted by DCF is attached and both are provided to the judge prior to public distribution. The court monitor meets with the judge to discuss the report, noting progress made during the quarter and any areas of concern, but the judge does not edit or make changes to the report contents. The monitor also will informally let the parties know the overall findings before the official release of the final report for the quarter.

Following the report's release, the court monitor meets with both DCF staff and the plaintiffs to discuss the results in detail. At present, attention is focused on issues underlying outcomes related to treatment planning and needs met but there is always feedback between the parties and the monitor on all areas covered by the exit plan. The court monitor notes one of his key responsibilities is ensuring information is shared among all the parties and everyone involved has an opportunity to provide input.

Based on the findings presented in the quarterly report, the department will make adjustments to policies and procedures and develop action steps to improve compliance with the exit plan outcomes. Although the court monitor is not required to approve corrective actions planned by DCF, he generally is involved in agency meetings and staff discussions related to exit plan activities, and sometimes brings along TAC members to provide advice and assistance.

The department's exit planning staff oversees efforts to improve compliance with all outcome measures, under the direction of the agency's former chief of staff, now the commissioner's director of strategic initiatives and organizational development. In the past, various staff throughout the agency have been assigned lead responsibility for action steps on particular outcomes and progress has been tracked through a “status of work” section of the DCF quarterly report to the court monitor. This process began with the department's POC plan and has been further developed and refined over time. Beginning with the quarterly report for the first quarter of 2007, the department is focusing its status reporting on implementation of the Juan F. Action Plan strategies concerning treatment planning.

However, the agency's website currently has up-to-date information on the compliance status of each measure and performance reports also can be accessed for each area office. The exit planning staff maintain a variety of quality improvement information on-line for agency staff and the public (quarterly reports, forms, guides, practice standards, policy updates, contacts for assistance, etc.).

Also, the court monitor's recently implemented Juan F. Action Plan includes: regular meetings with DCF staff, the plaintiffs, provider groups, and other stakeholders to examine the impact of the plan's action steps; selected site visits; targeted reviews of critical elements in the plan; ongoing analysis of monthly point-in-time and other selected data reports; and attendance at a variety of meetings on specific initiatives outlined in the plan.

According to the court monitor, targeted reviews will look at specific populations of children with permanency and placement issues to develop information to promote better practice and better inform the parties about the results of agency programs and services. The revised case review methodology developed for Outcome Measure 15 will be applied to these targeted reviews and several additional qualitative methods (interviews with children and families, for example) will also be incorporated.

Other activities. In addition to regular data analysis and case reviews related to quarterly progress reporting, the Office of the Court Monitor periodically carries out studies on topics of particular importance to agency compliance with the Juan F. consent decree. In the past, the court monitor has done program reviews of the DCF quality assurance function, adoption practices, investigations functions, and use of flexible funding.

Most recently, the court monitor participated with the Office of the Child Advocate and the department's continuous quality improvement bureau in the ad hoc study of Riverview Hospital. Ongoing activities related to this project include review and monitoring of the hospital's new strategic plan, updates with DCF staff, facility visits, analysis of facility data, attendance at advisory group meetings, and meetings with the unions representing the hospital staff.

The court monitor also undertakes occasional comprehensive case reviews of large samples to develop qualitative and quantitative information on overall compliance progress and on each exit plan measure. For example, the court monitor's office conducted a comprehensive targeted review of approximately 2,500 cases and issued a report in September 2007 with the final results, in addition to the regular quarterly exit plan report. The monitor's basic case review methodology of pairing DCF and its own staff in review teams is employed for comprehensive reviews. The larger random sample size makes the findings more generalizable and allows analysis by area office or program.

The monitor has access to any and all meetings held at DCF and regularly attends:

By observing and sometimes participating in agency meetings, the monitor believes he has a better understanding of where the department is placing its efforts and can give the plaintiffs a more accurate picture of the work that is being done in the field as well as in the DCF central office.

Organization and resources. The monitor for the Juan F. consent decree is appointed by and solely responsible to the U.S. district court trial judge for the case. All expenses of the court monitor, including staff, consultants, equipment, supplies, and space, upon approval by the trial judge, must be paid by the state. For FY 07, the proposed budget of the Juan F. Court Monitor's Office totaled approximately $665,000.

During the committee study, the office was staffed by three full-time and 14 part-time employees. The full-time staff included the court monitor, a monitoring specialist, and an office manager. Six part-time positions were case reviewers who worked under contract to the court monitor's office as needed. In most cases, the contracted case reviewers were retired DCF social workers. The other part-time personnel were current Department of Children and Families staff who were assigned as liaisons to the monitor's office to carry out case review activities as needed.

Technical Advisory Committee. As noted above, provisions of the October 2005 Revised Monitoring Order created a Technical Advisory Committee of national experts to assist the court monitor with the methodologies and data collection used to report on DCF performance under the Juan F. consent decree. In collaboration with the court monitor, the TAC is also responsible for advising the department on practice, infrastructure, or other functions concerning members of the Juan F. class that need improvement. Specific issues subject to this advisory function include: ensuring children's needs are met; ensuring an appropriate treatment planning process; and permanency needs of children in foster care.

Emily J.

Emily J. v. Rell, another federal class action lawsuit, was brought by the Connecticut Civil Liberties Union, Center for Children's Advocacy, Yale University Jerome N. Frank Legal Services Organization, and Center for Public Representation on behalf of seven children placed in juvenile detention centers operated by the Judicial Branch. Originally filed in 1993 as a “conditions of confinement” case, it sought to address serious problems of overcrowding, unacceptable housing, sexual and other assaults of detainees, and inadequate medical, mental health, educational, and recreational services found in the Bridgeport, New Haven, and Hartford Juvenile Detention Centers.

In February 1997, the court approved a consent agreement reached by all parties. The members of the defense (which included the governor, the DCF commissioner, the state department of education commissioner, the director of detention services, the chief court administrator, and the supervisors of the Bridgeport, Hartford, and New Haven Juvenile Detention Centers) signed off on the settlement, although most of the stipulations applied primarily to the Judicial Branch. The settlement established requirements for juvenile detention centers concerning: living conditions; housing; recreation and programming; staffing and staff training; education; medical and mental health services; behavior management; and family support and interaction. The agreement also required the Judicial Branch to establish a minimum number of residential and nonresidential community placements as alternatives to incarceration and pretrial community support services.

In addition, the 1997 settlement required that if a detainee was a DCF client, the assigned caseworker visit that youth at least once a month and work closely with the youth's attorney and probation officer to assist in placement decisions that involve alternatives to detention center confinement. The settlement also required that an independent monitor be appointed to ensure the above mentioned requirements were met.

A revised stipulated agreement and a corrective action plan that the Judicial Branch and DCF had developed was approved by the court in June 2002 and replaced the 1997 agreement. The court acknowledged the accomplishments of the Judicial Branch in improving conditions within the detention centers; however, it ordered the defendants to focus on four main areas for children with mental health needs: screening; assessment; planning; and services.

Monitoring history. Unlike the original agreement, where the Judicial Branch had primary responsibility for compliance, DCF and the Judicial Branch were jointly responsible for making improvements under the revised stipulated agreement and corrective action plan. In addition, a written memorandum of agreement (MOA) between the Judicial Branch and DCF was developed to reserve 20 beds at Riverview Hospital for psychiatric evaluations of court-ordered children (i.e., children involved with the juvenile court as delinquent or FWSN and for whom the judge ordered an inpatient evaluation).

The 2002 agreement again specified that an independent monitor be appointed to conduct general inspections and program reviews that result in a summary report to be done more than twice a year. As a mechanism to ensure compliance, the monitor hired mental health consultants who made recommendations to both DCF and the Judicial Branch Court Support Services Division (CSSD). For example, the mental health consultants reviewed and proposed changes to the Juvenile Justice Intermediate Evaluation (JJIE) program. As a result, DCF developed a more comprehensive child assessment program with stronger family and community involvement.

In June 2005, just before the 2002 agreement was set to expire, a third agreement was negotiated by the parties and approved by the court. The purpose of this settlement was to provide supplemental, community-based services that would reduce the number of children placed in detention. Examples include but are not limited to: multidimensional treatment foster care slots; therapeutic mentors; and comprehensive, home-based behavioral health treatment and other supports known as “wraparound” services.

Under the 2005 agreement, DCF was also required to conduct a comprehensive review of a child's needs prior to adjudication. Better needs assessments were intended as another effort to divert juveniles from detention and long-term, out-of-home placement by providing wraparound services in the community.

The agreement further required DCF to provide outcome reports that contain statistical information for evaluating the success of the various additional services on a quarterly basis to the plaintiffs and the court monitor. These reports provide both program and child-specific outcome measures for the following programs: wraparound services; group homes; adolescent substance abuse; outpatient; multidimensional treatment foster care; flex funding for educational success; general flex funding; wraparound training; protocols for DCF-involved detainees; and general outcome measures. Examples of some of the reported outcome measures are:

During the committee study, the independent court monitor responsible for reviewing compliance with the Emily J. agreement found satisfactory progress had been achieved by the state. The case was closed by the federal court in October 2007.

Emily J. services. Overall, DCF spent approximately $15 million since FY 03 to meet its Emily J. obligations.²³ The department recently reported the Emily J. settlement resulted in the development and implementation of $6.9 million in new or expanded community-based services for the targeted class of children (i.e., children who are in detention or who have recently been in detention, and who are determined to be at imminent risk for residential placement). The services began as a pilot program in Hartford in October 2005 and during FY 07 were expanded statewide.

The new services included: multidimensional treatment foster care; a gender-specific therapeutic group home for girls; family-based substance abuse treatment; flex funding; and therapeutic mentoring (which is paid for with flex funding). Flex funding provisions allow DCF caseworkers to allocate discretionary funds for a variety of purposes based on a child's particular needs. As established under the Emily J. settlement, flex funds may be used to pay for after school care, music instruction, Boys Club memberships, or other services that address factors related to delinquency.

For identified Emily J. class members, DCF and CSSD convene a meeting to identify and develop an appropriate placement diversion plan. For all DCF-involved children (who are those committed to the department or part of an open case), a “triage” meeting is convened within three days of their being detained in the judicial system. The intention of the meeting is to develop service plans for the court's consideration, with a goal of reducing the number of days the children are held in detention.

Children found by the triage team to be at risk for residential treatment typically receive a bundle of services that may include: recreational services (e.g., dance, basketball, art, or music lessons); individual therapy; vocational services; after school programs; treatment services from private providers, such as medication management through the HomeCare Program; and other community-based treatment services such as Multi-Systemic Therapy and family advocacy.

Monitoring and evaluation activities. As required by the court, an independent monitor oversaw and reported on implementation of the Emily J. settlement agreement provisions since 1997. The monitor's main activities included: holding status conferences with the judge; reviewing reports from DCF; talking with the plaintiffs; meeting with children from the Emily J. class; conducting site visits; meeting with department staff; and publishing quarterly compliance reports. Over time, the monitor became less directly involved in agency operations as the departments (both DCF and the Judicial Branch CSSD) succeeded in developing a comprehensive internal monitoring and evaluation process for activities covered by the Emily J. service system.

Internal process. In an effort to plan and implement services and to develop outcome measures as defined in the settlement agreement, an Emily J. implementation team was formed in June 2005. DCF convened the team as a way of managing the compliance process going forward. The team consisted of representatives from the two state agencies involved, DCF and the Judicial Branch Court Support Services Division, as well as staff from the Connecticut Center for Effective Practice and the University of Connecticut Health Center.

The health center staff collected assessment data from CSSD, DCF, and direct service providers, including retrospective data on the children who were served in the first year of services, to provide a comprehensive look at outcomes. In accordance with the settlement agreement, quarterly reports based on data gathered by the implementation team were submitted to the court monitor and the state attorney general's office.

DCF also assigned responsibility for internally managing progress and monitoring compliance with the Emily J. settlement to a team of agency staff that included: a program director, full-time social work supervisor, three part-time detention liaisons, and one clerical worker. Results were tracked by the DCF team on a weekly, monthly, and quarterly basis through reports provided by providers, CSSD, and the agency's own automated systems. Examples include: monthly utilization, outreach, and program development reports that all Emily J. providers are required to prepare; reports on triage and diversion efforts; and weekly client outcome reports. These reports were reviewed and analyzed with DCF management and then summarized and submitted to the court monitor and the plaintiffs.

In preparation for the expiration of the current settlement, the Emily J. implementation team developed a plan for sustaining progress in the spring of 2007. This sustainability plan formed the basis of a subsequent memorandum of agreement between the Judicial Branch and DCF aimed at ensuring: a) continuous improvement of services; and b) continued success in diverting children from residential treatment and helping them remain in the community.

W.R.

The W.R. v. Connecticut Department of Children and Families lawsuit was filed as a federal class action in 2002 on behalf of a group of children with mental health needs in the care of DCF. The group certified by the court as the W.R. class is described as all mentally ill children aged 0-21 and/or youth with serious behavioral issues, who are in the care of DCF, and:

The plaintiffs -- several youth in DCF care and/or their parents -- were represented by Connecticut Legal Services and joined by the Office of the Child Advocate. They claimed the department failed to provide a continuum of placements appropriate to the class members' clinical needs and was relying on overly restrictive institutional placements and foster care placements that lacked adequate clinical supports. DCF denied the allegations, and for a number of years the parties were unable to reach agreement on any issues, including the definition of the class.

Frustrated by the lack of progress in resolving the case, the trial judge appointed an outside mediator to work with the plaintiffs and the department in 2006. With the mediator's help, the parties reached a three-year settlement agreement in April 2007. The agreement, which requires the department to put in place policies and procedures to improve services for all W.R. class members as well as address the specific needs of several individual plaintiffs, was approved by the legislature, effective July 2007. The final agreement also required that implementation of its provisions be monitored by an outside consultant agreed upon by the parties.

On August 8, 2007, the U.S. District Court held a fairness hearing to review the terms of the approved agreement and allow class members an opportunity to object. No comments in opposition to the agreement were received and the settlement agreement went into effect for a three-year period that concludes June 30, 2010. The lawsuit is considered ended and no independent, on-going monitoring, beyond the consultant activities called for in the approved agreement, is required.

Settlement provisions. Under the settlement agreement, DCF agreed to take the following steps to increase its ability to serve all members of the W.R. class:

The settlement requires the department to increase EMPS funding by $1 million per year for three years, using the additional money to increase staffing during peak and expanded hours to “allow maximum mobility and faster response times to crisis calls.”²⁴ It also outlines the allocation of the additional resources across geographic areas and requires the consultant to periodically review the allocation of EMPS services, and if the parties agree, DCF may reallocate funding to areas where need is greatest.

The consultant is also required to help DCF develop and implement the new ICBO program, establish the program's eligibility criteria, and institute an appropriate transition process for participation in the program. DCF must commit $1,312,500 in the first year of the settlement to provide W.R. class members with certain services including: therapeutically supported living; crisis supports; and related services to help maintain them in the community. In the second and third years of the agreement, the department must commit $2 million annually for such ICBO services.

Guidance on W.R. services, which is to be provided by DCF in consultation with the consultant, must address: transition planning beginning at age 14; unconventional or “out-of-the-box” planning options, understanding there is a “no eject/no reject” policy for services for all DCF clients; and the availability of increased EMPS, group home, and ICBO services under the agreement. The consultant additionally must help DCF develop transition planning and policy to help older adolescent class members prepare for adulthood, and, if appropriate, transition to DDS or DMHAS services.

For four of the plaintiffs, the department is required to fund small ($5,000) special needs trust funds. For two plaintiffs who have already aged-out of the DCF system, the department is required to cover the cost of a case manager, educational and vocational mentors, living expenses, and educational, health care, and other related expenses up to a total of $199,953 per year for three years. Another plaintiff currently receiving voluntary services for serious mental illness will remain eligible for services and receive the same priority as a committed DCF youth until age 23. The department also will pay $150,000 for the plaintiffs' attorney fees and costs.

Monitoring. Other duties of the W.R. consultant include reviewing and reporting on implementation of the settlement agreement. Quarterly reports on EMPS and ICBO services, services provided in group home settings related to the W.R. class, and the nature and extent of guidance provided to DCF by the consultant must be submitted to the agency and the plaintiffs' counsel during each of the three years of the agreement. The consultant is also responsible for identifying best and promising practices for clinical and supportive services provided in group homes and for EMPS and ICBO services, and making recommendations for improvements. Overall, the guidance provided to DCF by the consultant is intended to improve department staff and contractor practice, particularly in the way they deal with and plan for W.R. class members.

Legislative Oversight

Legislative oversight of state agencies is the primary function of the General Assembly's Program Review and Investigations Committee. In that role, PRI has conducted a number of evaluations of the Department of Children and Families and its mandates and major programs. The General Assembly's committees of cognizance over the department, which include the legislature's human services and judiciary committees, as well as the Select Committee on Children, have ongoing authority for monitoring and evaluating the department's performance and compliance with legislative intent.

A key way the legislature oversees and assesses DCF and other state agencies is through the appropriations process. The appropriations committee's recently established Results-Based Accountability (RBA) project, in particular, is focused on monitoring and evaluating the progress agencies are making in achieving their policy and program goals. DCF's participation in the RBA process as well as recent DCF monitoring and evaluation activities of the children's committee are highlighted in this section.

As another mechanism for tracking agency progress in meeting its goals, DCF is required by law to provide a number of reports and plans to the legislature. Current statutory reporting requirements for the department are also presented in this section.

Children's committee activities. In regard to DCF, the children's committee over the past five years has held a number of informational forums on areas of concern including the Juan F. exit plan, the Connecticut Juvenile Training School, and Riverview Hospital. The forums have provided committee members and other legislators with opportunities to discuss issues related to children's services in detail with officials and key program staff from DCF and other state agencies, as well as representatives of various stakeholder groups (e.g., private service providers, advocates, and parent organizations). The committee has also used the forums to monitor agency progress in meeting the exit plan goals and to address performance problems at CJTS and Riverview Hospital identified through various internal and external evaluations and investigations.

One significant resource for the children's committee oversight efforts is the Commission on Children, a legislative entity established in 1985 with 25 members representing all three branches of government, advocates for children, and private service providers and professionals who work with children. By law, the commission is responsible for: studying and providing information on the status of children and children's programs in Connecticut; and identifying programs and policies needed to improve the development of children and strengthen families.

The children's commission has focused its research and policy development efforts on prevention, particularly in the areas of early childhood and positive youth development. It views its role as advising the legislature and working in partnership with DCF and other state agencies and interest groups to improve services and policies for children.

The Commission on Children has no oversight authority over DCF; its monitoring activities are limited to looking at data and general trends related to outcomes for children and providing that information to policymakers. For example, the commission supported the development of the state's annual social health index, a tool that looks at long-term trends in 11 indicators of social well-being including child abuse, youth suicide, and high school dropout rates.

Results-Based Accountability. Results-Based Accountability is an approach for planning, implementing, and managing programs and policies in terms of desired outcomes and performance measures. It was developed by Mark Friedman of the Fiscal Policies Studies Institute; at present, the RBA process is used, to some extent, in over 40 states, predominantly at the county and municipal levels.

In 2005, the General Assembly's appropriations committee co-chairs established a work group to carry out a pilot project to try to apply the RBA framework to the state's budget process. Two program areas (early childhood education and Long Island Sound water quality) were selected for the initial test of the process during the 2006 legislative session. A consultant, The Charter Oak Group, was retained to help the work group adapt RBA principles to the legislature's appropriations process and implement the pilot project.

The main steps in the first RBA budget process included: identifying overall population goals (i.e., “quality of life results”); developing a standard template for providing data on results achievement (indicators), as well as key budget information, for use during the appropriations subcommittee hearings; and subcommittee presentations by the budgeted agencies that discussed the results data and plans for improving performance (i.e., “turning the curve” to meet the program goal). After evaluating the programs according to measurable goals, committee members then could make funding decisions (either increases or cuts in appropriations) based on the results data.

Positive feedback from all participants in the pilot project led the appropriations committee to continue its Results-Based Accountability approach, and expand it to include more programs and agencies during the 2007 budget process. As one of the added agencies, the Department of Children and Families applied the committee's RBA framework to four of its programs. DCF prepared templates for two programs related to early childhood, an area targeted for inclusion by the appropriations committee work group, and for two key agency functions, foster care services and general child protection services activities.

The department noted in its budget hearing testimony to the appropriations committee that participating in the RBA process was very similar to its experience with the Juan F. consent decree exit plan. In fact, the program results information DCF submitted in its RBA templates for foster care and child protection includes performance measures similar to several of the 22 exit plan outcome measures, as Table III-3 indicates.

Statutory reporting requirements. DCF is required by law to report on matters that cover all mandate areas of the agency as well as on agencywide activities. Overall, there are more than a dozen different state plans and reports the department must prepare and submit periodically to the legislature. Each of these statutory reports is summarized now briefly summarized.

Agencywide activities. Public Act 79-165 required DCF to prepare and submit to the legislature a five-year master plan on an annual basis; a 1986 amendment changed the plan to a biennial requirement. By law, the master plan must include: long range goals and the current level of attainment of the goals; a detailed description of the types and amount of services provided; a forecast of future service needs; a written plan for the prevention of child abuse and neglect; a comprehensive mental health plan for children and adolescents; and an overall assessment of the adequacy of children's services.

Biennial master plans including this information have never been prepared by the department. Periodically, DCF has created multi-year strategic plans that have partially addressed this requirement; the last five-year plan was produced in 2000. Now, however, the

department considers its action plan for meeting the outcomes of the Juan F. consent decree exit plan as the agencywide strategic planning document.

Behavioral health. Statutory requirements in the behavioral health mandate area date back to 1981, when quarterly hospital reports to DCF were required concerning psychiatric care. More recent legislation included reporting requirements for the KidCare program and, subsequently, a variety of evaluation and assessment reports related to the state's Behavioral Health Partnership (BHP).

To meet the 1981 mandate, DCF initially provided monthly reports to the legislature on hospital admissions, diagnosis, discharge, and demographic information. This type of reporting is currently handled by the BHP's Administrative Services Organization, which began its behavioral health service authorization and utilization management functions for DCF and DSS in January 2006.

With the enactment of the KidCare program in 2000, the legislature required annual self-evaluations of the program's community care collaboratives and mandated a five-year independent longitudinal evaluation of the implementation of this children's behavioral health reform. Periodic status reports on the KidCare collaboratives and services, in addition to the outside, contracted longitudinal reports on the program were completed in accordance with statute. However, these reporting requirements were revised in 2003 and in effect replaced by a variety of Behavioral Health Partnership reports.

Under P.A. 05-280, an annual report is due each March 1 by the Behavioral Health Partnership Oversight Council (BHPOC). The current report, which includes subcommittee updates on annual progress and council recommendations, can be found on the partnership's website. The BHPOC may also conduct an independent external evaluation of the BHP. The RFP was recently issued for this project and a “report card” is expected in the coming year. Additionally, the BHP must report annually to legislative committees on the estimated cost savings of the BHP as well as provide an annual evaluation report. The first annual evaluation was expected to be completed by the end of 2007; the partnership is still working on the methodology for determining the cost savings.

Another advisory group, the Connecticut Behavioral Health Advisory Committee (CBHAC) must provide the DCF State Advisory Council with annual reports on the local systems of care and biennial recommendations on behavioral health services. As of August 2007, only one report, completed in 2003, had been done to meet both statutory requirements.

Protective services. State statute (C.G.S. §17a-91) requires DCF to provide an annual report to the legislature on all committed children. However, 2001 was the last year for which this was completed. Additionally, DCF must establish a central registry of all children for whom a permanency plan has been formulated and in which adoption is recommended. According to the department, the intent of this report is met by the registry of children awaiting adoption found on the DCF website.

State law also requires all licensed residential, child care facilities to submit annual reports. Standardized reports containing the following six items are provided to the department: 1) number of children currently in residence; 2) number of children in residence one-year ago; 3) number of children served during the year; 4) number of admissions during the year; 5) number of discharges during the year; and 6) number of deaths during the year. The information required in these reports is collected in a variety of other ways by the department through licensing, contracts, and the ASO; it entails duplicative work by the agencies.

Prevention. DCF must annually provide an update to OPM on its activities related to the Child Poverty and Prevention Council 10-year plan. The department's Director of Prevention submits annual updates to the council on current DCF prevention programs such as the Positive Youth Development Initiative, Suicide Prevention, and Prevention of Shaken Baby Syndrome. The updates include long-term goals and the number of children and families served along with measurement and outcome information.

Juvenile justice. Under C.G.S §17a-6b, the CJTS advisory group is to provide an ongoing review of the training school with recommendations for improvement or enhancement. The statute outlines nine items that must be contained in the report, including but not limited to: a review of the program and policies of the facility; the percentage of residents in need of substance abuse treatment; and demographic information on the residents. The department currently prepares the report, which is then reviewed by the advisory group.

Other reporting requirements. Under C.G.S. §17a-37, DCF must provide an annual evaluation on its school district (Unified District #2) to the commissioner of education. When PRI staff inquired about these reports, the department could not document fulfilling this specific requirement. However, similar to other school districts in the state, the DCF unified district submits annual reports concerning special education services it provides and strategic school profile information to SDE.

Under another statute, C.G.S. §17a-3 (6), DCF must “… conduct studies of any program, service or facility developed, operated, contracted for or supported by the department in order to evaluate its effectiveness.” Currently, the department partially fulfills this mandate through the program review and evaluation functions of its Bureau of Continuous Quality Improvement. However, to date, much of the bureau's focus is on residential facilities and protective services, with an emphasis on process rather than outcomes.

Since 1999, the department has been required to respond to actions taken regarding recommendations put forth by the advisory committee promoting adoption and provision of services to minority and difficult to place children. The last year the department fulfilled this requirement was 2003 and this advisory group has no members and does not exist at present.

Accrediting Body Oversight

Accreditation is intended to put forth standards against which an organization is measured to assure a minimum level of care. It has been reported that accreditation has the benefit of formalizing and clarifying policies and procedures. It is also useful as a credential signifying organizational quality to consumers, funders, and other key stakeholders. Accreditation usually requires an organization to submit evidence of adherence to required standards (the “self-study”) and undergo a site visit by inspectors of the accrediting body. Areas found to be out of compliance require correction before accreditation or reaccreditation is granted.

The Department of Children and Families currently receives accreditation for Riverview Hospital through the Joint Commission (formerly the Joint Commission on Accreditation of Healthcare Organizations). Additionally, the Connecticut Juvenile Training School is pursuing accreditation by the American Correctional Association. Further, the Connecticut General Assembly passed legislation in 2005 (P.A. 05-246) requiring DCF to apply for accreditation by the Council on Accreditation within a reasonable time. Each of the accrediting bodies is now described.

The Joint Commission. The Joint Commission, until 2007 the Joint Commission on Accreditation of Healthcare Organizations (JCAHO), currently accredits Riverview Hospital, Connecticut's only state-run psychiatric hospital for children between the ages of 5 and 17. The Joint Commission is a U.S.-based nonprofit organization formed in 1951 with a mission “to continuously improve the safety and quality of care provided to the public through the provision of health care accreditation and related services that support performance improvement in health care organizations.”²⁵ The commission currently accredits approximately 80 percent of all hospitals in the country.

The Joint Commission accreditation steps include preparation of an in-depth self-study followed by a site visit. There are several hundred standards for accreditation that fall into 11 areas:

During the site visit, the performance of the hospital is compared to the relevant standard for that area. Riverview Hospital was first accredited as a psychiatric hospital following a survey on December 13-15, 2003. Prior to 2003, Riverview Hospital was accredited as a behavioral health facility.

Site visits occur at least once every 39 months. Unannounced site visits may occur at any time, as was the case for Riverview Hospital in October 2004. Unannounced site visits may be prompted by at least one patient care concern received from the public. The Joint Commission does not tell the hospital what the complaint is about; however, the targeted inspections give some indication of the areas of concern.

The Joint Commission now uses a tracer methodology. Upon arrival, Joint Commission site reviewers, called “surveyors,” request case records to review. Based on the records reviewed, the surveyors will trace a child's stay at Riverview Hospital. Any of the services used during the child's stay may be assessed according to Joint Commission standards. Internal services used to support the child's stay at the hospital, such as building safety issues, may also be reviewed.

In addition to the hospital accreditation process, the Joint Commission began recognizing hospitals for meeting National Patient Safety Goals in 2005. The purpose of the goals was to highlight problematic areas in health care and describe evidence- and expert-based solutions to these concerns. The national patient safety goals for hospitals in 2006 were:

Additionally, because Riverview Hospital uses a consulting pharmacy rather than an on-site pharmacy, the Joint Commission requires monthly documentation that demonstrates compliance with standards for pharmacy practice.

The cost of accreditation by the Joint Commission includes the direct fee paid annually to the Joint Commission ($2,500) to maintain accreditation, and $5,900 to McLean Hospital for a data comparison required by the Joint Commission. The Oryx submission of data requires five days per month from an Information Systems staff person. There is an additional on-site survey fee of $15,000 on years when the reaccreditation site visit occurs.

The Riverview Hospital Quality Assurance Manager is primarily focused on Joint Commission activities 40 hours per week. This includes Infection Control Coordinator activities, survey readiness activities of the Quality Assurance department, submission of the annual periodic performance review, tri-annual application process, follow-up survey reports, and monitoring. In essence, efforts for Joint Commission accreditation is part of the everyday operation activities of the hospital, largely determining the committees formed, areas monitored, and policies and protocols written.

Council on Accreditation. In 2005, the Connecticut General Assembly enacted legislation requiring DCF to become accredited by the Council on Accreditation (P.A. 05-246). The act directed the commissioner to apply, within a reasonable time, for accreditation of the department by the Council on Accreditation. A failed bill (SB 334) during the 2007 regular session attempted to amend the statute to require the commissioner to apply for accreditation no later than October 1, 2007. To date, there is no statutory deadline and the department has not officially sought accreditation from the Council.

The Council on Accreditation is an international, independent, not-for-profit organization that accredits child and family serving agencies, and behavioral and healthcare organizations. The Council has been in existence since 1977 when it was co-founded by the Child Welfare League of America and Family Service America (now the Alliance for Children and Families). The standards are based on best practices in the field. The accreditation process requires a self-study (self-evaluation) followed by a site visit. The accreditation lasts for a four-year period.

There are standards for accreditation of the department overall in such areas as:

Beyond the generic standards, accredited public agencies must also adhere to standards specific to services such as:

Currently, the state child welfare agencies in Arkansas, Illinois, Kentucky, Louisiana, and Maryland are accredited; the COA Public Agency Accreditation Report of June 2007 also identifies six additional state administered child welfare agencies that are currently going through the accreditation process (Maine, Missouri, Ohio, Tennessee, Washington, and West Virginia).

In preparation for DCF becoming COA-accredited, the Director of Planning, Policy and Program Development, the Director of Policy and Accreditation, and several other DCF staff attended basic accreditation training in March 2007. The training focused on how to calculate the staffing needed to complete the COA process. The department has prepared an estimate for DCF and its 14 area offices and facilities to become accredited. Unlike other states with a strong county system where each county may get accredited separately, Connecticut's 14 offices make this function quite spread out, and accreditation of one entire state a more involved process. Beyond the area offices, there are also DCF-run facilities that would need to be visited and brought into line with COA accreditation standards. DCF estimates that it will cost as much as $909,675 to become accredited, based on 7-8 part-time positions ($415,000-$475,000) and accreditation fees ($434,675). Funding would then be needed to make improvements required to meet accreditation standards, and additional funding to prepare for subsequent reaccreditation processes.

The Policy and Accreditation Unit of the Planning, Policy and Program Development division of the Bureau of Continuous Quality Improvement will be responsible for shepherding DCF through accreditation. Concern has been expressed that COA standards change frequently and the accreditation process will be very time consuming. Some believe that states with accredited child welfare agencies are no better than other states that do not have the accreditation. Another concern is that the department is very focused on meeting the Juan F. exit plan outcome measures and preparing for the upcoming Child and Family Services Review; getting ready for accreditation on top of these other efforts could be overwhelming.

In an effort to identify any deficiencies, a comparison is currently being done part-time by one DCF staff person of COA standards with current department policies and procedures. The Director of Policy and Accreditation believes that human resources and LINK are two areas that will require significant change in order for DCF to meet COA standards. The department also has an opportunity to go through a mock COA review that would help DCF identify areas of weakness and help prepare for accreditation.

Commission on Accreditation for Corrections. The Commission on Accreditation for Corrections is a private, nonprofit organization that administers the only national accreditation program for all components of adult and juvenile corrections. Accreditation standards are set by the American Correctional Association (ACA). The Connecticut Juvenile Training School is considering accreditation by the commission as a juvenile correctional center (juvenile detention centers have different ACA standards). According to the American Correctional Association, organizations seek accreditation to ensure that they are in compliance with national standards, and to demonstrate to key stakeholders that they are operating at acceptable professional levels. The Commission on Accreditation for Corrections is made up of 28 corrections professionals from throughout the nation to ensure that the commission is independent and impartial. The main purpose of the commission is to conduct accreditation hearings to verify that agencies applying for accreditation meet the relevant standards.

The association's Standards and Accreditation Department develops new standards, revises existing standards, and coordinates the accreditation process including the semi-annual accreditation hearings. The Standards and Accreditation Department also provides technical assistance to agencies and training for consultants participating in the accreditation process.

The standards are a national benchmark for the effective operation of correctional systems, addressing services, programs, and operations essential to good correctional management. Operations examined pertain to:

The association has 21 different manuals of standards, each of which applies to a particular kind of correctional facility or program. The accreditation process usually takes up to 18 months. Accreditation lasts for a three-year period.

All programs and facilities conduct a self-assessment of operations and complete a Self-Evaluation Report that specifies the agency's level of standards compliance. The Self-Evaluation Report is submitted to the American Correctional Association for review. A standards compliance audit can only occur if all of the mandatory standards and at least 90 percent of the non-mandatory standards are met.

The compliance audit is administered by trained American Correctional Association consultants who have an average of 18 years experience in the corrections field. The audit is usually done by three consultants during a three-day period, during which time they will determine whether the policies described in the self-assessment have actually been implemented.

An accreditation decision by the American Correctional Association Board of Commissioners is then made. The DCF Bureau Chief for Juvenile Services estimates that there are over 400 standards, about 40 of which are mandatory; 80 percent of the remaining 350 standards must also be met for a facility to be accredited.

Accreditation hearings, which are conducted by a panel of three to five commissioners, are held three times per year at three different conferences sponsored by ACA. Concerns are addressed with the facility representatives that attend the accreditation hearing. Issues that could prevent accreditation would be known prior to the accreditation hearing through the unofficial report given to the facility by the auditors before they leave the facility. The facility would then have the opportunity to change the audit to a “technical visit” and request an extension and re-audit six months later. Accreditation denial almost never occurs at accreditation hearings.

The Bureau Chief of Juvenile Services noted that just 33 juvenile correctional centers in the entire country are ACA accredited, and accreditation of CJTS would be a source of pride to staff, and recognition by external stakeholders.

Annual certification statements to the American Correctional Association are required once an organization has become accredited. These statements contain the following:

A monitoring visit may occur during the initial three-year accreditation period to ensure continued compliance with the appropriate standards. Accredited agencies then apply for reaccreditation approximately nine months prior to accreditation expiration.

The Judicial Branch Court Support Services Division (CSSD) went through the accreditation process for their juvenile detention centers. While the future of the Connecticut Juvenile Training School is up in the air, the Bureau Chief for Juvenile Services believes that preparation now for accreditation will serve as a foundation for future ACA accreditation regardless of whether there is a single training school or several smaller facilities. Policies and procedures are currently being compiled in preparation for ACA accreditation.

The cost of ACA accreditation for CJTS includes the direct fee paid to the American Correctional Association ($10,000), which covers the costs of three auditors visiting CJTS for three days, and one CJTS staff person to attend the ACA conference to represent the facility at the hearing and to receive the accreditation. Additional costs associated with the requirements for maintaining accreditation include the assignment of one quarter to one half of the time of a manager to act as the ACA manager. The DCF bureau chief noted that after the initial accreditation, the standards become part of the facility operation and the cost would become negligible.

Other Regulatory Monitoring and Evaluation Activities

Additional monitoring and evaluation of DCF includes review and reporting requirements carried out by several other federal and state agencies. Recent activities of the federal Centers for Medicare and Medicaid Services (CMS), the U.S. Drug Enforcement Administration (DEA) and the Connecticut Department of Public Health (DPH) are now briefly described.

CMS. Riverview Hospital has to be approved by the federal Centers for Medicare and Medicaid Services. In 2000, Conditions of Participation (COPs) standards were introduced for hospitals receiving Medicaid and Medicare reimbursement.

The Centers for Medicare and Medicaid Services conduct unannounced site visits and stringent reviews. In Connecticut, the Department of Public Health performs the CMS reviews on behalf of the federal government. Riverview Hospital is also required to submit information to CMS notifying them, for example, on the purchase of a new Glucometer machine for patient testing. Riverview Hospital is also required to submit a report to CMS whenever there is an adverse reaction to medication.

DEA. Riverview Hospital is required to maintain a controlled substance license through the federal Drug Enforcement Administration. The DEA has offices in each state and inspectors conduct unannounced spot visits periodically. The visits entail a visual check of where medications are stored and secured. The last inspection occurred in May 2007 during which inspectors witnessed the destruction of controlled drugs.

DPH. In 2007, the Wilderness School was licensed as a youth camp by the Department of Public Health. The purpose of this licensure is to assure the health and safety of campers. Licensure requires adherence to 121 requirements including standards in the areas of physical plant, staff qualifications, safety, and administration of medications. Licensure site visits to the Wilderness School occur annually. Additionally, DPH requires the Wilderness School to report any positive medical diagnoses (e.g., strep or hospital admission).

Riverview Hospital is required to report information to DPH including infection control and immunizations. The Riverview Hospital Immunization Coordinator and Pediatrician, for example, submit all information regarding vaccinations to DPH on a monthly basis. Additionally, a form outlining treatment and follow-up care is sent to DPH whenever there is positive identification of a patient with tuberculosis.

Chapter IV

Outside Investigations and Reviews

Several state entities have independent oversight roles related to children's services and the Department of Children and Families. These include the Office of the Child Advocate and the Child Fatality Review Panel, both of which have statutory investigatory powers and duties related to programs and services provided to children by DCF and other state agencies. The Office of the Attorney General (OAG), under the provisions of the state “whistleblower” law (C.G.S. § 4-61dd), also has investigatory responsibilities concerning reports of mismanagement or misconduct occurring in any public agency, including the Department of Children and Families. The DCF monitoring and evaluation functions of all three entities are described in this chapter.

Office of the Child Advocate

The Office of the Child Advocate was established in 1995 to monitor and evaluate services provided to children and families by DCF and other state agencies (P.A. 95-242). Concerns over accountability for protecting children and their rights, reinforced by the tragic death of an infant in a child abuse case, led the legislature to create OCA as an independent agency with strong oversight authority.

The OCA enabling legislation also established an advisory committee for the child advocate's office and a Child Fatality Review Panel, of which the state child advocate is a member. The oversight duties and activities of the Office of the Child Advocate are summarized below, followed by a description of the Child Fatality Review Panel.

Statutory requirements. The state child advocate is appointed by the governor from a list submitted by the OCA advisory committee and subject to legislative approval. The individual appointed to the position must have knowledge of the child welfare system and legal system, and be qualified by training and experience to perform the duties of the office. These specific statutory duties include:

State statute grants the child advocate broad authority to access any information, even confidential records, necessary to carry out the office's duties. According to OCA, it is the only state agency authorized to review information from all aspects of a child's life, including DCF court files, school and health care records. Information obtained or generated by OCA in the course of an investigation, as well as the identity of persons making reports to the advocate, is confidential and may be released by the advocate only if it is deemed to be in the best interest of a child or the public.

The child advocate may issue subpoenas to compel the production of books, papers, and other documents as well as the attendance and testimony of witnesses. The child advocate is also authorized to bring actions on behalf of any child before a court or state agency, provided a good faith effort has been made to resolve issues or problems through mediation. Each year, the child advocate must submit a detailed report analyzing the work of the office to the governor and legislature.

By law, the seven-member child advocate advisory committee²⁶ must meet with the advocate and OCA staff three times per year to assess:

Authorization by the advisory committee also is needed for the advocate to initiate legal actions against the state. The advisory committee is required to provide an annual evaluation of the effectiveness of the child advocate's office. To date, this has been issued in the form of a cover letter to the OCA annual report from the committee chairman, which briefly assesses the office's accomplishments over the prior year.

In practice, the OCA advisory committee meets four times a year to help set priorities for the office and to review the status of ongoing work. The child advocate considers the multidisciplinary committee a useful resource and has called on members for their expertise and technical assistance. For example, the psychologist who serves on the committee was asked to review and evaluate CJTS surveillance videos obtained during the OCA/OAG review of that facility.

Activities. According to its annual report, the mission of the Office of the Child Advocate is to oversee the care and protection of children and advocate for their well-being. Its purpose is: to monitor public and private agencies that care for children and evaluate state agency policies and procedures to ensure they protect children's rights and promote their best interests.

The main activities of the child advocate's office, discussed briefly below, include: ombudsman activities; reviews and investigations of facilities and programs; and special projects. In addition, OCA conducts public education and legislative advocacy, and recommends policy changes and system reforms based on its reviews and ombudsman activities. The child advocate also serves on the Child Fatality Review Panel, which is described later in this chapter.

Ombudsman activities. A primary OCA function is to receive and review inquiries and complaints from citizens about the state's child-serving systems and programs. One assistant advocate serves as the intake coordinator, screening initial calls, providing guidance and information about available services, programs, and policies, and making referrals to other agencies. All OCA professional staff share responsibility for follow up on calls and work on cases opened for investigation. In addition to helping children and families access services and resolve problems, OCA uses its ombudsman process to identify trends and areas of concern, and to set priorities for its oversight efforts.

The numbers of calls received and cases opened by the child advocate's office over the past three fiscal years are shown in Table IV-1. Of the approximately 1,100 calls to the office from the public during FY 06, about 300 only needed general information while around 800 required more follow up. Most of the calls needing follow up (over 75 percent) were taken care of through referral or with additional information; OCA opened cases for investigations regarding the remainder (172).

The information presented in Table IV-1 is based on estimates because of limitations of the OCA call management database. While the advocate is working with the Department of Information Technology to improve the system, little progress has been made, mainly due to a lack of funding and staff resources.

As a result, it continues to be difficult for OCA to compile data routinely on the nature of complaints received. An analysis of citizen concerns was carried out on calls received in FY 04. That review found the majority of calls were made about child welfare issues, most frequently about DCF child abuse investigations and case management. The second largest category of calls was legal, which involved concerns about the rights and representation of children and families in abuse and neglect proceedings. The legal category also included calls