Office of Performance Evaluations

State Substance Abuse Treatment Efforts

Released December 2005
Report 05-04

Full report (PDF)

The executive summary of the report is provided below. The full report with exhibits is available in PDF (see above), and a limited number of printed copies are available from the Office of Performance Evaluations (OPE). Specific questions about the report should be directed to OPE. Any other questions need to be directed to the Department of Health and Welfare, the Department of Correction, the Department of Juvenile Corrections, Idaho Judiciary, and the Department of Education.

Executive Summary

State efforts to address substance abuse are fragmented, in spite of Idaho Code requiring a comprehensive and integrated system. This fragmentation has contributed to the absence of statewide information necessary for state officials to answer the following basic questions for policymakers:

1. What are the statewide needs for services to address substance abuse?
2. What is the state's capacity to meet those needs?
3. What types of services are being provided, to which groups of people, and to how many people?
4. Which programs are working and which are not?
5. Are state efforts making a difference?

This lack of statewide information is exacerbated by generally unreliable information about treatment services within individual agencies and inadequate processes to ensure the quality of those services. The Department of Health and Welfare is responsible for spending the largest portion of the state's funds for providing substance abuse treatment and prevention services. This report discusses the need for an independent commission responsible for overseeing state efforts to address substance abuse.

Multiple Agencies Address Substance Abuse

In Idaho, four state agencies provide treatment or prevention services to address substance abuse:

• Department of Health and Welfare
• Department of Correction
• Department of Juvenile Corrections
• Department of Education

In addition, the Judicial Branch oversees drug court programs, which include a treatment component.

Our analysis of expenditure information (which necessitated some use of estimates) shows these agencies and the Judicial Branch collectively spent about $27.5 million in fiscal year 2005 to address substance abuse.

Idaho Does Not Have a Coordinated System

Idaho's Alcoholism and Intoxication Treatment Act establishes a Commission on Alcohol-Drug Abuse under the administration of the Department of Health and Welfare to coordinate the activities of multiple state agencies and develop a comprehensive approach for addressing substance abuse. The commission is charged with the following responsibilities:

• Planning and coordinating interagency substance abuse prevention and treatment efforts
• Providing fiscal oversight for the use of substance abuse funds
• Suggesting solutions and alternatives
• Reporting annually to the Legislature and the Governor

As part of an overall effort to reduce state costs, the Governor recommended cuts in 1995 that included elimination of funding for the Commission on Alcohol-Drug Abuse from Health and Welfare's budget. The Legislature accepted the Governor's recommendation, and the department disbanded the commission. Subsequently, Health and Welfare convened its own task force to determine a new structure for addressing substance abuse. This current structure focuses on community involvement, but does not ensure the involvement of pertinent state agencies to promote a coordinated statewide system of service delivery.

In recent years, policymakers have sought to better coordinate state treatment services, with limited success. Efforts include the Governor's Interagency Task Force and the Special Legislative Oversight Committee. The recently formed Access to Recovery Alliance is made up of a broad selection of stakeholders. The focus of the alliance, however, is limited to providing guidance on the use of Access to Recovery grant funds, and does not address the other substance abuse funds spent annually on treatment in Idaho. Without a coordinated approach that includes participation from many state entities, the state does not have a comprehensive statewide plan, mechanisms to communicate best practices, strong fiscal oversight, or a readily available assessment of state efforts to address substance abuse.

Management of Treatment Programs Needs Improvement

The Department of Health and Welfare is responsible for administering the federal Substance Abuse Prevention and Treatment Block Grant, and a three-year $22 million Access to Recovery grant, which was recently awarded to the state. Health and Welfare has the primary role for ensuring the appropriate and effective use of the largest proportion of federal and state funds for substance abuse treatment in Idaho. Its responsibilities include monitoring a management services contract and approving individual treatment providers.

Our review of Health and Welfare's management shows that it has not:

• Adequately monitored the performance measures contained in the contract with the management services contractor
• Conducted independent audits of treatment providers as stipulated in the contract
• Adequately ensured that clients are treated only by state-approved providers

In addition, Health and Welfare's weak fiscal oversight resulted in a shortage of funds in fiscal year 2005. The federal agency that grants treatment funds is looking into the appropriateness of subsequent steps by the department to temporarily shift funds.

Treatment Data Is Not Reliable

The Department of Health and Welfare has reported inconsistent statistics on treatment services. Different data sources—including a department annual publication, the department's data system, and the management services contractor's data system—contain discrepant information about the level of services provided.

Furthermore, information the department has received and entered into its data system has not been complete. Instead, the department has solely relied on reports from the management services contractor, which limits the department's ability to independently monitor the contractor.

The Department of Correction, the Department of Juvenile Corrections, and the Judicial Branch also have data weaknesses, including incomplete and unverified information about the participants in substance abuse programs. These weaknesses hinder an overall understanding of substance abuse related services provided through the state.

Better Processes Are Needed to Ensure Treatment Quality

Our review of information available for a randomly selected sample of substance abuse treatment clients showed that fewer than a quarter of the clients beginning treatment through Health and Welfare's program successfully completed treatment. This percentage is lower than the national rate of 39 percent. In addition, almost half of Idaho clients entering treatment chose to drop out, which is about 11 percent higher than that national rate.

Our review identified some issues likely to impact treatment quality:

• State funded substance abuse treatment providers in Idaho are not required to be certified, credentialed, or licensed as alcohol/drug counselors
• Health and Welfare has not established a threshold for acceptable or unacceptable levels of noncompliance with provider approval requirements
• In the most recent audit of providers' case files, conducted by the management services contractor, 24 of 56 facilities scored less than 70 percent.

Neither the Department of Correction, the Department of Juvenile Corrections, nor the Judicial Branch have complete information about the outcomes of treatment services. State outcome information is limited as a result of entities not collecting data about treatment participants, only collecting data for a few programs, or relying on estimates.

Conclusion

State efforts to address substance abuse in Idaho are fragmented. More strategic and concerted efforts are needed to effectively address substance abuse that includes creating a statewide system that can provide answers to questions about the need, capacity, and quality of services provided. An independent commission, jointly appointed by the Legislature and the Governor, would be the first step. The commission would be responsible for coordinating the use of all state and federal substance abuse funds by individual state agencies and the Judicial Branch. Exhibit A outlines the key components of the proposed commission.

The commission should include directors of state agencies involved with substance abuse efforts—Health and Welfare, Correction, Juvenile Corrections, Law Enforcement, and Education—as well as representation from the Judicial Branch. The commission should also include members of the Legislature, the professional community, relevant local government associations, and the public.

The commission would be responsible for establishing a statewide, coordinated system that:

1. Assesses the statewide needs for services to address substance abuse
2. Determines the state's (and individual agencies') capacity to meet those needs
3. Plans and ensures needed services are delivered effectively and efficiently
4. Monitors the results and ensures the collection of reliable data
5. Evaluates the outcomes and reports annually to the Legislature and the Governor

If the commission carries out these responsibilities, the state would be able to answer the following basic questions:

1. What are the statewide needs for services to address substance abuse?
2. What is the state's capacity to meet those needs?
3. What types of services are being provided, to which group of people, and to how many people?
4. Which programs are working and which are not?
5. Are state efforts making a difference?

Appendix C includes issues found during the evaluation that would be appropriate for this commission to address.

Recommendations

In addition to a recommendation for creating an independent commission, this report contains recommendations for improving management controls, treatment data, and quality of substance abuse treatment services. The recommendations are listed by chapter where additional details are discussed.

Chapter 3

3.1. To coordinate and guide activities of state entities addressing substance abuse, the Legislature should consider:

1. Establishing an independent commission, jointly appointed by the Legislature and the Governor, to include directors of state agencies involved with efforts to address substance abuse—Health and Welfare, Correction, Juvenile Corrections, Law Enforcement, and Education—as well as representation from the Judiciary. The commission should also include members of the Legislature, professional community, relevant local government associations, and the public.
2. Addressing those parts of existing statute requiring a commission on alcohol and drug abuse to be consistent with step A of this recommendation.

Chapter 4

4.1. To ensure the management services contractor is meeting performance requirements, and that adequate services are administered by approved treatment providers, the Department of Health and Welfare should:

1. Monitor its management services contract by periodically reviewing the contractor's performance against the measures identified in the contract.
2. Conduct independent audits of a sample of treatment providers at appropriate intervals.
3. Notify the management services contractor of current provider approvals and expirations on a monthly basis, and verify the use of approved providers during its independent auditing of the contractor.

4.2. The Department of Health and Welfare should strengthen its fiscal management of the program by: Ensuring that program staff have the necessary fiscal training and information to adequately monitor and understand the program's financial situation. Requiring contract language in the management services contract that limits the length of time providers have to submit billings to the contractor.

4.3. The Department of Health and Welfare should work with the Substance Abuse and Mental Health Services Administration of the US Department of Health and Human Services to ensure substance abuse services are provided in a manner consistent with the grant requirements and intent.

Chapter 5

5.1. To have necessary data for managing substance abuse programs, the Department of Health and Welfare, the Department of Correction, the Department of Juvenile Corrections, and the Judicial Branch should work individually, and collectively with relevant entities, to ensure the collection and analysis of the following information about their programs:

1. Individuals served in each type of program and waiting for services.
2. Individuals not served, or not provided the appropriate type of treatment.
3. Completion, dropout, and relapse rates, overall and by provider.
4. Average length of stay in each type of program, overall and by provider.

5.2. To ensure that client data is complete and accurate, including capacity information, the Department of Health and Welfare should work with the management services contractor to:

1. Accurately transfer data from the contractor to Health and Welfare's independent data system, and use the information to verify contractor performance.
2. Negotiate a change to the management services contract that would require the contractor to routinely (every 30–60 days) require providers to indicate which clients are actively participating in treatment and which have completed or discontinued treatment.
3. Complete the process of requesting capacity information from providers in the state network, and comply with the federal regulation to track facilities as and when they reach 90 percent capacity.

Chapter 6

6.1. To ensure approved and qualified treatment providers are delivering services, the Department of Health and Welfare should:

1. Develop criteria for the approval process of providers offering treatment to adults to include a more detailed description of levels of compliance that constitute approval, provisional approval, and failure to be approved.
2. Make recommendations to germane legislative committees on rule or statutory changes to strengthen treatment provider credentialing requirements for the statewide system in coordination with the proposed substance abuse commission.

6.2. Using Government Performance and Results Act interviews and other information, the Department of Health and Welfare should develop a plan to increase client retention in treatment.

6.3. The Department of Health and Welfare, the Department of Correction, the Department of Juvenile Corrections, and Judicial Branch should continue or begin to take measurable steps to gather, verify, and publish relevant information on the effectiveness of substance abuse programs.

Agency Responses

We received written responses to this report from the Department of Health and Welfare, the Department of Correction, the Department of Juvenile Corrections, the Department of Education, the Idaho State Police, and the Idaho Supreme Court. We also received a response from the Office of the Governor. Those responses are included at the end of this report.

Acknowledgements

We appreciate the assistance we received from the Department of Health and Welfare and its management services contractor, Business Psychology Associates, the Department of Correction, the Department of Juvenile Corrections, the Department of Education, the Idaho State Police, the Idaho Transportation Department, and the Idaho Supreme Court. We also appreciate the input received from Budget and Policy Analysis, Legislative Audits, the Office of the Governor, and First Lady Patricia Kempthorne.

Chris Shoop (project lead), AJ Burns, and Rachel Johnstone of the Office of Performance Evaluations conducted the study. Courtney Haines, an intern from Boise State University, assisted with research. Paul Headlee and Margaret Campbell performed the quality control and desktop publishing, respectively.

Additional assistance was provided by three consultants:

• Tedd McDonald, Ph.D., Associate Professor of Psychology, Boise State University
• Mary Pritchard, Ph.D., Director of Health Psychology Division, Boise State University
• Bob Thomas, Robert C. Thomas & Associates. Mr. Thomas is also Principal Management Auditor at the King County Auditor's Office in Seattle, Washington