Research & Evaluation
Assessment of Risks and Needs
For the past 10 years, my research team has conducted research on how to assess risks and needs (mental health and substance use) in a criminal justice context, as well as response to intervention (called “responsivity” in the criminal justice literature).
Below you will see the dissertations my students conducted evaluating risk, needs, and response to intervention. Curtis (2018) and Meeks (2020) both studied risk, Gerber (2022) studied needs (specifically substance use), Bell (2020) studied response to intervention (“responsivity”), and Cochran (2018) studied how to conduct cost-benefit studies in criminal justice settings. Chroback (2022) is completing her dissertation studying violence risk and substance use, which she will defend in early August.
In addition to those dissertations, I will summarize other unpublished research I have led in criminal justice risk, needs, and responsivity, and what I see as current best practices and future directions.
Since my dissertation in 1998 I have studied the assessment of criminal risk, with a particular focus on risk of violence. In 1996 I was the Principal Investigator (PI) in a series of validation studies of the Texas Risk Assessment System (TRAS), and in 2021 the Texas Department of Criminal Justice (TDCJ) asked me to specifically study sexual violence and the utility of continuing to use the assessment of sexual violence risk vs. creating a new assessment using modern statistical modeling techniques (sometimes called data mining or predictive modeling or machine learning). [I prefer the more general term “statistical modeling,” but because “machine learning” has gained the most acceptance, I will use that term to represent statistical techniques which are used to predict a specific outcome—in this case criminal behavior.]
Because I have studied assessment of risk the most, I will focus this first blog post on what we know about assessing risk of felony arrest, violence, and sexual violence:
Using actuarial assessment tools such as the TRAS, Level of Supervision Inventory – Revised (LSI-R) or any of a handful of other similar instruments is more accurate than using structured or unstructured human judgment to assess criminal risk.
The most accurate and least biased risk assessment tools are created using machine learning algorithms on large, representative samples. These algorithms accurately identify important predictors/items as well as more accurately weight these predictors/items in order to assess risk. See Curtis (2018) and Meeks (2020) below for detailed, technical explanations.
For general recidivism (such as arrest or conviction for a felony), actuarial assessment tools are able to accurately predict at roughly 60% accuracy, while our research shows that machine-learning-based assessments can accurately identify over 80%. See reports from the TRAS Validation Studies for more information.
For risk of violence, the discrepancy in accuracy between actuarial assessments and machine-learning-based assessments, is larger because actuarial assessments are less accurate predicting violence (due to a lower base rate of violence relative to felony arrest). See TRAS reports, Curtis (2018) and Meeks (2020).
For risk of sexual violence, the discrepancy is even wider (again because the base rate of sexual violence is lower than for “general” violence). For example, our latest research shows that actuarial tools used specifically to identify sexual violence have a 20% accuracy rate, while our machine-learning-based assessments have approximately 85% accuracy. See the TDCJ Sex Offender Study for more details.
In addition to substantial accuracy benefits, machine-learning-based assessments have the added advantage of being less biased for both race and gender across all types of criminal risk (i.e., felony arrest, violence, sexual violence), and because of its operational efficiencies machine-learning-based risk can be conducted quickly and cheaply (and thus more frequently).
In summary, current best practices for assessing criminal risk for felony arrest, violence, and sexual violence all involve creating a machine-learning-based risk assessment. These machine-learning-based approaches are more accurate, less biased, fast to complete, and cheaper than traditional actuarial risk assessments such as TRAS or LSI-R.
— Dr. Eugene Wang
“Numerous studies have been conducted over the last twenty years investigating the effectiveness of cognitive intervention programs on reducing reconviction rates among offenders. Whereas cognitive intervention programs have demonstrated promising results, there is a gap in the literature surrounding who would be most responsive. Knowing what type of offender would most benefit from attending this program could potentially reduce recidivism by providing the proper rehabilitation programming to offenders before they are released.”
Bell, B. K. (2020). Responsivity in rehabilitation programming for offenders [Doctoral dissertation, Texas Tech University]. TTU DSpace. https://hdl.handle.net/2346/86565
“Over the last decade state prisons have experienced unprecedented growth and many demographic changes. At the same time, courts are requiring states to provide mental health screening and treatment to prisoners. Findings from recent studies indicate that the prevalence of mental illness is higher in prisons than in the community, and comorbidity is common. Our ability to generalize from these studies is limited, however, because of major shifts in the demographic mix in prisons during the past decade. New studies on the prevalence of mental illness in prisons, which consider these recent changes would help planners allocate funds and staff to more effectively meet the needs of these individuals.“
The prevalence of mental illness in prison: Review and policy implications. Administration & Policy in Mental Health, 29(1), 21-40. Diamond, P. M., Wang, E. W., *Smith, J. L., Holzer, C. E. III, Cruser, d. A., & Thomas, C. R. (2001).
“In 1990, the Texas Department of Criminal Justice instituted a specialty aggression treatment program called the Program for the Aggressive Mentally Ill Offender (PAMIO). It has beds for 458 violent male offenders and offers multidisciplinary medical and behavioral treatment. This study examined the effectiveness of this treatment program by comparing the annual rates of disciplinary offenses (e.g., staff assaults, inmate assaults, and good time lost) of 66 offenders prior to and following treatment. Paired t tests showed a significant decrease after PAMIO treatment in the annual rates of total disciplinaries, staff assaults, inmate assaults, and good time lost. This improvement appeared to be directly related to treatment effects.“
The effectiveness of rehabilitation for persistently violent male prisoners. International Journal of Offender Therapy and Comparative Criminology, 44(4), 505-514. Wang, E. W., Owens, R. M., *Long, S. A., Diamond, P. M., & *Smith, J. L. (2000).
“The authors used structural modeling to predict institutional aggression among male mentally ill offenders using the predictors of anger, antisocial personality style, current violent offense, ethnicity, and impulsivity. Measures included the Barratt Impulsiveness Scale, the Buss-Perry Aggression Questionnaire, the Personality Assessment Inventory, age, ethnicity, current violent offense, victim injury from current offense, and institutional incidents of physical and verbal aggression. The model fit the data, and accounted for 94% and 87% of the variance of physical and verbal aggression, respectively. Results indicated anger, antisocial personality style, and impulsivity are stronger predictors of institutional aggression than are ethnicity and current violent offense; anger was the best predictor. Results suggest dynamic variables such as anger can be targeted for clinical intervention to reduce institutional violence.“
Empirically identifying factors related to violence risk in corrections. Behavioral Sciences and the Law, 17, 377-389. Wang, E. W., & Diamond, P. M. (1999).
“Indicators of feigned PAI profiles were derived from comparisons of simulators instructed to feign and genuine patient groups. Concerns are raised regarding whether these indicators are applicable to forensic and correctional populations and can be cross validated with a known-groups comparison. Compiling data on 57 malingerers and 58 genuine patients from two forensic and correctional sites, three primary indicators of feigning, Negative Impression (NIM) scale, Malingering Index, and the Rogers Discriminant Function (RDF) were investigated. Results suggested that the RDF was not applicable to forensic referrals. However, NIM 2 77T appeared to be a useful screen for forensic samples. In addition, convergent evidence of feigning was found across designs (simulation and known-groups) and samples (non-forensic and forensic) for extreme elevations on NIM (> 1101) and Malingering Index (? 5).“
The PAI and feigning: A cautionary note on its use in forensic-correctional settings. Assessment, 5, 399-405. Rogers, R., Sewell, K. W., Cruise, K. R., Wang, E. W., & Ustad, K. L. (1998). doi:10.1177/107319119800500409.
“Examined the usefulness of the PAI (L. C. Morey, 1996), in assessing problematic behaviors in a corrections-based psychiatric hospital. As evidence of criterion related validity, selected PAI scales were compared to (1) evidence of malingering on the Structured Interview of Reported Symptoms (R. Rogers, et al 1992), (2) suicidal threats and gestures, and (3) ratings of aggression on the Overt Aggression Scale (S. C. Yudofsky, et al 1986). Clinical records were reviewed systematically for the time period of September 1, 1996–February 10, 1997. All records containing PAI protocols were examined in relation to other psychometric measures. In general, results support the use of the PAI for the assessment of these problematic behaviors. The PAI scales, subscales, and derived scales are appended. (PsycINFO Database Record (c) 2016 APA, all rights reserved).“
A pilot study of the Personality Assessment Inventory (PAI) in corrections: Assessment of malingering, suicide risk, and aggression in male inmates. Behavioral Sciences & the Law, 15, 1-14. Wang, E. W., Rogers, R., Giles, C. L., Diamond, P. M., Herrington-Wang, L. E., & Taylor, E. R. (1997).
“We looked at whether or not the offender got employed, how long it took, whether the individual was employed one year after gaining initial employment, and whether or not the individual was rearrested or reincarcerated within three (3) years post-release.”
“The bulk of this report evaluates WSD academic, career and technical education (CTE), and life skills programs against these outcomes: academic achievement gains, HSED and industry certifications, and reentry outcomes.”
“S.B. 213 Sec. 19.0041, titled “Program Data Collection and Biennial Evaluation and Report”, requires Windham School District (WSD) to compile and analyze information to determine whether its programs are meeting its goals, to make changes to the programs as necessary, and to submit a report to the Board, the Legislature, and the Governor’s office.”