Predictive analytics has a wide variety of applications, from predicting financial markets to personalized medicine. The potential of predictive analytics in the services sector is to help government programs, nonprofit service providers or businesses identify those clients who could most benefit from targeted interventions. This targeting can facilitate effective service delivery at an efficient cost.
Why just focus on predictive analytics for improving services? Predictive analytics has vast applications. Different applications raise different issues with respect to project scoping, ethics and data and technical considerations. We focus on improving services because it allows us to build a framework that is driven on how results will be used. Also, my background is in using predictive analytics (as well as other data science and statistical methods) to strengthen social services, so it is the application to which I have given the most thought. But at the same time, many of the lessons we will learn will be helpful for other applications as well, and there will be flexibility in the data you use for your projects.
Some examples of using predictive analytics to improve social services:
During my time at MDRC, the nonprofit research organization from which the framework for these materials originated, my colleagues and I explored and applied predictive analytics in various ways:
Predicting K-12 academic outcomes: School districts often utilize “early warning systems” (EWS) that rely on student-level data. These systems help identify students who might be at risk of not achieving important educational milestones, such as timely graduation or passing state exams. For instance, the probability of not graduating high school is often gauged using the ABC indicators, which stand for attendance, behavior, and course performance. This indicator system represents a simpler form of predictive analytics. However, as schools increasingly collect richer, data sets with frequent data updates of many student measures, including daily attendance, exam scores, and course marks, there has been opportunity to compute more accurate, frequent and nuanced predictions of student risk.
Providing early warnings of not reaching milestones in an employment training program. This project focused on harnessing granular, longitudinal administrative data to build a system for ongoing, advanced analytics that support the continuous improvement process at the Center for Employment Opportunities (CEO). The goal was for these early warnings to be transmitted, practically in real time, to front-line case workers and leaders. The information would be part of standard dashboards and data protocols. CEO planned to train staff members to act on this information, and to work with MDRC to design, implement, and test new interventions based on insights provided by the predictive analytics results. Unfortunately, this project was interrupted by the COVID-19 pandemic.
Identifying TANF participants who were most likely to find employment. TANF is the Temporary Assistance to Need Families program that provides time-limited support for families’ basic needs. The federal government provides grants to states to run the TANF program. As part of MDRC’s TANF Data Collaborative Project, a team at the Virginia Department of Social Services (VDSS) sought to develop analytic tools to help TANF case workers customize education/employment-related services to increase the likelihood of participants’ labor market success after they leave the program. The team investigated whether demographic characteristics, household compositions, receipt of other public benefits, and past education/employment-related activities could predict success, and how to construct an unbiased predictive tool using such variables.(MDRC, 2023)
Identifying families at risk of disengaging from a home visiting program. Child First is a home visiting program that aims to promote high-quality relationships between caregivers and children in families experiencing challenges related to caregiver mental health and child behavior. Staff members provide intensive in-home clinical services to both the caregiver and child. They also connect families to additional services such as financial and housing support, health care, and treatment for disorders such as substance abuse. To accomplish its goals, Child First must ensure that families remain consistently engaged in program services over time. When families leave before being officially discharged from the program, they receive truncated interventions that are likely to be less effective at improving outcomes. Early disengagement is also expensive, given the large, fixed costs of enrolling new families into the program. Child First prioritizes collecting high-quality data to understand the population of families it serves. For example, the program collects information on a range of family characteristics assessed at intake, including sociodemographic information on both the caregiver and child, health-related information like insurance status and child DSM-5 diagnoses, as well as several risk measures and assessments such as adverse childhood experiences (ACEs) that capture experiences with violence, abuse or neglect, and household instability. The availability of these data provided researchers with a singular opportunity to explore whether predictive analytics could be a useful tool to summarize the large amounts of information the program collects and use it to help staff members identify families at particularly high risk of early disengagement, defined as being enrolled in the program for fewer than 90 days. That information could allow Child First staff members to triage families better at intake and provide more intensive support and services to those families at risk of early disengagement.(Xia, Htet, Porter, & McCormick, 2023)
Pretrial justice: Many jurisdictions across the United States are rethinking the “front end” of the criminal justice system — the pretrial period between an arrest and the disposition of a criminal case. Often these reforms focus on the initial decisions that judges and other court stakeholders make about whether to detain individuals in jail while they are awaiting trial, and on the use of money bail as a tool for ensuring that people will show up to court hearings. In most jurisdictions, the majority of people in jail at any point in time are awaiting trial, and many are there because they cannot afford to post bail. Jurisdictions are looking for fairer, more cost-effective approaches to the pretrial phase of the system. To assist jurisdictions in making better initial decisions, the Laura and John Arnold Foundation (now Arnold Ventures) developed the Public Safety Assessment (PSA), a tool that uses data on an individual’s history with the justice system and the current offense to predict the probability that the person will show up to hearings or be arrested for a new crime if released. The PSA aims to help judges make more informed, less subjective decisions about pretrial detention. It is currently used in nearly 40 jurisdictions across the nation.(Redcross & Henderson, 2019) (Golub, Redcross, & Valentine, 2019)
Later, we will return to some of the above examples to discuss various issues, including ethics, project scoping or set-up, the trade-offs of different modeling approaches, and more. Here are some other examples of predictive analytics being used in the social service sector with the goal to strengthen programs’ services:
Child welfare: An example of PA being used in child welfare is the Allegheny Family Screener Tool. First implemented in 2016, it was developed in a partnership between researchers from Auckland University of Technology and the Allegheny County Office of Children, Youth, and Families, a Pennsylvania child welfare system. The research team wanted to use predictive analytics to help inform and improve decisions made by staff when determining whether reports of possible child abuse and neglect should be marked for further investigation, rather than replace human decision making altogether. The tool summarizes vast amounts of information across multiple databases to provide a risk score to child welfare call screeners. The researchers worked closely with the child welfare agency and partner organizations to discuss implementation of the tool and results, and feedback from community meetings informed how the tool was developed. An independent evaluation found that it increased the staff’s ability to accurately screen reports and pursue investigations. Also, the tool did not increase the rate of children screened in for investigation. That is, using it resulted in a different pool of children being identified as needing child welfare intervention, but did not substantially increase the proportion of children investigated among all children referred for maltreatment. The model and its implementation have been updated over time so that its predictions reflect contemporary information on families currently being served.(Human Services, 2019)
Lead poisoning prevention: The Chicago Department of Public Health partnered with the Data Science for Social Good initiative at the University of Chicago to help find the homes that are most likely to still contain lead-based paint hazards. From their website: “By building statistical models that predict exposure based on evidence such as the age of a house, the history of children’s exposure at that address, and economic conditions of the neighborhood, CDPH and their partners can link high-risk children and pregnant women to inspection and lead-based paint mitigation funding before any harm is done.” This integrated and innovative system will ensure resources are used most efficiently, and ultimately will mean healthier Chicago children. This short video summarizes the project.
Criminal justice: In addition to applications in the pretrial periods, predictive analytics is being applied to other parts of the criminal justice system. For example, predictive analytics has been used for “predictive policing” based on forecasting future crime at the community level, for guiding sentencing and probation, for detecting fraud, for assessing young people’s risk of becoming involved in crime and more. As you are surely aware, bias is an enormous concern and this topic alone could take up a whole courses. We will not have the time to delve is as deep as is warranted due to time constraints, but we will discuss this topic later, and I will provide some ideas for further reading. For now, this semi-recent New York Times article provides a brief overview of some applications and concerns about bias. As an optional, longer read, you can check out this Pro-Publica story about algorithmic bias in sentencing.
Health care: The adoption of electronic health records (EHRs) by most US health care systems for patient care has led to an explosion of predictive analytics in health care - with applications aimed at improving health outcomes, care coordination, and quality of care. Health care systems and insurance companies harness patient demographics, insurance claims data, and clinical characteristics in EHRs to create statistical models of future health care risks and resource utilization. There are also efforts to incorporate social and behavioral determinants of health (SBDH), which include measures of diet and physical activity as well as characteristics of patients’ neighborhoods, such as food access and transportation.
The above examples do not capture the breadth of growing applications of using predictive analytics to improve social services. To read about some additional examples, see the following short articles and blog posts. These are OPTIONAL.
Some examples of using predictive analytics to improve consumer services:
I do not have first-hand experience in using or reviewing predictive analytics to improve consumer services, but of course there are many! Applications in social services have been inspired by applications in consumer services. Here are a few articles that provide some good summaries and discussion. Not all applications discussed focus on estimating binary outcomes, but many, such as predicting customer churn, product purchase or customer satisfaction can be examples of predicting binary outcomes.
Questions for discussion:
- What examples do you find interesting for using predictive analytics to improve services? Why?
- What promise do you see in the examples you read about or know about?
- What concerns do you have?
Back to topReferences
Golub, C., Redcross, C., & Valentine, E. (2019). Evaluation of pretrial justice system reforms that use the public safety assessment effects of new jersey’s criminal justice reform.
Human Services, A. C. D. of. (2019). Impact evaluation summary of the allegheny family screening tool.
MDRC. (2023). Virginia pilot: TANF data collaborative.
Redcross, C., & Henderson, B. (2019). Evaluation of pretrial justice system reforms that use the public safety assessment.
Xia, S., Htet, Z., Porter, K. E., & McCormick, M. (2023). Exploring the value of predictive analytics for strengthening home visiting: Evidence from child first.