Sources of bias
Earlier, we discussed a wide range of ethical considerations that pertain to predictive analytics. Some of these considerations are related to bias, or fairness, of predictive models and how their results are used. This is an enormous, complex and extremely important topic.
Broadly, in the context of predictive analytics, bias refers to systematic errors or inaccurate predictions, often stemming from historical inequalities, flawed data sources, or model assumptions, which can result in unfair or discriminatory outcomes for certain groups. However, as you will read in this section, there are many different ways to define and measure bias. We will get more specific in a bit.
In this section, we provide an overview of key concepts. We begin with a discussion of the potential sources of bias in predictive analytics. The development and use of predictive analytics to improve services involves several steps. Each step presents different sources of bias:
Step 1: Estimating predicted probabilities of the outcome.
Step 2: Translating results into recommendations for action.
Step 3: Implementing decisions based on recommendations.
The following pages discuss how bias may be introduced in each of these steps. To illustrate, we will use an example from the criminal justice system, when the consequences of bias are substantial for the lives of people affected. In particular, we will focus on the pretrial period. Across the country, release and detention decisions for defendants in the pretrial period are increasingly guided by algorithmic risk assessments. These assessments rely on data to estimate defendants’ risks of negative outcomes if they are released pending trial. Negative outcomes include failing to appear for a court date, or being charged with new criminal activity. The algorithm’s results are generally used by a judicial body to help determine whether a defendant will be released while waiting for a case to be resolved, and if so, under what conditions. The content is this section is summarized from Porter, Redcross, & Miratrix (2020).
As we discuss the potential sources of bias in pretrial assessment, we’ll focus on bias that can affect Black defendants for simplicity, although any of the ideas could be applied to other demographic subgroups.
