Algorithms learn from historical data, and US history includes redlining, segregated hiring and unequal policing. When those patterns are built into a model, the model repeats them at scale while appearing neutral.
Research has documented the effects. A 2016 ProPublica investigation of the COMPAS risk score used in criminal courts found Black defendants were nearly twice as likely as white defendants to be wrongly labeled high risk. A 2019 study in Science found a widely used health care algorithm underestimated how sick Black patients were because it used past medical spending as a stand-in for need.
Bias can enter through data, through the choice of what to predict, or through how tools are used. People harmed by these systems often do not know an algorithm was involved. Advocates push for audits, transparency and limits on automated decisions in high-stakes areas like bail, policing and housing.