Algorithmic bias is a systematic tendency of an algorithm to produce unequal or distorted results for people or groups. In AI systems, bias can arise from the data, the labels used in training, the way the problem was defined, or the context in which the tool is applied.
How it works
A model learns patterns from the examples and objectives it receives. If the data does not adequately represent the people affected, if it carries historical unfair decisions, or if it uses a variable that acts as a proxy for another sensitive characteristic, the system can reproduce or amplify inequalities.
Bias can also appear after training. A metric chosen to optimize the model can favor one type of error over another. A system can be used on a population different from the one evaluated, or an automated decision can be applied without considering individual circumstances.
Finding bias requires looking beyond the code. The team needs to ask who is represented in the data, who is not, which errors are most harmful, and how results vary across relevant groups. Fairness metrics depend on the task and are not always satisfiable at the same time; therefore, it is necessary to make choices explicit, document limitations, and listen to impacted people.
The topic is linked to ethics in AI and responsible AI, which deal with principles and governance. Technical analysis can reveal disparities, but the decision about what is fair also involves social context, rules, and human responsibility.
Practical example
Consider a system that prioritizes resumes for a job opening. If historical examples reflect hiring practices that favored certain profiles, the model may learn patterns that reduce visibility for qualified candidates from other groups. The team should not assume that removing explicit fields, such as age or gender, eliminates the problem, because other variables may be correlated.
A careful evaluation compares errors and outcomes across relevant groups, reviews the data, and checks whether the tool actually helps the process. It also defines how a person can contest a decision and how atypical cases are analyzed. The analysis must respect privacy and applicable rules.
What it is for
Identifying algorithmic bias serves to assess whether a tool distributes errors and opportunities in an acceptable way. This is important in hiring, credit, health, education, moderation, and other contexts where decisions affect rights or access to services.
A team can reduce risks by combining data auditing, pre-deployment testing, continuous monitoring, and human review proportional to impact. It should record the system's purpose, the groups evaluated, the metrics chosen, and the limitations found.
There is no single test that proves an algorithm is impartial in any situation. The goal is to make risks measurable and debated, correct identified disparities, and maintain channels for contestation. When potential impact is high and there is no adequate mitigation, limiting or not using the system can also be the responsible decision.