Skip to content
Tutorials Center
My learning

Topics

Lesson 33 of 45 ยท Applied Ethics: Life, Technology and Environment

Technology and AI: privacy, bias and responsibility

Learn the ethics of technology and AI: privacy, surveillance, algorithmic bias, why fairness metrics conflict, responsibility gaps and regulation, with runnable demos.

A hospital uses software to decide which patients receive extra care. A bank uses a model to approve loans. A city uses cameras and facial recognition. A chatbot answers a child's question. Each of these systems makes or shapes decisions that affect real people, and each raises ethical questions that older technologies did not raise in the same form.

This lesson gives you a map: where in an automated system problems can enter, which ethical ideas apply, and why some trade-offs are mathematical facts, not matters of opinion.

A pipeline of four stages: data, model, decision and people. Under each stage a warning card names a typical ethical problem: skewed data, a wrong target, opaque reasons and unequal impact.
Follow the data: four places where an automated decision can go wrong.

What you will learn

  • Why privacy matters and how data collection changes power.
  • How bias enters automated systems and how to look for it.
  • Why several reasonable fairness definitions cannot all hold at once.
  • What a "responsibility gap" is, and who may be responsible for AI harms.
  • How regulation and professional codes address these issues.

Key terms

Algorithmic bias: systematic and unfair differences in a system's outputs or errors across groups of people.

Proxy: a measurable quantity used in place of the one you really care about (for example, cost as a stand-in for health need).

False positive rate: the share of people who do not have the property who are wrongly flagged.

Calibration (predictive value): whether a score means the same thing for every group, for example that a 70% risk score corresponds to about a 70% chance in each group.

Explainability: the ability to say, in terms a person can understand, why a system produced a result.

Privacy and surveillance

In 1890 Samuel Warren and Louis Brandeis wrote an article on the "right to privacy" in response to instant photography and gossip newspapers. Today the same worry applies to data brokers, location tracking and cameras everywhere. Why does privacy matter?

  • Autonomy. People who are watched change how they think and speak, and may be less willing to experiment, dissent or seek help.
  • Dignity and control. Information about you is part of your identity; control over it is part of respect.
  • Power. Detailed data allows manipulation, discrimination and pressure, especially when held by large firms or governments.
  • Trust. Medicine, journalism and therapy depend on confidentiality.

Against these stand benefits: safety, public health, research and convenience. Many regulations, such as the European Union's General Data Protection Regulation (applied from 2018), try to strike a balance through consent, purpose limits, data minimization and rights of access and deletion.

How bias enters a system

Bias is not only the fault of "bad programmers." It can enter at several stages.

  1. Data. Historical records may reflect past discrimination. If past hiring favored some groups, a model trained to imitate past hiring will learn that pattern.
  2. Target choice. A system must predict something measurable. In a 2019 study published in Science, Obermeyer and colleagues examined an algorithm used to identify patients needing extra care. It predicted healthcare cost as a proxy for need. Because less money had historically been spent on Black patients at the same level of illness, the system underestimated their needs. Changing the target reduced the disparity.
  3. Design and deployment. The system may be used in contexts it was not tested for, or its output may be treated as more objective than it is.
  4. Feedback. Predictions can change the world they measure, as when more policing in a district creates more recorded crime there.

Fairness is not one thing

Suppose a tool gives each person a risk score and flags those above a threshold. Several definitions of fairness are natural:

  • Equal error rates: the false positive and false negative rates are the same for each group.
  • Calibration/predictive parity: a flagged person has the same chance of being truly positive in each group.
  • Demographic parity: the same share of each group is flagged.

In 2016 ProPublica analyzed a criminal-risk tool called COMPAS and reported that Black defendants who did not reoffend were more often wrongly labeled high risk than white defendants. The company replied that the scores were equally predictive across groups. Both were right about their own metric. Mathematicians (Kleinberg, Mullainathan and Raghavan 2016; Chouldechova 2017) then proved that when the groups have different base rates of the outcome, a score cannot in general satisfy equal error rates and predictive parity at once, except in trivial cases. So choosing a fairness criterion is a moral choice, and designers must explain it, not hide it in code.

Responsibility and transparency

When a system that learns from data causes harm, who is responsible? Andreas Matthias (2004) named the responsibility gap: the more autonomous and opaque a system, the harder it is to hold any human accountable. Candidates include the developers, the company deploying the system, the operators who rely on it and regulators. Many ethicists argue that responsibility must stay with identifiable people and organizations: a machine cannot be punished or feel remorse, and "the algorithm did it" is not an excuse.

Related ideas:

  • Transparency and contestability. People affected by high-stakes decisions should receive reasons and a way to appeal.
  • Human oversight. A person who nominally "reviews" every decision may in practice rubber-stamp it. Real oversight needs time, information and authority.
  • Self-driving cars. The Moral Machine experiment collected preferences about crash dilemmas, but most engineers argue that the main ethical work is in preventing crashes, testing and being honest about limits, not in solving trolley problems in advance.
  • Misuse and dual use. A tool built for good (face recognition for finding missing children) can be used for harm (tracking dissidents).

Rules and codes

The European Union's AI Act, which entered into force in August 2024, sorts uses of AI by risk: some practices are banned, high-risk systems face requirements on data quality, documentation, human oversight and testing, and lower-risk uses carry lighter duties. Professional codes (such as those of computing societies) emphasize honesty, avoiding harm and public interest. Critics of regulation worry about slow rules and stifled innovation; critics of self-regulation worry about weak enforcement. Most ethicists argue for combining both.

Applying the four lenses

  • Outcomes: What are the benefits and harms across all users and non-users? Are errors costly, and who bears them?
  • Duties and rights: Do people have a right to privacy, to non-discrimination, to an explanation? Was consent meaningful?
  • Character: Do designers show honesty about limits, humility and care for the vulnerable?
  • Care: Who is affected, especially people with little power, and have they been listened to?

Common misconceptions

"Algorithms are objective." They encode choices about data, targets and thresholds.

"Fairness can be fixed by removing the sensitive variable." Other variables can act as proxies, and removing the variable can hide the problem.

"If the model is accurate overall, it is fair." Overall accuracy can mask large differences across groups.

"There is one correct fairness formula." Mathematical results show that common formulas conflict.

Try it

JS Why equal errors and equal predictive value clash

Press Run (or Ctrl + Enter) to see the console output here.

No output.

Drag the threshold, watch the trade-offs

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. In the Obermeyer et al. (2019) study, why did an algorithm underestimate the needs of Black patients?

    The proxy target carried historical inequality into the predictions.
  2. 2. What did Kleinberg and colleagues and Chouldechova show?

    This is an impossibility result, so choosing a fairness criterion is a value choice.
  3. 3. What is a responsibility gap?

    Matthias (2004) named it; many ethicists argue responsibility must remain with identifiable humans and organizations.
  4. 4. Which is a reason privacy matters?

    Privacy protects autonomy, dignity, trust and limits on power.
  5. 5. Why is 'remove the sensitive variable' not enough to guarantee fairness?

    Correlated features such as location can carry the same information.

Summary

  • Privacy protects autonomy, dignity, trust and limits on power, and must be balanced with other goods.
  • Bias can enter through data, target choice, design, deployment and feedback.
  • Common fairness definitions conflict mathematically when base rates differ, so choosing one is an ethical decision.
  • Responsibility should stay with identifiable people and organizations; transparency and contestability matter.
  • Regulation and professional codes work best together, with the four lenses guiding analysis.

Next: Chapter review