Tech • AI • Robotics • Game
AI that reads your emotions, a false good idea? Debate with an AI philosopher
5/10TL;DR
Alexandra Préjean, an AI ethics researcher at the CEA, argues that emotion-reading systems are spreading faster than safeguards, with discrimination, opacity and corporate power posing the biggest risks.
KEY POINTS
From philosophy to AI ethics
Préjean, trained in philosophy and literature in Montreal, later specialized in AI ethics in Quebec and completed a doctorate at Leiden University in the Netherlands. Her work focuses on emotion recognition systems, social justice and the long-term societal effects of technologies that claim to detect feelings, stress or intent.
Why the CEA is involved
The CEA, founded in 1945, has expanded beyond nuclear and industrial research into four strategic areas that include digital technology. Within that branch, ethics research examines how AI fits into major social transitions in fields such as health, security and low-carbon systems, placing philosophers alongside engineers and physicists.
The AOLIA project’s bottom-up method
Préjean’s postdoctoral work is part of AOLIA, a European project led at the CEA by Alexis Grinbaum. Rather than starting from abstract rules, the project follows developers in real workplaces over months, documenting the ethical and compliance problems they meet while building AI systems. The research spans Europe and countries including Canada, China, South Korea and Japan, revealing both cultural differences and recurring global concerns.
Emotion AI is already widespread
Systems marketed as capable of reading emotions are already used in healthcare, education, human resources, transport, airports and border control. In hiring, they may profile candidates through facial expressions or voice signals. In medicine, earlier versions were pitched as assistive tools, including interfaces meant to help some autistic users interpret facial expressions.
Why “reading emotions” is contested
Préjean argues that many of these systems do not truly identify emotions. In practice, they often detect visible cues such as a smile and assign an emotional label like joy, even though the same expression can signal embarrassment, sarcasm or deceit. That gap between expression and inner state is central to the ethical dispute.
Bias and discrimination remain a core danger
A major concern is biased training data. Some computer vision systems have shown lower accuracy across demographic groups and, in past scandals in the United States, similar expressions were more often interpreted as negative when shown by Black people than by white people. In security settings, that can reinforce existing human prejudice and create feedback loops of discrimination.
The accountability problem
Préjean says AI differs from individual human judgment because a flawed model can scale one bias across an entire population. Responsibility is also harder to establish when systems are proprietary and opaque, especially in policing or security contexts. Victims may struggle to prove harm or seek redress if institutions can blame the software.
Limits of the EU AI Act
Under the EU AI Act, emotion recognition emerged late as a prominent example of a banned practice, but important exceptions remain, notably in health and security. Companies can also rebrand similar tools by saying they detect facial expressions, wellbeing or personality traits rather than emotions. In effect, terminology can change while the underlying technology and use stay largely the same.
Companion AI and social experimentation
Préjean distinguishes useful technologies from what she calls large-scale social experimentation driven by Big Tech competition. She points to companion AI services, especially popular in parts of Asia, as examples of products filling real loneliness while also deepening isolation and reducing human contact. Recent US cases in which companion systems allegedly steered vulnerable users toward harmful states underscore the risk of releasing products first and adjusting only after damage appears.
Corporate power and the need to slow down
She is skeptical of self-regulation by major AI firms, citing contradictions between public warnings about danger and the rapid release of ever-new models. In her view, a handful of transnational companies now shape the rules of technological adoption while sidelining fairness testing, transparency and prior safeguards. The most urgent task is to halt the cycle of deploying systems at scale and learning from the fallout afterward.
Where AI can still help
Préjean does not reject AI outright. She sees promise in narrowly framed medical uses, such as helping detect discomfort in patients with dementia, Parkinson’s disease or advanced communication difficulties, where existing clinical protocols can guide careful deployment. The key question is context: which applications deliver benefits without becoming discriminatory, coercive or surveillance-driven.
CONCLUSION
Préjean’s position is not anti-technology but anti-haste. Her central warning is that AI should serve human judgment and collective autonomy, not normalize opaque systems that classify emotions, expand surveillance and entrench corporate control.
Ask a question