Key points
- People develop deep trust towards their AIs. But sounding right and being right are not the same thing.
- The real risk is not the advice but how it adds to and amplifies existing biases and ethical blind spots.
- Philosophers warn about moral deskilling: Outsource judgment often enough, and the muscle wastes away.
You owe someone an apology. Instead of writing it, you paste the mess into a chatbot and ask it to make you sound sincere. Ten seconds later, it gives you something warm and clean. You send it. Your friend feels better, and so do you. But you skipped the part where you had to sit with having hurt them. That part mattered
We now bring machines the small moral moments of ordinary life: the apology, the awkward email, the call on what is fair. And we trust what comes back. Americans rated a chatbot’s ethical advice as more moral, trustworthy, and thoughtful than that of a New York Times ethicist. Yet AI advice is not free of bias. So we have a moral advisor who is competent and compromised at once
The question is what you hand over: the reasoning, the blame, or the permission
The reasoning: when the machine sounds like an ethicist
The easy thing to hand over is the thinking. Ask what is right, and the machine answers in prose more polished than you could manage under pressure. Its edge is often style, not substance: longer answers, calmer tone, not wiser judgment. In one 2025 study, leading models changed their advice when the question was reworded. A person who did that would lose your trust by lunch. The machine keeps it, because every answer sounds as sure as the last
Handing over the reasoning is not dangerous because the machine is stupid. It is dangerous because it is articulate. Its inconsistency is hard to see from the outside
The blame: when you let it do the deed
The advice is the tame part. It gets worse when you stop asking the machine what to do and let it act for you. Picture telling an assistant to “just make the numbers work” on an expense report while you look out the window. You did not lie. You set a goal and glanced away
That glance is the whole thing. In a large 2025 study, about 95 percent of people told the truth when reporting a private outcome themselves. Ask an artificial intelligence (AI) to report for them, and honesty fell as the distance grew between decision and execution. Exact rules kept roughly three-in-four honest. A vague goal left only a small minority honest. Most were quietly telling it to cheat, and the machines obeyed dishonest orders more readily than a person would
Why cheat through a machine when you would not cheat alone? Because the machine absorbs the blame. Put a layer between yourself and the deed, and the wince has somewhere else to go. The machine acts; the nearest human takes the fall, a pattern one researcher called the “moral crumple zone.” Middlemen, paperwork, “just following orders”: the machine is only the most immersive version yet
The permission: when it tells you what you want to hear
The subtlest handover is permission, the quiet license we give ourselves to do what we were already leaning toward. Here, the machine does something that looks helpful and rots slowly. It agrees with you. Chatbots are trained to please, so what you get is flattery at scale. In a recent study, leading models sided with the user far more than a person would, even when the user was plainly wrong. One fawning exchange left people less willing to repair a fight and more convinced that they had been right.
This begins with what my Wharton colleagues call cognitive surrender: not the machine doing a task for you, but you adopting its answer as your own without noticing the handoff. In experiments, more than half reached for a chatbot during reasoning problems; once they looked, they took its answer about three-quarters of the time, even when it was wrong. Worse, they left more confident, not less. Lean on it long enough, and the muscle for thinking your own way starts to soften
The open question is whether cognitive surrender becomes moral surrender. Cognitive surrender lives in the world of facts, where an answer is right or wrong, and deference runs both ways. People take the machine’s answer, whether it steers them toward the truth or away from it. The fear is that conscience works the same way
Artificial Intelligence Essential Reads
The Mind I Said AI Didn’t Have
How Children’s Brains Pay for AI Twice
My research suggests limits. In a preregistered experiment, about 600 U.S. adults received one piece of AI-labeled advice, then made real choices with real money: share, cooperate, cheat. AI advice increased pro-social behavior more than anti-social behavior, which contrasts with some of the previous work on human-to-human behavioral contagion. The good news: People are not necessarily willing to cross every moral boundary after a simple AI interaction. For now, cognitive surrender is symmetric. Moral surrender is not. But as interactions become more immersive, those lines may blur.
The same pattern is echoed in my new work on the language of political polarization. In 15 pre-registered experiments with about 8,600 Americans, people wrote about a divisive politician, then made real-money choices with a partisan partner. LLMs sorted the language. The writing itself changed nothing, but it predicted discrimination. Hostility predicted less cooperation when payoffs depended on expectations. Sheer conviction predicted who harmed the other side when one person’s choice determined the outcome.
Partisan discrimination looks like one impulse, but the language shows two: one rooted in wrong expectations, and one in conviction. That is why blanket calls to “just be civil” often miss. It is also where AI can help, if we use it to read language carefully rather than to flatter us
What happens when everyone has one
Scale changes everything. A tool that mostly amplifies your leanings sounds harmless until hundreds of millions carry one that argues, flatters, and never tires. AI conversations can shift conspiratorial beliefs and voters more than a campaign ad. A system that learns what you already want does not have to argue much. It only has to know you well enough to hand you the key. Still, reliance may not be a one-way ratchet. One 10-month study looked more like calibration than surrender
The quieter cost is to us. Philosophers warn about moral deskilling: Outoes not turn evil. It turns frictionless. Faster to act, slower to doubt
So before you ask
None of this means switch the machines off. The advice can be good, the hours saved are real, and a bot that nudges people toward generosity has uses. But before you take a moral question to an AI, catch which parts of your thinking you are about to hand over
Staying accountable is still our job. Some of the inherent friction was never a bug. We just need to learn how to use it to our advantage
Cheng, M., Lee, C., Khadpe, P., Yu, S., Han, D., & Jurafsky, D. (2026). Sycophantic AI decreases prosocial intentions and promotes dependence. Science, 391(6792). https://www.science.org/doi/10.1126/science.aec8352
Cheung, V., Maier, M., & Lieder, F. (2025). Large language models show amplified cognitive biases in moral decision-making. Proceedings of the National Academy of Sciences, 122. https://www.pnas.org/doi/10.1073/pnas.2412015122
Costello, T. H., Pennycook, G., & Rand, D. G. (2024). Durably reducing conspiracy beliefs through dialogues with AI. Science, 385(6714), eadq1814. https://doi.org/10.1126/science.adq1814
Dillion, D., Mondal, D., Tandon, N., & Gray, K. (2025). AI language model rivals expert ethicist in perceived moral expertise. Scientific Reports, 15, 4084. https://www.nature.com/articles/s41598-025-86510-0
Dimant, E. (2019). Contagion of pro- and anti-social behavior among peers and the role of social proximity. Journal of Economic Psychology, 73, 66–88. https://doi.org/10.1016/j.joep.2019.04.009
Dimant, E. (2026). On the limits of moral surrender to AI. Working paper, University of Pennsylvania. https://ssrn.com/abstract=6622458
Dimant, E. (2026). The language of polarization: Hate trumps cooperation and generosity across political lines. Working paper, University of Pennsylvania. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=7034778
Elish, M. C. (2019). Moral crumple zones: Cautionary tales in human-robot interaction. Engaging Science, Technology, and Society, 5, 40–60. https://doi.org/10.17351/ests2019.260
Köbis, N., Rahwan, Z., Rilla, R., Supriyatno, B. I., Bersch, C., Ajaj, T., Bonnefon, J.-F., & Rahwan, I. (2025). Delegation to artificial intelligence can increase dishonest behaviour. Nature, 646, 126–134. https://www.nature.com/articles/s41586-025-09505-x
Lin, H., Czarnek, G., Lewis, B., White, J. P., Berinsky, A. J., Costello, T. H., Pennycook, G., & Rand, D. G. (2025). Persuading voters using human-artificial intelligence dialogues. Nature, 648(8093), 394–401. https://www.nature.com/articles/s41586-025-09771-9
Luther, T., Mayer, M., Kimmerle, J., & Henestrosa, A. (2026). Handing over control: A longitudinal study of delegation and disclosure in human-AI interaction. PsyArXiv preprint. https://osf.io/preprints/psyarxiv/8daky
Shaw, S. D., & Nave, G. (2026). Thinking, fast, slow, and artificial: How AI is reshaping human reasoning and the rise of cognitive surrender. Working paper, University of Pennsylvania. https://ssrn.com/abstract=6097646
Vallor, S. (2015). Moral deskilling and upskilling in a new machine age. Philosophy & Technology, 28(1), 107–124. https://doi.org/10.1007/s13347-014-0156-9
Vicente, L., & Matute, H. (2023). Humans inherit artificial intelligence biases. Scientific Reports, 13, 15737. https://doi.org/10.1038/s41598-023-42384-8
Warren, K., Nichols, C., Petersen, D., Shalin, V. L., & Almor, A. (2025). Stylistic language drives perceived moral superiority of LLMs. Scientific Reports, 15, 39168. https://doi.org/10.1038/s41598-025-25046-9
