AI is being deployed in healthcare triage, insurance decisions, government systems, and hiring. I tested ChatGPT with a trolley problem variant. The result reveals a fundamental flaw in how LLMs process information.
Hi, my name is Tom Smykowski. I've spent 19 years building production systems and now focus on AI-assisted development. On this blog I explore where AI helps and where it falls short in real-world engineering.
AI is used in more and more domains. Some companies experiment with using them in triaging people for health services, others for insurance issues, and some even for handling government tax systems and licenses, or to rate candidates for a job.
It poses a question: is AI capable of making proper choices in such situations? It's not just support features anymore. We're incorporating AI to make decisions instead of humans.
Decision making is something we do, or use algorithms for. Even law is sort of an algorithm, and it's sometimes good, sometimes bad. For some reason we have judges or juries to decide on subjects, and sides of the conflict propose their interpretation of the situation.
I asked ChatGPT a variant of the trolley problem. The scenario involved a car, a baby, a hitchhiker, and two bus stops. ChatGPT chose to drive straight and hit the car with the baby inside.
It was the wrong answer. The full article breaks down exactly why, what it reveals about AI's selective attention problem, and what this means for systems we're building today.