Not all bias comes from the LLM

Some of AI’s most powerful decisions aren’t made by algorithms. They’re made by humans.

We spend a lot of time discussing data quality, model bias, and the ethics of AI at scale. And rightly so.

But some of the most influential decisions don’t happen deep inside the machine. They happen literally on the surface - and they are human made from product and design people:
We are talking about default states in AI interfaces. Defaults are easy to overlook.They feel like neutral ground. A convenience. A helpful suggestion.In reality, they’re one of the most powerful framing tools we use.

Consider ChatGPT. Many of us started by asking it for poems, not because we love poetry, but because the prompt field suggested it: “Write a poem about...” That default seeded the behavior. It shaped expectations before we even thought about what we wanted. In finance, it gets more consequential. A preselected “low-risk ETF” in a robo-advisor flow doesn’t feel like advice, it feels like common sense.
But it's not. It's a design decision, made by someone, passed along to everyone.When defaults are presented as obvious, we stop questioning.And that’s the risk - or the opportunity.

However we frame it... So maybe the real challenge isn’t only improving the model.It’s becoming more conscious of how we humans frame the interaction before AI is even involved.