What form should AI-driven advice take?
Originally posted on The Horizons Tracker. This article is part of an ongoing series looking at AI in KM, and KM in AI.
As AI has boomed in recent years, it’s become increasingly common for us to include it in our decision-making. We’re using it to spot defects in products or potential fraud in financial transactions. Many of these recommendations are quite overt in nature, but other times they can be more subtle attempts to nudge us in a certain direction.
The advice typically falls into either attention signals, which highlight the importance of an issue, without necessarily offering a recommendation, or an action signal, which does. Both are commonly used, but research1 from The Wharton School explores which is the most effective and whether there are costs associated with relying on AI-driven advice too much.
Smart decisions
The researchers used chess as their Petri dish, with AI providing a range of recommendations to help players improve their game. The game was chosen because it’s easy to gauge whether decisions are high quality or not, and therefore, the quality of the recommendations can be measured.
They worked with around 300 players, ranging from amateurs to elites, with around 36 official chess master titles between them. The players competed under three conditions, with some receiving action-oriented signals, whereby the AI would show them the best move at certain parts of the game. Others received attention-oriented signals, with the AI highlighting key points in the game, but not providing next steps. The third, control group, received no help at all.
Perhaps unsurprisingly, action signals proved most effective in improving player’s performance. These gains were not without costs, however, with the performance of players declining after they received the help. They happily leaned on AI to support them, but struggled without its support once it was withdrawn.
Uncharted waters
It’s a process the researchers refer to as the “uncharted waters effect”, which describes how AI can be initially helpful but can fundamentally change our cognitive process in a way that makes us less well prepared for the future.
Interestingly, the attention-oriented support had the opposite effect. This support didn’t provide people with answers, but, instead prompted players to think for themselves. This resulted in better performance, albeit not to the same extent as with action-oriented support, but importantly, it also resulted in stronger performances in the future too.
It’s perhaps worth pointing out that both forms of advice improved performance more than not receiving any at all, which underlines the value AI can provide, albeit in the relatively narrow confines of chess. What’s important, however, is how the type of AI support affected how we think, both in the immediate period around the support being given and in the future.
Guiding decisions
With AI increasingly being deployed in a decision support capacity, the findings are a timely reminder that not all forms of deployment are going to have an equal impact.
At a time when so much focus is on the capabilities of AI itself, the results also remind us that its impact is often not based on the reliability and accuracy of the recommendations. The study clearly shows that AI support nearly always improves decision-making, but this boost isn’t always sustainable.
This matters as decisions are seldom once-and-done affairs. Indeed, even if we are using AI to help guide us, it’s important that we’re able to scrutinize what we’re presented with, and if using AI dulls our thought processes, we become less capable of working, even if we’re receiving AI support. This can be especially problematic when it comes to areas in which the AI isn’t capable, such as in rare cases for which the system doesn’t have reliable data.
Design matters
The researchers argue that this isn’t something confined to chess, and is just as applicable in a wide range of other domains. They highlight recruitment, for example, where participants were found to make the best decisions when AI made recommendations only during moments where they were uncertain or had made a mistake. In other words, it wasn’t offering constant guidance, but targeted support.
Similarly, studies2 have shown that the introduction of Hawk Eye into tennis officiating impacted how line judges behaved. While their accuracy generally improved, they also let borderline decisions slide for fear of being embarrassed in public. In other words, errors weren’t eradicated, they just moved to a different part of the process.
Similarly, a study3 from the University of Michigan found that an AI tool used to detect early signs of sepsis often flagged patients after clinicians had already suspected sepsis. In practice, some doctors delayed action while waiting for AI confirmation, slowing care instead of speeding it.
These studies remind us that while the AI companies talk heavily about the capability of the tech, the benefits rely far more on how the tech is designed and deployed than on its raw power. If the technology is poorly designed, we run the very real risk of over-reliance, under-scrutiny, or even outright rejection. If it’s designed well, however, then it’s capable of stepping in at the precise time we’d benefit from it the most, giving us a boost not only to our immediate performance but also our long-term capabilities.
The risk is that we build systems that deskill us, nudge us into passivity, or make us more risk-averse. The opportunity is to build systems that genuinely augment us, sharpening our judgment and helping us improve even when the AI isn’t there.
AI doesn’t just shape what decisions we make. It shapes how we think. If we want technology that makes us more capable, not less, the real frontier isn’t in model size or processing speed. It’s in design choices: when advice is given, how it’s framed, and whether it encourages us to lean in or switch off.
Article source: What Form Should AI-Driven Advice Take?
Header image source: Adapted from “US Open line judge” by Kate Tann on Flickr, CC BY-SA 2.0.
References:
- Poulidis, S., Ge, H., Bastani, H., & Bastani, O. (2025). Action vs. attention signals for human-ai collaboration: Evidence from chess. The Wharton School Research Paper. ↩
- Almog, D., Gauriot, R., Page, L., & Martin, D. (2024, July). AI oversight and human mistakes: evidence from centre court. In Proceedings of the 25th ACM Conference on Economics and Computation (pp. 103-105). ↩
- Kamran, F., Tjandra, D., Heiler, A., Virzi, J., Singh, K., King, J. E., … & Wiens, J. (2024). Evaluation of sepsis prediction models before onset of treatment. NEJM AI, 1(3), AIoa2300032. ↩
