Leonore Röseler, Ingo Scholtes, Bernhard Sendhoff, Aniko Hannak, "Willing to revise? Confidence and Recommendation Adoption in AI-Assisted Image Recognition", International Conference on Hybrid Human-Artificial Intelligence (HHAI 2022), 2022.

Artificial intelligence (AI) is increasingly used to assist humans in various aspects of everyday life, including high-stakes decision-making. Nevertheless, the question how to design human-AI teams that optimally integrate the strengths of both parties, while mitigating their respective weaknesses, is still open. This work investigates how different mechanisms for the integration of AI-generated recommendations influence the performance of AI-assisted humans in an online experimental study. To this end, we study the willingness of human participants to revise their decision in an image recognition task, which can be modulated such that it exhibits different levels of difficulty. The preliminary findings of our study reveal no significant differences in decision accuracy between participants in the groups of the experiment, suggesting that the different mechanisms to present AI-generated recommendations did not influence the performance of humans. We further find that presenting participants with an AI recommendation led them to revise their decision more often compared to a recommendation presented as originating from a human. As expected, low confidence correlated with a higher willingness to revise a decision and adopt a recommendation, both from humans and AI. The study highlights the potential to improve machine decisions by means of an inclusion of observational data on human decisions, such as confidence, possibly observed in a less obtrusive way.