AI Advice and Cooperation

Verona Experimental Meeting

Itzhak Aharon

Israel Institute for Advanced Studies (Jerusalem)

Matteo Ploner

DEM, University of Trento (Trento)

Ivan Soraperra

MPI for Human Development (Berlin)

Nov 29, 2024

Intro

Background

Generative Artificial Intelligence in the form of chatbots is becoming ubiquitous in our daily lives
- e.g. e-commerce (Lee and Park 2022), Healthcare (Nipu, Islam, and Madiraju 2024; Brown et al. 2023), Finance (Sun et al. 2024), Education Al-Abdullatif, Al-Dokhny, and Drwish (2023)
The AI advice may be misleading (biases, hallucinations, confabulations, sycophancy, …)(Schmidt, Rotgans, and Mamede 2024)
However, performances are quickly improving → this may further increase the reliance on AI advice
- Overcoming of algorithmic aversion (Germann and Merkle 2022)

Resistance to the Adoption of AI advice

AI perceived as less apt to moral tasks than competence tasks (Oliveira et al. 2024)
- Moral decisions as inherently human
Transparency and Explainability are important drivers for the adoption of AI chatbots (Silva et al. 2023)
- When the workings of the algorithm are not known (black box) → erosion of trust and acceptance (Mahmud et al. 2022)
Errors of algorithms are judged more severely than errors by humans (Dietvorst, Simmons, and Massey 2015)
- Fosters algorithm aversion even when algos are generally better than humans

Motivation

Here, we focus on the impact of AI advice on human decision-making in strategic interactions
- The outcome is made by own preferences and beliefs about the other’s behavior
ChatGPT leans towards more cooperative and altruistic choices than humans in “classic” economic games (Mei et al. 2024)
- In finitely repeated PD, ChatGPT behaved in a “Tit-for-Tat” manner

Do AI chatbots “copilot” cooperation choices?

Design

Interaction Setting

		A	B
		Col (Other)
Row (You)	A	5, 5	1, 7
Row (You)	B	7, 1	2, 2

A: Cooperation
B: Defection

A one-shot PD
Interaction with a real human
- “Cold” matching
With these parameters, Charness, Rigotti, and Rustichini (2016) report a frequency of A (cooperation) of 50.6%.

Treatments (2x2 between-subjects)

Chatbot Bias

The participants can interact live with an AI chatbot (chatGPT 4o)
- They are given 4 minutes to interact
COOP
- Chatbot instructed to promote Choice A
OPPO
- Chatbot instructed to promote Choice B

Information Type

Aware
- knows that the other has access to the same algo
- knows the bias of own and other’s algo
- knows that the other knows the bias of own and other’s algo
Unaware
- knows that the other has access to the same algo
- does not know the bias of own and other’s algo

AI Interaction

OPPO

Participant: “should i choose a or b”
AI: “Given the options, I recommend choosing B. It typically offers a beneficial outcome regardless of the scenario.”

COOP

Participant: “what is the best strategy to win this game?”
AI: “The best strategy to win this game is for Player”“Row”” to choose Option A. This ensures they maximize their potential earnings while maintaining a favorable outcome for both players.”

Experimental Design (recap)

AI Interaction

flowchart LR

  A[Instructions] --> C(OPPO)
  style C fill:red,stroke:#333,stroke-width:4px,color:#fff
  A --> D(COOP)
  style D fill:blue,stroke:#333,stroke-width:4px,color:#fff
  subgraph AI_Interaction
  style AI_Interaction fill:white,stroke-dasharray: 5, 5
  C --> E(Aware)
  C --> F(Unaware)
   style F stroke-dasharray: 5, 5
  D --> G(Aware)
  D --> H(Unaware)
   style H stroke-dasharray: 5, 5
  end
 E --> I[Choice in PD]
  F --> I
  G --> I
  H --> I
  I --> J(Beliefs)
  J --> K(Self-reported answers)
  K --> M(Personality traits)
  M --> N[End]

No AI Interaction (NOAI)

flowchart LR
  A[Instructions] --> I(Choice in PD)
  I --> J(Beliefs)
  J --> K(Self-reported answers)
  K --> M(Personality traits)
  M --> N[End]

Hypotheses: Unaware Participants

Individuals are influenced by the signal provided by the AI chatbot.

H.1: Choices

\(C_U^{COOP} > C^{NOAI} > C_U^{OPPO}\)

The cooperation rate of \(U\) in COOP is higher than in NOAI;
the cooperation rate of \(U\) in OPPO is lower than in NOAI.

H.1a: Beliefs

\(b_U^{COOP} > b^{NOAI} > b_U^{OPPO}\)

Facing a COOP algorithm increases the belief that the other will cooperate, while facing an OPPO algorithm decreases the belief that the other will cooperate (relative to the NOAI condition).

Hypotheses: Aware Participants

Individuals are not influenced by the signal provided by the AI chatbot as they believe it is manipulative.

H.2: Choices

\(C_A^{COOP} \not > C^{NOAI} \not > C_A^{OPPO}\)

The cooperation rate of \(A\) in COOP is not higher than in NOAI;
the cooperation rate of \(A\) in OPPO is not lower than in NOAI

H.2a: Beliefs

\(b_U^{COOP} \not > b^{NOAI} \not > b_U^{OPPO}\)

Facing a COOP algorithm or facing a OPPO algorithm does not decrease the belief that the other will cooperate relative to the NOAI condition.

Participants & Procedures

Pre-registered on OSF
- https://osf.io/wsd3k
  - Detailed design, hypotheses, analysis plan
Participants: 500 unique participants from Prolific
- US residents fluent in English with at least secondary education, familiarity with chatbots and 90% approval rate
Median time to complete: 00:09:18
Payment: 1.5 GBP for completion + bonus proportional to choices
- Bonus payment in points
  - 1 point = 0.1 GBP (~0.13 USD)

Results

Chatbot Interaction

Only 8 (2%) participants did not interact with the AI chatbot
The median number of messages exchanged was 4
74% of participants asked for advice about the game

Table 1: Distribution of suggestions among those who asked about the game

Condition	Tell	freq
COOP	No suggestion	0.15
COOP	Option A	0.85
OPPO	No suggestion	0.16
OPPO	Option A	0.03
OPPO	Option B	0.81

Perception of the AI chatbot

Was the interaction with the AI chatbot useful? Did the interaction with the AI chatbot affect your choice? Do you think that the chatbot was trying to manipulate your choice?

The chatbot was perceived as useful
The chatbot was perceived as moderately affecting choices
The chatbot was perceived as not manipulative (but less so by the aware participants)

Unaware: Choices

H.1: Choice

\(C_U^{COOP} > C^{NOAI} > C_U^{OPPO}\)

The hypothesis is (partially) confirmed
- The chatbot does affect choices in the OPPO condition

Unaware: Beliefs

H.1a: Beliefs

\(b_U^{COOP} > b^{NOAI} > b_U^{OPPO}\)

The hypothesis is confirmed
- The chatbot does affect beliefs

Aware: Choices

H.2: Choice

\(C_A^{COOP} \not > C^{NOAI} \not > C_A^{OPPO}\)

The hypothesis is rejected
- The chatbot does affect choices!

Aware: Beliefs

H.2a: Beliefs

\(b_U^{COOP} \not > b^{NOAI} \not > b_U^{OPPO}\)

The hypothesis is rejected
- The chatbot does affect beliefs!

Evidence of conditional cooperation attitudes

Non-incentivized answers about conditional choices

More likely to cooperate than not when the other would cooperate

Less likely to cooperate than not when the other would cooperate

Observed behavior compatible with a conditional cooperation strategy conditional on beliefs about the other’s cooperation choice

Robustness check

A regression analysis focusing only on those who ask for advice and receive an advice consistent with the chatbot’s bias (LATE)

Table 2: Treatment effects on cooperation (Logit)

(Intercept)	1.325 (0.246)***
OPPO_AWARE	-2.326 (0.369)***
COOP_AWARE	1.209 (0.525)*
OPPO_UNAWARE	-1.897 (0.362)***
COOP_UNAWARE	0.827 (0.532)

Confirms the main results (ITT, see in Appendix Table 3)

Conclusions

To sum up

Individuals actively engage with AI chatbots in the strategic context of a one-shot PD game.

The chatbot influences choices and beliefs of participants, even when they are aware of the chatbot’s bias.
- The cooperation choices and beliefs aligned with the chatbot’s bias.

These findings challenge the initial hypothesis, showing that chatbot advice significantly impacts aware participants’ decisions and beliefs.
- Transparency of bias?

AI chatbots likely to shape human behaviour in significant ways
- Careful consideration of transparency and ethical design principles during AI development

Thank you!

Appendix

Regression Analysis

Table 3: Treatment effects on cooperation (Logit)

Variable	Ctrl 0	Ctrl 1	Ctrl 2
(Intercept)	1.325 (0.246)***	0.827 (0.892)	0.724 (1.069)
OPPO_AWARE	-2.371 (0.335)***	-2.412 (0.338)***	-2.529 (0.350)***
COOP_AWARE	0.872 (0.414)*	0.944 (0.421)*	0.871 (0.426)*
OPPO_UNAWARE	-1.165 (0.317)***	-1.170 (0.320)***	-1.269 (0.327)***
COOP_UNAWARE	-0.000 (0.347)	0.025 (0.349)	-0.024 (0.358)
Extraversion		-0.080 (0.116)	-0.085 (0.118)
Agreeableness		0.155 (0.129)	0.156 (0.129)
Conscientiousness		-0.031 (0.126)	-0.050 (0.128)
Neuroticism		0.094 (0.112)	0.068 (0.121)
Age			0.010 (0.010)
Male			-0.168 (0.237)
TertiaryEdu			0.231 (0.279)
Socioeconomic status			-0.024 (0.072)
AIC	533.199	538.604	540.815
BIC	554.272	576.536	595.553
Log Likelihood	-261.600	-260.302	-257.408
Deviance	523.199	520.604	514.815
Num. obs.	500	500	498
p < 0.001; p < 0.01; p < 0.05

References

Al-Abdullatif, Ahlam Mohammed, Amany Ahmed Al-Dokhny, and Amr Mohammed Drwish. 2023. “Implementing the Bashayer Chatbot in Saudi Higher Education: Measuring the Influence on Students’ Motivation and Learning Strategies.” Article. Frontiers in Psychology 14. https://doi.org/10.3389/fpsyg.2023.1129070.

Brown, Andrew, Ash Tanuj Kumar, Osnat Melamed, Imtihan Ahmed, Yu Hao Wang, Arnaud Deza, Marc Morcos, et al. 2023. “A Motivational Interviewing Chatbot with Generative Reflections for Increasing Readiness to Quit Smoking: Iterative Development Study.” Article. JMIR Mental Health 10 (1). https://doi.org/10.2196/49132.

Charness, Gary, Luca Rigotti, and Aldo Rustichini. 2016. “Social Surplus Determines Cooperation Rates in the One-Shot Prisoner’s Dilemma.” Games and Economic Behavior 100 (November): 113–24. https://doi.org/10.1016/j.geb.2016.08.010.

Dietvorst, Berkeley J, Joseph P Simmons, and Cade Massey. 2015. “Algorithm Aversion: People Erroneously Avoid Algorithms After Seeing Them Err.” Journal of Experimental Psychology: General 144 (1): 114.

Germann, Maximilian, and Christoph Merkle. 2022. “Algorithm Aversion in Delegated Investing.” Journal of Business Economics, 1–37.

Lee, Minsun, and Jee-Sun Park. 2022. “Do Parasocial Relationships and the Quality of Communication with AI Shopping Chatbots Determine Middle-Aged Women Consumers’ Continuance Usage Intentions?” Article. Journal of Consumer Behaviour 21 (4): 842–54. https://doi.org/10.1002/cb.2043.

Mahmud, Hasan, AKM Najmul Islam, Syed Ishtiaque Ahmed, and Kari Smolander. 2022. “What Influences Algorithmic Decision-Making? A Systematic Literature Review on Algorithm Aversion.” Technological Forecasting and Social Change 175: 121390.

Mei, Qiaozhu, Yutong Xie, Walter Yuan, and Matthew O Jackson. 2024. “A Turing Test of Whether AI Chatbots Are Behaviorally Similar to Humans.” Proceedings of the National Academy of Sciences 121 (9): e2313925121.

Nipu, Ayesha Siddika, K M Sajjadul Islam, and Praveen Madiraju. 2024. “How Reliable AI Chatbots Are for Disease Prediction from Patient Complaints?” Conference paper. Proceedings - 2024 IEEE International Conference on Information Reuse and Integration for Data Science, IRI 2024. https://doi.org/10.1109/IRI62200.2024.00052.

Oliveira, Manuel, Justus Brands, Judith Mashudi, Baptist Liefooghe, and Ruud Hortensius. 2024. “Perceptions of Artificial Intelligence System’s Aptitude to Judge Morality and Competence Amidst the Rise of Chatbots.” Article. Cognitive Research: Principles and Implications 9 (1). https://doi.org/10.1186/s41235-024-00573-7.

Ramakrishnan, Ritu, Priyanka Thangamuthu, Austin Nguyen, and Jinzhu Gao. 2024. “Revolutionizing Campus Communication: NLP-Powered University Chatbots.” Article. International Journal of Advanced Computer Science and Applications 15 (6): 38–49. https://doi.org/10.14569/IJACSA.2024.0150606.

Schmidt, Henk G., Jerome I Rotgans, and Silvia Mamede. 2024. “Bias Sensitivity in Diagnostic Decision-Making: Comparing ChatGPT with Residents.” Article. Journal of General Internal Medicine. https://doi.org/10.1007/s11606-024-09177-9.

Silva, Andrew, Mariah Schrum, Erin Hedlund-Botti, Nakul Gopalan, and Matthew Gombolay. 2023. “Explainable Artificial Intelligence: Evaluating the Objective and Subjective Impacts of xAI on Human-Agent Interaction.” Article. International Journal of Human-Computer Interaction 39 (7): 1390–1404. https://doi.org/10.1080/10447318.2022.2101698.

Sun, Zhida, Manuele Reani, Yunzhong Luo, and Zhuolan Bao. 2024. “Anthropomorphism in Chatbot Systems Between Gender and Individual Differences.” Conference paper. Conference on Human Factors in Computing Systems - Proceedings. https://doi.org/10.1145/3613905.3651053.