AI as a team member? Lessons from an experiment

Artificial intelligence (AI) is often presented as the driving force behind the future of work. But what happens when we use AI not just as a tool, but treat it as a full-fledged team member?

In the article The Cybernetic Teammate, Ethan Mollick discusses an interesting experiment conducted in collaboration with researchers at Wharton and Procter & Gamble (P&G). The study examines how generative AI (specifically GPT-4) affects team collaboration and performance.

While the results of the study seem promising at first glance, some caution is in order. The study is based on a controlled experiment with a limited group of participants in a single organisational context. That makes it a valuable first step, but not yet a broad conclusion about how AI fundamentally changes organisations.

The experiment in a nutshell

In the study, P&G employees were divided into four conditions:

1. Individuals without AI

2. Teams without AI

3. Individuals with AI

4. Teams with AI

Each participant or group was given a complex task, involving both technical and commercial interests. The goal was to develop a balanced and thoughtful proposal. In the variants with AI, participants worked with GPT-4, integrated into their workflow as a kind of virtual teammate.

Already intriguing findings

The results of this experimental study are certainly intriguing and a great start to broader research on the effects of AI use in professional organisations.

1. Improved performance when using AI

In this particular experiment, both individuals and teams performed better when using AI. The best performing group consisted of teams using AI. They scored on average 0.39 standard deviation higher than the control line. Individuals with AI also scored significantly better than those without.

Still, it is important to keep in mind here that this was a simulation, where performance was judged by a panel of experts. How this translates to everyday work situations in other industries or team structures remains uncertain for now.

2. Blurring of expertise boundaries

A striking result was that participants, regardless of their background (technical or commercial), arrived at more balanced solutions using AI. AI seemed to help bridge functional boundaries between disciplines. Even more so than in teams that did not use AI.

The researchers suggest that this may allow AI to help with multidisciplinary thinking - but again, this requires follow-up research. The question remains whether AI can actually replace or complement deep domain knowledge in the longer term.

3. Positive emotional impact

Participants reported feeling more energized, more engaged and less frustrated when using AI. This may indicate that AI plays a role in reducing mental strain in complex tasks.

But again, context is important. Working with AI in a controlled experiment is different from working in a busy work week full of emails, meetings and deadlines. Whether this kind of positive experience holds up in practice is still unknown.

Limitations of the study

Although the study provides interesting insights, it is worth noting its limitations:

• Small, specific target group: The participants were all employees of P&G, an organisation with a particular corporate culture and high degree of professionalism. Thus, the results are not easily transferable to other sectors or job levels.

• Simulation, no real-life workload: The assignments were carried out in an experimental setting, without the complexity and dynamics of real projects, customer requests or internal processes.

• Third-party assessment: Performance was assessed by outside experts, which is useful, but also prone to subjectivity.

These limitations do not diminish the value of the research as an exploratory study, but make it important to be cautious about grand conclusions.

Cautious optimism: opportunities as well as questions

The study shows that AI can play a positive short-term role in supporting collaboration and improving output in specific contexts. AI seems able to help employees look beyond their own area of expertise, and can help formulate balanced proposals.

Yet the research also raises questions:

• What does it mean for the division of responsibility when AI acts as a team member?

• How does AI affect long-term collaboration, trust and decision-making within real teams?

• What skills do employees need to work effectively with AI?

These are questions that organisations need to address - preferably on a small scale, with room for learning and reflection.

AI: smart assistant or full-fledged team member?

The idea of a “cybernetic teammate” is intriguing, and this research represents a valuable initial exploration. But it is important not to overestimate AI. We are still at a stage where experimentation, critical evaluation and careful implementation are crucial.

AI has potential as an enhancer of collaboration and decision-making, but it is too early to speak of a fundamental shift in how teams function. For now, AI may be better thought of as a smart assistant rather than a full-fledged team member.

The next few years will be decisive: can organisations experiment sensibly with AI in teams? And will they succeed in focusing not only on technology, but also on the human side of collaboration?

Read the original article The Cybernetic Teammate by Ethan Mollick.