Enhancing Team Effectiveness with AI: A Squadify Case Study

In this article, I'll share insights from my work as CTO at Squadify, where we've been pioneering the integration of AI into team performance solutions. I'll walk you through our journey—from initial concept to implementation—highlighting the technical challenges we encountered and the valuable lessons we learned along the way. For those who prefer a video, you can watch the full presentation on the Mindstone platform.

Over the past year at Squadify, I've led our technical strategy to harness the power of large language models to transform how teams understand and improve their performance. What follows is the story of that transformation.

The Squadify Platform

Team performance is critical to organisational success. McKinsey memorably stated in their October 2024 report "Go, teams: When teams get healthier, the whole organisation benefits:

“...teams generate value as a primary unit of performance.”

– McKinsey

Traditional solutions for team performance development—executive coaching, leadership development, engagement surveys—often fall short in maximising team potential. This is where Squadify comes in.

Squadify is a SaaS platform and services solution that rapidly and continuously builds engagement and performance through data-driven coaching and AI-led development. We survey teams and use data to pinpoint areas of strength and opportunities for improvement. Unlike conventional feedback mechanisms, Squadify provides clear, actionable insights.

The Three Cs Framework

Our platform measures foundational metrics driving team performance through what we call "The Three Cs":

1. Clarity: How clear is the team about its mission, and how clear are team members about their role in delivering it;
2. Climate: What's it like to work in the team? This examines the team's working environment;
3. Competence: Does the team have the skills and resources to deliver on their mission.

Beyond these core metrics, Squadify also analyses key team dynamics from Alignment to Psychological Safety. Each metric shows teams where they are, where they need to be, and how they compare to other teams, with the ability to drill down for deeper insights.

The AI Integration Challenge

While Squadify has proven effective—with some teams experiencing up to 30% performance gains—we identified a significant limitation: turning data insights into action required additional expertise that not every team had access to. According to a Kings Fund study, transforming a group of people into a team yields a 20% performance improvement. We wanted to make that transformation accessible to everyone.

In March 2023, four months after ChatGPT's release, OpenAI launched their API and GPT-4. The significant capabilities of GPT-4 prompted us to explore how AI might support teams in making their Squadify data actionable.

Our AI Implementation Journey

Challenge #1: Getting from Numbers to Insights

Our first challenge was converting numerical data into meaningful insights. When we fed team data to the LLM and asked for useful feedback, the results were unreadably full of numbers—sometimes to many decimal places.

The breakthrough came with implementing a multi-stage process:

1. First, we asked the AI to evaluate the team data—identifying strengths and areas needing improvement
2. Then, we summarised these insights for human consumption, creating much more "human-friendly" content

We implemented this using Langchain, an open-source framework for developing LLM applications. Despite criticisms that Langchain is overly abstracted, our experience was positive. It facilitated multi-stage processing and allowed easy switching between different LLMs for testing.

Challenge #2: Remembering What We'd Forgotten We Knew

As we iterated on our prompts, we repeatedly discovered we needed to explicitly tell the AI things we had internalised. For example, we had to help the AI understand whether a score was good or not.

We found that without guidance, the AI might criticise a team for their lowest score, even if that score was objectively excellent (like a 4.7 out of 5). To solve this, we developed extensive prompt boilerplate to help the AI interpret numbers correctly, calibrated based on our large dataset of team results.

We also provided guidance on tone, how to address the team (using "you" instead of "we"), and avoiding advice about things teams couldn't directly control (like "be happier" or "get more resources").

Challenge #3: Delivering Actionable Guidance

Rather than simply describing team states, we wanted to provide potential focus areas and actions for improvement. We structured our solution to generate focus areas based on the team's state description, then derive coaching questions and recommended content to help teams progress.

For content recommendations, we implemented Retrieval Augmented Generation to identify relevant clips from Squadify's "We Not Me" podcast, including playback links to specific segments.

We're now extending this to recommend varied resources, including integration with learning management systems used by our enterprise clients.

To test that the guidance was good, we worked with our network of experienced Squadify coaches. We developed a scoring system for the LLM's output, and then iterated with the coaches from a starting position of around 5/10 until the average score got to 8.3 out of ten. We decided that this was good enough to go live.

Challenge #4: Moving to Production

Transitioning to production raised several concerns, particularly around monitoring and managing the AI system. We implemented Langsmith, a service from the Langchain developers.

This tool:

• Helps unpick complex AI interactions in a chain;
• Makes it easy to filter and examine results;
• Tracks LLM costs;
• Stores end-user feedback (thumbs up/down on AI-generated items);
• Provides a Prompt Hub for managing our prompts with full versioning and testing support.

This approach gave us flexibility in prompt management without disrupting our development team's workflow.

Current State and Future Plans

Today, every team using Squadify receives expert analysis of their report and a set of actionable recommendations with coaching questions and resources to improve team functioning. Our system is flexible, maintainable, and powerful.

Looking ahead, we're working on:

• Time-based analysis and insights—adding AI to existing graphical displays of data
• Guided journeys—AI handholding through a team's development path
• Enhanced recommendations—Learning Management System integration
• Data-driven team coaching chatbot

Onwards and upwards

Our journey integrating AI into Squadify taught us valuable lessons about translating data into insights, transferring domain knowledge to AI systems, and creating actionable guidance teams can use independently. By combining human expertise with AI capabilities, we've created a solution that makes team performance improvement more accessible and effective for organisations of all sizes.

If you'd like to learn more about how Squadify can help your teams, please visit the website.

If you'd like to talk to me about implementing AI in your organisation, please get in touch using the button below.

Footnotes

1. I was in Episode 16 - Teams in the Metaverse ↩