Chat Decipher

Use AI to extract questions from chat transcripts such as Slack and group together ones that are similar or duplicates.

Last updated June 7, 2023
ProductStack Overflow for Teams

When evaluating Stack Overflow for Teams, people often ask two questions:

  • How big of a problem are repeat questions? In other words, do I really have a problem that Teams can fix?
  • I know I have a problem, but how can I identify which questions should be in Teams to start?

This tool uses AI to read Slack transcripts and pulls out the questions contained within them, then groups together the ones that are duplicates or similar. This helps to address both concerns above. If you have a lot of duplicate questions, you'll better understand how much time and effort could be saved. Second, with both unique and duplicate questions, you'll be able to more easily pull them into a central repository to search and evaluate versus reviewing a random set of Slack messages. Customers can use the updated list to pre-seed questions and answers on Teams. Making the case internally of the amount of time wasted on answering the same set of questions in Slack or call-out content that hadn’t been socialized or distributed broadly enough throughout the company to enable teams to mitigate these knowledge distribution challenges quickly.

What we learned from the alpha

Every customer we spoke with saw the potential value in the tool. Knowledge silos in Slack are everywhere, and people are approaching this problem using manual efforts. Even the slightest improvement over a manual process is positive.

Running the app locally

Decipher was an app that runs locally via a Docker container. Getting this to run on a customer's machine was much more seamless than we suspected. Within one 30-minute meeting, we were able to download/install/run the app.

Question detection

The tool did a great job of actually detecting questions in Slack messages. It was not pulling in generic Yes/No questions; it was capturing the more complex ones.

Question summary statistics

Related to the above, the handful of summary statistics for questions was very interesting to the customers we tested with. Simply seeing the trend and volume of questions help them quantify the problem they are trying to solve with Stack Overflow for Teams.

The data

By far, the biggest challenge was having the customers get access to the raw Slack exports - it usually required them to go through their IT department. We believe that streamlining the export process of getting this data into the tool should be a top priority.

Question clustering

The other main challenge was another foreseeable one. After all the questions were detected, they are passed into a series of unsupervised machine learning algorithms. Unsupervised means the model does not actually have labels, so it does not know what is the ground truth about what questions are actually related to each other.

When testing with customer data, the clusters were not as usable as we would have liked. Luckily the customers were technical enough to feel comfortable adjusting some of the parameters. But they would have to do additional curation before handing them over to people.

Have ideas or opinions? Join the discussion on Meta
The meta sites are our sites behind the sites - they’re places to discuss site policy and ask questions about how the sites work. The Company also uses meta sites for two-way communication directly with community members. It’s our way of getting bug reports and feature requests as well as where we share upcoming changes, particularly on ‘Meta Stack Exchange’, the network-wide meta site.
View Meta post

About Labs

Since 2008 Stack Overflow has pioneered open source conversations in the technology community, helping us become the most visited, most trusted destination for developers in the world. In 2017, we unleashed the same productivity gains inside companies with Stack Overflow for Teams.

Knowledge sharing between peers and experts is fundamental to software development ‒ you can see it happening in Slack, in meetings, or quick hangs. Advances in technology, like GenAI, puts everyone in learning mode and knowledge sharing is at the core of that experience.

Stack Overflow for Teams sits at the very intersection of curiosity and innovation, a place to ask & answer your peers’ questions, learn from other experts within the company, and keep up with ‒ or be the driver of ‒ all new developments.

Our guiding principles

Find new ways to give technologists more time to create amazing things.
Accuracy is fundamental. That comes from attributed, peer-reviewed sources that provide transparency.
The coding field should be accessible to all, including beginners to advanced users.
Humans should always be included in the application of any new technology.

With these in mind, starting over the next few months, we will be sharing our ideas, opinions, designs, research and product ideas which combine emerging technologies with our platforms and services.