Understanding customer satisfaction is more important than ever — the CSAT (“Customer Satisfaction”) metric is a popular tool, as part of a balanced picture, that many organizations take to help them get there. 

We had the pleasure of hosting Jill Kahn, VP Customer Support Engineering at Fastly, and Charlotte Ward, Head of Customer Support at Snowplow Analytics, to weigh the pros and cons of CSAT as a customer experience metric. At Fastly, a powerful edge-cloud platform that developers use to build groundbreaking apps,  Jill oversees the global Tier 1 through Tier 3 technical support teams and has achieved industry-leading CSAT. At Snowplow Analytics, a behavioral data management platform, Charlotte manages a global support team responsible for traditional front-line support, site-ops, and tech-ops support and preparing her team for a CSAT rollout later this year.  Charlotte also runs a fantastic podcast called Customer Support Leaders. Here are our top three takeaways from our discussion about the role of CSAT in measuring customer experience. 

CSAT's Simplicity Is a Double-Edged Sword

The fact that CSAT is a number is both a strength and a weakness. As a popular benchmark, it’s easy for most people to understand, even outside a Support environment. Jill likes “its binary nature where customers are either satisfied or dissatisfied because it makes it easier to triage over shades of gray.” Conversely, Charlotte says, because it’s a number, “it hides a lot of complexity and nuances in the underlying qualitative feedback. Often, customers are trying to tell you about their experience with your product vs. support or what kind of day they’re having. There’s no richness to it, and you really need to dig into what’s going behind that.” 

CSAT’s simplicity also leaves it open to customer interpretation, and disparate elements of customer experience often seep into CSAT ratings.  While CX teams can ask more specific questions, Charlotte said, “there’s only so much you can actually ask a customer, particularly in a transactional context in these frequent interactions across the regular touchpoints that we value.” Of the feedback Jill comes across in CSATs, she noted, “Roughly 50% is addressable by my team, and the other 50% is cross-functional. In addition to the Support feedback, it’s also my responsibility to triage that feedback to cross-functional teams.” 

While CSAT’s simplicity makes it attractive, Jill cautioned that “CSAT doesn’t tell you about the bigger picture of satisfaction across the customer relationship. You need to take into account much more about the customer’s health than one moment in time.” 

Furthermore, Charlotte highlighted an essential distinction between satisfied customers and successful customers. “A successful customer is probably satisfied generally, aside from the occasional hiccup,  whereas a satisfied customer is probably returning good CSAT most of the time, but isn’t necessarily entirely successful with your organization. By and large, one probably sits inside the other, but a CSAT score only speaks to the satisfaction and doesn’t speak to the success.” Jill agreed, “It would be shortsighted to operationalize CSAT as success,” given the many moving parts in the customer experience. 

Operationalize CSAT With a Grain of Salt

There are learning opportunities in every ticket, regardless of the rating. Charlotte noted that “we should approach good CSAT ratings the same way we approach bad CSAT ratings. We don’t want to glamorize or demonize one or the other because of all of the complexity. It’s not a target we use to compensate our front-line staff.” 

Jill’s team does set CSAT targets, but importantly, the targets are team-based. “ I don’t think we will ever incentivize an individual based on their CSAT because of the small sample size — one or two outliers can completely throw it off, and that may be because of something entirely outside of the team member’s control.” 

The transactional nature of  CSAT can also provide instant gratification and much-needed positive reinforcement to Support teams that spend their days fire-fighting. For example, Jill highlighted Fastly’s internal #caughtdoingsomethinggood Slack channel. The team highlights timely examples of good CSATs, prompting Fastly’s CRO to call client services employees individually and congratulate them on a job well done. 

Jill noted that in the rare instances where they receive bad CSATs, “I do pay special attention to those moments, and we’ll do a post-mortem on those tickets. The most important thing I need to take away from these instances is, ‘is the customer’s issue solved, or do we still have an issue?’ Separately, if the learnings are cross-functional, then we’ll follow out workflows for feedback loops. If there are coaching opportunities, they flow into one-on-ones for managers and the team member on the ticket.” 

The Best CX Measurement Is Predictive and Reflects Quality Standards 

It’s crucial to support CSAT with core quality standards and predictive technology that helps you see the complete picture and ensures you don’t miss anything important. 

One nuance of CSAT surveys can be customers’ reluctance to throw agents under the bus with poor ratings, even if their experience was poor.  Charlotte and Jill agreed that quality metrics like response and resolution times help them navigate this phenomenon. Charlotte noted that “quality will always be more important than CSAT. It’s also a leading indicator because we know what good Support looks like.” Jill described her approach — “we calibrate CSAT internally with our quality assurance monitoring metric, and evaluate Support Engineers on a rubric for handling metrics like response and resolution time. Our CSAT results are strongly correlated with our quality monitoring metrics.” 

Jill explained that Fastly has achieved outstanding customer satisfaction because Support is not a cost center. The company has made a significant investment in conversational support with Slack channels across their enterprise customer base, which means that their team is continually engaged in a collaborative discussion with their customers. Another key contributor to this is Fastly’s investment in AI-powered technology to predict escalations, which leads to fewer bad CSAT ratings. “We’ve used Frame AI to predict the sentiment and urgency of tickets. Even if a customer doesn’t submit a ticket as an emergency, it gets labeled as a predicted emergency or predicted issue so that we can prioritize those tickets above lower priority ones. We take a proactive approach to preventing bad CSAT and increasing customer satisfaction.”  

Charlotte emphasized the advantage that seizing opportunities to get ahead of customers affords — “the ability to understand things before they happen is key. Any opportunity you have to get ahead at scale is an opportunity you should take!” 

Incorporating AI into your CX strategy can also help you categorize feedback into relevant buckets for Support and other cross-functional teams. Charlotte said, “My dream is to simply invite customers to tell me anything that they want to tell me. I want to do the heavy lifting for them, whether it’s reading every interaction or using AI. It would be wonderful to then organize that feedback into areas that are relevant for me as a Support leader.” 

Thank you so much to Charlotte Ward and Jill Kahn for an insightful discussion about the role of CSAT in CX! Check out the full recording below for more.