“A new way of thinking first requires a fresh way of seeing or measuring the world around us.” 

Every business is competing on customer experience, yet nearly 90% of companies lack strong metrics to measure and improve. TS Balaji, VP of Experience Design and Customer Experience at Cox Communications, says that AI and analytics can add much-needed clarity, depth, and automation to CX measurement. Over the last 15 years, TS has played an instrumental role in building winning experiences in both B2C and B2B companies. He highlighted some of the most noteworthy trends shaping the future of CX measurement and offered several examples of how he has used AI and analytics to improve CX. Here are three top takeaways from TS’ presentation:

Surveys Have Peaked as a Standalone Measurement Solution

Surveys routinely bombard us in both our personal and professional lives. Many organizations are reacting to immense pressure to improve customer experience by sending more surveys to more customers. “We’ve gone completely overboard,” said TS. “Companies believe that having more feedback is better, but don’t always have a clear mission as to how that information will be used, or an understanding of the context in which they are getting that feedback, or what they will do once they get it.” Too often, companies squander valuable opportunities to engage with customers on survey questions they should be equipped to answer themselves. TS cited the ubiquitous airport bathroom rating touchpads as a familiar example — surely, the sanitation provider need not depend on hundreds of visitors touching a dirty screen to know whether or not the bathroom needs cleaning. By the time the sanitation provider gathers enough survey data and decides to clean the bathroom, they have already exposed many customers to a substandard experience. 

In a well-meaning attempt to capture as much information as possible, many organizations have simply gotten too carried away with asking questions. Building on the airport analogy, TS questioned what a worthwhile outcome could be to ask travelers to rate their passport control experience with a smiley face?

Doubling down on surveys is yielding diminishing returns. Not only are response rates low and flat as customers grow increasingly disillusioned with surveys, but worse, there is often a severe disconnect between the ratings that CX teams do receive and the actual underlying experiences. CSAT surveys typically limit companies’ ability to understand the true customer experience due to quarantine rules, scale, and other rules created by organizations. Many well-intentioned customers give positive ratings despite negative experiences, complicating organizations’ ability to discover and address the cause. Conversely, many customers unwittingly shoot the messenger — traditional measurement methods struggle to attribute a strong sentiment to service quality, product quality, or another unknown that may have influenced the rating. 

Without the deep coverage and context that AI-based analysis and operational data afford, surveys fail to provide meaningful insight. In the airport example, important operational data could include the timestamp of the last bathroom cleaning or traffic volume throughout the day. In a B2B context, important operational data could consist of ticket resolution times for certain types of issues or the volume of tickets across known and unknown issues. “We often found ourselves becoming NPS historians or survey detectives — we are always going back and asking, ‘Why did the score go up? Why did it go down?’” Without clear, high-quality data to align your efforts around, improving the customer experience becomes far more complicated than it should be. A combination of AI-based analysis, operational data, and surveys allow Cox to focus on actions that improve the customer experience instead of investigating the status quo. 

AI and Analytics Provide the Strongest CX Measurement Foundation 

While there’s still an important role for surveys, leading CX organizations ground measurement in AI and analytics. Where surveys often disappoint with sparse data, AI offers a significant scale advantage by delivering insight from the customer voice across every single customer interaction. TS walked us through how Cox is using AI-based sentiment analysis to automate measurement across call center and chat interactions. “The beauty of AI is that it allows us to capture meaningful insight across 100% of our customer interactions, whereas, with surveys, it’s very easy to get hung up on the rules-based logic, such as who you’re going to survey, what questions you’re going to ask, and when.” For many organizations, defining and iterating on these rules consumes a great deal of time and resources that are arguably better spent on improving the customer experience.

AI-based sentiment analysis produces passively assigned scores that consider factors such as tone and silences and populate across interactions in real-time. TS reported that the ability to provide real-time suggestions to support agents makes an enormous impact on the likelihood of delighting customers. 

In addition to replacing CSAT with AI-based sentiment analysis, TS offered an approach for reimagining CES, or Customer Effort Score, using analytics. Traditionally, companies measure CES with a survey question that asks customers to rate how easy or difficult it was to engage with support. Rather than rely only on surveys, TS recommended leveraging analytics for a complete picture of your customer experience. In TS’ experience, his team sought to improve content efficacy across support articles, blogs, and curated community-generated solutions. His framework starts with how many customers visit the Support site and how many people initiate chat and phone conversations. 

A typical customer support journey often starts with visiting a company’s website, which may offer some support resources like FAQs and articles. The customer may then initiate a chat if they don’t find what they’re looking for, and then the chat may escalate into a phone call if the issue is still not resolved. How are customers finding the content? Is it relevant to their questions, and if yes, did they feel like the provided solution fully addressed their issue? If not, did they call us?

Support engagement data is used to maintain a hierarchy of content performance, and used in conjunction with questionnaires, aggregates up to a weighted index across all support content. This index became a successful mechanism for TS’ team to continually evaluate and launch more appropriate content for the site, “without having to constantly ask our customers questions about how well we are doing.” From there, the team was able to build on small improvements, such as changing some articles from plain text to rich text and using more videos in conjunction with written content. 

Improve Measurement at the Transactional Level to Improve CX Across the Entire Customer Journey 

While there are seemingly endless AI applications in CX, TS explained that replacing transactional CSAT surveys should be a natural starting point for many companies. “Using AI to measure experience at the transactional level makes the most sense because of the volume you’re dealing with.” Additionally, examining survey effectiveness across different use cases, TS highlighted that the transactional level is often the weakest link. 

With CSAT surveys, you’re never going to have a great set of data that can actually talk to that particular agent who provided the service in question. The effectiveness of any given survey goes down as you get to an individual level. In a call center environment, one agent may only get a few surveys per month, and that’s not a great way to provide feedback. So what’s an effective way for agents to understand how they’re doing and how customers are feeling without having to depend on a survey response?

In terms of evolving Cox’s ability to measure customer experience, TS saw the initial move “from transactional surveys to using AI for 100% of interactions as a stair step.” Having rock-solid measurement at the transactional level also allows organizations to better understand the end-to-end customer journey, instead of looking at individual touchpoints, as too many organizations do. To date, CX measurement has focused on the transactional level and the brand level, traditionally measured by CSAT and NPS surveys, respectively — the journey level is between the transactional and brand level. TS explained that many companies fail to understand the journey level due to ineffective measurement methods across the transactional interactions that comprise each part of the journey. 

If there are three interactions in a journey, the likelihood of success across that journey is a product of those underlying interactions. As a result, it’s harder to see success across a journey than across an individual interaction. Achieving positive brand perception is even more challenging because of the multitude of interactions a customer could have had with your company. 

High-quality measurement at the transactional level is essential because these interactions are foundational, not only to customers’ perception of their overall experience but also to organizations’ ability to measure the journey level.

I believe the next evolution of experience measurement is a focus on the journey level. You can start by breaking your experience into smaller journeys like activation, order to install, or issue to resolution. Then, you can find insights into what’s driving satisfaction in every journey. Start with improving measurement at the transactional level, and build up from there to measure and improve journeys.

While the benefits of leveraging AI and analytics for CX measurement are clear, the transition from a survey-first approach can seem daunting for many organizations. At Cox, TS’ first step to replacing transactional CSAT surveys with AI-based sentiment analysis was a tightly scoped pilot — this allowed an initial group of managers to build confidence in the accuracy of the analysis before abandoning the familiar surveys. 

We asked, ‘Is the AI analysis representative of the interactions we’re having?’ A critical consideration was also interpretability — it was essential to ensure that people understood what the analysis suggested, compared to the survey measurement. Once we got great feedback from the pilot, we moved into a trial with the entire call center, from which we determined that sentiment would scale across the organization at large. Now with the launch, now that we’ve aligned on sentiment as a measure, we can start to incorporate that into KPIs, scorecards, dashboards, etc. And next, there’s the change management piece, where we work on getting everyone from individual contributors to functional leaders educated on how to read these scores.

 TS offered a vision for CX teams to ultimately operate like their own small, nimble SaaS companies operating at great speed, even in the context of massive organizations. “AI presents a great opportunity to drive the pace of innovation. My core belief is that purpose-built AI is going to win the marketplace.” Acknowledging the presence of AI skeptics in some companies, TS emphasizes that “using AI solutions in a way that can be tied into operations is going to be the key to success. If you can start to use that to show operational and business results, that’s going to be the easiest way to win over skeptics and smooth over headwinds.”

A big thank you to TS Balaji for his generously contributing his exceptional insight on CX measurement. Check out the video below for more, get on the list to make sure you don't miss the next one!