Customer Support in the Age of LLMs

This is the first post in my "In the Age of LLMs" series, where I look at how AI is reshaping different industries. I am starting with customer support because I have firsthand experience in this space.

My Background in Customer Support

In 2019, I worked at a startup focused on customer support quality. We used machine learning to analyze text customer support tickets -- including those from a $1B fitness startup -- and evaluate them on quality criteria such as empathy, friendliness, and helpfulness.

Before our solution, the entire process was carried out manually, if at all. Reviewers could only review about 1% of tickets, leading to sampling error and bias. The gap between what was being measured and what was actually happening was enormous.

That experience gave me a front-row seat to the problems in customer support -- and why LLMs are such a big deal for this space.

What LLMs Change

The adoption of Large Language Models in customer support represents a significant leap toward more efficient and empathetic customer interactions. Here is what is changing:

Automated Resolution

LLMs can now handle a substantial portion of routine inquiries autonomously, allowing human agents to focus on complex and nuanced issues. Intercom, which has positioned itself as AI-first, launched their AI chatbot Fin, which solves up to 50% of support questions instantly. That is a massive reduction in ticket volume and response time.

Quality Analysis at Scale

Remember the 1% review problem? LLMs solve this entirely. You can now analyze 100% of support interactions for quality criteria -- empathy, accuracy, helpfulness, tone -- in real time. No more sampling error. No more bias from reviewing a tiny fraction of tickets.

Language and Localization

One simple but powerful application: I built a ChatGPT app that converts Hinglish (Hindi-English mix) to proper English text for customer support email responses. This is a gamechanger for teams where English is not the first language. The quality of written communication goes up dramatically with almost no effort.

The Platforms

Intercom

Intercom has gone all-in on AI. Their Fin chatbot is powered by GPT-4 and can answer customer questions using your knowledge base. Within hours of GPT-3.5's release, they began experimenting, and just four months later launched Fin. They committed $100M to replatforming the business around AI. Fin resolutions are billed at $0.99 each -- if your chatbot handles 1,000 queries monthly, that is an extra $990.

Zendesk

The legacy platform has also shipped AI features -- AI-powered agents, Agent Assist for summarizing tickets, drafting responses, and detecting intent for better routing. Zendesk's AI is more focused on agent productivity and workflow optimization rather than direct customer-facing automation.

Others

New entrants are popping up constantly, offering specialized AI support solutions for different verticals and use cases.

Build vs. Buy

This is the critical question. A large company with sizeable customer support volume -- think DoorDash or Swiggy -- will likely decide to build in-house. They have the volume to justify the investment, and customer support quality is directly tied to their core business.

But startups? Opt for off-the-shelf products. Customer support is important, but it is not core to your offering. Use Intercom, Zendesk, or one of the newer AI-native platforms and focus your engineering resources on your actual product.

Measuring Quality

AI does not just answer tickets -- it helps you understand the quality of your entire support operation:

• Collect objective and subjective feedback through CSAT, NPS, or other metrics and surveys
• Summarize, tag, and organize the text responses into a system that makes sense
• Draw correlations between customer support quality and end business metrics like revenue, profitability, and retention

This feedback loop is where the real value lies. It is not just about deflecting tickets -- it is about understanding what your customers are struggling with and feeding that back into product development.

Challenges

The transition is not without challenges:

• Accuracy and reliability: AI can hallucinate or give wrong answers. You need guardrails.
• Brand voice: Making AI sound like your company, not like a generic chatbot
• Escalation handling: Knowing when to hand off to a human seamlessly
• Customer trust: Some customers do not want to talk to a bot, period

The Bottom Line

LLMs in customer support are not a future prediction -- they are happening now. The benefits in cost savings and improved customer satisfaction are compelling. The companies that figure out the right balance of AI automation and human empathy will have a significant competitive advantage.

The key insight: AI should augment your support team, not replace it. Use AI to handle the routine, measure quality at scale, and free up your human agents to do what they do best -- handle the complex, emotional, high-stakes interactions that require genuine human connection.