Jump to section
In our latest RAG Brag episode, we had the pleasure of speaking with John Wang, Co-founder and CTO of Assembled, who shared valuable insights into how Assembled Assist, the company's latest AI-powered tool, is transforming the world of customer support. Below, you'll find highlights from our conversation, capturing John's perspectives on the challenges and innovations in this space.
Assembled helps major companies like Stripe, Etsy, and DoorDash manage their customer support operations more efficiently. With a focus on optimizing support teams, Assembled provides solutions that help customer service agents handle everything from simple inquiries to complex problem-solving, all within an intuitive system that accelerates customer support operations.
Exceptional customer support goes beyond just having the right people answering queries; it requires quick access to accurate information— something not all support teams have. "Support operations have always been at the core of our strategy," says John Wang, Co-founder and CTO of Assembled. "We had been using data-driven approaches for workforce management and performance tracking. But there were gaps in the support journey that we wanted to fill: speeding up response times and providing more accurate answers."
An Inside Look at Assembled and Their Use of Pinecone
At the heart of Assembled customer support offerings is Assembled Assist, a powerful AI automation engine that analyzes incoming customer support tickets and generates high-quality, contextual responses. These responses can then be seamlessly integrated into the support workflow, either as suggestions for human agents to review and refine, or in some cases, automatically delivered to customers
"Assembled Assist was born out of a desire to fill the gaps in the support journey. We wanted to find ways to speed up response times and provide more accurate answers, all while keeping the human element at the core of what we do." - John Wang, Co-founder and CTO of Assembled.
The core of Assembled Assist is built on large language models (LLMs) like those from OpenAI and Anthropic. These AI models provide the language understanding and generation capabilities that allow Assembled Assist to interpret support requests and craft relevant responses.
But Assembled doesn't stop there. They've paired vector search using Pinecone with keyword search from Algolia to optimize the accuracy and relevance of the information surfaced for each support ticket. This hybrid approach leverages the strengths of both semantic and keyword-based search, enabling the retrieval of relevant results that might otherwise be missed by either technique alone. Through the use of Reciprocal Rank Fusion, Assembled has developed a flexible and scalable method for combining these diverse search results, ensuring that the most relevant information is surfaced quickly and accurately.
The result is a customer support solution that drastically reduces response times while maintaining high-quality interactions. With Assembled Assist, tasks that once took 40 minutes are now completed in just 2 minutes — a ~95% reduction of time, freeing up human agents to focus on more complex issues.
Challenges and Lessons in Getting Started with AI
When building Assembled Assist, the team had to navigate key decisions, including whether to focus on fine-tuning language models or use a retrieval augmented generation (RAG) approach.
John explains that the Assembled team ultimately found the flexibility and speed of updates offered by RAG to be more valuable than fine-tuning. As he states, "if you're betting on LLMs improving consistently in the world, which we are, being able to update prompts, and having a pretty generalized infrastructure, we're able to update our prompts very quickly with RAG." This approach not only enables fast iteration but also makes it easier to scale across customers by focusing on data quality rather than managing multiple techniques.
Another area of focus has been on the evaluation and feedback loop. When implementing AI-powered solutions, it’s crucial to identify key business metrics and measure the impact. John describes being able to leverage the data generated solving customer support tickets as a form of “free evaluation” of their Assembled Assist product. This involves analyzing whether a customer support ticket was handled by Assembled Assist, manually written by an agent, or through a combination of both. By evaluating performance across each scenario, they gain valuable insights into the performance of their product, helping them to continuously refine and improve the quality of the responses generated by Assembled Assist.
Finally, John shares some practical tips for others building AI products:
- Practice Writing Prompts: Writing effective prompts is a critical part of the process at Assembled. Engineers with experience in crafting prompts are significantly more effective, highlighting the importance of this often underrated skill.
- Leverage LLMs for Prompt Tuning: Assembled uses LLMs to help refine and optimize their prompts. This technique, known as "meta prompting," allows for rapid improvements by utilizing the LLMs to provide feedback and enhance prompt quality.
- Stay Connected to the AI Research Community: Maintaining strong ties with the AI research community is vital. Engaging with experts and staying updated on the latest developments in LLMs and generative AI helps integrate cutting-edge insights and keep the team at the forefront of AI innovation.
“Pinecone was a no-brainer for us. We needed to move quickly, and Pinecone was the leader in the vector database space. Its cost-effectiveness and ease of integration have been significant advantages, allowing us to focus on delivering value rather than managing infrastructure. We can test and adjust on the fly, which is crucial for maintaining high search quality and continuously enhancing our support solutions.” - John Wang, Co-founder and CTO of Assembled.
Assembled Assist shows how AI can revolutionize customer support. By combining LLMs with a hybrid search system, Assembled has created a fast and accurate product that delivers top-quality responses to customer inquiries.
The key takeaways? Mastering AI prompt engineering, staying flexible in how data is gathered and applied, and always focusing on enhancing customer experiences. AI products will continue to evolve, and so will their applications in customer support. But the goal remains constant: making support better, faster, and more helpful.
For more insights about John and Assembled, be sure to check out the full recording of our conversation or head over to their website. We will continue the RAG Brag series with more engaging talks featuring leaders in the AI industry. So stay tuned for those upcoming episodes!