Rashmi Shetty on Building AI Agents for Complex Tasks

Rashmi Shetty, Senior Director of the Generative AI Platform at Capital One, shared insights on the evolution and development of AI agents during a recent TWIML AI Podcast episode. The conversation, hosted by Sam Charrington, explored the transition from traditional machine learning models to more sophisticated, multi-agent AI systems capable of performing complex tasks.

Rashmi Shetty's Background

Rashmi Shetty holds a significant position at Capital One, leading the Generative AI Platform. Her background is rooted in academic research, with her doctoral thesis focusing on pervasive computing and decision-making systems. This academic foundation has informed her approach to developing AI solutions in enterprise settings. Her work at Capital One places her at the forefront of leveraging AI for practical business applications, particularly in the financial sector.

The full discussion can be found on TWIML's YouTube channel.

The Evolution from Generative AI to AI Agents

Shetty described the progression in AI development, moving from a focus on generating responses to enabling AI systems to take action. While generative AI models excel at producing text or other outputs based on prompts, AI agents are designed to understand a goal, break it down into discrete steps, and execute those steps to achieve the objective. This shift represents a move from passive information generation to active problem-solving.

Shetty elaborated on this transition, stating, "We moved from a classic ML world to a world where we have LLMs generating responses, and now we want to move on to a world where actions need to be taken." This highlights the growing demand for AI systems that can not only understand but also act upon complex information and achieve specific goals.

The Core of AI Agents: Planning and Reasoning

The fundamental difference between generative models and agents, according to Shetty, lies in their ability to plan and reason. While generative models are primarily concerned with outputting relevant content, agents must be able to deconstruct a problem, devise a strategy, and execute a series of actions. This requires a sophisticated understanding of the problem space and the ability to adapt based on intermediate results.

Shetty explained the underlying principle: "When the problem that we are working on is a complex one, that's where multi-agentic comes into play." This means that for intricate challenges, a single AI model may not suffice. Instead, a system composed of multiple specialized agents, each handling a part of the problem, becomes necessary. This distributed approach allows for more nuanced and effective problem-solving.

Capital One's Journey and the "Chat Concierge" Project

Shetty shared Capital One's experience in developing and deploying such systems. She highlighted their "Chat Concierge" project as an example of applying these principles. The goal was to bridge the gap between the customer's intent and the dealership's inventory, making the car buying process more seamless. This involved understanding the customer's needs, identifying suitable vehicles, and facilitating actions like scheduling test drives.

Shetty detailed the reasoning behind their approach: "The principle is that we are a tech organization that does banking, and we have always been at the forefront of modernization... Similarly, the AI journey began a few years ago, like early 2023, when we embarked on this mission. We had our first agent pilot which went live in 2023, and now we have presence in the generative AI presence in the past two years." This indicates Capital One's sustained commitment to integrating advanced AI capabilities into their operations.

The Role of Observability and Evaluation

A critical aspect of developing robust AI agents, as discussed, is observability and evaluation. Shetty emphasized that the agents need to be designed with mechanisms to monitor their performance and provide feedback. This includes evaluating the agents' ability to execute tasks accurately, efficiently, and safely within the operational constraints.

She stated, "The key differences are that in the past, we were focused on model performance metrics, but today, we need to look at the agent's performance, which is essentially the end-to-end execution of a task, and then we need to evaluate that based on the outcomes." This shift highlights the need for a more holistic approach to AI system evaluation, focusing not just on individual model capabilities but on the overall system's ability to achieve its intended goals.

Challenges in Agent Development

Shetty also touched upon the inherent challenges in developing AI agents, particularly in highly regulated industries like banking. The need to ensure that agents operate within strict compliance frameworks, maintain data privacy, and adhere to ethical guidelines is paramount. The ability to reason, plan, and execute actions must be balanced with robust governance and safety measures.

"The reason why we have been so forward-thinking is that we have always considered our data advantage, our AI advantage, and we have always considered our platform strategy," Shetty remarked, underscoring the importance of a strong foundational strategy that integrates AI capabilities seamlessly into the broader business operations.

The Future of AI Agents

The conversation concluded with a look towards the future, emphasizing the continued evolution of AI agents. As these systems become more sophisticated, they will play an increasingly critical role in automating complex tasks, enhancing decision-making, and driving business value across various sectors. The focus will likely remain on building agents that are not only intelligent but also reliable, transparent, and aligned with human values.