Despite the excitement from executives about the potential of artificial intelligence agents to handle an array of office tasks with relentless efficiency, the technology itself remains fragile and could become a financial liability.
This week, at two distinct gatherings in Silicon Valley, leaders and engineers explored both the thrill and the hurdles tied to AI agents. Kevin McGrath, CEO of the AI startup Meibel, highlighted a significant issue in the field: the common misconception that every task must go through a large language model (LLM).
He cautioned against mindlessly allocating resources to AI systems, saying, “Just give all of your tokens and all of your money to an AI Claw bot that will just waste millions and millions of tokens,” urging companies to be cautious and strategic about which tasks are appropriate for AI utilization.
The surge of interest in AI agents has been fueled by innovations like OpenClaw, a platform enabling developers to utilize different AI models for building and overseeing teams of digital assistants, which have been touted as the next big breakthrough in the tech world. Nvidia's CEO Jensen Huang even characterized this development as the evolution beyond ChatGPT during a conversation with CNBC's Jim Cramer in March.
However, discussions at the Generative AI and Agentic AI Summit in San Jose made it clear that deploying and managing AI agents is fraught with complexities. A session featuring Google software engineer Deep Shah emphasized the need for new methodologies aimed at controlling the operational costs associated with numerous AI agents.
Running these digital assistants involves expenses, and a poorly structured system to oversee their activities can result in unexpected financial drains rather than savings. Shah elaborated, “If you think of a machine learning system or any multi-agent system, there are multiple challenges you will find when you try to deploy that system at scale. The first one is the inference cost.”