EY is taking steps to optimize its internal AI infrastructure by implementing an "invisible" router, which serves as a smart intermediary for directing employee queries to the most suitable AI models. This innovation, unveiled globally in April, was designed to manage expenses associated with AI usage, according to Dan Diasio, EY's global consulting AI leader, in an interview with Business Insider.
This router operates behind the scenes of EY's specialized AI tools, seamlessly guiding users to the most appropriate AI model for their needs. "To the users, it makes no apparent change," Diasio explained. Users simply access the frontier model, input their query, and receive a response, while the router efficiently determines the optimal path for processing that request.
Interestingly, this routing system can often yield quicker results than using the heavier models directly, as the larger models require more computational effort and can have slower response times. Diasio noted, "Many times, it's faster than the frontier models."
With AI usage costs becoming a pressing issue for many organizations—exacerbated by AI providers like OpenAI and Anthropic shifting to token-based pricing—EY has recognized the need for cost management. The EY US's AI Pulse survey revealed that 82% of senior leaders in AI-investing companies expressed concerns over rising token consumption. This survey, which collected responses from 534 decision-makers between April and May, highlights the financial implications of inefficient AI tool usage.
Diasio pointed out that excessive bills often stem from a small percentage of employees misusing the available AI tools. The router effectively mitigates this issue by selecting the most suitable model for each task, reducing unnecessary token expenditure and ensuring that more costly AI resources are reserved for high-value functions. The implementation of this system has been particularly impactful in specific departments, such as tax and risk, rather than across all AI tools at EY.
Since the router's global deployment, EY reports a significant reduction in token use, achieving a decrease of 60% in consumption as of April, underscoring its effectiveness in enhancing AI token management.
As the corporate landscape evolves, the focus has shifted from merely encouraging employee participation in AI technology to strategically controlling and optimizing its application. Major companies like Disney and JPMorgan have begun tracking AI usage through dedicated dashboards, while new startups are emerging to facilitate task routing and monitor expenditures.
To further manage AI costs effectively, EY has implemented token budgets tailored to various roles and departments. Employees who surpass their token limits can request additional allowance through an approval process. The AI Pulse report indicated that 64% of senior leaders are now actively monitoring AI token consumption and have established spending limits, a figure that Diasio anticipates will increase to about 90% in the coming months.
Developing a robust internal knowledge base and crafting better prompts using company-specific data are also vital strategies for optimizing AI investments, according to Diasio. Companies should concentrate their resources on areas where AI can significantly enhance productivity, rather than indiscriminately applying funds across all functions.
As businesses navigate the next phase of AI deployment, it will become increasingly essential to focus on value and measurable outcomes, moving beyond traditional profit assessments. Diasio suggests organizations should ask deeper questions: Is AI helping employees work more efficiently? How can businesses encourage employees to take on new challenges that extend beyond their standard job descriptions?
