Over the last couple of years, businesses have invested heavily in training their workforce on the use of AI technology, primarily focusing on fundamental skills like prompting. This training has enabled employees to become familiar with AI tools and recognize their potential benefits. However, as AI systems advance towards greater autonomy, this foundational level of interaction may not suffice.
Today’s AI systems are evolving beyond mere task execution—they can now set goals, utilize tools, make suggestions, initiate actions, and transfer responsibilities between systems. Consequently, the role of employees must adapt as they transition from simple users to individuals tasked with guiding and managing these AI agents. Gianpaolo Barozzi, Cisco's 3P Chief Technology Officer, notes that agentic AI alters the dynamics between humans and technology, demanding new approaches to setting boundaries, calibrating trust, and ensuring human accountability.
Unfortunately, many organizations have not yet shifted their training focus to address this evolution. Instead of preparing employees to lead AI initiatives, companies are primarily teaching them how to optimize AI tools. This oversight may create a significant skills gap as enterprises move into the next phase of AI adoption. Without a solid foundation in AI management, employees may inadvertently grant their AI agents more independence and trust than warranted by both the task and the technology.
The concern isn't that AI agents will consistently operate poorly; rather, the risk lies in their ability to perform helpfully, promptly, and convincingly. This capability may lead individuals to lower their critical judgment at precisely the moment it is most necessary to remain vigilant. A 2025 study by MIT Sloan Management Review and BCG revealed that 76% of executives view agentic AI more as a collaborative colleague than merely a tool. This sentiment indicates a growing perception of AI agents as teammates who actively participate in decision-making, coordination, and execution of tasks traditionally reserved for humans.
However, the term "teammate" describes various relationships and dynamics. As AI agents become involved in work processes, it's essential for humans to clearly understand the roles these agents play and manage them accordingly. Research indicates that people tend to engage with AI agents through one of five mental models: as a tool, intern, service provider, teammate, or expert. Each perspective carries distinct expectations regarding the agent's competence, autonomy, trustworthiness, oversight, and accountability.
When employees regard AI agents as tools, they anticipate them to complete specific tasks on demand, evaluating the results while retaining ultimate responsibility. While effective for straightforward, repetitive jobs, this approach may fall short as agents gain more autonomy. For instance, an employee might utilize an AI tool to summarize a meeting or format data, yet this structure fails as AI capabilities expand.
If an AI agent is treated like an intern, humans will generally provide context, closely monitor performance, correct errors, and gradually expand the agent's responsibilities. Though capable, the agent is still learning and requires guidance on the organization's standards and workflow. An example includes a manager asking an AI to draft a client briefing, then offering feedback to hone the agent’s skills.
Wharton professor Ethan Mollick has likened this relationship to that of an AI intern, as like any new staff member, AI requires a defined role, comprehensive context, clear tasks, and ongoing evaluations of reliability. The human must assess the AI's output, providing the nuanced judgment the agent lacks.
When functioning as a service provider, the AI executes tasks based on clearly established outcomes, scope, deliverables, and performance metrics defined by the human. The agent operates with a level of discretion within these parameters, ensuring visibility and oversight. For example, a procurement agent might manage sourcing processes adhering to set quality and compliance criteria while escalating significant issues for human intervention. Hala Jalwan, co-founder and CEO of Rivio.ai, emphasizes that as agents take charge of work, the human role transitions to one of oversight, where objectives and boundaries must be established to ensure accountability.
If the relationship is akin to that of a teammate, collaboration is key. The AI contributes ideas, coordinates work, adjusts to feedback, and engages in collective problem-solving, adding value through interaction. Despite this collaborative nature, the human retains accountability for outcomes—an agent may, for example, assist throughout product launches by developing strategies or following up on decisions.
At BNY, the company's approach highlights the importance of a structured human-agent dynamic. Leigh-Ann Russell, BNY's CIO, explains that when they launched their first digital employee, they applied the same governance principles used for other enterprise systems. This framework ensures that digital agents are continuously monitored and governed, reinforcing that while these agents contribute significantly, ultimate responsibility remains with their human counterparts.
When an AI is treated as an expert, the interaction shifts to a consultative model. Here, humans rely on the agent for specialized insights or analysis that might outstrip their own knowledge. The main concern in this model is the tendency toward deference; because the agent appears knowledgeable, users may hesitate to scrutinize its recommendations. For instance, a manager might ask an AI to analyze complex data sets and suggest where operational risks lie, but the human must still validate that the AI’s advice fits within the business's broader context.
Research at Procter & Gamble demonstrated how AI can excel in an expert role while contributing to collaborative efforts. In a study focused on product innovation, AI facilitated the generation of solutions that expanded employees' thinking beyond their areas of expertise. Less experienced team members improved their contributions through AI support, while the final judgment about the recommendations remained with the human participant.
While each model of human-agent interaction offers benefits, improper application can introduce unique risks. An employee perceiving an AI agent as an expert might overlook its shortcomings, while treating it as a teammate could lead to misplaced assumptions about its understanding of organizational context. Delegating outcomes without sufficient insight could be problematic if the agent is seen merely as a service provider, while excessive oversight could undermine efficiency if the agent is treated like an intern.
Moreover, an AI agent shouldn't be confined to one role. Depending on the task and context, the same AI can function as an expert for data analysis, a service provider for defined workflows, a teammate during problem-solving tasks, or an intern while learning a new process. Different tasks and risks dictate the appropriate model, and recognizing the agent's capabilities is vital to effective management. By labeling an agent as a teammate, organizations may encourage participation in work beyond mere output generation, but they must also determine the level of authority, supervision, and trust placed in it.
Companies must therefore focus on developing human-agent fluency, which entails understanding the dynamics of the relationship and managing the AI appropriately. This goes beyond traditional prompt engineering, which seeks only to extract better responses. Instead, human-agent fluency encourages critical thinking about the agent's role and the associated responsibilities of the human. Tom Lamberty, Senior Consultant at Cisco’s 3P Tech Office, articulates that a well-defined role for an agent influences how individuals interact with it, including the authority granted, the degree of questioning, and the boundaries of judgment retention.
Recent studies on agent framing emphasize this notion, revealing that individuals identified significantly fewer errors when they perceived artificial intelligence output as coming from an "AI employee" rather than a simpler chatbot. This discovery should not lead to a universal dismissal of teammate terminology, but rather to a deeper commitment to its implications. When AI functions as a teammate, it calls for established standards, challenges, oversight, and accountability—overtrusting a teammate without proper management compromises the collaborative effort.
Hence, organizations must clarify expectations for each model of interaction, delineating what decisions the agent may make independently, how its work will be evaluated, and the criteria for shifting roles. Employees need to cultivate the awareness to balance trust granted to AI agents, ensuring that it aligns with their capabilities and the task at hand.
The distinction between simply using AI and being prepared to engage with it effectively lies in this understanding. A workforce is only properly equipped when staff can identify the necessary relationship between humans and AI, manage it accordingly, and adjust as project goals evolve. Throughout all modes of interaction, maintaining human accountability for outcomes is essential.
