The Peril: What are the main risks of AI agents?
Deploying AI agents isn’t without risks. In fact, underperforming or hallucinating agents can lead to serious consequences for enterprises that put too much trust in a faulty system. Much like hiring a new employee, building trust in an AI agent relies on two key factors: competence and character.
Competence risks
A weak memory layer spells foundational failure
The memory layer is the single most important component to get right. It is the foundation upon which the entire agentic AI system is built. It ensures your agent has the right knowledge to perform its tasks reliably and accurately.
Without a scalable, flexible, and secure memory layer, even the most advanced AI will fail to deliver trustworthy results, which could cause serious disruption. This can look like fundamentally misunderstanding the steps necessary to complete a task by relying solely on parametric reasoning (especially for smaller or legacy models), as well as the inclusion of hallucinated responses and facts into the content of the task the agent is executing.
EXAMPLE
Imagine deploying an AI travel agent for booking flights and filing expenses automatically. If the agent hallucinates a corporate discount or fabricates nonexistent ticket availability, your team could arrive at the airport with invalid tickets. The result? Missed opportunities, reputational damage, and a financial mess to untangle.
The memory layer acts as the foundation for any agentic AI system – making it the most critical component to build correctly.
Traditional LLMs struggle with complex reasoning
Agentic AI reasoning occurs at the cortex layer, where the LLM plans a course of action to accomplish its task. Advanced reasoning models, such as OpenAI’s o series or DeepSeek’s R1, exhibit strong performance solving complex, multistep problems. These models are specifically trained to optimize the chain of thought—the sequence of steps the model brainstorms in the planning phase—to increase performance. However, these sophisticated models can be very costly, especially when deployed at an enterprise scale.
For budget-conscious teams or builders of early-stage agentic systems, traditional, non-reasoning-centric LLMs may seem like an attractive alternative due to cost. However, these models often struggle with tasks that require nuanced, multi-step reasoning and execution. When applied to flexible or generalizable agents, these models may fall short, leading to poor task planning and eventual failure, regardless of how well the rest of the agentic system is configured.
To strike a balance between cost and functionality, teams should consider connecting their LLM to a knowledge base of best practices and standard operating procedures (SOPs). This strategy combines the cost-efficiency of smaller or legacy models with a minimized risk of failure.
Shifts happen, action errors erode efficiency and derail operations
Failure at the action layer is the most straightforward of the system component failures. Even with a perfectly planned approach and flawless recall, a task can fail if the computer use or RPA component shifts the mouse click by just a few pixels. Such small errors can prevent the task from being executed correctly.
Traditional RPA solutions attempted to address this issue by becoming increasingly brittle. However, advancements from large AI labs like Anthropic and OpenAI have introduced more flexible multimodal models. These models significantly enhance visual understanding of user interfaces, offering a promising solution to such action-layer failures.
As this technology continues to evolve, these failures will become increasingly rare. Until then, agent builders need to strike a balance between automation and human oversight. For example, shifting a task from requiring full manual effort to a process where the employee only needs to provide final verification still delivers significant value. This approach preserves efficiency while mitigating potential failures.
Character risks
Generative AI lies
LLMs focus on language fluency rather than on factual accuracy (i.e. they know how to speak, not what to say). This limitation means that AI models may generate outputs that sound convincing but are detached from reality.
Compounding this issue is the fact that real-world representations are encoded into the models’ parametric memory. As a result, LLMs may produce responses that align with human perceptions of truth rather than an objective or unbiased reality. In enterprise or government contexts, this poses serious risks.
Grounding has become an essential prerequisite for deploying these models effectively, particularly in agentic systems. In conversational applications, answers are delivered directly to a human user, providing a natural "gut check". However, in agentic systems, hallucinations and inaccuracies can flow directly into automated workflows, leading the agent to not only say the wrong thing, but do the wrong thing, amplifying the potential for errors.
AI can steal or leak your data
To improve accuracy, AI agents are often grounded in proprietary and personal data sources. While this provides better performance, it also introduces risks of exposure or breaches of sensitive data, requiring those sources to be properly handled and protected.
Ensure your system has appropriate guardrails to comply with local or national data privacy standards, respect role-based access control levels (ACL), and include measures to mask personally identifiable information (PII).
Keep in mind that AI agents function as integrated systems. Whether you’re building the system yourself or purchasing components from a vendor, every component must adhere to your data privacy standards and be architected to safeguard your information.
