Understanding AI Agent Hype: Promises versus Potential

How organizations should think about the hype cycle's latest darling agentic AI

AI Agents are fueling the generative AI media outlets with a new wave of hype, quickly turning the conversation away from pure LLM capabilities and algorithms to the new “intelligent” future of autonomous AI. In this promised future, AI agents will act independently of humans to accomplish complex tasks using algorithmic “reasoning” and any frameworks governing the algorithm to determine the most probable course of action. 

Major enterprise software providers like Salesforce, Microsoft, and IBM are quickly moving to integrate agents into their offerings, and LLM providers such as Open AI and Claude are providing agentic enabling algorithms. The hype is exciting and more grounded on business needs, accomplishing hard, costly, and/or rote tasks without human oversight. However, enterprises and organizations need to proceed carefully with AI agents, as there are significant barriers to entry.

First, the hype here is significant. From the overambitious billing of AI agents as intelligent bots, the capability of acting autonomously of humans is an aggressive positioning statement in the kindest of lights. Further, the agentic future of AI is just becoming capable and is not here yet on a widespread, accessible basis. While initial prototypes appear and initial vendor offerings become available, agentic AI is early in capability and actual use cases.

Secondly, the road to implementing AI agents is not simple. Big technology companies’ marketing often positions AI as pixie dust that can solve challenging human problems. However, these systems are not yet capable or easy to build. In addition to limited but rapidly evolving technological capabilities, data, and resource challenges can ground AI agent deployments right out of the gate. Let’s take a deeper look. 

What Are AI Agents, and Are They Intelligent?

In the promised future of the hype cycle, AI agents will act independently of humans to accomplish complex tasks using algorithmic “reasoning” and any frameworks governing the algorithm to determine the most probable course of action. An AI agent does not check with the human for the next steps; it simply executes against the original human-provided direction (e.g., prompt) within the confines of its governing rules and the quality of the training data it uses.

Reasoning, like intelligence, is an interesting word because it infers sentience and the arrival of artificial general intelligence. Indeed, it is to the advantage of certain AI providers like OpenAI to achieve this accomplishment, but this is not that moment. Candidly, I doubt we will see sentient AI in my lifetime.

That being said, agentic AI is the next step in probability-based decision-making. I prefer the more grounded Salesforce definition of AI agents and how they handle reasoning: “Using sophisticated machine learning models, AI agents analyze the collected data to identify patterns and make decisions... This decision-making process is enhanced by the agent's ability to learn from previous experiences and refine its responses over time.”

In layperson's terms, the AI uses probability based on the data to determine the most likely answer. The algorithm improves with time because it can use past answers to train itself. This is not a conscious being but rather an algorithm operating on powerful frameworks that allow it to choose the most likely best answers and next steps to meet the users’ needs.

Several moving parts create capable AI agents:

• Multiple original algorithms include but are not limited to large language, machine learning, and natural language processing models.

• Fine-tuning via the form of governance frameworks that limit the AI to acting within certain

• The removal of barriers, such as seeking permission for a next step or the creation of approved next-step decision trees (another type of AI model) for the agent

• Experienced data science teams engaged in agent training that refine the AI to achieve successes

The potential for AI agents is excellent. In reviewing some of the initial new AI agent packages that are out there, most initial offerings are domain-specific, acting within very confined task paths such as customer response agents or sales development assistants embedded in CRMs. Narrow AI scoping is expected as enterprise software platforms will create somewhat customizable agents that serve the broadest swath of their customers possible. 

For example, in a hypothetical example, SAP would create an AI agent to help an accounting department process and review expense reports. An SAP customer could customize the AI agent to meet its internal financial and data governance needs and any applicable regulatory reporting requirements.

AI Agent Challenges

In most cases, building AI Agents is not as easy as signing up for Claude or the latest GPT version and customizing an automation to unique organization needs. The challenges are diverse and range across various issues, including actual resources – both data and human – and the complex building of the agent. 

First, the sophisticated technology used in an agentic AI model necessitates robust data. If an agent is going to string together several automations based on machine learning reasoning, then the data fueling the probability determinations becomes exponentially more critical. Why? 

Consider that instead of one interaction or result, several actions depend on reasonable accuracy throughout the agent model. Agents lack human feedback during the intermediate stages of task-solving to meet the original human request. That creates a waterfall effect if a computational error or hallucination occurs. Certainly, foundational LLMs used in agents have natural hallucination challenges, and the fastest way to realize wrong answers is to use suspect data. Garbage in, garbage out.

That puts increased emphasis on good data handling and governance practices. In addition, guardrails for the agent need to be defined, and in many cases, developers recommend building decision frameworks to guide the AI’s decision-making process. 

As a result, Agentic AI architectures are convoluted, requiring multiple algorithm models, advanced RAG [retrieval augmented generation] stacks, advanced data architectures, strong governance, and frameworks, all requiring specialized expertise. This is not the work of a conventional CIO team member. Instead, strong data science team members are needed to manage the AI agent development process.

The technological complexity necessary to successfully build an AI agent almost always necessitates hiring a development firm or using a software provider’s professional services team. The risk of failure is so high that Forrester Research warns, “Three-quarters of the organizations that try to build AI agents in-house will fail.” For many organizations with already limited technical staff members, losing months of resource time, not to mention financial resources, is too much of a risk to consider.

Given the choice, most organizations will hire a firm or existing vendor. However, the costs will be high. The current technological sophistication, data cleanup, and fine-tuning required to create a helpful agent require a certain level of AI maturity, too. For example, organizations that have yet to successfully deploy a private instance of an LLM probably cannot implement a mission-specific AI agent. Building an agent without that maturity will require an organization to rely on vendors to build AND maintain the implementation thoroughly.

Concluding Thoughts and Guidance