A. Dina
Agentic AI systems under the EU AI Act are regulated less by what they are and more by what they do. That distinction matters, because the Act was deliberately drafted as a technology-neutral, risk-based regulation, meaning Brussels chose not to chase buzzwords. “Agentic AI” may dominate conference stages and venture decks, but in legal terms it is usually just another AI system with unusually high autonomy, persistence, and capacity to act.
Under Regulation (EU) 2024/1689, an AI agent would generally fall within the definition of an AI system if it can infer from inputs and generate outputs such as decisions, recommendations, content, or actions that influence digital or physical environments. In plain English: if it reasons, plans, decides, or acts with some autonomy, it is inside the perimeter.
The real regulatory issue is therefore not nomenclature. It is risk profile plus operational behavior.
An agent that schedules meetings or summarizes emails may face relatively modest obligations, mainly transparency and general product-law duties. An agent that screens job applicants, allocates insurance pricing, manages access to education, controls industrial processes, or supports law enforcement can move quickly into high-risk territory under Annex III or through integration into regulated products.
Once classified as high-risk, obligations become significantly heavier. Providers may need to demonstrate risk management systems, data governance, technical documentation, logging, human oversight, robustness, cybersecurity, accuracy, and post-market monitoring. This is where many “autonomous agent” narratives collide with European compliance reality.
The central tension is simple: autonomy creates value, but also legal exposure.
A conventional software system behaves within narrow, predictable boundaries. An agentic system may chain tools, create sub-goals, adapt strategies, retain memory, or discover workflows not explicitly scripted. Investors call this leverage. Regulators may call it uncertainty.
One of the hardest issues is what you referred to as behavioural drift. If an agent materially changes how it operates after deployment, especially in ways that affect risk, then prior conformity assessments may no longer reflect real-world behavior. Under the AI Act, substantial modifications to high-risk systems can trigger renewed obligations. A provider cannot credibly certify one system and deploy another emergent one six months later.
That is particularly relevant for agents with:
Persistent memory that changes decision logic over time.
Dynamic tool access that expands capabilities post-deployment.
Self-optimizing workflows.
Multi-agent coordination producing unforeseen outputs.
Real-world execution powers such as purchasing, hiring, approving, blocking, or controlling infrastructure.
The law was built largely around assessable systems with documentable functions. Agentic architectures challenge that model because action pathways can be context-driven rather than fully pre-specified.
Another key issue is human oversight. The AI Act repeatedly requires that humans remain able to supervise and intervene where necessary, especially in high-risk contexts. Many companies market agents precisely by removing human bottlenecks. That may be commercially attractive, but if no meaningful human control remains, compliance becomes harder, not easier.
There is also the matter of traceability. If an AI agent denies someone a loan, rejects a candidate, or executes a harmful transaction, organizations must often explain what happened, reconstruct logs, and assign accountability. “The agent decided” is not a recognized legal defense, despite being common in badly managed boardrooms.
For GPAI and foundation-model-based agents, there can be layered obligations as well. If an agent is built on top of a general-purpose AI model, responsibilities may be split between upstream model providers and downstream deployers, depending on who controls what, modifies what, and places what on the market.
The most pragmatic compliance strategy is not to obsess over whether something is “truly agentic.” That is mostly marketing theology. Instead, organizations should map five concrete dimensions:
What external actions can the system take?
Who can be affected by those actions?
What sector/use case is involved?
How much autonomy exists at runtime?
Can behavior be monitored, constrained, audited, and overridden?
That is where sophisticated governance frameworks outperform slideware.
A robust operating model for agentic systems in Europe increasingly requires:
Scoped permissions rather than unlimited tool access.
Approval gates for irreversible or high-impact actions.
Runtime monitoring and anomaly detection.
Immutable logs of decisions, tool calls, and outputs.
Clear responsibility allocation between provider, deployer, and operator.
Reassessment triggers when capabilities materially evolve.
In other words, move from “What model are we using?” to “What powers does this system exercise?”
That shift is strategic. The market still rewards demos where an AI books flights, negotiates contracts, fires vendors, and perhaps invades Poland by Q4. Regulators reward controllability.
So how do you reconcile unpredictable reasoning with rigid legal robustness?
You do not regulate internal cognition directly. You regulate capabilities, permissions, outputs, and impact surfaces. Europe is unlikely to bless black-box autonomy with unchecked authority simply because Silicon Valley finds it elegant.
The likely future is not a ban on AI agents, but a tiered market:
Low-risk bounded agents proliferate.
Enterprise agents survive behind controls.
High-impact autonomous agents face heavy assurance burdens.
Uncontrolled “YOLO agents” become excellent case studies for enforcement lawyers.
That may disappoint evangelists promising digital employees with infinite initiative. But history is cruel to technologies that confuse possibility with admissibility.
The sharper summary is this: under the EU AI Act, agentic systems are regulated as AI systems whose autonomy increases scrutiny. The more an AI can decide, act, adapt, and affect rights or safety, the more governance must surround it. Freedom for the machine means paperwork for the company.
Paper: https://arxiv.org/pdf/2604.04604
