14 Agentic AI Terms Every Newcomer Should Know

What's the difference between an AI agent and an agentic workflow? What does a forward deployed engineer actually do? Why does everyone keep talking about harnesses and guardrails?

Agentic AI comes with its own vocabulary. Terms like ontology, RAG and HITL show up in vendor pitches, board decks and project kickoffs, often without a definition. If you're a business leader trying to follow the conversation, or you're new to the field, that vocabulary can get in the way.

This guide covers 14 of the terms you're most likely to hear. Each one starts with a plain-language definition, then explains how it shows up in real work.

1. What is agentic AI?

Agentic AI describes AI systems that can take a goal, plan the steps to reach it, act in other systems, and check the results, with limited step-by-step direction from a person. Each of these systems is called an AI agent.

The easiest way I found to picture it: a deep research tool answers your question and waits for the next one. An agent takes the next step itself. It could look up a record, draft a reply, update a system, and then ask a person to approve the result.

Once an AI can do things, you need to decide what it can know, and who will check (and be responsible for) its work.

What's the difference between an AI agent and an agentic workflow?

An AI agent is a single AI system that works toward a goal and decides its own next steps. An agentic workflow is a defined business process that connects agents, systems and people in a set order to get a larger piece of work done. The agent is one worker. The workflow is the process that worker is part of.

Using the bid response example from later in the post: one agent might read the tender and pull out the requirements. The agentic workflow covers the whole job, from reading the tender to drafting each section to routing the final draft to a person for approval. Some steps use an agent, some follow fixed rules, and some need a human sign-off.

The trade-off comes down to freedom and predictability. Agents have more room to decide how to reach a goal. Workflows set the path, which makes them easier to test and audit, and that's often what enterprise and public sector clients need.

2. What is an ontology in AI?

An ontology is a structured map of the things a business cares about and how they relate to each other. This one that kept coming up again and again in internal conversations, and every time I asked for a definition, each person had a slightly different explanation. The definition above is my distillation.

An ontology defines your business's entities (a customer, a well site, a permit, an invoice), their properties, and the rules that connect them. AI agents use this structure to understand what your data means, so they can understand your business, make decisions, and flag concerns - the same way your team does.

I vaguely recall hearing this word in a philosophy class, where it broadly meant something along the lines of "the study of what exists." In AI it's much more practical. Think of it as the shared vocabulary between your systems and your agents. If one database calls them "clients," another calls them "accounts," and your finance team calls them "customers," the ontology is where you map out that they're all the same thing.

Why do ontologies matter for agents?

An agent that knows a purchase order belongs to a vendor, and that a vendor has a contract with an expiry date, can answer "which of our vendor contracts are up for renewal this quarter?" without someone writing a custom query.

3. What is a context layer in agentic AI?

A context layer is the part of an AI system that gives an agent the business knowledge it needs at the moment it acts. It pulls together definitions, data sources, permissions, history and policy, and hands the relevant pieces to the agent for each task.

The ontology and the context layer took me a while to separate. To use a hockey analogy, the ontology is the playbook, covering every position, every play and how team members should move up the ice. The context layer is the coach's word on the bench before a shift: who's going on next and what the other team just did. It answers questions like "which data is this agent allowed to see?", "what does 'active customer' mean here?" and "what happened the last time this request came in?"

Large language models know a lot about the world and nothing about your company. The context layer closes that gap, and it's a big reason an informed agent will give a useful answer while another gives a confident, generic one - or worse, a hallucination. Speaking of hallucinations...

4. What is an AI hallucination, and what is grounding?

A hallucination is when an AI model produces an answer that sounds confident but is false or unsupported. Grounding is the practice of tying a model's answers to verified sources, such as your documents or system records, so every claim can be traced back.

For most people outside the field, hallucination is one of the first worries about AI. Grounding answers it in a practical way: when an agent has to show where each fact came from, a reviewer can check its work quickly, and an answer with no source gets flagged instead of shoehorned in.

RAG (retrieval-augmented generation) is one of the most common grounding techniques.

5. What is RAG (retrieval-augmented generation)?

RAG, short for retrieval-augmented generation, is a technique where an AI model first searches a set of documents or data for relevant information, then uses what it found to write its answer. It lets a model answer from your policies, manuals or records instead of only what it learned in training.

It's like an open-book exam. Without RAG, the model answers from memory, or worse, makes a best guess. With RAG, it looks up the right pages first and incorporates them into its answer, all with the ability to point you to the source(s) it used.

RAG also keeps answers current, since you can update the documents without retraining the model. And it connects back to the context layer: RAG is one of the ways that layer finds the right information to hand an agent.

What is the difference between RAG and fine-tuning?

Fine-tuning retrains a model on your own examples so it learns a new style or specialty. RAG leaves the model as is and gives it the right information at the moment it answers.

Fine-tuning changes what the model is; RAG changes what it can see.

In the open-book exam analogy, fine-tuning is studying for weeks until the material becomes second nature. It works well for a consistent format or a specialized task. RAG is better when the facts change often or when you need to cite a source. Many production systems use both, often starting with an open-weight model.

6. What is an open-weight model?

An open-weight model is an AI model whose trained parameters (its "weights") are published so anyone can download, run and fine-tune it on their own infrastructure. Examples include Meta's Llama, Mistral's models and OpenAI's gpt-oss. You get the finished model, usually without the training data or code used to build it.

What's the difference between an open-weight and an open source model?

I assumed "open weight" and "open source" meant the same thing. The Open Source Initiative draws a line between them: open weights share the final trained model, while open source AI also shares the data details and code needed to study or rebuild it.

For the organizations we work with, the appeal is control. An open-weight model can run on your own servers or in a Canadian data centre, so sensitive data stays where you put it, and you're less exposed to a vendor changing prices or retiring a model.

7. What is task decomposition in AI agents?

Task decomposition is breaking a large goal into smaller, ordered steps that an AI agent (or a group of agents) can complete one at a time. Each step has a clear input and output, so the agent can check its work, use the right connector, and hand off to the next step.

This one felt familiar. It's what a good project manager does with a messy request. Say the goal is "prepare a bid response for this tender." Decomposed, that might look like:

1. Read the tender and pull out the requirements.

2. Find past proposals that answered similar requirements.

3. Draft a response to each section.

4. Flag gaps a person needs to fill.

5. Assemble the draft for review.

Smaller steps are easier to test and fix, and they make it clearer which ones a person should own. (For more on this, see "What does HITL (human in the loop) mean?" below.)

8. What is MCP (Model Context Protocol), and what are connectors?

MCP, or Model Context Protocol, is an open standard that lets AI agents connect to outside systems (a CRM, a document library, a database) in one consistent way. A connector is the link between an agent and one of those systems. It's what lets an agent read your data or take an action somewhere else.

Anthropic introduced MCP in November 2024, and in December 2025 it moved to the Linux Foundation's new Agentic AI Foundation, with OpenAI and Block as co-founders. It's now supported by ChatGPT, Claude, Gemini and Microsoft Copilot, among others.

The common comparison is USB-C. Before a common standard, every device needed its own cable - I certainly remember the days of asking a friend for a charger, and them having a lightning cable, when what I needed was a micro-USB!

With MCP, like with USB-C, a system built to the standard can plug into any agent that supports it. You'll sometimes see these called tools, but connectors is the more common term.

Connectors are where the context layer gets its information, and they're also where access control lives: an agent can only act in systems it has a connector for.

9. What is an AI harness?

An AI harness (often called an agent harness) is the software that wraps around an AI model so it can do real work. It connects the model to data and connectors, keeps track of what has happened so far, runs each step, checks the results and enforces the guardrails. Put simply, an agent is a model plus its harness.

At AltaML, the harness is part of what AltaForge, AltaML's proprietary agentic AI platform, provides, so each new agent starts with the connectors, memory and guardrails already in place.

This one clicked for me once I realized the model is only one piece. The model does the reasoning and decides what to do next. The harness handles everything around it: pulling in the right context, calling the right connector, remembering the last step, and stopping the agent when it tries something it shouldn't. Google Cloud describes it as the environment that turns a model's text into action.

The name comes from a horse's harness. The horse provides the power, and the harness is what lets you steer it and put that power to work.

It also explains why two products built on the same model can perform so differently. As Databricks puts it, a strong harness around a mid-tier model can outperform a weak harness around a stronger one. Building and tuning the harness is a big part of the engineering behind any agent, which leads into the next term.

10. What does a forward deployed engineer do?

A forward deployed engineer (FDE) is a software engineer who works directly with a client's teams to design, build and deploy AI solutions inside that client's environment. They sit close to the people doing the work, learn the domain, and take a solution from first conversation to production.

Palantir made the title popular, and AI labs including OpenAI now hire FDEs to embed with enterprise customers. At AltaML we call this role an AI Architect. Our AI Architects design and deploy agentic AI systems from prototype to production on AltaForge. Their job covers how an agent reasons, which connectors and data it can use, how its work gets evaluated, and whether it moved the business metric it was built for.

What surprised me most is how much of the job is listening. Before anyone writes code, an AI Architect spends time with the people whose work will change, learning where the real friction is.

11. What are accelerators in AI?

In AI services, accelerators are pre-built, reusable components that let a team start a project partway done. At AltaML, our accelerators are built on AltaForge, which lets us deploy them in weeks.

Each accelerator is a starting point. An AI Architect still tailors it to a client's data, connectors and approval steps, so each client gets something proven that still fits the way they work.

One piece that momentarily tripped me up: you'll also see "accelerators" used for hardware: chips such as GPUs that are designed to run AI workloads quickly. If the word appears next to "sovereign compute" or "data centre," it's usually referring to hardware, not solutions.

12. What is AI governance?

AI governance is the set of policies and roles an organization uses to decide how AI gets built and used, and who is accountable for the results. It covers questions like what data an AI system can use, who approves it before launch, and how it gets monitored once it's live.

In the public sector, governance is often written into policy. Federal departments in Canada, for example, work under the Treasury Board's Directive on Automated Decision-Making, which requires an algorithmic impact assessment before an automated system makes decisions affecting the public.

In practice, governance is carried out through three things: people who review key decisions (HITL), automatic limits on what an agent can do (guardrails), and tests that prove it works (evals).

What does HITL (human in the loop) mean?

HITL, or human in the loop, means designing an AI system so a person reviews or approves its work at defined points before it takes effect. The agent handles the repetitive steps, and a human makes the calls that need judgment or sign-off.

In the bid response example above, a person would likely review step 4 (the gaps that have been flagged) and approve step 5 (the final draft) before anything goes to a client. In a regulated setting, HITL might mean a reviewer signs off on every decision the agent recommends.

Where the human sits in the loop is a design decision, made early, based on risk. High-stakes steps get a person. Low-risk, high-volume steps might not need one. Every correction a reviewer makes also becomes feedback that helps the system improve.

What are AI guardrails?

Guardrails are the rules and automatic checks that limit what an AI agent can do or say. They work alongside HITL: guardrails catch problems automatically, every time, and people handle the judgment calls.

Examples include blocking an agent from sharing personal information, limiting which systems it can change, capping how much it can spend, or stopping any answer that isn't backed by a source. Guardrails decide what an agent is allowed to try. HITL decides which of its results need a person's approval.

What are evals in AI?

Evals (short for evaluations) are structured tests that measure how well an AI system does its job, using real examples with known good answers. They're scored against criteria set before launch, such as accuracy or how often the agent needs a person to step in.

I think of evals as the report card that gets written before the student starts. The team decides what "good" looks like, builds a set of test cases, and runs the system against them every time something changes: a new model or a new connector. If scores drop, the change doesn't ship.

At AltaML, defining and enforcing evaluation standards is part of the AI Architect role. Evals are also how you answer the question every executive asks eventually: how do we know it's working?

13. What is the difference between horizontal and vertical AI?

Horizontal AI solutions do one type of job across many industries, while vertical AI solutions are built for the specific workflows and rules of one industry. A general writing assistant or meeting summarizer is horizontal. An agent that reviews construction change orders or drafts energy regulatory filings is vertical.

You'll often hear these called horizontal and vertical "applications." At AltaML we say "solutions," which covers everything from a full product to an agent working behind the scenes in a client's workflows.

Most organizations end up with both. The interesting question for any team is which of their problems are common enough for a horizontal tool, and which are specific enough to need a vertical solution.

14. What is sovereign compute?

Sovereign compute is AI computing infrastructure (data centres and the chips inside them) that is owned and operated within a country, so its data and AI systems stay under that country's laws and control. For Canadian organizations, it means training and running AI on Canadian soil.

For our public sector and enterprise clients, sovereignty can be a requirement before a project starts. It shapes where data is stored, which models can be used (this is where open-weight models come back in), and who can access the system.

Putting it all together

Most of these terms describe one question from different angles: how do you give an AI agent enough of the right knowledge, and the right checks, to do useful work inside a real organization? Once I saw that, the vocabulary started to connect. The ontology, context layer and connectors give an agent knowledge and reach, and the harness ties it all together around the model. RAG and grounding help it find what it needs and show its sources. Task decomposition and governance decide how the work gets done and how you know it's right. AI Architects and accelerators get it built, horizontal and vertical describe how broadly it applies, and open-weight models and sovereign compute decide where it all runs.

My list is still growing. If there's a term you keep hearing and can't pin down, send it my way and I'll add it to the next one.

Want the short version?

Download the agentic AI glossary as a one-pager.
All 15 terms, grouped by what each one does for an AI agent. Print it, pin it or share it with your team before your next AI planning session.

Frequently Asked Questions

What is agentic AI?
AI systems that can take a goal, plan the steps, act in other systems and check the results, with limited step-by-step direction from a person.

What is the difference between an AI agent and an agentic workflow?
An agent is one AI system that decides its own steps toward a goal. An agentic workflow is a set business process that connects agents, systems and people to complete a larger job.

What does a forward deployed engineer do?
A forward deployed engineer works directly with a client's teams to design, build and deploy AI solutions in the client's environment. At AltaML, this role is called an AI Architect.

What are accelerators in AI?
Pre-built, reusable AI components that let a team start a project partway done. At AltaML, our accelerators run on AltaForge and can be deployed in weeks.

What is an ontology in AI?
A structured map of a business's entities, their properties and how they relate, which AI agents use to understand what data means.

What is a context layer?
The part of an AI system that gives an agent the business knowledge, data access and policy it needs at the moment it acts.

What is MCP (Model Context Protocol)?
An open standard that lets AI agents connect to outside systems in one consistent way. Each of those links is called a connector.

What is an AI harness?
The software that wraps around an AI model so it can act: it connects the model to data and connectors, tracks its progress, runs each step and enforces guardrails. An agent is a model plus its harness.

What is the difference between open-weight and open source AI?
Open-weight models publish the trained model's parameters. Open source AI also publishes the training code and data details needed to study or rebuild it.

What is RAG in AI?
Retrieval-augmented generation: the model searches your documents or data first, then answers using what it found.

What is the difference between RAG and fine-tuning?
Fine-tuning retrains a model on your examples to change how it behaves. RAG gives an unchanged model the right information at the moment it answers.

What is an AI hallucination?
A confident answer that is false or unsupported. Grounding, which ties answers to verified sources, is the main way to reduce it.

What is task decomposition?
Breaking a large goal into smaller, ordered steps an AI agent can complete and check one at a time.

What is AI governance?
The policies and roles an organization uses to decide how AI gets built and used, and who is accountable for the results. It's carried out through HITL, guardrails and evals.

What does HITL mean in AI?
Human in the loop: a person reviews or approves an AI system's work at set points before it takes effect.

What are AI guardrails?
Rules and automatic checks that limit what an AI agent can do or say.

What are evals in AI?
Structured tests that measure how well an AI system does its job, using real examples with known good answers.

What is the difference between horizontal and vertical AI?
Horizontal AI does one type of job across many industries. Vertical AI is built for one industry's workflows and rules.

What is sovereign compute?
AI computing infrastructure owned and operated within a country, so data and AI systems stay under that country's laws and control.