Gartner: 80% of Enterprise GenAI Apps Will Use RAG on Existing Data Platforms by 2028

What Is RAG, and Why Does Gartner Care So Much?

RAG stands for Retrieval-Augmented Generation — an architectural pattern that fuses two things together:

1. Retrieval — the system searches the organization's own sources (documents, databases, reports) for information relevant to the user's question
2. Generation — the LLM uses the retrieved material as context to produce an answer grounded in real data, rather than one it invented on its own

How this differs from a traditional LLM:

Dimension	Traditional LLM	RAG
Knowledge source	Training data up to the cutoff date	Live enterprise data + training data
Hallucination	High risk — it can make things up	Far lower — answers come with evidence
Updating information	Requires re-fine-tuning — costs time and money	Updating the in-house database is enough
Audit trail	Not auditable — no way to know where the answer came from	Auditable — citations point back to the source
Compliance	Risky — data can leak through fine-tuning	Safer — the data stays inside the organization
Cost per query	Heavy token use, billed by context size	Sends only the relevant slice of data

The Full Gartner Prediction — How to Read It Without Misinterpreting It

Gartner's wording is: "By 2028, 80% of business generative AI applications implemented by RAG approach will use organizations' existing data management platforms (including LDW/lakehouse) as the knowledge source, increasing from less than 20% today."

There are a few points people frequently get wrong:

• 80% does not mean "80% of all organizations will use RAG" — it means "80% of the GenAI apps that already use RAG will run on an existing data platform." It is a statement about architecture choice, not adoption rate
• Gartner issued a separate prediction on LLM observability — by 2028, 50% of GenAI deployments will invest in Explainable AI, up from 15% today
• The goal is to "cut development complexity and time by 50%" — by reusing existing tooling instead of building everything from scratch

Why RAG Is Becoming the Standard

1. It Addresses the Hallucination Problem Baked into LLMs

A pure-generation LLM will invent an answer when it does not know one — the classic hallucination problem that makes AI untrustworthy for high-stakes work. Adding RAG makes it far easier to verify the accuracy of an AI's answers, because each answer carries citations pointing back to the source documents.

2. Compliance and Data Privacy

Fine-tuning an LLM on internal data carries a data leakage risk, because that data becomes embedded in the model weights and cannot simply be "deleted", which conflicts with the right to be forgotten under PDPA/GDPR.

RAG solves this head-on: the data stays in the organization's database and the LLM merely "reads" it at answer time. When a user requests deletion, you delete it once at the source.

3. Updates Are Orders of Magnitude Easier

When internal information changes (a new policy, new pricing, a new manual version), a traditional LLM has to be fine-tuned again at real cost in time and money. With RAG, you simply update the database — the very next answer already reflects the new information.

4. You Can Use the Infrastructure You Already Own

This is the heart of the Gartner prediction: instead of investing in an entirely new AI stack, organizations can use the databases, data warehouses and lakehouses they already have as the knowledge source for RAG — lowering cost, lowering risk and shortening implementation time.

RAG Architectures — The Options and How to Choose

In 2026 there are several RAG architectures to choose from, each suited to a different kind of use case:

Pattern	Characteristics	Best for
Naive RAG	Vector search, then the LLM answers directly	FAQs, manuals, straightforward documents
Advanced RAG	Adds re-ranking and query rewriting	Systems that demand high precision
Hybrid RAG	Combines vector + keyword + SQL	Mixed structured and unstructured data
GraphRAG	Uses a knowledge graph instead of vectors alone	Data with complex relationships
Agentic RAG	An AI agent decides what to retrieve and which tools to call	Multi-step work — Agentic AI

The Risks and Limits of RAG

• Garbage in, garbage out — RAG rests entirely on data quality; wrong documents produce wrong answers
• Added latency — retrieval happens before generation, so it is slower than a plain LLM call
• Embedding indexes need managing — new or edited documents must be re-embedded
• Vector database cost — embedding storage grows with the document corpus
• Retrieval security — if permissions are handled poorly, users can pull documents they should never see

RAG and ERP Systems — The Obvious Use Cases

An ERP system makes an excellent knowledge source for RAG, because it holds a large volume of structured data:

Use case	Data from the ERP	Benefit
CFO question-answering assistant	GL, cost centers, budget	Answers grounded in real figures, not guesswork
Procurement assistant	Vendor history, contracts	Compares against prices actually paid before
HR assistant	Policies, employee handbook	Answers your organization's own policy, not generic advice
Period-close assistant	Trial balance, journal entries	Spots anomalies in the actual data
Automated summary reporting	Sales, inventory, AR/AP	Executive summaries that cite real numbers

What Thai Organizations Should Prepare Before 2028

1. Audit your data quality — what percentage of the documents, policies and databases in your existing systems are current, complete and structured? How good RAG turns out to be depends on the source data
2. Define data governance — who owns each body of data, and who authorizes AI access to it? This matters for AI governance
3. Pilot on a small use case first — start where risk is low, such as FAQs and manuals, before moving into finance or operational systems
4. Plan observability from the outset — log every question and answer and track the hallucination rate over time. Gartner's 50% LLM observability prediction for 2028 is significant here
5. Watch out for vector database vendor lock-in — choose something that supports open standards (for example pgvector on PostgreSQL) or that lets you switch vendors
6. Choose on-premise vs. cloud based on data sensitivity — financial, tax and customer data belongs in-house; Ollama + RAG for running inside your internal network is a compelling option

Why This Matters for Organizations Running Saeree ERP

Saeree ERP has stored everything on PostgreSQL for more than 20 years — master data, transactions and reports alike. That is precisely the kind of "existing data management platform" Gartner is describing:

• The Saeree AI Assistant is being developed on a RAG architecture — using real data in the ERP as its knowledge source rather than fine-tuning a model (current status: in training)
• PostgreSQL supports pgvector — an extension that turns PostgreSQL into a vector database in its own right, simplifying the stack compared with running a separate vector DB
• Saeree master data can be maintained by an admin — adding, inactivating or setting valid from-to dates on account codes, VAT rates, vendors and customers is done in-house, so the RAG knowledge base updates immediately without waiting for a vendor patch
• The data stays on the customer's network — both on-premise and cloud are supported, which is PDPA-friendly and means financial data never has to leave the organization

Summary — When RAG Fits Today, and When It Doesn't

Ready for RAG now	Prepare the data first
Plenty of internal documents (manuals, policies, reports)	Documents scattered and unstructured
ERP/CRM data used consistently	Low data quality, outdated information
Compliance and an audit trail are required	No data governance policy yet
Already running PostgreSQL/lakehouse	Closed-source legacy systems
An AI budget whose ROI must be measurable	You want purely generative AI, unanchored to existing data

RAG is not a new version of AI — it is the way to make the AI you already have use your own organization's data as the foundation of its answers. So 2028 is not "the year AI gets smarter"; it is the year AI starts being able to answer questions about your organization — provided you have the data ready.

- Saeree ERP Team

Summary — What to Do Now

1. Inventory your internal documents and data — what exists, where it lives, and when it was last updated
2. Start a RAG pilot on a small use case — FAQs and the employee handbook, before touching real production systems
3. Plan observability from day one — log the query, the source and the answer every single time
4. Invest in data quality before investing in AI — data quality is what separates a poor AI result from an excellent one, many times over
5. Choose an architecture that reuses your existing data platform — as Gartner recommends: lower cost, lower vendor lock-in risk

If your organization runs Saeree ERP, or is planning to apply AI/RAG to internal data and wants to assess its readiness, the Saeree team is ready to advise on AI and ERP, on planning the ROI of an AI investment, and on connecting your existing data to an AI assistant. Feel free to contact our consultants directly.

References

• Gartner Predicts by 2028, 80% of GenAI Business Apps Will Be Developed on Existing Data Management Platforms — Gartner Press Release (2 June 2025)
• Gartner Predicts By 2028, Explainable AI Will Drive LLM Observability Investments to 50% — Gartner Press Release (30 March 2026)
• Strategic Predictions for 2026 — Gartner