Using Ollama for Real

Choose the Right Model for Your Task

Ollama supports over 100 models, but you don't need to try them all — choosing the right one for your task matters more. Here's a recommendation table by use case:

Use Case	Recommended Model	Size	Why It Fits
General Chat / Asian Languages	qwen2.5	7B / 72B	Best Asian language support among open-source models
All-round	llama3.1	8B / 70B	By Meta, strong all-round with high benchmarks
Coding	qwen2.5-coder	7B / 32B	Excellent at coding, supports 80+ languages
Reasoning (Complex Thinking)	deepseek-r1	8B / 70B	"Thinks before answering" with visible Thinking Process
Image Understanding	llama3.2-vision	11B / 90B	Read images, analyze charts, read receipts
Ultra Fast (Instant Replies)	gemma2:2b	2B	Smallest and fastest, great for simple tasks
Document Summarization	mistral	7B	From France, concise and to-the-point summaries
Data Analysis / Math	phi4	14B	By Microsoft, excellent at Reasoning + Math

System Prompt — Define Your AI's Role

A System Prompt is a "behind-the-scenes instruction" that tells the AI who to be, how to respond, and what not to answer — like briefing a new employee before they start. Without a System Prompt, AI responds generically and may miss the point.

In Ollama, you can set a System Prompt both at runtime and in a Modelfile:

Method 1: At Runtime (Temporary)

ollama run llama3.1 --system "You are an ERP expert for organizations. Answer concisely with real-world examples."

Method 2: In a Modelfile (Permanent)

This is better because you don't have to retype it every time — explained in the next section.

Good System Prompt Examples

Use Case	System Prompt
ERP Assistant	"You are an ERP system expert. Explain steps clearly in numbered lists. If unsure, say so directly."
Coding Helper	"You are a senior developer. Write clean, well-commented code. Always explain your approach before writing code. Use TypeScript by default."
Document Summarizer	"Summarize the given content in no more than 5 bullet points. Focus on key takeaways only. No opinions."
Chart of Accounts Review	"You are a senior accountant. Review the chart of accounts provided. Flag anomalies or non-compliance with IFRS standards and suggest corrections."
Translation	"Translate the given text to Thai. Keep technical terms in English. Maintain the original tone and formatting."

Modelfile — Build Your Own Custom Model

A Modelfile is like a Dockerfile for AI — a text file that defines how a model should behave, which base model to use, what System Prompt to set, and what parameters to configure. Then you build it into a custom model ready to use.

Example: Build an ERP AI Assistant

# File: erp-assistant.modelfile

FROM qwen2.5

PARAMETER temperature 0.3
PARAMETER top_p 0.9
PARAMETER num_ctx 8192

SYSTEM """
Your name is "Saeree AI" — an ERP system assistant for the organization.
- Always explain steps in clear numbered lists
- For financial/accounting questions, reference IFRS standards
- If unsure, say directly: "I'm not sure, please consult a specialist"
- Never fabricate numbers or data that doesn't exist
"""

Build and run:

# Build Custom Model
ollama create erp-assistant -f erp-assistant.modelfile

# Run
ollama run erp-assistant

# Test
>>> What are the steps for monthly financial closing?

More Examples

# AI Coding Assistant
FROM qwen2.5-coder
PARAMETER temperature 0.1
SYSTEM "You are a senior full-stack developer. Write clean code with comments. Prefer TypeScript, Python, and PostgreSQL."

# AI Document Summarizer (fast)
FROM gemma2:2b
PARAMETER temperature 0.2
PARAMETER num_ctx 4096
SYSTEM "Summarize content in no more than 5 bullet points. Concise and to the point."

# AI Risk Analyst
FROM phi4
PARAMETER temperature 0.4
SYSTEM "You are an enterprise risk management expert. Analyze the data provided. Identify risks, severity levels, and recommendations."

Essential Parameters

Parameters control "how" the AI responds — creative vs precise, long vs short:

Parameter	Range	Effect	Recommended
temperature	0.0 - 2.0	Low = precise, repetitive / High = creative, varied	Code/Accounting: 0.1-0.3, Chat: 0.7, Writing: 0.8-1.0
top_p	0.0 - 1.0	Limits responses to the most probable tokens	0.9 (default, works well for all tasks)
top_k	1 - 100	Limits the number of tokens AI picks at each step	40 (default)
num_ctx	2048 - 131072	Context Window — how much text AI remembers in a conversation	4096 (default), 8192-16384 for long documents
repeat_penalty	1.0 - 2.0	Prevents AI from repeating itself	1.1 (increase slightly if AI loops)

Multi-modal — Let AI Read Images

Some models can "see" images — read data from receipts, analyze charts, or read ERP system screenshots:

# Run a vision-capable model
ollama run llama3.2-vision

# Send an image to AI (drag file into Terminal or type path)
>>> Read data from this receipt /Users/me/receipt.jpg
>>> Analyze this sales chart /Users/me/sales-chart.png

Models with vision capability:

• llama3.2-vision (11B/90B) — By Meta, best open-source vision model
• llava (7B/13B) — Lightweight and fast, great for inventory document verification
• moondream (1.8B) — Tiny, runs even on low-end machines

GUI — No Terminal Required

For those who prefer not to use the command line, or want everyone in the organization to use AI via a web browser, several GUIs integrate with Ollama:

GUI	Type	Highlights	Best For
Open WebUI	Web App (Docker)	ChatGPT-like UI, user management, RAG support	Organizations, shared server
Chatbox	Desktop App	Easy install, fast model switching, history saved	Individual use, personal
Enchanted	macOS / iOS App	Native Apple UI, Siri Shortcuts support	Apple ecosystem users
Continue.dev	VS Code Extension	AI Coding Assistant in VS Code connected to Ollama	Developers using VS Code

Tips — Get the Most Out of Ollama

1. Pick the right model size: 7-8B for general tasks, 14B for smarter responses, 70B for complex work — don't use oversized models unnecessarily, they'll just be slow
2. Set temperature correctly: Tasks requiring precision (code, accounting, numbers) use 0.1-0.3 / creative tasks use 0.7-1.0
3. Use Modelfiles: Create custom models for frequent tasks — saves time from retyping System Prompts
4. Increase num_ctx for long documents: Default 4096 may not be enough for 10+ page documents. Set 8192 or 16384
5. Use GUI for organizations: Open WebUI is perfect for letting all employees use AI — no Terminal training needed

Ollama Series — Read More

"The best AI isn't the smartest one — it's the one configured to match the task at hand. Modelfile is the key."

- Saeree ERP Team

References

• Ollama Modelfile Documentation (Official)
• Ollama Model Library
• Open WebUI — GitHub
• Chatbox — Desktop AI Client