The Hallucination Problem Is Not Random
When I first prototyped an AI chatbot for this portfolio, I made the standard mistake: I gave the model a system prompt with a paragraph about me and asked it to answer questions. It worked for broad questions. For specific ones — what was your thesis topic, which companies have you worked at, what tech stack does your current project use — the model either said it did not know or confidently made something up.
The fabrications were the worse outcome. A visitor who gets a made-up job title or a hallucinated publication does not come away with a neutral impression; they come away with a wrong one.
The problem is not that LLMs are unreliable. The problem is that system-prompt context is treated as a vague, blurry input that the model interpolates from. When the facts run out, interpolation takes over. The fix is to make the model fetch facts rather than invent them.
Tool Calling as the Grounding Mechanism
The chatbot uses OpenAI-compatible tool calling through OpenRouter, so the model itself is a configuration value rather than something baked into the code. Instead of pre-loading everything about me into the context window, the model gets four tools, each returning one slice of my profile:
get_about— my bio, current role, positioning, the domains I have worked in, and a few credibility anchorsget_experience— every role with its dates, location, highlights, and skills, plus my formal educationget_projects— the project case studies (problem, approach, how it works, what I built, impact, metrics, stack) and the skill matrixget_publications— my journal and conference papers, plus research impact metrics
When a visitor asks "what stack did you use at Accenture," the model calls get_experience, reads the result, and answers from it. The tool output is the source of truth.
This is a different mental model from RAG. I am not embedding text and retrieving chunks by semantic similarity. I am giving the model a few well-named tools and letting it decide which ones to call. For a personal portfolio with bounded, well-structured content, that is simpler and more predictable than a retrieval pipeline.
Here is what the tool schema looks like:
{
"type": "function",
"function": {
"name": "get_experience",
"description": "Returns every role with its dates, location, highlights, and skills, plus formal education.",
"parameters": {
"type": "object",
"properties": {},
"required": []
}
}
}No parameters. Each tool is intentionally coarse-grained — it returns the full slice, and the model extracts what it needs. This keeps the schema trivial and avoids the meta-problem of the model needing to know what to query before it knows what the answer looks like.
One Source of Truth for the Site and the Bot
The tools do not read a separate knowledge base. They import the same typed TypeScript content modules that render the pages you are reading: the persona, the experience timeline, the project case studies, the publication list. Each tool formats its slice as compact, labeled plain text — Role:, Period:, Highlights: and so on — which models read reliably and which costs few tokens.
The practical effect is that the chatbot cannot drift from the site. When I update a date on the experience page or add a case study, the next answer reflects it. There is no "refresh the embeddings" step, and no second copy of my CV to keep in sync.
The Request Loop
A question goes through two phases on the server.
Phase 1 — tool-calling loop:
conv = [{"role": "system", "content": SYSTEM_PROMPT}] + messages
for _ in range(4): # cap at 4 rounds
resp = client.chat.completions.create(
model=MODEL,
messages=conv,
tools=TOOLS,
tool_choice="auto",
)
msg = resp.choices[0].message
if not msg.tool_calls:
break # model has everything it needs
conv.append(msg)
for tc in msg.tool_calls:
result = run_tool(tc.function.name)
conv.append({
"role": "tool",
"tool_call_id": tc.id,
"content": result,
})Phase 2 — streaming the answer:
stream = client.chat.completions.create(
model=MODEL,
messages=conv,
tools=[], # disable tools for the final answer
stream=True,
max_tokens=800,
)
for chunk in stream:
delta = chunk.choices[0].delta.content or ""
if delta:
yield json.dumps({"delta": delta}) + "\n"The two-phase design is deliberate: tool calls are not streamed (they need to be complete before the result is useful), but the final answer is streamed token by token so the browser can start rendering immediately.
The Rules the Model Follows
The system prompt is short and strict. It tells the model to call the relevant tools before answering any question about me rather than relying on prior knowledge, to only answer questions about me and politely decline anything else, to never fabricate facts, credentials, dates, employers, or numbers beyond what the tools return, and — when the tools do not contain the answer — to say so instead of guessing.
That still leaves room to be useful. "Which of your projects is closest to building a voice agent?" is not a lookup, but the model can answer it by comparing the case studies get_projects returns. What it may not do is fill gaps with what a typical AI engineer's CV might contain. Replies follow the visitor's language, English or Indonesian.
Rate Limiting and Cost Control
A public LLM endpoint on a personal site carries two risks: abuse and cost. Both are handled on the server before any model call is made:
- Input caps — up to 1,000 characters per message, 40 messages of history, and 20,000 characters in total. Anything larger is rejected.
- Two rate limits — a per-IP window (20 requests per minute by default) and a per-conversation window (10 requests per five minutes), keyed by a random ID the browser generates for each chat. Over the limit, the API returns a 429 and the chat shows a friendly "please wait a moment" message instead of an error.
- Bounded output — the final answer is capped at a configurable token limit, and the tool loop is capped at four rounds.
The limiter is an in-memory fixed window, implemented without an external dependency:
class RateLimiter:
def __init__(self, max_requests: int, window_seconds: int):
self.max_requests = max_requests
self.window_seconds = window_seconds
self._windows: dict[str, list[float]] = defaultdict(list)
self._lock = threading.Lock()
def is_allowed(self, key: str) -> tuple[bool, int]:
now = time.time()
with self._lock:
window = self._windows[key]
# evict expired entries
self._windows[key] = [t for t in window if now - t < self.window_seconds]
if len(self._windows[key]) >= self.max_requests:
oldest = self._windows[key][0]
retry_after = int(self.window_seconds - (now - oldest)) + 1
return False, retry_after
self._windows[key].append(now)
return True, 0The site runs as a single Node process (the actual implementation is TypeScript, but the logic is identical), so in-memory is enough. A restart resets the counters, which is acceptable for a personal site. The API key stays on the server, which is the main reason the site is server-rendered rather than a static export.
What I Would Change
If the content grew significantly — say, fifty blog posts and twenty detailed write-ups — returning a whole section per tool call would get expensive. I would add search_blog and search_projects tools backed by semantic search. Tool calling and retrieval compose cleanly: the structured tools would stay for factual questions, and the search tools would handle open-ended exploration.
The current version does not need that, because the content is bounded. That is the design principle I keep coming back to: build the simplest architecture that handles the content you actually have, not the content you might have one day.
The approach works for any domain-specific assistant where the facts are finite and well structured. Tool calling shifts the model's job from remembering to reasoning: it looks things up, and it does not invent them.
Companion code — a complete FastAPI implementation with rate limiting, streaming, and all four tools: github.com/dzakybd/grounded-llm-tool-calling