How I built an AI assistant for shanedrumm.com, reduced costs by 94%, and learned important lessons about RAG, token optimization, and system prompt engineering.

While studying The Complete Agentic AI Engineering Course (2025) | Udemy I have built my first agent that is based on some of my blog posts and linkedIn account.

Outstanding

Add sql for FAQ that LLM can read/write from
Add caching
Tools to collect email address
Tool to email me when somebody uses it
Usage limit set on OpenAI
Deploy to hugging face
Add as popup and custom CSS with avatar
Publish to GitHub (my first commit)

Updated System Prompt from lab 4

If you don’t know the answer to any question, use your record_unknown_question tool to record the question that you couldn’t answer, even if it’s about something trivial or unrelated to career. \
If the user is engaging in discussion, try to steer them towards getting in touch via email; ask for their email and record it using your record_user_details tool.

Building a Personal Website Chatbot: From 10,000 Tokens to 600

How I built an AI assistant for shanedrumm.com, reduced costs by 94%, and learned important lessons about RAG, token optimization, and system prompt engineering.

The Vision

I wanted to add an AI chatbot to my personal website that could answer questions about my professional background, athletic achievements, travel experiences, and side projects. The goal was simple: let visitors have a conversation instead of scrolling through pages.

But as I quickly discovered, “simple” doesn’t mean easy.

Starting Point: The Knowledge Base Problem

My first challenge was converting my information into a format an LLM could understand. I had:

A CV in PDF format
LinkedIn profile information
Personal achievements, travel experiences, and interests scattered across notes

Decision 1: Markdown as the Single Source of Truth

I converted everything into two structured markdown files:

shane-drumm-cv.md (18KB) – Professional experience, certifications, achievements
shane-drumm-personal.md (6KB) – Athletics, travel, books, side projects

This gave me clean, structured data I could easily update and version control.

First Attempt: The “Stuff Everything In” Approach

My initial implementation was straightforward:

# Load everything
system_prompt = load_comprehensive_instructions()  # 16KB
cv_content = load_cv()                             # 18KB  
personal_content = load_personal()                 # 6KB
# Send it all to the API
full_context = system_prompt + cv_content + personal_content
response = llm.complete(full_context + user_question)

Result: 10,059 tokens per request.

Problem: Token limit errors and high costs ($1.51 per 1,000 requests).

This approach hit token limits immediately. I needed a better strategy.

The Token Crisis: Discovering the Real Problem

When testing with Gradio, I got this error:

BadRequestError: Error code: 400
Token limit exceeded

The math was brutal:

System prompt: 3,985 tokens
Full CV: 4,487 tokens
Full personal info: 1,587 tokens
Total: 10,059 tokens per request

For context, GPT-4o-mini has a 16K token context window. I was using 63% of it before the user even asked a question.

Solution 1: Smart RAG (Retrieval Augmented Generation)

Instead of sending everything, I implemented keyword-based context selection:

def select_context(message):
    professional_keywords = ['work', 'job', 'career', 'portfolio', 'agile']
    personal_keywords = ['ironman', 'marathon', 'travel', 'books']
    
    # Count keyword matches
    if question_is_professional(message):
        return cv_content  # Only send CV
    elif question_is_personal(message):
        return personal_content  # Only send personal info
    else:
        return cv_summary + personal_summary  # Send both summaries

Result: Reduced to 2,500-5,000 tokens depending on question type.

But I could do better.

Solution 2: Creating Summaries

I created condensed versions of both knowledge files:

CV Summary (225 tokens vs 4,487):

**Current Role:** Strategic Advisor at Woodside Energy
**Key Achievements:** $750K value delivered, $12M portfolio management
**Certifications:** PMP, SAFe LPM, CSPO, CSM, AWS AI Practitioner
**Industries:** Energy, Technology, Finance, Government

Personal Summary (157 tokens vs 1,587):

**Athletic:** 4x Ironman (PB: 11:08), 3x Marathon (PB: 3:59)
**Side Projects:** The Pivot News (3,100 subscribers), PM-Training.net (sold 2024)
**Travel:** 30+ countries, based in Perth, Australia since 2017

Token savings:

CV: 95% reduction
Personal: 90% reduction

Solution 3: Minimizing the System Prompt

My system prompt went through multiple iterations:

Version	Tokens	What It Contained
Original	3,985	Everything including examples and redundant info
Short	905	Essential instructions, fewer examples
CTA-Optimized	1,050	Added CTA instructions (mistake – see below)
Minimal	222	Instructions only, no redundant knowledge

Key insight: The system prompt should contain instructions, not knowledge.

I was duplicating information:

# System Prompt (WRONG):
**Professional:** 12+ yrs portfolio management, $12M portfolio...
**Personal:** 4x Ironman, entrepreneur...
# KNOWLEDGE BASE (already has this!):
[Full professional details]
[Full personal achievements]

Removing redundant summaries from the system prompt saved 137 tokens (38% reduction).

The Call-to-Action System

I wanted every response to include relevant links:

Professional questions → LinkedIn
Ironman questions → Training blog + Strava
Travel questions → Instagram

Initial approach (wrong): I added CTA instructions to the system prompt:

## Automatic Call-to-Action System
CTAs are added automatically after your response.
Professional topics → LinkedIn
Ironman topics → Blog + Strava

The mistake: Since CTAs are appended in Python code AFTER the LLM generates its response, the LLM never sees them. These instructions were completely useless and wasted 145 tokens.

Correct approach:

# LLM generates response
answer = llm.complete(prompt)
# Python appends CTAs based on keywords
ctas = select_cta(message)
final_answer = answer + "\n\n---\n" + ctas

No system prompt instructions needed!

Issues Encountered and Fixes

Issue 1: UnboundLocalError

def select_context(message):
    if professional:
        context = cv_summary
    elif personal:
        context = personal_summary
    # No else clause!
    return context  # ❌ Error if no conditions match

Fix: Initialize variables first:

def select_context(message):
    # Guarantee variables exist
    context = default_summary
    source = "summary"
    
    if professional:
        context = cv_summary
    elif personal:
        context = personal_summary
    
    return context, source  # ✓ Always works

Issue 2: Duplicate CTAs

The LLM was adding its own CTAs (“Learn more…” / “Contact Shane…”), then Python was adding more:

[Answer...]
If you want more details, feel free to ask!  ← LLM added
Learn more: LinkedIn                          ← LLM added
Learn more: LinkedIn                          ← Python added

Fix: Add explicit instruction to system prompt:

**Never add "Learn more", contact links, or CTAs - they're added automatically.**

Issue 3: Declining Valid Questions

Question: “What’s Shane’s greatest achievement?” Response: “I’m here to answer questions about Shane Drumm. Is there something about his background I can help with?”

The bot incorrectly treated this as off-topic.

Fix: Make the system prompt explicit about what to answer:

## What TO Answer
✅ Shane's career, achievements, skills, certifications
✅ Questions like "greatest achievement", "best accomplishment"
## What NOT to Answer
❌ Weather, coding help, other people

Issue 4: Missing CTAs on Some Responses

The CTA selection logic wasn’t matching all keywords properly.

Fix: Expanded keyword lists and added debug logging:

if 'ironman' in msg or 'triathlon' in msg or 'tri' in msg:
    return [ironman_blog, strava]

Final Architecture

After all optimizations, here’s what I ended up with:

┌─────────────────────────────────┐
│ System Prompt (222 tokens)      │  Instructions only
└─────────────────────────────────┘
            ↓
┌─────────────────────────────────┐
│ Smart Context Selection         │  Choose CV or Personal
└─────────────────────────────────┘
            ↓
┌─────────────────────────────────┐
│ Knowledge Summaries             │  CV: 225t, Personal: 157t
└─────────────────────────────────┘
            ↓
┌─────────────────────────────────┐
│ LLM Generates Response          │  Clean answer
└─────────────────────────────────┘
            ↓
┌─────────────────────────────────┐
│ Python Appends CTAs             │  Based on topic
└─────────────────────────────────┘

Token breakdown per request:

System Prompt:        222 tokens
Context (Summary):    225-382 tokens (depending on question)
User Message:         ~20 tokens
─────────────────────────────────
TOTAL INPUT:          ~604 tokens
RESPONSE:             ~200 tokens
─────────────────────────────────
TOTAL PER REQUEST:    ~804 tokens

Cost comparison:

Original: $1.51 per 1,000 requests
Final: $0.12 per 1,000 requests
Savings: 92%

User Experience Improvements

Gradio Example Questions

Instead of starting with a blank chat, I added 8 clickable example questions:

examples = [
    ["What's Shane's current role?"],
    ["Has Shane completed any Ironman races?"],
    ["Where has Shane traveled?"],
    ["Tell me about Shane's side projects"],
]

Users can now click a question instead of typing “hi” or figuring out what to ask.

Automatic CTAs

Every response includes relevant next steps:

Ironman question →

Learn more:
- Ironman training tips for beginners
- Follow Shane's training on Strava

Career question →

Learn more:
- View Shane's full background on LinkedIn

Key Lessons Learned

1. Token Efficiency Matters

Going from 10,000 tokens to 600 tokens wasn’t just about cost—it enabled the entire project. Token limits are real constraints.

2. Smart RAG > Context Stuffing

Don’t send everything. Analyze the question and send only relevant information.

3. Summaries Are Your Friend

Create condensed versions of your knowledge base. 95% token reduction with minimal quality loss.

4. System Prompts Should Only Contain Instructions

Don’t duplicate information between system prompt and knowledge base. If something happens after the LLM’s response (like CTAs), don’t put it in the system prompt.

5. Be Explicit About Guardrails

“Decline off-topic questions” is too vague. List specific examples of what to answer and what to decline.

6. Initialize Your Variables

Python’s scoping rules can bite you. Always initialize variables before conditional logic.

7. Test Edge Cases

“hi”, “greatest achievement”, generic greetings—test unusual inputs to find gaps in your logic.

8. Separation of Concerns

System Prompt:  How to behave
Knowledge Base: What to know
Python Code:    What to add (CTAs)

Each layer has one job.

Testing Strategy

I tested systematically across categories:

Guardrails:

“hi” → Should decline politely ✓
“what’s the weather” → Should decline ✓
“help me code Python” → Should decline ✓

Professional:

“current role” → Should answer with CV context ✓
“portfolio management” → Should include LinkedIn CTA ✓

Personal:

“ironman time” → Should answer with personal context ✓
“travel” → Should include Instagram CTA ✓

Edge Cases:

“greatest achievement” → Should answer, not decline ✓
Empty message → Should handle gracefully ✓

Results

The chatbot is now live on shanedrumm.com with:

✅ 94% token reduction (10,059 → 604 tokens) ✅ 92% cost savings ($1.51 → $0.12 per 1K requests) ✅ Smart context selection (only sends relevant info) ✅ Automatic CTAs (drives traffic to LinkedIn, Strava, blog) ✅ Example questions (better UX than blank chat) ✅ Proper guardrails (declines off-topic, answers Shane questions)

The Code

All the code and documentation is available in my GitHub repo:

System prompts (Original, Short, Minimal)
Knowledge base (CV + Personal summaries)
Smart RAG implementation
CTA selection logic
Gradio interface with examples

What’s Next?

Potential improvements I’m considering:

Analytics – Track which questions are most common
Feedback loop – Let users rate responses
More granular summaries – Break CV into even smaller chunks
Semantic search – Replace keyword matching with embeddings
Multi-turn conversations – Better context retention across messages

Conclusion

Building a personal chatbot taught me that the hard part isn’t the AI—it’s the engineering around it. Token optimization, smart context selection, and clear separation of concerns made the difference between a broken prototype and a production-ready assistant.

The key insight? Every token counts. Not just for cost, but for making your application actually work.

If you’re building something similar, start with the minimal viable context, test extensively, and optimize based on real usage. And remember: if something happens outside the LLM’s response generation, don’t put it in the system prompt.

Want to try it? Visit shanedrumm.com and ask the chatbot anything!

Questions? Reach out at shane@shanedrumm.com or connect on LinkedIn.

Built with: OpenAI GPT-4o-mini, Python, Gradio
Development time: ~3 days
Token reduction: 94%
Cost savings: 92%