None of those products are bad. The problem was the space between them. It took too much tab switching to build a useful picture of what was happening, and that made regular reviews slower than they needed to be.

So I built Site Analytics Tool, a self-hosted control center that brings Google Analytics 4, Google Search Console, and Bing Webmaster Tools into one private portfolio view.

Explore the project · View the public demo · Browse the source

One review, too many tabs

The old workflow was fragmented by design:

• Google Analytics is useful for traffic and engagement, but a review is focused on one property at a time.
• Search Console has its own view of queries, clicks, impressions, and pages.
• Bing Webmaster Tools is another separate place to check visibility and site performance.

That becomes a problem when the question is not simply “How did one site do?” but “Which site needs attention, and why?” A traffic change might be a search issue, a content issue, a stale data feed, or nothing important at all. Seeing the signals in isolation makes that judgment slower.

I wanted a single place to start the review, identify the outliers, and then drill into the site or source that needed attention.

A portfolio view first

The home screen is designed around the decisions I make during a regular review: what moved, what needs attention, and whether the data is current.

It combines portfolio-level traffic and search KPIs with top movers, anomaly signals, action items, and data-coverage warnings. The goal is not to recreate every report from the source products. It is to surface the useful signals first, then make it easy to investigate.

Built for the way I actually work

This is intentionally not a multi-tenant SaaS product. It is a self-hosted tool for a solo publisher or a small portfolio operator.

There is no public signup, billing flow, organization workspace, or role-management screen. That keeps the product focused on the practical job: connecting the sites I own, reviewing their performance, and deciding what to work on next.

Each site has its own integration status, so it is clear whether Google Analytics, Search Console, and Bing are connected and healthy.

The site view also makes it possible to inspect performance at the source level, including Search Console queries and pages, instead of relying only on a portfolio summary.

Reliable data matters as much as charts

A dashboard is only useful when you can trust that its data is current. The tool supports scheduled daily syncs and manual per-site syncs, then keeps a visible history of what happened.

I added stale-data detection, integration health, sanitized error messages, and data-coverage indicators because an empty chart can mean many things. It might be a genuine traffic drop, a missing integration, a failed sync, or a source that has not reported yet. Those are very different problems, and the interface should make that distinction visible.

Private by design

The dashboard contains real performance data, so security and privacy are part of the product rather than an afterthought. It uses an allowlisted admin account and requires TOTP multi-factor authentication before dashboard data can be read.

There is also a privacy mode for screen sharing. It masks site names, domains, account details, and timestamps, which makes it possible to discuss the product or share screenshots without exposing the portfolio behind it.

The public demo uses synthetic data only. It shows the product experience without reading from the private production setup.

From dashboard to report

Some reviews need to leave the browser. Site Analytics Tool can export data as CSV or ZIP files and generate a PDF performance report for a selected site and period.

That makes it easier to keep a record of progress, review results later, or share a focused update without manually assembling screenshots from several dashboards.

How it is put together

The frontend is built with React and TypeScript, hosted as a static application. Supabase provides authentication, Postgres, Row Level Security, Edge Functions, and scheduled jobs. The sync functions pull read-only data from Google Analytics 4, Google Search Console, and Bing Webmaster Tools.

That setup keeps the browser focused on presenting the data while credentials and provider calls stay on the server side. It also makes the tool portable: it can run in a project I control instead of depending on a new hosted analytics service.

What I am learning from using it

The main lesson so far is that analytics becomes more useful when it starts with a decision, not a dashboard. I do not need more charts for their own sake. I need a fast way to answer which site changed, what may have caused it, and what deserves attention next.

Site Analytics Tool is still evolving through day-to-day use, but it already turns a scattered review process into one focused workspace. If you want to see it in action, try the public demo or explore the repository.

View the public demo · Explore the project · Browse the source

See the live onboarding page here

The Problem

You join a new project. The repo has millions of lines of code, hundreds of services, and a directory structure that looks like it evolved over a decade. Someone points you at a README that was last updated six months ago and says “just read the code.”

We have all been there. And it is a terrible experience.

I wanted to see if AI could solve this. Not by explaining one file at a time, but by analyzing an entire repository and producing something a new engineer could actually use on day one.

The Idea

Take a large, real-world open source project. Point an AI coding tool at it. Ask it to produce a complete onboarding guide, with architecture diagrams, folder explanations, common recipes, and getting started steps.

I picked Grafana because it is genuinely complex. Go backend, React/TypeScript frontend, plugin system, multiple database backends, gRPC communication, feature flags, CUE schemas, Yarn workspaces. If the approach works here, it works anywhere.

The tool I used was Claude Code, but the same concept applies to ChatGPT, Cursor, Copilot, or any AI tool that can read a codebase.

What I Asked For

The prompt was straightforward. I asked Claude Code to:

1. Analyze the repository structure, entry points, and architecture
2. Generate Mermaid diagrams for the system architecture and data flow
3. Produce a single self-contained HTML page with everything a new engineer needs

The key was being specific about the output structure. I asked for a hero section, an overview, architecture diagrams, component explanations, key folders, important files, getting started steps, and known gotchas.

What It Produced

The first pass gave me a solid foundation. A clean dark-themed HTML page with:

• A high-level architecture diagram showing how the browser, backend, database, and plugins connect
• A data flow sequence diagram tracing a request from the browser through middleware, API handlers, and service layer to external data sources
• Cards explaining each major component (HTTP API, service layer, plugin system, frontend SPA)
• A table of key directories with what lives in each one
• A list of the 10 most important files and what they do
• Step-by-step getting started instructions
• A command reference block with the most common dev commands

All generated from actually reading the codebase. Not hallucinated, not generic. It found the real entry points (pkg/cmd/grafana/main.go, public/app/index.ts), the real dependency injection setup (pkg/server/wire.go), and the real route definitions.

Making It Better

The first version was good but targeted at experienced engineers. I pushed it further by asking “what would make this useful for someone less experienced?”

That added:

Concept explainers. Plain-English descriptions of things like dependency injection, middleware, ORMs, gRPC, and Redux. Each one uses analogies instead of jargon. Middleware becomes “security checkpoints at an airport.” Dependency injection becomes “instead of building your own database connection, someone hands you one.”

A mental model section. The most important paragraph on the whole page: “This repo has millions of lines of code. That is OK. You are not expected to understand all of it.” Then a simple apartment building analogy. Frontend is the apartment. Backend is the plumbing. Plugins are appliances.

A “Where Do I Look?” decision tree. Ten common tasks (“I want to change what a page looks like”, “I want to add an API endpoint”) mapped directly to the files you need to touch.

Common recipes. Step-by-step instructions for adding a backend endpoint, adding a frontend page, adding a feature toggle, and adding a database migration.

A debugging guide. A table mapping symptoms (backend won’t compile, API returns 500, blank page) to where to look and what to try.

Role-Based Views

Then I had another idea. Different roles care about different things. A software engineer needs to know how to build and debug. An engineering manager needs to understand team boundaries and PR patterns. A product manager needs to understand what features exist and how they ship.

So I added a role picker at the top of the page. Four tabs: Everyone, Software Engineer, Engineering Manager, Product Manager. Clicking one filters the entire page to show only relevant sections.

The implementation is pure CSS. Each section has a data-roles attribute, and the body class toggles visibility. No framework needed.

For EMs, it shows a service ownership map, what to look for in PRs (scope creep, migration risk, Wire changes), and deployment model notes.

For PMs, it shows core product areas (dashboards, explore, alerting), how features ship through feature flags, and where to find product context like routes, config, and user docs.

Each role also gets its own first-week checklist with practical tasks.

The Finishing Touches

A few more iterations added:

• Grouped dropdown navigation. The sticky nav went from 15+ flat links to 4 clean groups: Architecture, Understand, Build, Reference. Each expands on hover.
• Copy buttons on code blocks. One click to copy any command. Shows “Copied!” feedback.
• Back-to-top button. The page is long. A floating arrow appears after scrolling.
• Reading time badges. Each section header shows an estimated reading time calculated from word count.
• FAQ accordion. Nine questions every new hire asks: “Do I need Docker?”, “How do I reset my local DB?”, “How do I enable a feature flag?” Each expands to show the answer.

All of this is a single HTML file. No build step, no dependencies, no npm install. Open it in a browser and it works.

What the AI Got Right

The architecture analysis was surprisingly accurate. It correctly identified:

• The modular monolith pattern with services in pkg/services/ wired via Google Wire
• The newer App SDK pattern in apps/ using Kubernetes-style APIs
• Two query paths: built-in data sources via pkg/tsdb/ and external plugins via gRPC
• The frontend boot sequence from public/app/index.ts through initApp.ts to app.ts
• Feature toggle management in pkg/services/featuremgmt/
• The fact that frontend and backend deploy independently

It also correctly identified that yarn test runs in watch mode by default (a gotcha that trips up every new hire) and that the first build takes about 3 minutes.

What the AI Might Miss

This is important to be honest about.

Tribal knowledge. The AI can read code but it cannot capture unwritten team conventions, “why we did it this way” context, or which parts of the codebase are actively being rewritten.

Ownership accuracy. The team-to-directory mapping is inferred from code structure. Real ownership might differ.

Staleness. The codebase changes daily. File paths and patterns described in the guide might be outdated by the time someone reads it.

Oversimplification. Any diagram that fits on a screen is leaving things out. Service boundaries and internal APIs are more complex than what is shown.

I added an “AI Limitations” section to the page itself so readers know this upfront.

How to Do This for Your Project

The approach is not Grafana-specific. Here is how to replicate it:

1. Pick your tool. Claude Code, ChatGPT with file upload, Cursor, GitHub Copilot in the IDE. Anything that can read multiple files.

2. Give it context. Point it at the root of your repo. If using Claude Code, it reads the filesystem directly. If using ChatGPT, upload key files (README, main entry point, config, route definitions).

3. Be specific about output. Do not just say “explain this codebase.” Ask for a structured onboarding page with specific sections: architecture diagram, key folders, getting started steps, common recipes.

4. Iterate. The first pass will be decent but generic. Push it: “What would help a junior engineer?” “Add a debugging guide.” “Add role-based views.” Each round makes it significantly better.

5. Add honest limitations. Tell the AI to include a section about what it might have gotten wrong. This builds trust with readers.

6. Keep it maintainable. A single HTML file is easy to regenerate. When the codebase changes significantly, run the process again. It takes minutes, not days.

The Bigger Picture

The interesting thing here is not the specific output. It is the pattern.

Every company has onboarding docs that are either nonexistent, outdated, or written for someone who already understands the system. AI tools can now read an entire codebase and produce something useful in minutes.

This does not replace a good mentor or a well-maintained wiki. But it fills the gap between “here is the repo, good luck” and “here is a curated 20-page guide that took someone a week to write.”

The onboarding page I built for Grafana has architecture diagrams, concept explainers, role-based filtering, interactive FAQ, debugging guides, and step-by-step recipes. It took an afternoon of iterating with Claude Code. Doing this manually would have taken a week of reading code and writing docs.

That is the real value. Not replacing human knowledge, but generating a solid first draft that gets a new team member from zero to oriented in 30 minutes instead of 3 days.

Try It Yourself

If you want to try this on your own project, start with something like:

“You are a senior software architect creating an onboarding guide for a new engineer. Analyze this repository and generate a complete onboarding page with architecture diagrams, key folders, important files, getting started steps, and common gotchas.”

Then iterate from there. You will be surprised how good the first pass is, and how much better it gets with a few rounds of feedback.

Live demo · Repo

The Three Agents

• Ava PO (Product Owner): generates one tool idea per day, scores candidates, creates a GitHub issue with acceptance criteria
• Eve SE (Software Engineer): picks up the issue, implements the tool, runs tests, opens a PR
• Nora QA (QA Reviewer): validates the PR against acceptance criteria, auto-merges on pass

There’s no shared runtime or message queue. They talk through GitHub: issues, labels, comments, repository_dispatch events. All state lives in the repo, so you can look at any issue and see exactly what happened.

How It Flows

Ava fires at 9 AM UTC. She generates three tool candidates via OpenAI, scores each one based on value, effort, confidence, and a novelty penalty that discourages repeating recent ideas. The winner becomes a GitHub issue. Ava dispatches se_ready, Eve picks it up, generates the tool under site/tools/, runs tests, and opens a PR. Eve then dispatches se_pr_ready. Nora waits 15 minutes before starting (without that delay she’d sometimes evaluate a PR before GitHub had finished processing it), then runs the test suite, checks the acceptance criteria from the original issue, and auto-merges if everything passes. The merge triggers the Pages deploy.

There’s also a watchdog that runs hourly and re-dispatches any agent that’s been sitting idle too long. It’s what keeps the pipeline from silently stalling after a transient failure.

What’s Shipped So Far

• JSON Formatter / Minifier / Key Sorter
• Markdown Table Builder from CSV
• SQL Formatter and Pretty Printer
• Cron Expression Explainer

Small, focused, no-install tools. Exactly what the scoring formula favors.

A Few Things I Learned

I didn’t need a message queue or a custom orchestration framework. GitHub Issues + labels + dispatch events turned out to be enough to coordinate three independent agents. That surprised me.

The part I underestimated was how much the PO prompt matters. Ava’s acceptance criteria get re-read by Nora later to drive QA decisions. When Ava writes something vague, Nora makes vague decisions. Getting that first prompt right had more impact than anything I did in the SE or QA workflows.

I also added fallbacks everywhere. Ava falls back to a seed idea file when OpenAI fails, Nora retries failed merges with exponential backoff. An autonomous system that hard-crashes on any API hiccup isn’t really autonomous.

Still alpha, but it ships something every day. That was the goal.

Even though the repo name says “analytics,” the project includes both:

• Google Search Console data for search performance
• Google Analytics data for traffic and behavior

The goal is simple: connect these sources to MCP-compatible AI tools so you can ask questions in plain language instead of jumping through five dashboards every time.

Why I Built It

I kept running into the same problem: the data existed, but getting quick answers still took too many steps.

Usually the flow looked like this:

• Open Search Console for query and ranking data
• Open GA for sessions, channels, and engagement
• Export or copy numbers
• Try to connect everything manually

That works, but it is slow when you just need a fast decision. MCP makes it easier to query both systems in one place and keep momentum.

What It Shows in Practice

Here is the dashboard-style overview from the project.

You get high-level visibility first: impressions, clicks, CTR, and average position, followed by top pages and performance trends. It gives you a solid snapshot before you drill deeper.

Google Search Console + Google Analytics Together

This is the part that matters most to me.

From Search Console, you can quickly inspect:

• Top queries by clicks and impressions
• CTR by query
• Average position trends

From Google Analytics, you can inspect:

• Channel mix (organic, direct, referral, social)
• Top pages by sessions
• Engagement metrics like average session duration

When both are available through the same MCP flow, it becomes much easier to connect ranking changes with traffic and engagement outcomes.

Detailed Views

Example of a query-level Search Console breakdown:

Example of analytics views mixed with SEO context:

This is exactly the type of view I wanted: one place to inspect visibility, clicks, channel distribution, and page behavior without switching tools every minute.

Where This Helps Most

I see this being useful in three common scenarios:

• Weekly SEO reviews: get quick summaries of what moved up or down
• Content prioritization: find pages with high impressions but low CTR
• Growth experiments: compare ranking wins with actual traffic quality

It does not replace deep analysis in GA or Search Console UI, but it is very good for fast exploration and decision-making.

Repo

If you want to test it, contribute, or adapt it for your own setup, the repository is here:

https://github.com/jafforgehq/google-analytics-mcp

I’ll keep improving it and share follow-up updates once I add more capabilities.

That’s exactly what I built with SplitDecision. You type in two options, hit compare, and a full-blown debate unfolds in real-time, token by token, complete with rebuttals, a final verdict, and a confidence score.

Here’s what I learned and how it all works under the hood.

What Is SplitDecision?

You enter two options you’re torn between. Four AI agents debate them in two rounds, then a synthesizer agent delivers a final verdict with a confidence score. The whole thing streams live.

The stack:

• Frontend: Next.js 15 (App Router), React 19, TypeScript
• Styling & Animations: Tailwind CSS, Framer Motion
• AI: OpenAI API (GPT-4o Mini / GPT-4.1 Nano / GPT-4.1 Mini)
• Rate Limiting & Storage: Upstash Redis
• Deployment: Vercel

The Four Agents

Each agent has a fixed archetype that shapes how they approach any comparison:

• The Analyst (Blue) - Data-driven. Cites specs, benchmarks, and numbers. Won’t give you a vague opinion.
• The Contrarian (Red) - Defends the underdog. Exposes hidden costs and overlooked advantages.
• The Pragmatist (Green) - Your experienced friend who’s actually used both options.
• The Wildcard (Purple) - Sees angles nobody else does. Future trends, philosophical implications, second-order effects.

The interactions between them are where it gets interesting. The Contrarian tears apart The Analyst’s data, The Wildcard reframes the entire discussion, and The Pragmatist brings everyone back to earth.

How the Debate Works

The debate flows through four phases:

1. Validation - A validation agent checks whether the comparison makes sense. Temperature set to 0.0 for deterministic output, plus OpenAI’s moderation API for content safety.

2. Round 1 - Initial Takes - Each agent responds sequentially with 400 tokens. Temperature at 0.9 for personality-rich responses.

3. Round 2 - Rebuttals - Each agent receives the full Round 1 transcript and responds directly to the others by name. 250 tokens to keep things punchy.

4. Verdict - A synthesizer agent reads everything and produces a structured verdict with a winner, confidence score (50-95%), and conditional recommendations.

The orchestration is a simple loop on the client side:

for (const agentKey of AGENT_ORDER) {
  const msgId = `r1-${agentKey}`;
  setMessages(prev => [...prev, {
    id: msgId, agentKey, round: 1, text: '', isStreaming: true
  }]);

  let fullText = '';
  for await (const chunk of streamChat(apiKey, {
    type: 'agent', agentKey, ...
  })) {
    fullText += chunk;
    setMessages(prev => prev.map(msg =>
      msg.id === msgId ? { ...msg, text: fullText } : msg
    ));
  }

  round1Results[agentKey] = fullText;
}

Each agent streams token by token. When Round 1 finishes, the full transcript gets bundled into Round 2 prompts so agents can reference each other.

Prompt Engineering

This is where the real work happened.

Personality Through System Prompts

Vague instructions like “be analytical” don’t work. You need to tell the model exactly how to think. Here’s the Analyst’s default system prompt:

You are The Analyst, a data-driven, no-nonsense comparison expert. Focus exclusively on specs, numbers, benchmarks, cost, market data, and measurable differences. Quantify everything you can. Your tone is professional and concise. Never give vague opinions, back claims with data or concrete reasoning. Keep your response under 150 words.

And the same agent in “Startup Bros” theme:

You are The Analyst, a growth-obsessed startup metrics guru. You talk in terms of TAM, CAC, LTV, burn rate, and runway. Every comparison is framed as a market opportunity. You reference Y Combinator, a16z, and Series A benchmarks. You say things like ’the unit economics here are clear.’ Keep your response under 150 words.

Same archetype, completely different personality. This is how I support 9 debate themes with 72 unique agent prompts total.

Making AI Sound Human

I added global writing rules to every prompt to avoid AI slop:

• Use contractions (don’t, can’t, it’s)
• Vary sentence length
• Never use em dashes or semicolons
• Avoid hedge words (arguably, essentially, fundamentally)
• No filler phrases (at the end of the day, when it comes to)

These small constraints made a massive difference. The responses feel like actual personalities instead of four variations of “certainly, here’s my analysis.”

Temperature and Token Budgets

Lower temperatures made all four agents sound the same. 0.9 gave each personality room to breathe.

Token limits shape how agents communicate too. Round 1 gets 400 tokens for real arguments, Round 2 gets 250 to force direct engagement instead of restating positions, and the Verdict gets 500 for structured synthesis.

Streaming

I built dual-mode streaming: direct browser calls when users bring their own API key, and server-proxied calls for the free tier.

Direct Browser Streaming

async function* streamDirect(
  apiKey: string, req: StreamRequest
): AsyncGenerator<string> {
  const client = new OpenAI({
    apiKey,
    dangerouslyAllowBrowser: true
  });
  const stream = await client.chat.completions.create({
    model, messages, max_tokens, temperature, stream: true
  });
  for await (const chunk of stream) {
    const text = chunk.choices[0]?.delta?.content;
    if (text) yield text;
  }
}

The AsyncGenerator pattern lets the UI consume tokens with a simple for await loop, keeping orchestration code clean.

Server-Proxied Streaming

export async function POST(req: Request) {
  const ip = req.headers.get('x-forwarded-for');
  const limit = await rateLimit(ip);
  if (!limit.ok) return new Response('Rate limited', { status: 429 });

  const stream = await client.chat.completions.create({
    model, messages, max_tokens, temperature, stream: true
  });

  return new Response(new ReadableStream({
    async start(controller) {
      for await (const chunk of stream) {
        const text = chunk.choices[0]?.delta?.content;
        if (text) controller.enqueue(encoder.encode(text));
      }
      controller.close();
    }
  }));
}

Same experience for the user, but the API key stays on the server. Rate limiting uses Upstash’s fixed-window algorithm, 50 comparisons per IP per 24 hours.

The Verdict System

The verdict isn’t just “Option A wins.” I enforce a structured format in the prompt:

WINNER: [Option A or Option B]
CONFIDENCE: [50-95]%

[3-4 sentence synthesis]

WHAT WOULD FLIP THIS: [1-2 sentences]

PICK [Option A] IF: [1 sentence]
PICK [Option B] IF: [1 sentence]

Then parse it with regex as the verdict streams in:

export function parseVerdict(
  text: string, optionA: string, optionB: string
) {
  const winnerMatch = text.match(/WINNER:\s*(.+)/);
  const confMatch = text.match(/CONFIDENCE:\s*(\d+)/);

  let winner = null;
  if (winnerMatch) {
    const raw = winnerMatch[1].trim().toLowerCase();
    if (raw.includes(optionA.toLowerCase())) winner = optionA;
    else if (raw.includes(optionB.toLowerCase())) winner = optionB;
  }

  const confidence = confMatch
    ? Math.max(50, Math.min(95, parseInt(confMatch[1], 10)))
    : null;

  return { winner, confidence };
}

Confidence is clamped to 50-95%. Below 50% doesn’t make sense for a binary choice, and above 95% feels dishonest for subjective comparisons.

The Theming System

There are 9 debate themes that completely reshape agent personalities:

• Default Panel - Professional experts giving measured takes
• Startup Bros - Everything framed as market opportunity with VC lingo
• Academic Panel - Peer-reviewed discourse with citations
• Bar Argument - Friends arguing over drinks, one of them definitely googled it
• Shark Tank - Is this comparison worth investing in?
• Reddit Thread - Upvotes, hot takes, and “this is the way”
• Courtroom Trial - Legal drama with expert witnesses and objections
• Sports Commentary - Play-by-play analysis of the matchup
• Philosophy Seminar - Existential deliberation on the nature of choice

Each theme rewrites all four agent prompts for both rounds, 72 total plus validation and verdict prompts. A “React vs Svelte” debate feels completely different as a Courtroom Trial versus a Bar Argument.

What I Learned

Constraints create character. Without strict writing rules, token limits, and specific vocabulary guidance, all four agents sound the same. The more constraints I added, the more distinct each voice became.

Agent ordering matters. The first agent sets the frame and everyone else reacts to it. The Analyst always goes first, which means it has outsized influence on every debate.

Structured LLM output is fragile. LLMs don’t always follow format instructions perfectly. Clear regex patterns with sensible fallbacks are essential.

Token budgets shape behavior. The Round 2 limit of 250 tokens was the breakthrough. It forced agents to actually engage with each other instead of restating their position.

Streaming changes perception. Watching agents “think” token by token feels like a live event. It’s fundamentally different from waiting for a complete response.

Claude Code made this viable as a solo project. From streaming logic to 72 unique agent prompts, AI-assisted development made the scope manageable.

Wrap Up

Multi-agent systems don’t need to be complicated infrastructure projects. Good prompt engineering, clear agent archetypes, and a streaming-first architecture can create AI interactions that feel like watching a real discussion.

The biggest takeaway? Personality in prompts matters more than model selection. GPT-4o Mini with a great system prompt produces more engaging debates than a larger model with a generic one. The constraints you put on your agents are what give them character.

If you’re interested in building multi-agent systems, start with clearly defined roles, invest time in prompt writing, and let your agents actually interact with each other’s output. That’s where the magic happens.

Claude Code is an impressive agentic coding tool that can read, modify, and execute code in your working directory. But there’s one catch: it usually requires paying for Claude API calls. What if I told you that you can run the exact same tool with completely free, local models?

That’s exactly what I’ve been experimenting with lately, and I wanted to share how you can do it too.

What you get with this setup:

• Cost: Zero API fees. Run as many coding sessions as you want.
• Privacy: Your code never leaves your machine. No cloud API calls.
• Offline: Work without internet once models are downloaded.
• Learning: Compare open-source coding models with Claude’s performance.

What Is Claude Code?

Claude Code is Anthropic’s official CLI tool that brings AI-powered coding assistance to your terminal. It can understand your codebase, make edits across multiple files, run commands, and help debug issues - all through a conversational interface.

Normally, it connects to Anthropic’s API and uses Claude models (Sonnet, Opus, Haiku). But thanks to an Anthropic-compatible API bridge, you can point it at Ollama instead.

Prerequisites

Before we start, you’ll need:

1. Ollama installed - If you don’t have it yet, check out my earlier post on running LLMs locally.

2. Claude Code CLI - Install from npm:

npm install -g @anthropics/claude-code

3. Enough RAM - Claude Code needs models with large context windows (minimum 64k tokens). Plan for at least 8GB of free RAM.

Step 1: Choose Your Model

Not all models work well with Claude Code. You need coding-specific models with large context windows. Here are the recommended ones:

Best options:

• qwen2.5-coder:7b - Strong coding performance, good balance
• qwen2.5-coder:14b - Better quality if you have the RAM
• deepseek-coder-v2:16b - Excellent at following instructions
• codellama:13b-instruct - Solid baseline for coding tasks

Pull your chosen model:

ollama pull qwen2.5-coder:7b

Tip: The first pull downloads several GBs and can take a few minutes depending on your connection.

Step 2: Quick Setup (Recommended)

The easiest way to get started is using Ollama’s automatic configuration:

ollama launch claude

This command will:

1. Start the Ollama server if it’s not running
2. Configure the necessary environment variables
3. Launch Claude Code pointed at your local Ollama instance

You’ll be prompted to select which model to use from your installed models.

Step 3: Manual Setup (Alternative)

If you prefer more control, you can configure it manually:

Set these environment variables:

export ANTHROPIC_API_KEY="ollama"
export ANTHROPIC_BASE_URL="http://localhost:11434/v1"

Then launch Claude Code with your chosen model:

claude --model qwen2.5-coder:7b

Note: The API key can be any string when using Ollama locally - it just needs to be set.

Using Claude Code with Local Models

Once launched, you can use Claude Code exactly as you would with Claude models:

Ask it to create a new feature:

Debug an issue:

Refactor code:

The model will:

• Read relevant files in your project
• Understand your codebase structure
• Make necessary edits
• Run commands to test changes
• Explain what it did

Performance: Local vs Cloud

Here’s what I’ve noticed after using both:

What works well locally:

• Simple bug fixes and code generation
• Following clear, specific instructions
• Working within a focused codebase area
• Standard coding patterns and frameworks

Where Claude models still lead:

• Complex multi-file refactoring
• Nuanced architecture decisions
• Understanding implicit requirements
• Edge case handling

For most day-to-day coding tasks, the local models are surprisingly capable. You might be pleasantly surprised.

Troubleshooting

Out of memory errors?

• Switch to a smaller model (7b instead of 14b)
• Close other applications
• Check ollama ps to see memory usage

Model responses are slow?

• Local inference is slower than API calls - that’s expected
• GPU acceleration helps significantly (Ollama auto-detects)
• Smaller models respond faster

Context window errors?

• Some models have smaller context than advertised
• Try reducing the number of files you’re working with
• Use more focused prompts

Connection refused?

• Make sure Ollama server is running: ollama serve
• Check the base URL is correct: http://localhost:11434/v1

Cost Comparison

Let’s do some rough math:

Claude API pricing (Sonnet 4.5):

• Input: $3 per million tokens
• Output: $15 per million tokens
• A typical coding session might use 50k-200k tokens
• Cost: roughly $0.50-$3 per session

Local Ollama setup:

• One-time: Electricity cost (negligible for CPU, ~$0.10/hour for GPU)
• Ongoing: $0

If you’re doing regular coding sessions, the local setup pays for itself quickly.

When to Use Each

Use local Ollama when:

• Learning and experimenting
• Working on personal projects
• You want complete privacy
• Budget is a concern
• You’re offline

Use Claude API when:

• Working on complex production systems
• You need the absolute best performance
• Speed matters more than cost
• You don’t want to manage local infrastructure

For me, I’ve been using local models for 80% of my coding tasks and only switching to Claude API for the really tricky stuff. That hybrid approach gives me the best of both worlds.

Advanced: Switching Between Models

You can easily switch between different local models to compare them:

# Try a smaller, faster model
claude --model qwen2.5-coder:7b

# Or a larger, more capable one
claude --model deepseek-coder-v2:16b

This is great for finding the right balance between speed and quality for your specific use case.

Integration with Existing Workflow

Claude Code works in any directory, so you can:

cd ~/projects/my-app
claude --model qwen2.5-coder:7b

Then start asking it to help with your project. It will read your code, understand the structure, and make intelligent suggestions.

Works great with:

• Git repositories
• npm/yarn/pnpm projects
• Python virtual environments
• Any codebase really

Wrapping Up

Running Claude Code with Ollama is one of those setups that sounds too good to be true but actually works. You get a powerful agentic coding assistant running completely locally and free.

Is it as good as Claude Opus or Sonnet 4.5? Not quite. But for most everyday coding tasks, it’s more than capable. And the privacy, cost savings, and learning value make it absolutely worth trying.

If you’re already using Ollama for other AI tasks (which I covered in my Ollama guide), adding Claude Code is just one more command away.

Give it a try and let me know what you think. Happy coding!

Further Reading

• Official Ollama Claude Code docs: https://docs.ollama.com/integrations/claude-code
• Claude Code repository: https://github.com/anthropics/claude-code
• Ollama model library: https://ollama.com/library
• My post on MCP integration for even more AI tool capabilities

No servers. No API keys (except initial HuggingFace model download). No monthly bills. Just pure on-device intelligence.

Why I Wanted an On-Device LLM

I’ve been experimenting with running AI locally for a while, first on my MacBook, then on Android. Some reasons why you could want this:

• Prompts and code stay on your device.
• It works offline (flights, remote areas, anywhere).
• No rate limits, no API costs, no waiting on network calls.

When I learned Gemma 3n could be quantized to ~3GB and run on modern phones, it clicked:

“I can ship a full LLM inside an Android app.”

So I did. And it works.

Open LLMs Make This Possible

Models like:

• Meta Llama
• Google Gemma
• Mistral

…gave us something we didn’t have before: public weights that anyone can run.

Gemma 3n in particular is wild for its size. 3 billion parameters, quantized to ~3GB, runs smoothly on a flagship and “usable” on mid-range devices.

Real-World Use Cases

Here’s where on-device AI shines:

• Students → Study help without sending homework to cloud servers
• Developers → Code assistants that keep proprietary code local
• Medical professionals → Draft notes without risking patient data
• Lawyers → Generate documents without leaking client info
• Remote/offline workers → AI that works on a plane or in a tunnel

Basically: if privacy matters, or internet sucks, on-device AI wins.

How PocketLLM Is Built

PocketLLM uses MediaPipe Tasks GenAI to load and run Gemma 3n on Android. Architecture-wise, it’s clean and simple:

• Domain layer → business logic
• Data layer → MediaPipe integration + storage
• Presentation layer → Jetpack Compose UI

Add MediaPipe to Your Project

dependencies {
implementation("com.google.mediapipe:tasks-genai:0.10.27")
}

Requirements:

• Android API 26+
• ARM64 device
• Enough RAM (4GB+ recommended)

For secure key/model storage, you’ll want:

implementation("androidx.security:security-crypto:1.1.0-alpha06")

Core MediaPipe Integration

This is where the magic happens, inside ChatRepositoryImpl.

Initialize the model

llmInference = LlmInference.createFromOptions(
context,
LlmInference.LlmInferenceOptions.builder()
.setModelPath(modelPath)
.setMaxTokens(currentSettings.maxTokens)
.build()
)

Generate responses

session.addQueryChunk(prompt)
val response = session.generateResponse()

Customizable Model Behavior

Users can tweak how the model responds through settings:

• Temperature (0.0-1.0): Controls randomness.
  • Lower = more focused,
  • Higher = more creative
• Top K & Top P: Fine-tune token sampling
• Max Tokens: Control response length (128-2048)

These are passed when creating the chat session, giving users control over the model’s personality.

Gemma 3n uses ~3GB of memory.

Model Downloads

WorkManager handles:

• downloading the model from HuggingFace
• progress updates
• error handling
• retries

Credentials get stored with EncryptedSharedPreferences.

Performance

Cold start: ~2-4 seconds depending on phone. After that, responses are snappy.

• Flagships: smooth
• Mid-range: usable
• Low-end: don’t bother, the model is too large.

What I Learned

• On-device LLMs are 100% usable today.
• MediaPipe GenAI has clean APIs and surprisingly good docs.
• Gemma 3n performs well for its size.

The future is clear: smaller models + better quantization = AI that lives on your device, not on the cloud.

Next steps for the project might include:

• streaming responses
• conversation memory
• multiple model support

…but honestly even v1 is already super functional.

Try It Yourself

Full source code is here: https://github.com/jafforgehq/pocketllm

If you care about privacy, offline AI, or you just want to build something cool with MediaPipe, give it a try. It’s clean, fully tested, and easy to extend.

This is the direction AI is moving, and it’s fun to be early.

Most people use ChatGPT or Gemini in the cloud, but what if you could run AI right on your own laptop or PC? That’s what I’ve been experimenting with lately, and I like it. And if you’re into self-hosting more broadly, tools like Coolify make it just as easy to spin up your own AI services alongside other apps.

What I like after playing with this:

• Privacy: Prompts never leave your machine.
• Cost: No monthly fees once models are downloaded.
• Learning: You’ll touch concepts like context window and quantization instead of treating AI as a black box.

What Is Ollama?

Ollama is a tool that lets you download and run open‑source LLMs locally with a simple CLI. You pull a model once, then chat with it offline.

Tip: The first run of a model downloads several GBs and can take a few minutes.

Install Ollama

macOS or Linux (Homebrew):

brew install ollama
ollama --version

Alternative (official script):

curl -fsSL https://ollama.com/install.sh | sh

Alternative (official website for macOS, Linux or Windows): https://ollama.com/download/

Run Your First Model

Start small with Gemma 2B:

ollama run gemma:2b

That command will pull the model if it’s not present, then open an interactive prompt.

Good Models to Try

• gemma3: Latest Google model, available in multiple sizes (1B,4B,12B,27B).
• llama3:8b: Strong general-purpose baseline.
• mistral:7b:Efficient, good reasoning for size.
• gemma:2b: Fast and light for laptops.
• phi3:mini: Tiny but capable.

See what you have locally:

ollama list

Specialized Models

Beyond general chat models, there are domain‑specialized models you can run locally:

• Coding: Trained on code and repos. Better at writing/reading code and following tool‑use prompts. Try codellama:7b-instruct as a lightweight starting point.
• Medical/Legal/Finance: Domain‑tuned models exist on Hugging Face for specialized terminology and compliance language. Quality varies validate outputs and check licenses before use.
• Vision: Multimodal models like llava let you ask questions about images (screenshots, charts, UI states).
• Speech: whisper models handle local transcription without sending audio to cloud services.

Tips:

• Prefer *-instruct variants for chat/assistant use.
• Start with q4 quantization for laptops increase to q5/q8 if you have RAM and want quality.
• Always test on your real tasks (sample codebase, sample note set, or representative documents).

Quantization Explained

When browsing models you’ll see two kinds of size info that are easy to mix up:

• Model size (e.g., 2B, 4B, 7B/8B, 12B, 70B): The number of trainable parameters. Bigger models generally reason better but need more memory and run slower.

• Quantization (e.g., q4_0, q5_1, q8_0): How many bits are used per weight when loading the model. Lower bits = smaller memory footprint and faster load on CPUs, at the cost of some quality.

Example tag: llama3:8b-instruct-q4_0

• 8b: ~8 billion parameters (model capacity / quality indicator).

• instruct: Chat-tuned variant for conversational use.

• q4_0: 4-bit quantization preset (lighter memory use, faster inference).

Memory Usage

Actual usage depends on quantization preset, loader, and whether you run on CPU or GPU but these ranges give you a feel:

• 2–4B q4: ~1–2.5 GB

• 7–8B q4: ~3.5–5 GB

• 7–8B q8: ~7–9 GB

• 13B q4: ~6–8 GB

Run a specific quantized build:

ollama run llama3:8b-instruct-q4_0

If a tag without quantization is used, Ollama selects a sensible default from the library you can always pin an explicit q4/q5/q8 for predictability.

Browse models:

• Ollama Library: https://ollama.com/library
• Hugging Face Hub: https://huggingface.co/models

Use Ollama via API

Ollama also exposes a local API, so you can call models from apps like JabRef or even wire it into your own projects. If you’re curious about the bigger picture, I wrote a post about MCP , which shows how standards like MCP make it easier to connect local LLMs with other AI tools in a consistent way.

Start the server (if not already running):

ollama server

Example curl call:

curl -s http://localhost:11434/api/generate -d '{
  "model": "llama3:8b",
  "prompt": "Give me three AI project ideas."
}'

You can also use OpenAI-compatible clients by pointing the base URL to http://localhost:11434/.

Call Ollama from Postman

Prefer to test APIs visually? You can call Ollama directly from Postman.

• Method: POST
• URL: http://localhost:11434/api/generate
• Headers: Content-Type: application/json
• Body (raw JSON):

{
  "model": "llama3:8b",
  "prompt": "Summarize why local AI can be useful in 3 bullets.",
  "stream": false
}

If stream is false, Postman shows the full response at once. With true, you’ll see a stream of events. Here is how it looks in Postman:

Troubleshooting

• Out of memory? Try a smaller or more quantized model (e.g., llama3:8b-instruct-q4_0).
• Downloads slow? First pull can be several GB let it finish once.
• Performance feels laggy? Close other heavy apps, or switch to a 2B–7B model.

Use GUI Apps (No Terminal)

If you prefer a GUI app, you can use:

• AnythingLLM: https://anythingllm.com/
• LibreChat: https://www.librechat.ai/
• LM Studio: https://lmstudio.ai/

A major trend in the world of AI tools right now is something called the Model Context Protocol (MCP). After researching and implementing it in one of my sample projects, I wanted to share an overview of MCP in this blog post.

What is Model Context Protocol

Model Context Protocol (MCP) is a flexible standard that allows AI tools to connect not just to databases, but to a wide range of resources such as APIs, file systems, document stores, and even real-time event streams. Its goal is to make it easy for AI applications to interact with all sorts of data and services using a common approach, so you can add new capabilities without a lot of extra work. While MCP can power integrations with web services, files, and custom plugins, in this post I’ll focus specifically on how it can be used to connect AI tools to databases, since that’s where I’ve found it most immediately useful and impactful. Another great feature is that it enables users who aren’t familiar with writing queries to perform various operations on a database. Based on your request, it will automatically generate and execute the appropriate query directly on the database.

First thing first, we need a database

To showcase this example, we need to create a database. We’ll create a Postgres database and deploy it on our machine using the Coolify platform, which we covered in this article: https://jafforge.com/posts/coolify/. This allows us to deploy a database in just two clicks.

Since this is a demo database, we’ll create it with standard data and enable public access.

Disclaimer: Public access is not recommended for production apps, but it’s acceptable for this example to showcase how it works.

Database Ready

Now that our database is ready, we’ll use DBeaver to connect and add some dummy data.

After connecting, we’ll run the following query to create the table:

CREATE TABLE users (
    id SERIAL PRIMARY KEY,
    name VARCHAR(100),
    email VARCHAR(100) UNIQUE,
    city VARCHAR(100),
    address VARCHAR(150),
    age INT,
    is_active BOOLEAN,
    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);

Next, let’s insert some dummy data:

INSERT INTO users (name, email)
VALUES 
  ('Diana Prince', 'diana.prince@example.com'),
  ('Ethan Hunt', 'ethan.hunt@example.com'),
  ('Frodo Baggins', 'frodo.baggins@example.com'),
  ('Grace Hopper', 'grace.hopper@example.com'),
  ('Hank Moody', 'hank.moody@example.com'),
  ('Isla Fisher', 'isla.fisher@example.com'),
  ('Jack Sparrow', 'jack.sparrow@example.com'),
  ('Katniss Everdeen', 'katniss.e@example.com'),
  ('Luke Skywalker', 'luke.skywalker@example.com'),
  ('Maria Hill', 'maria.hill@example.com'),
  ('Neo Anderson', 'neo.anderson@example.com'),
  ('Olivia Wilde', 'olivia.wilde@example.com'),
  ('Peter Parker', 'peter.parker@example.com'),
  ('Quentin Beck', 'quentin.beck@example.com'),
  ('Rachel Green', 'rachel.green@example.com'),
  ('Steve Rogers', 'steve.rogers@example.com'),
  ('Tony Stark', 'tony.stark@example.com'),
  ('Uma Thurman', 'uma.thurman@example.com'),
  ('Viktor Reznov', 'viktor.reznov@example.com'),
  ('Wanda Maximoff', 'wanda.maximoff@example.com');

Now let’s check the database to see if all the data was saved correctly.

Everything looks good. Now let’s proceed with adding this database to Cursor AI via MCP. (I’m using Cursor AI since I’m currently experimenting with it, but the logic is similar for all AI tools that support MCP.)

Adding postgres via MCP to Cursor

First, open Cursor IDE, go to Settings, and navigate to Tools and Integration.

Add a new MCP server and fill in all the details.

Note: For the connection, you can either provide a full connection string (including host, password, port, user, etc.) or specify a config object. Cursor supports both. For this example, I used a connection string.

Once saved, if all credentials are correct, Cursor will be able to access your database.

Let’s retrieve the tables present

Now let’s retrieve all users from users table

Next, let’s showcase an example of filtering:

Note: MCP Postgres connections only support read operations. The database is treated as a read-only model source.

Wrap up

As you can see, MCP is a really interesting concept that enables us to create new data sources for our AI agents. This technically allows both engineers and non engineers to use natural language to execute any possible query on a database without writing a single line of SQL.

I’ve tested this on another project, and the Postgres MCP integration can even suggest improvements to your tables or queries, which can have a significant impact on large databases.

All in all, I expect that more and more tools will create their own versions of MCP servers that you can integrate into your favorite AI IDEs. You can even create your own MCP if you want!

That’s all for now. Thank you for your attention!

That’s when I found Coolify and set it up on a Hetzner VPS - simple, powerful, and fully under my control. Technically, you could host all of this on something like a Raspberry Pi, an old laptop, or whatever, but for now, let’s keep it on Hetzner.

So, let’s dive into the setup process.

Hetzner

As I already said, for our server we will use a Hetzner VPS, this one in particular:

As you can see, it is a VPS with 4 vCPU, 8 GB RAM, and 80 GB SSD with 20 TB traffic included. This is more than enough for my needs. There are even cheaper options like CX11 starting from 4€ per month, but I think this is a good balance between price and performance. After you choose your machine, Linux version, and location, we can proceed. I chose Ubuntu 24.04 (latest) and Helsinki as the location.

VPS Setup

Once my Hetzner VPS was up, I connected via SSH using the root user. The first thing I did was update all system packages to ensure everything was up to date:

sudo apt update && sudo apt upgrade -y

Since I am not a Linux expert at all, I did some Googling and getting guidance from ChatGPT and found some best practices to secure my machine. First, I changed the default SSH port and disabled root login so I am only able to access my server via SSH.

sudo nano /etc/ssh/sshd_config

And changed the following lines:

Port #Some random port above 1024
PermitRootLogin prohibit-password

After that, I restarted the SSH service:

sudo systemctl restart sshd

There are other things I could do, like creating a new user and adding it to the sudo group, but for now, I think this setup is enough. Anyway, I have snapshots enabled so I can always restore my system to a previous state if something goes wrong. For now, we can say that we are ready to install Coolify. There’s also a great firewall built into the Hetzner console where you can open any port needed. I skipped setting up UFW directly on the Linux machine and instead, I am using the one integrated in the Hetzner console. I guess two firewalls would be overkill for now.

Coolify Setup

Run this in the terminal:

curl -fsSL https://cdn.coollabs.io/coolify/install.sh | sudo bash

This is how it will look after installation:

After that, you need to follow the onboarding steps and create an admin user, and then you can access Coolify at http://<your-server-ip>:8080

Now for the sake of not breaking anything, I used a neat feature of Hetzner to create a snapshot backup that I can restore anytime:

P.S. I also added a custom domain to Coolify and enabled 2FA (I am really surprised that they have 2FA available for free). Also, Coolify is handling free SSL certificates for your domain and automatically renewing them.

Basically, to add a custom domain, you just need to add it in Coolify under Settings/Instance Settings/Instance Domain and adjust your DNS records to point to your server IP, depending on your domain registrar.

Add an A record pointing to your VPS IP and, optionally, a CNAME if you’re using subdomains.

After that, you can access Coolify at https://yourdomain

That’s it Coolify is up and running, and you’re ready to start deploying projects. If you’re thinking about self-hosting your own apps, I highly recommend trying this setup.

Benefits

One of the things I really like about Coolify is how easy it is to connect it with GitHub. I set it up so this blog gets automatically deployed every time I push to the main branch.

Right now, I have all of this running at the same time on the same server:

• This blog (Hugo static site)

• An n8n instance

• A backend app built with Spring Boot

• A Postgres database

So in summary, Coolify gives you:

• Push-to-deploy workflows for your code

• Instant-ready runtimes for a wide range of languages and frameworks

• One-click apps + managed databases

It’s genuinely everything you need for a modern self-hosted PaaS.

That’s all for now. I’ll keep sharing more as I continue building, experimenting, and learning new things along the way. Whether it’s another backend app, some automation with AI, or just a fun side project, I plan to document it here. Until the next cool thing, thanks for reading and see you in the next post!