PocketLLM

PocketLLM ships a full language model inside an Android app. It runs Google’s Gemma 3n locally using MediaPipe Tasks GenAI — no servers, no API keys, no monthly bills, and no data ever leaving the device. On a flagship phone, responses are smooth; cold start is a couple of seconds.

Why on-device

• Prompts and conversations stay on your phone.
• It works anywhere — flights, remote areas, no signal required.
• No rate limits, API costs, or network latency.

When Gemma 3n became small enough (~3GB quantized) to run on modern phones, the idea clicked: a genuinely private AI assistant that lives entirely in your pocket.

How it works

MediaPipe’s LlmInference API loads the quantized model and drives chat sessions, while WorkManager handles the one-time model download from Hugging Face with progress, retries, and error handling. Credentials are stored with EncryptedSharedPreferences.

Users can tune how the model behaves — temperature, top-K, top-P, and max tokens are all adjustable in settings and applied per chat session, so the same model can be focused and precise or loose and creative.

The code is clean, fully tested, and easy to extend — the build story covers the architecture, the MediaPipe integration, and real-world performance across device tiers.