AI
Small Models and the Embedded Gap
At 1B parameters, a Raspberry Pi model answers like a 10-year-old. At 2B, it generates JSON. At 3-4B, it reliably calls Python functions. The gap between too dumb and useful is narrow. Open-source models from Qwen, Meta’s Llama, HuggingFace’s SmolLM, and Google’s Gemma now hit the 3-4B range. That gave PamirAI, a San Francisco startup founded by ex-Microsoft and Qualcomm engineers, a target: squeeze those models into badge-sized hardware.
The Biohacking Village Badge
PamirAI built a Raspberry Pi 5 badge for DEF CON’s Biohacking Village. The device fits inference into a wearable form factor with four RGB LEDs and a voice interface. Ask for red, green, or blue and the model fires the right LED reliably. Ask for pink or purple and it fails. Those colors require mixing RGB values, a multi-step calculation the model cannot one-shot. The failure marks the ceiling of useful embedded reasoning at this parameter count.
Med-Gemma and Clinical Reasoning at the Bedside
Google’s Med-Gemma is a 4B open-source model trained for clinical reasoning and medical knowledge answering. PamirAI ran it on their Raspberry Pi badge. At DEF CON, a visitor asked how many milligrams of antibiotics to give a 2-year-old. The model answered within the correct range. Google’s own research applied Med-Gemma to chest X-ray report generation with decent results. Running locally, the device handles basic triage questions without sending patient data to a remote server.
Cloud AI Writing Hardware Drivers
Giving a cloud model full control of a Linux system, PamirAI found the agent handles real hardware autonomously. A Flipper Zero plugged in: the agent found the CLI SDK and wrote a screensaver. An Arduino connected: the agent discovered Arduino’s IDE CLI, downloaded board definitions, and flashed firmware. A thermal printer had no Linux driver; the agent reverse-engineered the protocol and wrote one. A Roomba with a USB port got Xbox controller support.
The Road to Fully Local Inference
Cloud models work today, but medical devices cannot depend on them. Biometric data should not leave the device. A paper on multi-step planning, the pre-act paper, shows that fine-tuning a 70B model on reasoning and replanning data brings it close to the largest cloud models. PamirAI expects that level of capability on a MacBook or Mac Studio within two to three months. For medical automation specifically, drop in a machine’s manual or SDK and let the model operate it.
Notable Quotes
knows more medical stuff than me PamirAI · ▶ 6:45
didn’t know that But the AI found that PamirAI · ▶ 9:06
I’m using like I don’t know one trillion PamirAI · ▶ 11:02
Key Takeaways
- At 3-4B parameters, small models reliably call Python functions and control basic hardware sensors.
- Med-Gemma on a Raspberry Pi badge answered antibiotic dosage questions within the correct clinical range.
- AI agents given Linux system access wrote full hardware drivers without a single line of human code.
- Biometric data should not reach a cloud provider; local inference is the prerequisite for medical device AI.
About the Speaker(s)
Kevin and Tianqi are veteran engineers, formerly at Microsoft Surface devices and Qualcomm. Their work focuses on miniaturizing enterprise-grade inference into badge-sized hardware. Their device, Distiller, packs 3-billion-parameter language models into a 10-watt, pocket-safe form factor for bedside clinical use, giving clinicians private access to AI reasoning without a network dependency.