Independent quality baselines for AI services

Local AI is moving beyond text chat into multimodal retrieval and domain workflows. EmbeddingGemma 2 maps text, code, images, video and audio into one space while reporting a 567 MB phone footprint; MiniCPM-V adds compact vision; a health explorer runs a medical explainer in the browser; and an independent builder split a 27B model across two Android phones. Janas reports streaming model experts from SSD, while the fast-growing Strata repository packages a local inference engine for consumer hardware. Most numbers are self-reported, but the deployment boundary is broadening across browser, phone and laptop.

The local-runtime line now has concrete private-search applications rather than only model ports. An open Mac application indexes files, code and agent sessions for retrieval by an agent; a separate Mac product offers on-device file search; and a Windows builder indexes documents, images and audio with local embeddings. A user independently describes the remaining demand for a reliable, polished local AI search experience. These are early launches and a request, not adoption or performance measurements, but together they identify private retrieval as a practical deployment layer.

Local AI is expanding from generation into private multimodal retrieval. Google's EmbeddingGemma 2 maps text, images, audio and video into one on-device embedding space; a GGUF conversion appeared immediately; and a practitioner discussion shows active demand for local personal-memory retrieval without hosted APIs. This extends the local-runtime line into the indexing layer that decides what private context reaches a model. The launch and conversion belong to one model ecosystem, while independent quality, battery and latency comparisons are still absent.

Local inference is pushing beyond conventional memory limits. Strata packages Qwen3.8 for consumer hardware, Edge0 reports running a 35B mixture-of-experts model in about 2.5 GB by streaming weights from storage, and a browser implementation streams a 37 GB model on a 24 GB Mac. Separate work cuts Qwen3-Omni time to first audio from 213 ms to 23 ms on one RTX 5090, while a 16.9 MB speech model targets phones and microcontrollers. The shared mechanism is tiered memory and specialized runtimes. Benchmarks are self-reported and need independent power, latency and quality comparisons.

Model quality is increasingly separated from the serving stack. New work optimizes memory movement, benchmarks production inference, exposes hidden serving confounds, builds robotics-specific runtimes and adds confidential execution. Deployment, latency, cost and privacy are becoming product surfaces in their own right.

Agent evaluation is shifting from final-answer scoring to process, regression and checker quality. New work isolates trainable failure states, selects cheaper trajectory subsets, evaluates computer-use processes and mutates benchmark oracles; an operator message explicitly pairs online evaluation and offline benchmarking with production infrastructure. Reliability is becoming an always-on engineering layer.