Developers and AI enthusiasts can now run capable local AI agents entirely on their own machines, keeping code, private files, and credentials on the device while a wave of open-weight models arrives with day-zero support on NVIDIA hardware. Because the work runs locally, private tasks do not count toward cloud token limits, and agents can stay on around the clock. Throughout August, NVIDIA detailed new open-weight models, developer tools, and accelerated computing that support responsive agents on a single system.<\/p>\n
Local inference keeps sensitive code, proprietary data, and project context on a developer’s own hardware while AI assistance runs alongside it. Because inference happens on the device, workflows stay private and do not depend on cloud usage caps.<\/p>\n
Perplexity’s local agent app, Portable Computer, runs a personal AI agent on NVIDIA DGX Spark for private, local workflows that do not count toward token limits. It connects to apps including Google Drive, Gmail, Slack, and GitHub, and lets users switch between cloud models for complex work and local models for everyday tasks. DGX Spark provides fast large language model inference and 24\/7 always-on operation suited to long-running autonomous agents. Running locally, Perplexity offers one-click local inference setup with a specially post-trained Qwen 3.8 27B model, and a fine-tuned Nemotron 3.5 Lightning variant for faster responses is in the works. Support for GeForce RTX and RTX PRO GPUs, Windows, and DGX Station is planned.<\/p>\n
Qwen3.8-27B is a 27-billion-parameter open-weight model sized for a single GPU and built for responsive coding with local files, tools, and project context. As the local companion to Qwen3.8-Max, it keeps sensitive code on a developer’s own system. Optimized with multi-token prediction, it reaches 131 tokens per second on a single GeForce RTX 5090, giving fast local inference for agentic coding, tool use, and longer development tasks. Day-zero support is available on NVIDIA RTX GPUs, including RTX PRO GPUs and the GeForce RTX 5090, and developers can start with llama.cpp, Ollama, Unsloth, and LM Studio Bionic. The model also runs on DGX Spark, DGX Station, and, at the edge, NVIDIA Jetson, with NVFP4 support coming to all NVIDIA local AI devices.<\/p>\n
Meta’s Muse Glimmer is a 30-billion-parameter dense open-weight model with a 120K-plus context window, purpose-built for coding and local agentic AI. It delivers over 200 tokens per second on an RTX 5090 and is small enough to run on a PC with a single consumer GPU. Its dense architecture and hybrid attention keep processing and memory demands manageable as agents take on longer tasks. The model is designed for custom agents, private data processing, credential handling on the device, multistep tasks, and long-running projects that can pause and resume with context intact. Developers can run it with vLLM or llama.cpp, use DFlash speculative decoding to speed generation, build agents with NemoClaw, and fine-tune it on private data with NVIDIA NeMo Automodel.<\/p>\n
Several open-weight models run on NVIDIA GPUs and consumer AI hardware:<\/p>\n
Unsloth Desktop brings local model inference, image and video diffusion, fine-tuning, agent integrations, web research, and code execution into one open-source desktop app, and is the first desktop app to both train and run AI models locally. For larger models such as GLM 5.2 and DeepSeek V4 Flash, the NVIDIA Sync app and its Cluster Assistant configure two or more DGX Spark systems as a high-speed cluster over ConnectX-7 ports, routing workloads across nodes and providing secure remote access through Tailscale on Windows and macOS. Later in August, DGX Spark gains a one-click native ARM64 Linux build of Google Chrome from the DGX Dashboard, plus a Sync Resource Monitor that shows real-time and historical CPU and GPU usage across a single system or a full cluster.<\/p>\n
NVIDIA expanded its Nemotron 3 family with Nemotron 3.5 Lightning, a customizable open 30-billion-parameter mixture-of-experts model for always-on agents. It delivers up to 4x faster token generation and 30% faster time to completion compared with open models in its class. Because it ships as open weights, developers can fine-tune it on their own examples to match a preferred writing style, a specialty, or a specific workflow.<\/p>\n
It is a local agent app that runs a personal AI agent on NVIDIA DGX Spark for private workflows that do not count toward token limits. It connects to apps including Google Drive, Gmail, Slack, and GitHub, lets users switch between local and cloud models, and uses a specially post-trained Qwen 3.8 27B model with one-click local inference setup.<\/p>\n
Optimized with multi-token prediction, the 27-billion-parameter open-weight model reaches 131 tokens per second on a single GeForce RTX 5090, giving fast local inference for agentic coding, tool use, and longer development tasks.<\/p>\n
Muse Glimmer is a 30-billion-parameter dense open-weight model with a 120K-plus context window, purpose-built for coding and local agentic AI. It runs at over 200 tokens per second on an RTX 5090 and is small enough for a PC with a single consumer GPU.<\/p>\n