A new Go inference engine called dlgo lets developers run GGUF models locally without Python, CUDA, or external runtimes. The project handles text generation, vision tasks, and image synthesis through a single binary that targets both CPU and Vulkan GPU backends.
Running models without Python dependencies
The core promise is straightforward: install with go install github.com/computerex/dlgo/cmd/dlgo@latest and run dlgo run model.gguf to start an interactive chat session. Adding --gpu enables Vulkan acceleration. A server mode launches a web UI and OpenAI-compatible API at dlgo server --model model.gguf --gpu. The binary builds as a static executable using -ldflags "-linkmode internal" for CPU-only deployment, or with -tags vulkan for GPU support.
Vision and multimodal capabilities
The repository includes a vision pipeline built around an mmproj projector. The dlgo-vision tool preprocesses images through CLIP-style encoding, runs the projector, inserts the resulting embedding at an <image> token marker, and feeds the combined sequence to a compatible text backbone. The implementation validates that projector output dimensions match the language model embedding width at runtime. Synthetic fixtures (white, black, gray, color bars, primary colors) help verify projector behavior and token placement. A companion utility cmd/mmproj-encode inspects projector outputs.
Text-to-image diffusion pipeline
Beyond language and vision-language tasks, dlgo ships a local diffusion pipeline targeting Z-Image and Lumina2-style DiT checkpoints. The flow chains a Qwen text encoder, quantized DiT denoising with 3D RoPE, and a SafeTensors VAE decoder to produce PNG output. The dlgo-image binary validates dimensions and estimates peak memory before generation, releasing the text encoder after prompt embedding to free budget for the DiT and VAE. GPU acceleration applies to the DiT execution path via Vulkan. Model compatibility, image quality, determinism, VRAM use, and throughput must be validated per checkpoint, quantization, and resolution combination.
Performance compared to Ollama
Benchmarks on an RTX 4070 Ti SUPER show dlgo outperforming Ollama across multiple architectures. On CPU with AVX2+FMA SIMD, TinyLlama 1.1B Q4_0 reaches 423 tok/s versus Ollama at 187 tok/s (126% faster). Gemma 3 1B Q4_K_M hits 245 tok/s versus 116 tok/s (111% faster). Qwen 2.5 0.5B Q4_K_M achieves 394 tok/s versus 237 tok/s (66% faster). On GPU with Vulkan versus Ollama CUDA, Qwen3.5 0.8B Q8_0 runs at 287 tok/s versus 250 tok/s (15% faster). Qwen3.5 27B Q3_K_M reaches 6.4 tok/s versus 4.0 tok/s (60% faster). Tests use identical GGUF files, temperature 0, seed 42, and max tokens 64.
Architecture support and quantization
The engine supports LLaMA, Qwen2/3, Qwen3 MoE (including 128-expert Qwen3-Coder-30B-A3B), Qwen3.5 with hybrid Gated Delta Net and attention layers, Qwen3.5 MoE, GLM-4.7 with Multi-head Latent Attention, gpt-oss with attention sinks, Gemma 2/3, SmolLM2, Phi, and Whisper for transcription. Over 25 quantization formats are available: Q4_0 through Q8_0, K-quants (Q2_K–Q8_K), I-quants (IQ1_S–IQ4_XS), MXFP4, F16, BF16, and F32. Vulkan GPU inference uses quantized MatVec shaders, fused attention, and dp4a integer dot products. An automatic VRAM budget prevents out-of-memory errors with graceful fallback to partial GPU plus CPU execution. MoE models use fused multi-expert dispatch with GPU-side top-K routing and zero CPU-GPU synchronization. CPU inference leverages parallel worker pools and batched prefill GEMM.
System One API for structured decisions
Alongside generation, dlgo exposes a /v1/systemone endpoint for deterministic closed-set decisions: choice selection, binary verification (noul), and ordered scores. The API accepts a loaded model, an on-disk readout artifact, a state string, and one or more questions with semantic true/false criteria. Responses return typed probabilities with Server-Timing headers for queue and decision latency. A high binary probability reflects preference between candidates, not universal truth. Readout artifacts are trained from feature snapshots of the exact quantized backbone, context length, prompt format, and engine configuration. Three published unified heads exist for Qwen3.5 0.8B Q4_K_M (78.6% GPU, 78.9% CPU), Qwen3.8 2B Q4_K_M (84.9% GPU, 84.1% CPU), and Qwen3.5 9B Q3_K_M (93.7% GPU, 93.5% CPU). The trainer at cmd/systemone-train extracts features per device, union-trains for device-agnostic artifacts, and evaluates against held-out mechanism-disjoint examples.
What this means for Go teams
Teams can embed local inference in Go applications without dragging Python stacks or CUDA toolchains. The same binary serves chat, vision, diffusion, and structured decision workloads. Static linking simplifies deployment. The OpenAI-compatible server enables drop-in replacement for cloud APIs in internal tooling. Vulkan GPU support works on Linux and Windows without vendor-specific compute APIs. The System One head provides a path to calibrated, auditable decisions for routing, classification, and policy enforcement use cases where token generation is the wrong abstraction.