# INFERNO | GPU Mission Control ### Making AI inference visible. INFERNO is an interactive 3D GPU inference visualization and performance analysis dashboard built for the Akamai AI Inference Hackathon. ## The Problem AI applications often hide what happens behind an inference API. Developers can see model responses, but understanding GPU congestion, request scheduling, throughput, and latency is much harder. INFERNO makes these infrastructure behaviors visible. ## The Solution INFERNO provides an interactive visualization of an AI inference pipeline, allowing users to explore how increasing request concurrency affects GPU performance. ### Features - Interactive 3D inference pipeline visualization - Normal Traffic and Traffic Surge simulation modes - Animated request flow and simulated queue congestion - Benchmark replay using real Qwen3-4B inference measurements - Performance diagnosis based on benchmark results - Live vLLM telemetry integration through an Akamai Kubernetes port-forward - Interactive controls for concurrency, playback, and traffic conditions ## Real Benchmark Results Benchmarks were collected using Qwen3-4B served through vLLM on Akamai infrastructure. | Concurrency | Throughput | P50 TTFT | P95 TTFT | |-------------|------------|----------|----------| | 1 | 37.4 tok/s | 62 ms | 63 ms | | 8 | 255.2 tok/s | 127 ms | 232 ms | | 32 | 698.2 tok/s | 286 ms | 664 ms | | 64 | 788.5 tok/s | 1310 ms | 2227 ms | These measurements demonstrate the trade-off between throughput and latency. At concurrency 64, throughput is highest, but time to first token increases significantly. ## Architecture Frontend: React, TypeScript, Vite, Three.js Inference infrastructure: Akamai Cloud Kubernetes, NVIDIA GPU, vLLM, Qwen3-4B Monitoring: vLLM metrics endpoint and live telemetry proxy ## How It Works 1. Users select a concurrency level. 2. The 3D environment visualizes the inference pipeline. 3. Traffic Surge simulates increased load and queue pressure. 4. Benchmark Replay presents recorded inference measurements. 5. The diagnosis panel explains performance trade-offs. 6. Live Telemetry connects to the running vLLM endpoint when available. ## Simulation vs. Live Data INFERNO clearly separates simulated visualization from measured inference data. Benchmark Replay uses recorded BF16 experiment results. Traffic animations and simulated queue behavior illustrate inference concepts and are not live GPU measurements. Live Telemetry reads metrics from the Akamai-hosted vLLM deployment. The diagnosis panel currently uses simulated, benchmark-based analysis. Autonomous AWS Strands Agents integration is planned but not yet implemented. ## Future Roadmap - AWS Strands Agents integration - Autonomous inference bottleneck diagnosis - Configuration recommendations - Human-approved inference optimization experiments - Automated BF16 vs. FP8 comparisons ## Why INFERNO? Because AI inference should be observable, understandable, and optimizable. Built at the Akamai AI Inference Hackathon.
Built at Build AI agents that own their inference - #SFTechWeek
INFERNO | GPU Mission Control
# INFERNO | GPU Mission Control ### Making AI inference visible. INFERNO is an interactive 3D GPU inference visualization and performance analysis dashboard built for the Akamai AI Inference Hackathon. ## The Problem AI applications often hide what happens behind an inference API. Developers can see model responses, but understanding GPU congestion, request scheduling, throughput, and latency is much harder. INFERNO makes these infrastructure behaviors visible. ## The Solution INFERNO provides an interactive visualization of an AI inference pipeline, allowing users to explore how increasing request concurrency affects GPU performance. ### Features - Interactive 3D inference pipeline visualization - Normal Traffic and Traffic Surge simulation modes - Animated request flow and simulated queue congestion - Benchmark replay using real Qwen3-4B inference measurements - Performance diagnosis based on benchmark results - Live vLLM telemetry integration through an Akamai Kubernetes port-forward - Interactive controls for concurrency, playback, and traffic conditions ## Real Benchmark Results Benchmarks were collected using Qwen3-4B served through vLLM on Akamai infrastructure. | Concurrency | Throughput | P50 TTFT | P95 TTFT | |-------------|------------|----------|----------| | 1 | 37.4 tok/s | 62 ms | 63 ms | | 8 | 255.2 tok/s | 127 ms | 232 ms | | 32 | 698.2 tok/s | 286 ms | 664 ms | | 64 | 788.5 tok/s | 1310 ms | 2227 ms | These measurements demonstrate the trade-off between throughput and latency. At concurrency 64, throughput is highest, but time to first token increases significantly. ## Architecture Frontend: React, TypeScript, Vite, Three.js Inference infrastructure: Akamai Cloud Kubernetes, NVIDIA GPU, vLLM, Qwen3-4B Monitoring: vLLM metrics endpoint and live telemetry proxy ## How It Works 1. Users select a concurrency level. 2. The 3D environment visualizes the inference pipeline. 3. Traffic Surge simulates increased load and queue pressure. 4. Benchmark Replay presents recorded inference measurements. 5. The diagnosis panel explains performance trade-offs. 6. Live Telemetry connects to the running vLLM endpoint when available. ## Simulation vs. Live Data INFERNO clearly separates simulated visualization from measured inference data. Benchmark Replay uses recorded BF16 experiment results. Traffic animations and simulated queue behavior illustrate inference concepts and are not live GPU measurements. Live Telemetry reads metrics from the Akamai-hosted vLLM deployment. The diagnosis panel currently uses simulated, benchmark-based analysis. Autonomous AWS Strands Agents integration is planned but not yet implemented. ## Future Roadmap - AWS Strands Agents integration - Autonomous inference bottleneck diagnosis - Configuration recommendations - Human-approved inference optimization experiments - Automated BF16 vs. FP8 comparisons ## Why INFERNO? Because AI inference should be observable, understandable, and optimizable. Built at the Akamai AI Inference Hackathon.
AkamaiReact, TypeScript, Vite, Three.js, React Three Fiber, Python, Kubernetes, NVIDIA GPU, vLLM, Qwen3-4BKeep exploring what builders shipped.
HackerSquad project
Inference Health Monitor
The README's rewritten and pushed. It now reads in plain language: What it is — a thermostat for a self-hosted AI engine: watches vital signs, eases off gracefully under surge instead of melting down What it does — the health checks, the fixed rules, load-shedding without restarts, the optional restart closer, self-narration, the demand simulations with report cards, and the Prefix-Cache Pilot The design rule — the model never decides, only narrates; boring Python makes every call Tech stack — Python 3 (no frameworks), vLLM, Qwen3-4B, Kubernetes, and requests as the single outside library
HackerSquad project
Decision Lab on Akamai GPU
We show what is actually best (faster, cheaper, secure) to run on Akami GPUs: a JEV-style model (fine tune of Qwen3.5-9B in FP8 config with BF16 image-incoder; this model was fine tuned by Cloudflare and is called Clef). Specifically we show 3 demonstrations and compare Akamai GPU vs OpenAI GPT-6-Luna: 1) game playing AI 2) Email inbox filtering 3) Procurement So we show easy to demo game, and two real life business application.
AgentFight
AgentFight
One small GPU, a whole team of agents. Everything runs on one Qwen3-4B model, served by vLLM on an Akamai Cloud GPU. 1) Mixture-of-Experts router A fast router (thinking off) reads each message and picks one of six experts. They all share the same weights and the same GPU. Each expert just has its own system prompt, sampling settings and thinking mode. - 🧠 reasoner: math, logic and code, with thinking on - 🔧 tool_agent: calls tools over MCP (time, safe calculator, dice, fetch_url) - ✍️ storyteller, 🎭 character, 🌍 polyglot (100+ languages), 📐 formatter (strict JSON) You can switch modes per message with /think or /no_think. "Compare thinking ON vs OFF" runs the same question both ways and shows latency and tokens per second. You can also force an expert with @polyglot, @character and so on. 2) Agent Arena: Red vs Blue prompt-injection defense A victim agent fetches an internal memo through an MCP tool and summarizes it. Some rounds hide a prompt injection in the memo, in several languages, telling the agent to leak the restricted appendix. A Blue defender checks the document and the draft summary, then returns ALLOW or BLOCK. A plain Python referee decides leaks by checking for canary honeytokens, so no LLM grades another LLM. - Live Gradio UI: Red, Victim and Blue stream token by token - Scoreboard: attack success, block rate, false positives, Blue latency, defense cost in tokens per round, results by language - Blue has three modes: Think, Fast and Off. You see exactly what extra inference buys you in security. - Fully sandboxed: the secrets are fake canaries and send_email only writes to a local log Why it matters: one 4B model on one Akamai GPU can act as a router, a tool user, an attacker target and its own security guard. And every run measures what that inference costs. Stack: Akamai Cloud GPU · vLLM · Qwen3-4B · MCP · Qwen-Agent · Gradio · Python Run: pip install -r requirements.txt python app.py # MoE router UI python arena_app.py # Agent Arena UI