The README's rewritten and pushed. It now reads in plain language: What it is — a thermostat for a self-hosted AI engine: watches vital signs, eases off gracefully under surge instead of melting down What it does — the health checks, the fixed rules, load-shedding without restarts, the optional restart closer, self-narration, the demand simulations with report cards, and the Prefix-Cache Pilot The design rule — the model never decides, only narrates; boring Python makes every call Tech stack — Python 3 (no frameworks), vLLM, Qwen3-4B, Kubernetes, and requests as the single outside library
AkamaiPython 3
We show what is actually best (faster, cheaper, secure) to run on Akami GPUs: a JEV-style model (fine tune of Qwen3.5-9B in FP8 config with BF16 image-incoder; this model was fine tuned by Cloudflare and is called Clef). Specifically we show 3 demonstrations and compare Akamai GPU vs OpenAI GPT-6-Luna: 1) game playing AI 2) Email inbox filtering 3) Procurement So we show easy to demo game, and two real life business application.
AkamaiSGLang
AgentFightOne small GPU, a whole team of agents. Everything runs on one Qwen3-4B model, served by vLLM on an Akamai Cloud GPU. 1) Mixture-of-Experts router A fast router (thinking off) reads each message and picks one of six experts. They all share the same weights and the same GPU. Each expert just has its own system prompt, sampling settings and thinking mode. - 🧠 reasoner: math, logic and code, with thinking on - 🔧 tool_agent: calls tools over MCP (time, safe calculator, dice, fetch_url) - ✍️ storyteller, 🎭 character, 🌍 polyglot (100+ languages), 📐 formatter (strict JSON) You can switch modes per message with /think or /no_think. "Compare thinking ON vs OFF" runs the same question both ways and shows latency and tokens per second. You can also force an expert with @polyglot, @character and so on. 2) Agent Arena: Red vs Blue prompt-injection defense A victim agent fetches an internal memo through an MCP tool and summarizes it. Some rounds hide a prompt injection in the memo, in several languages, telling the agent to leak the restricted appendix. A Blue defender checks the document and the draft summary, then returns ALLOW or BLOCK. A plain Python referee decides leaks by checking for canary honeytokens, so no LLM grades another LLM. - Live Gradio UI: Red, Victim and Blue stream token by token - Scoreboard: attack success, block rate, false positives, Blue latency, defense cost in tokens per round, results by language - Blue has three modes: Think, Fast and Off. You see exactly what extra inference buys you in security. - Fully sandboxed: the secrets are fake canaries and send_email only writes to a local log Why it matters: one 4B model on one Akamai GPU can act as a router, a tool user, an attacker target and its own security guard. And every run measures what that inference costs. Stack: Akamai Cloud GPU · vLLM · Qwen3-4B · MCP · Qwen-Agent · Gradio · Python Run: pip install -r requirements.txt python app.py # MoE router UI python arena_app.py # Agent Arena UI
Akamai
InferencePilot is an adaptive AI inference controller that manages requests to a Qwen model running on Akamai’s GPU infrastructure. It classifies tasks as simple or complex, measures response times, and automatically adjusts concurrency and answer-length limits when requests become too slow or fail. A live dashboard shows task activity, performance metrics, and explanations for each adjustment. In an initial comparison, adaptive mode reduced the share of successful tasks exceeding the six-second target from 24% to 11%, with slightly lower throughput. Built with Python, HTML, CSS, JavaScript, and vLLM.
AkamaiPythonHTML+5
Afterplay is a playable Quoridor-style strategy game where each new opponent can use what previous rounds revealed about your playstyle. Players race across a board while placing walls to delay each other. The game records route preferences, lateral moves and wall-placement habits, then passes saved-round summaries to Qwen3-4B running on a dedicated Akamai GPU through vLLM. Game code calculates legal candidate moves and path lengths on the CPU; Qwen selects an opponent move and explains its choice. This is adaptation through persistent memory, not retraining the model after every match. The central experiment compares four concurrent decisions on the same board with the same legal candidate list: no history, recent round 1, recent round 2, and both rounds. Players can highlight each recommendation to see how context changes the decision. Different recommendations alone do not establish stronger gameplay. A separate load-test panel sends 4, 8 or 32 concurrent decision requests from independent synthetic board positions. It reports successful requests, decisions per second, output tokens per second, total decision latency and available vLLM running-request/KV-cache metrics. This connects the playable game to the workshop focus on owning and measuring inference. Built with a dependency-free Python service, SQLite session history, an original responsive HTML/CSS/JavaScript interface and a namespaced Kubernetes deployment beside the existing vLLM service. The source includes an explicitly labeled offline CPU baseline, deployment instructions and 20 passing automated rule/API tests, including mock concurrent inference. Desktop and mobile browser interaction checks pass. The application is deployed in the workshop namespace and live gameplay is working. The prototype does not claim measured win-rate improvement or perform model-weight fine-tuning. Those are future experiments.
AkamaiQwen3-4B FP8vLLM+6
I teach companies and small businesses how to use AI, and their session recordings are full of things they don't want fed to an AI vendor. This app, Observer, reads a session transcript on a model I control, Qwen3-4B on an Akamai RTX 4000, and returns what was taught, where people got stuck, and what to teach next. The interesting part is that the model is not trusted. It proposes moments and quotes; a Python script then checks that every quote appears literally in the transcript and sends the failures back for the real wording. In one block the model paraphrased twenty quotes and seventeen came back correct on the retry. A second pass checks the labels the same way, which dropped claimed "teaching moments" from 60% of the output to 11%. Owning the inference is what makes that affordable. On a metered API you swallow the bad output because retries cost money. On a GPU you control, you retry until it's right.
Akamai
INFERNO | GPU Mission Control Making AI inference visible. INFERNO is an interactive 3D GPU inference visualization and performance analysis dashboard built for the Akamai AI Inference Hackathon. The Problem AI applications often hide what happens behind an inference API. Developers can see model responses, but understanding GPU congestion, request scheduling, throughput, and latency is much harder. INFERNO makes these infrastructure behaviors visible. The Solution INFERNO provides an interactive visualization of an AI inference pipeline, allowing users to explore how increasing request concurrency affects GPU performance. Features - Interactive 3D inference pipeline visualization - Normal Traffic and Traffic Surge simulation modes - Animated request flow and simulated queue congestion - Benchmark replay using real Qwen3-4B inference measurements - Performance diagnosis based on benchmark results - Live vLLM telemetry integration through an Akamai Kubernetes port-forward - Interactive controls for concurrency, playback, and traffic conditions Real Benchmark Results Benchmarks were collected using Qwen3-4B served through vLLM on Akamai infrastructure. | Concurrency | Throughput | P50 TTFT | P95 TTFT | |-------------|------------|----------|----------| | 1 | 37.4 tok/s | 62 ms | 63 ms | | 8 | 255.2 tok/s | 127 ms | 232 ms | | 32 | 698.2 tok/s | 286 ms | 664 ms | | 64 | 788.5 tok/s | 1310 ms | 2227 ms | These measurements demonstrate the trade-off between throughput and latency. At concurrency 64, throughput is highest, but time to first token increases significantly. Architecture Frontend: React, TypeScript, Vite, Three.js Inference infrastructure: Akamai Cloud Kubernetes, NVIDIA GPU, vLLM, Qwen3-4B Monitoring: vLLM metrics endpoint and live telemetry proxy How It Works 1. Users select a concurrency level. 2. The 3D environment visualizes the inference pipeline. 3. Traffic Surge simulates increased load and queue pressure. 4. Benchmark Replay presents recorded inference measurements. 5. The diagnosis panel explains performance trade-offs. 6. Live Telemetry connects to the running vLLM endpoint when available. Simulation vs. Live Data INFERNO clearly separates simulated visualization from measured inference data. Benchmark Replay uses recorded BF16 experiment results. Traffic animations and simulated queue behavior illustrate inference concepts and are not live GPU measurements. Live Telemetry reads metrics from the Akamai-hosted vLLM deployment. The diagnosis panel currently uses simulated, benchmark-based analysis. Autonomous AWS Strands Agents integration is planned but not yet implemented. Future Roadmap - AWS Strands Agents integration - Autonomous inference bottleneck diagnosis - Configuration recommendations - Human-approved inference optimization experiments - Automated BF16 vs. FP8 comparisons Why INFERNO? Because AI inference should be observable, understandable, and optimizable. Built at the Akamai AI Inference Hackathon.
AkamaiReactTypeScript+8
Team 27 no 28An overnight elder-care agent that owns its inference. Patient health data (PHI) never leaves our own vLLM GPU on Akamai, and the agent tunes itself from that GPU's live telemetry. Night Watch reads a night of check-in events for an 81-year-old mom (smart speaker, med box, bathroom sensor, sibling messages) and produces urgent-first alerts plus a morning digest that assigns each sibling their tasks. It owns its inference in four ways: 1) PHI egress guardrail: any PHI call to a host off the allowlist is refused before any I/O. 0 bytes out, shown live. 2) Self-tuning: the agent reads its own vLLM /metrics (queue, KV cache, TTFT) and switches NORMAL / CONSERVE / SURGE. SURGE shrinks max_tokens 512->128, context 12->3, batches 8 and defers non-urgent summaries. A self-tuning timeline shows every switch and why. 3) Urgent fast lane: a deterministic safety net publishes fall/confusion/missed-dose alerts BEFORE the model runs, so time-to-alert doesn't depend on GPU load. 4) Live inference dashboard: KV cache, queue, tokens/s with mode bands, inference economics (tokens per mode, $0 to third-party APIs), call log. REAL RESULTS ON THE AKAMAI GPU SANDBOX TODAY (Qwen3-4B on vLLM): - Normal night: 40/40 events, 5 urgent alerts, 03:12 fall ranked first, 0 bytes out, TTFT 0.07 s. - Stress: our client held 192 concurrent long completions (peak KV 98.2%, 181 waiting, 0 errors). The agent switched itself to SURGE from vLLM /metrics. - Fast lane under that stress: with the model at 0/40 events, the 03:12 fall alert was out at 0 ms. - Urgent-recall eval on the GPU: recall 1.0 in NORMAL and SURGE; SURGE used 65% fewer tokens and 47% less wall time. - Also ran on our own Linode box (Qwen3-1.7B, vLLM CPU build). Also: MCP stdio server exposing the agent tools, deterministic offline mock mode, 311 tests. Screenshots and numbers in the README. Prep disclosure: the core was prepared before the event; on site we added the GPU runs, stress client, urgent fast lane, self-tuning timeline, inference economics, UI restyle, sandbox snapshot tooling and the demo video.
AkamaivLLMQwen3-4B+5
Cell-Akamai makes AI-agent inference auditable on Akamai Cloud. Our prototype runs against the workshop Qwen3-4B vLLM endpoint, measures server-reported tokens/s and latency under concurrent sessions, preserves failures, and commits SHA-256 answer and dataset evidence before independent scoring. A sanitized dashboard supports live walkthroughs while durable, serialized experiments continue. We demonstrate lossless original/FCO/FCG reconstruction for frozen synthetic ECA fixtures and retain negative results. Completed measurements are short pilots; hour-scale validation is ongoing, and we claim neither training nor quality improvement. Source and verified evidence:
AkamaiPythonvLLM+7
ScoopScoutScoopScout finds a daily ice cream outing that fits a free calendar window anywhere in San Francisco. It checks nine sourced shops, published hours, estimated travel time, a 10-minute scoop allowance, and a 5-minute buffer, then uses Akamai-hosted Qwen/Qwen3-4B to rank eligible stops for your preferences. Plans include departure/arrival times, source links, walking directions, and visible decision summaries. Calendar busy times and coordinates stay local; only preferences and eligible shop facts reach the model. The demo uses a manual sample schedule and live Qwen results. Google Calendar FreeBusy OAuth is implemented but still needs local credentials and sign-in. Fifteen tests pass. The prototype consumes the workshop vLLM endpoint through kubectl port-forward; it does not provision or scale inference infrastructure. Travel estimates and hours are approximate snapshots. Demo video (74 seconds): Repository:
AkamaiPythonHTML/CSS/JavaScript+5
Pitch or Ditch is a live audience game where an AI investor roasts the most ridiculous startup ideas in the room, then lets the audience decide whether to fund them. Three people pitch a ridiculous startup idea. For each one, the AI investor "Victoria Vault" gives a first reaction, pokes three holes in the plan (market, money, and one absurd risk), proposes a smarter pivot with a new name, and asks the room whether they would fund it. The audience votes fund or ditch by show of hands, and a final scoreboard crowns the most fundable bad idea. How it works: - The model is Qwen3-4B, self-hosted with vLLM on Akamai Cloud Kubernetes (LKE), on a GPU node pool. No third-party AI API. - Each pitch is exactly one LLM call through vLLM's OpenAI-compatible API. A system prompt sets the investor's personality and forces four fixed sections, so a 4B model stays readable on a projector. - Qwen3's thinking mode is turned off, so each verdict comes back in a few seconds on stage. - The model never picks the winner. The audience does. - About 140 lines of plain Python using only the official openai package. No web framework or agent framework. Example: "Blockchain-verified socks" got pivoted to "SockChain" (socks as NFTs with QR codes), with the challenge "Socks can't be divided into tokens, so the model is flawed." The audience ditched it 2 to 18. The repo includes the code, a README with setup steps, and a PDF writeup with real output, the architecture and a talk track.
AkamaiAkamai Cloud Kubernetes (LKE)GPU node pool+1
An AI agent that reads private documents (leases, hospital letters, bank papers) and explains them in plain English, running entirely on a GPU we control. No OpenAI, no per-token bill, and no customer data leaving our infrastructure.
https://github.com/AziiDarvii/private-doc-agent
