Built at ExecuTorch Hackathon — Compute Track
BlindShare keeps a screen share safe. It captures a window on this PC, reads the text with Windows OCR, and uses an AI model to cover secrets before that image is sent. Keys, passwords, email addresses, phone numbers, card numbers, SSNs, IBANs, and the keywords you type are masked. Ordinary sentences stay visible. Everything in the shipped app runs on the machine. There is no cloud call.
BlindShare keeps a screen share safe. It captures a window on this PC, reads the text with Windows OCR, and uses an AI model to cover secrets before that image is sent. Keys, passwords, email addresses, phone numbers, card numbers, SSNs, IBANs, and the keywords you type are masked. Ordinary sentences stay visible. Everything in the shipped app runs on the machine. There is no cloud call.
QualcommExecuTorchOpenRead
An open-source chest X-ray reader that shows its work. A vision language model reads the film, reports the regions it looked at in order, and drafts a radiology report. The UI shows that reading path live as a red cloud moving over the image, then gives you an editable Exam / Findings / Impression, a one-click differential diagnosis and management panel, and a two-button rating that feeds a per-model benchmark. Works with any OpenAI-compatible endpoint (OpenAI, OpenRouter, vLLM, Ollama), the Anthropic API, or Amazon Bedrock. Ships with a mock engine so it runs with no API key, and a fully on-device local engine built on ExecuTorch (below). Reading Research prototype. Not for clinical use. The radiologist decides. OpenRead is for exploring how vision language models read radiographs. It is not a medical device, it has not been validated, and nothing it produces should be used to diagnose or treat a patient. Hackathon plan: the five-task pipeline, the Snapdragon ladder, the Astra-reference benchmark and dataset notes are in docs/PLAN.md. Run it fully on-device (ExecuTorch Hackathon) ENGINE=local replaces the single vision language model with a pipeline of small specialist models that run on the laptop with no network: a chest X-ray classifier exported to ExecuTorch, a reading path built from its activations, and a small local LLM (or a template) that writes the report. Built for the ExecuTorch Hackathon compute track (Dell Latitude 7455, Snapdragon X Elite, Windows on ARM): QNN/HTP for the NPU, XNNPACK on the CPU as the fallback. image | v [1] findings model ......... py/cxr_worker.py (Python, spawned once, JSON lines) TorchXRayVision DenseNet-121 "densenet121-res224-all" as an ExecuTorch .pte -> 18 pathology probabilities + a CAM box per pathology (no .pte? falls back to PyTorch on CPU and says so) | v [2] reading path ........... server/lib/findings.js (JavaScript, deterministic) lungs overview -> each positive finding with its CAM box, zoom: true -> heart and mediastinum -> pleura and costophrenic angles -> bones and soft tissues streamed as `path` events with a small delay, so the cloud animates | v [3] report writer .......... LOCAL_LLM= executorch ExecuTorch Llama runner (llama_main) with a 4-bit Llama 3.2 3B or Qwen3 4B openai-compatible any local server (Ollama, LM Studio, vLLM) for testing on a Mac template no LLM: a deterministic report straight from the probabilities -> `token` events while it streams, then `report`, then `done` The same step answers "Show me differential diagnosis" (rule-based with template). Three model roles, three files you can swap independently: role model where swap for findings TorchXRayVision DenseNet-121, 18 outputs, 224 px models/cxr-densenet121.pte via scripts/export_cxr.py any classifier that returns probs + feature maps (edit py/cxr_model.py) reading path none, rules over the findings server/lib/findings.js a segmentation model for true lung fields report writer Llama 3.2 3B Instruct or Qwen3 4B, 4-bit models/*.pte via scripts/export_llm.md any instruct model the runner loads Quickstart on the Dell (details in scripts/setup-snapdragon.md) # 0. the pipeline with no models at all (canned findings, templated report): proves the plumbing ENGINE=local CXR_WORKER=py/cxr_worker_mock.py LOCAL_LLM=template npm run dev # 1. real classifier on CPU through PyTorch (downloads 30 MB of weights once) python -m venv .venv && . .venv/bin/activate # .venv\Scripts\activate on Windows pip install -r py/requirements.txt ENGINE=local LOCAL_LLM=template npm run dev # log says: falling back to PyTorch # 2. export to ExecuTorch and run the .pte (XNNPACK; add --backend qnn for the NPU, see the docs) python scripts/export_cxr.py # writes models/cxr-densenet121.pte + verification run ENGINE=local LOCAL_LLM=template npm run dev # log says: findings worker ready, backend=executorch # 3. a local LLM writes the report # a) tonight on a Mac: ollama pull llama3.2:3b, then ENGINE=local LOCAL_LLM=openai-compatible LOCAL_LLM_URL=http://localhost:11434/v1 LOCAL_LLM_MODEL=llama3.2:3b npm run dev # b) on the device with ExecuTorch: scripts/export_llm.md, then LOCAL_LLM=executorch LLM_RUNNER=... LLM_MODEL=... LLM_TOKENIZER=... # 4. a labelled sample set and the agreement check python scripts/fetch_samples.py --n 40 # NIH ChestX-ray14 sample via Kaggle (see samples/README.md) ENGINE=local npm run bench # per-label agreement of the classifier's top finding All variables are documented in .env.example under "local". npm test runs the local engine through py/cxr_worker_mock.py, so the test suite needs Python 3 but no PyTorch. Honesty notes The classifier is a research model (TorchXRayVision, trained on public datasets with NLP-mined labels). Its probabilities are not diagnoses; they are the model's calibrated outputs on its own operating point, and they are quoted as percentages so nobody mistakes them for certainty. The reading path shows where the classifier's activation was strongest for each finding (class activation map, thresholded at 60% of its max, largest region). That is not a lesion outline and not eye tracking. The anatomic order around those boxes is fixed by us, not chosen by a model. The report writer has not seen the image. It rewrites the classifier's findings. Negatives are phrased "not flagged by the classifier", never "normal". Nothing here is a medical device. The radiologist decides. 5-minute quickstart git clone <this repo> openread && cd openread cp .env.example .env # ENGINE=mock works with no key npm install npm run dev # API on :8787, web on :5173 Open http://localhost:5173, click Try the sample image, then Analyze. You will see the mock engine stream a six-region reading path and a report. To use a real model, edit .env: # OpenAI ENGINE=openai-compatible MODEL=gpt-4o OPENAI_API_KEY=sk-...
DTX
SplitLM rebuilds LLM inference on a Snapdragon X laptop around what each chip is good at, and pushes each phase toward its hardware ceiling. 1. Phase-split engine: long prompts are prefilled on the Hexagon NPU (about 4x the CPU's prefill speed), tokens are decoded on the Oryon CPU (which reads memory fastest), and the model state moves between them through shared memory in milliseconds. For big models the CPU decodes straight from the NPU's copy of the weights, so a 24B MoE (LFM2.5-24B-A2B) runs split in 14.7 GB instead of ~27 GB. Concurrent requests overlap: the NPU prefills while the CPU decodes a batch. OpenAI-compatible server and live dashboard. 2. Custom kernels (llama.cpp fork): fused NPU element-wise kernels (+26% Qwen3.5-2B NPU prefill vs stock kernels), a streamed, pipelined NPU MoE expert matmul and causal block skipping in NPU flash attention (LFM2.5 prefill +21% vs stock, then +9-19% more), CPU kernels that read NPU-tiled weights in place, decode-overhead cuts, and MTP speculative decoding (Qwen3.8-27B 1.7x). Results on a Dell Latitude 7455 (X Elite X1E80100, 32 GB): - Qwen3.5-2B, 4K-token request: ~2x faster than llama.cpp's default device choice, 1.4x NPU-only, 2.5x CPU-only (same build). - Demo: first token in 0.98 s vs 3.84 s CPU-only on a 1.6K-token document, same decode speed (~48 t/s). - NPU prefill reaches 39% of the 11.48 TFLOPS FP16 peak (2B); 1.2x Qualcomm's own GenieX runtime on the same model file. Every speedup is a same-session A/B with identical text or perplexity; methods and limitations are in the repo.
whales
Your platform for on-device automated, intelligent, and proactive assistance and agents that remember and seamlessly embed into your workflow. Leveraging a ExecuTorch's ability for efficient local model inference and combining it with heuristic AI systems and always-on knowledge ingestion, never lose anything again. Composed of the WhaleBrain to autonomously ingest and maintain knowledge, along with WhaleSense to proactively assist and embed itself into your system, and the WhaleRuntime that manages models exported and deployed in ExecuTorch, the Whale system provide the platform for future knowledge systems and seamless agentic experiences.