Switchboard Inference Challenge
Write the fastest inference server you can for Qwen3.6-35B. We run it against replayed traffic on identical hardware, and the highest throughput within the quality limit wins. A two-week competition from BME and E-Group.
Checking status…
Install the CLI (macOS, Linux)Get started →
Get started
Example output. The names and scores are made up.
The challenge
You get one slice of an NVIDIA B300 and the weights for Qwen3.6-35B-A3B, a mixture-of-experts model. Everything between the request and the GPU is yours to change. Patch SGLang or start from nothing, and try different batching, caches, speculation or arithmetic.
- Model
- Qwen3.6-35B-A3B, NVFP4 weights
- Hardware
- One B300 MIG 1g.34gb slice: 30.5 GiB, 18 SMs
- Host
- 8 CPU threads, no network
- Baseline
- Stock SGLang v0.5.22 on the same kind of slice
Memory on the slice
In our first test, stock SGLang had about 3.7 GB free after loading and ran at most 5 requests at once. It reached 468 tok/s with 5 users. With 16 users, the first token took about 5.6 s.
How runs are judged
Submission
Submit readable source and a switchboard.json that says how to build and start it. We read each person's first submission before running it.
Workload
We replay long agent sessions, chat bursts, requests that share a prefix, and very long contexts. Final runs use the same four kinds of traffic in a mix you won't see in advance.
Quality
You can trade accuracy for speed, up to a limit. Hidden audit requests are compared with a full-precision reference, and final runs add held-out tasks. One result over the limit fails the run.
Score
Your score is throughput divided by stock SGLang's, on the same requests and the same kind of slice, combined across load levels. In each kind of traffic, your 90th-percentile time to first token may be up to 1.5 times stock's. Past that, your throughput counts for less. We rerun the top entries and rank them by their median.
We'll publish the size of the quality limit before submissions open. The latency rule is still a draft.
Prizes
The four highest verified scores win BME's standard study-competition prizes.
Prize money
- First40,000 HUF
- Second30,000 HUF
- Third20,000 HUF
- Fourth15,000 HUF
A role at E-Group
E-Group may invite top finishers to interview for a role. If you're hired, the offer may include a signing bonus.
Places are final once we've rerun the top entries and read their code.
Leaderboard
Each person's best scored run. Scores are throughput relative to stock SGLang.
| # | Name | Score | Tok/s | First token p90 | Mean KL | Runs |
|---|
Rules
This is a draft. We'll publish any missing values before submissions open.
Taking part
- This is an individual competition, by invitation. Each person enters once, with their own verified GitHub account, and must accept these rules.
- It's open to current BME students. We check this when you register, and again before prizes.
- You get 1 scored run and 3 check runs a day. The limits reset at 00:00 UTC.
- We read your first submission before it runs. Automated screening may hold any submission for review, and may send its code to an outside AI provider to do so.
Submissions
- Submit readable source with a switchboard.json. We don't accept binaries, prebuilt GPU code, or compressed, encoded or obfuscated source.
- Builds run offline from a pinned image. Your server runs as an unprivileged user, with no network access and read-only weights.
- You must have the right to submit the code.
Fair play
- Don't try to escape the sandbox, reach other systems or read other people's work. Doing so ends your participation, and we may report it.
- Don't try to detect the judge or audit requests, tamper with timing, or replay stored outputs.
- All positions are provisional. We rerun the top entries and read their code, and we may remove runs that game the challenge.