- Do I need a full voice stack?
- No. ASR and LLM are the baseline and give you a Core Score. TTS and end-to-end are optional and open the Full Score. The leaderboard shows both, so a strong recogniser is never hidden behind a missing speech synthesiser.
- How long does integration take?
- Two to three hours for ASR, about a day for the full stack. You implement a WebSocket server and we connect to it; there is a scaffold and a conformance checker so you can get to green locally before exposing anything.
- Why do you call my endpoint instead of taking a file?
- Because latency and turn-taking cannot be measured from a file, and those are 25% of the score. A batch upload path exists as a fallback, but it is Core Score only and labelled on the leaderboard.
- What happens if my endpoint goes down mid-window?
- The run reports insufficient data rather than a bad score, and it does not consume your quota. Failures we caused are attributed to us and re-run for free. A downed endpoint is never published as a 100% error rate, because that would be a false statement about your system.
- Can I see which clips I got wrong?
- On practice runs, yes, with the reference transcript alongside your output. On scored runs you see per-item status and your own response, but not the ground truth.
- Who judges the LLM pillar?
- Two model families from different providers, three runs each, median taken. A judge scoring its own vendor is a conflict, so agreement between families is published. Before the first scored window the judge is validated against human experts and the correlation is published whatever it says.
- Who rates the speech?
- Paid native Cantonese listeners who pass a calibration set first. They never see which system produced a sample, playback order is randomised, and ratings are only accepted once the audio has actually finished playing.
- Will I be named?
- Only if you choose to be. Codename by default, opt-in to be named, reversible until publication.
- Can I train on the data?
- No. You may use it to produce submissions during the edition, and copies are deleted within thirty days of it closing.
- Why are three teams sharing rank 2?
- Because their confidence intervals overlap. With thirty clips a difference of a fraction of a percent is noise, and printing a strict order over noise is false precision. Tied means we cannot tell them apart, and saying so is more useful than inventing a winner.