Cantonese Voice Benchmark粵語語音基準

Cantonese Voice Benchmark 2026.1

How they did it.

Every entrant writes up their system before their Full Score is released. These are the ones whose teams chose to publish. A rank tells you who came first; this tells you what to try.

Edition
Cantonese Voice Benchmark 2026.1
Published write-ups
1
Full leaderboard
Full leaderboard
  1. Yardarm7competing under a codenameno committed score yetpublished 23 Sept, 18:16 HKT
    NotOnlyOffice enters Cantonese Voice Benchmark 2026.1 with the three Core pillars that define the published Core Score: speech recognition (ASR), call understanding (LLM), and Cantonese speech synthesis (TTS). We are not entering the optional end-to-end conversation pillar in the qualification round. Our submission is built for the benchmark’s stated setting—real Hong Kong insurance contact-centre calls on a single mixed mono channel, with background noise, overlap, code-switching, and spoken Hong Kong Cantonese rather than clean read speech. We implement the organiser’s WebSocket protocol (version 1.2) as a service we host; their orchestrator connects to us as the client. For portal registration we provide three wss:// URLs on one TLS host, using separate paths for ASR, LLM, and TTS, all served by one integrated server codebase with shared connection handling and logging. Each evaluation clip uses one connection only; we do not carry state across clips. We respond to hello with ready before any audio is streamed, keep the socket limited to protocol frames, and route work according to the pillar field and registered endpoint. For ASR and LLM, the organiser sends paced 16 kHz mono PCM in 100 ms binary frames after ready, then end_of_utterance. We must return a single transcript JSON frame before deadline_ms, which we read from hello. We use an internal time budget with margin before the hard deadline so clips rarely fail for lateness. ASR output is one plain text string in Hong Kong Traditional, reflecting spoken Cantonese as heard on the call. We transcribe both parties in chronological order, including overlapping speech, and we retain fillers such as 呃 and 嗯 because they are scored as speech. We do not add speaker labels or editorial tags. In scored mode we avoid aggressive cleanup that would drop short or quiet segments, since deletions count against character error rate. After recognition we apply OpenCC simplified-to-Hong-Kong-traditional conversion with protection for Cantonese-specific characters that generic conversion would distort, plus conservative repetition and hallucination suppression that does not remove genuine utterances. The LLM pillar receives the same audio stream. The task string in hello is passed unchanged into our prompt. Architecturally we use a cascade: streaming ASR produces a transcript, then a large language model generates the required reply. The response must answer the caller in natural Hong Kong Cantonese, grounded only in what the call contains—not a recap of the transcript. We aim for concise answers that still surface facts downstream processes care about: policy numbers, claim amounts, dates, medical terms, and what was agreed or promised, using Arabic numerals where those facts are spoken. When the call does not establish something, we state that limitation rather than invent coverage or amounts. Before sending, we check that the answer is not unduly echoing the transcript. For TTS, hello carries the text to speak and no input audio. The deadline starts at hello. After ready, we emit speaking as soon as the first synthesized audio byte exists, stream 16 kHz mono signed 16-bit PCM without a WAV header in fixed-size chunks, then send turn_done. For scored runs we use hosted and/or self-hosted components declared here: primary ASR via Alibaba DashScope Qwen3-ASR-Flash and/or self-hosted SenseVoice-small-yue; an optional second-pass recognizer on long clips when time allows (e.g. Qwen3-ASR-1.7B or GLM-ASR-Nano); LLM via Anthropic Claude (faster models on the shortest clips, stronger models when the deadline permits); TTS via a Cantonese neural service such as Azure zh-HK Neural or ElevenLabs with a fixed voice profile. TLS is terminated with publicly trusted certificates; compute is deployed in a nearby region (e.g. Hong Kong, Singapore, or Tokyo) with modest declared concurrency per pillar URL. We do not train, fine-tune, or adapt on benchmark evaluation or practice audio. We may use public insurance wording in prompts or hotwords for terminology only, never to assert facts absent from the call. Audio is processed solely to produce submissions and is not redistributed. Scored runs are fully automated with no human transcription or editing. Remaining risks include overlapped speech, digit and policy-number errors propagating from ASR into understanding, and TTS tone survival under the benchmark’s automatic checks.