ASR · 20% of the score · every entrant
Transcribing Hong Kong Cantonese: six tones, constant English code-switching, insurance vocabulary, and callers who are frightened or angry.
Cantonese carries six lexical tones. A tone error does not degrade a word, it replaces it. Every tile below is a word from the calls this benchmark scores — one per tone, with how often it is said in them. Say one on the wrong contour and you have said a different word, which is what the pairs further down price: 買 maai5 / 賣 maai6. Play them.
Pitch demonstration, synthesised from the Chao values shown. Not a recording of speech — the point is the shape, not the voice.
Low rising against low level. In a sales call this inverts who is doing what.
High rising against mid level. 保險 is a policy; 報險 is filing a claim against one. Both are said constantly on the same call.
High level against low level. A claims call is dense with both hospital words and read-aloud digits, so this one lands in the amount as easily as the diagnosis.
Substitutions plus deletions plus insertions, over reference characters. Corpus-level, weighted by length. Cantonese has no word boundaries, so Word Error Rate would be meaningless here.
The same measure, but Latin runs count as whole words. Without it, misreading one English word costs four errors while misreading a Chinese word costs one, and every code-switching system is punished for the wrong thing.