Announcing the VoiceMOS Challenge 2023!

Last year we ran the first edition of the VoiceMOS Challenge, a shared task for Mean Opinion Score prediction of synthesized speech. 22 teams from academia and industry participated, and we had a special session about the challenge at Interspeech. The main task was MOS prediction for a large-scale, diverse dataset of synthesized speech from text-to-speech and voice conversion systems, and an out-of-domain track focused on MOS prediction for a separate listening test using only a small amount of labeled data.

This year’s challenge will emphasize real-world and challenging zero-shot out-of-domain MOS prediction with three tracks for three different voice evaluation scenarios. In collaboration with this year’s Blizzard Challenge and Singing Voice Conversion Challenge, we will have a French speech synthesis track and a singing voice conversion track. Samples from BC and SVCC teams will be provided while those challenges’ listening tests are still ongoing, and the task is to predict MOS ahead of the true ratings becoming known. We will also have a third noisy and enhanced speech track with a leaderboard on CodaLab. Unlike last year, no MOS-labelled audio data from the target domains will be provided. This is to reflect a real-world MOS prediction scenario. Participants are encouraged to develop MOS predictors that are flexible and generalizeable to a wide variety of audio evaluation tasks. Like last year, the primary evaluation metrics for MOS prediction will focus on correct ranking of synthesis systems in each track. Teams may participate in any subset of the three tracks.


If you are interested in participating in the challenge, please fill out this Google form. Same as last year’s challenge, there is no fee to participate. Please note that this year, we require an institutional email address (e.g., university or company) for team registration.

Tentative schedule

The tentative schedule for the VoiceMOS challenge is as follows:


Suggested public datasets

Open-source MOS predictors:

These were the baseline systems in last year’s challenge:

Publication opportunities

We are happy to announce that this year’s challenge has been accepted as a Challenge Special Session at ASRU 2023!