Facebook researchers have developed an AI model that can make a better distinction between five voices speaking on one microphone at the same time than any existing system. The new approach could boost audio technologies like recording, conferencing, hearing aids, and voice assistants, in noisy spaces.

The researchers used a novel recurrent neural network to develop their model, an algorithm class that uses a memory-like internal state to process input sequences of variable lengths (e.g., audio). Their model leverages an encoder network that maps a latent representation of the raw audio waveforms. The researchers trained various models to separate two, three, four, and five speakers, feeding the input mixture into the model designed to handle up to five speakers so that the number of audio channels present could be identified.

They then performed the same procedure with a model trained for the number of active speakers and tested to see if any output channels were active, either stopping when all the channels were active or when the model with the lowest target speaker count has been found.