Evaluating the Performance of Automated Speech Recognition Systems for Black Users

Abstract

Automated Speech Recognition (ASR) systems understand and convert human speech into written text and
even transform it into its own speech, as found in applications like Siri or Alexa. While these technologies have
the potential to benefit the greater society, studies have shown they may not provide that benefit equitably
across demographic groups. To better understand the extent to which ASR technologies support the diaspora
of potential Black users, this systematic literature review covers existing research studies, performance
evaluations, and technical reports related to ASR systems. Based on an analysis of articles discussing this
topic, it was revealed that while efforts exist that aim to support Black users of ASR, there remain significant
differences in the performance of ASR for Black and African American users. This could be because the
primary focus of efforts thus far has been on the underrepresentation of African American Vernacular English
(AAVE) in existing datasets and the need for exposure to diverse linguistic patterns during training. This review
highlights important gaps in existing efforts to support the development of ASR technologies for the Black
community, such as the lack of diverse data collection efforts.