Effects of Foreign Language and Audiovisual Degradations on Degrees of Visual Influence in Speech Perception Among Native Speakers of Thai
Article Sidebar
Main Article Content
Abstract
This study examined 1) the effects of foreign language and audiovisual degradation on visual influence in speech perception, and examined the effect of degradation on 2) perception accuracy, and 3) reaction time in speech perception. Participants were 22 native speakers of Thai who completed a speech identification test with 5 options, comparing their perception of speech in their native language and a foreign language (Mandarin Chinese). Perception accuracy from audio-only trials, congruent audiovisual trials, and incongruent audiovisual trials was used to calculate degrees of visual influence. Reaction time to the trials was also measured. Analyses of experimental data were conducted to examine the effect of audiovisual degradations and foreign language. Statistical analyses of the data revealed that 1) participants presented significantly lower degrees of visual influence when listening to speech in Thai compared to Mandarin Chinese. 2) White noise significantly worsened perception across all modalities in Mandarin Chinese language blocks, but did so only in the incongruent audiovisual modality for native language (p < .001). In terms of visual degradation, blurry visuals caused a non-significant reduction in perception accuracy and are associated with significantly shorter reaction time (p = .021) only in incongruent audiovisual trials in the Mandarin Chinese language blocks. This study provides proofs that a foreign language plays a role in increasing the importance of visual information and the degradation analyses suggests that perception of a foreign language is more effective without noise and visual blur.
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
References
Best, C. (1995). A direct realist view of cross-language speech perception. In W. Strange (Ed.), Speech perception and linguistic experience: Issues in cross-language research (pp. 171–204). York Press.
Bidelman, G. M., Bush, L. C., & Boudreaux, A. M. (2020). Effects of noise on the behavioral and neural categorization of speech. Frontiers in Neuroscience, 14, 153. https://doi.org/10.3389/fnins.2020.00153
Broersma, M., & Scharenborg, O. (2010). Native and non-native listeners’ perception of English consonants in different types of noise. Speech Communication, 52(11), 980-995. https://doi.org/10.1016/j.specom.2010.08.010
Bsharat-Maalouf, D., & Karawani, H. (2022). Bilinguals’ speech perception in noise: Perceptual and neural associations. PLOS ONE, 17(2), e0264282. https://doi.org/10.1371/journal.pone.0264282
Burnham, D., & Dodd, B. (2018). Language-general auditory-visual speech perception: Thai-English and Japanese-English McGurk effects. Multisensory Research, 31(1–2), 79–110. https://doi.org/10.1163/22134808-00002590
Chen, Y., & Hazan, V. (2009). Developmental factors and the non-native speaker effect in auditory-visual speech perception. The Journal of the Acoustical Society of America, 126, 858-865. https://doi.org/10.1121/1.3158823
Davis, C., & Kim, J. (2006). Audio-visual speech perception off the top of the head. Cognition, 100(3), B21-31. https://doi.org/10.1016/j.cognition.2005.09.002
Hazan, V., Kim, J., & Chen, Y. (2010). Audiovisual perception in adverse conditions: Language, speaker and listener effects. Speech Communication, 52(11), 996-1009. https://doi.org/10.1016/j.specom.2010.05.003
Hazan, V., & Li, E. (2008, September). The effect of auditory and visual degradation on audiovisual perception of native and non-native speakers [Paper presentation]. INTERSPEECH 2008, Brisbane, Australia.
Lalonde, K., & Werner, L. A. (2019). Infants and Adults Use Visual Cues to Improve Detection and Discrimination of Speech in Noise. J Speech Lang Hear Res, 62(10), 3860-3875. https://doi.org/10.1044/2019_jslhr-h-19-0106
McGurk, H., & Macdonald, J. (1976). Hearing lips and seeing voices. Nature, 264(5588), 746-748. https://doi.org/10.1038/264746a0
Peirce, J., Gray, J. R., Simpson, S., MacAskill, M., Höchenberger, R., Sogo, H., Kastman, E., & Lindeløv, J. K. (2019). PsychoPy2: Experiments in behavior made easy. Behavior Research Methods, 51(1), 195-203. https://doi.org/10.3758/s13428-018-01193-y
Schwarz, J., Li, K. K., Sim, J. H., Zhang, Y., Buchanan-Worster, E., Post, B., Gibson, J. L., & McDougall, K. (2022). Semantic cues modulate children’s and adults’ processing of audio-visual face mask speech. Frontiers in Psychology, 13, 879156. https://doi.org/10.3389/fpsyg.2022.879156
Sekiyama, K. (1997). Cultural and linguistic factors in audiovisual speech processing: The McGurk effect in Chinese subjects. Perception & Psychophysics, 59(1), 73-80. https://doi.org/10.3758/BF03206849
Sekiyama, K., & Burnham, D. (2008). Impact of language on development of auditory-visual speech perception. Dev Sci, 11(2), 306-320. https://doi.org/10.1111/j.1467-7687.2008.00677.x
Sekiyama, K., & Tohkura, Y. (1991). McGurk effect in non-English listeners: few visual effects for Japanese subjects hearing Japanese syllables of high auditory intelligibility. J Acoust Soc Am, 90(4 Pt 1), 1797-1805. https://doi.org/10.1121/1.401660
Sekiyama, K., & Tohkura, Y. i. (1993). Inter-language differences in the influence of visual cues in speech perception. Journal of Phonetics, 21(4), 427-444. https://doi.org/10.1016/S0095-4470(19)30229-3
