Developing and Testing Audio Data Processing Modules in Python to Connect to and Data Be Scored by ASS Cloud Server

Xiaoqin Shi

doi:10.54691/fhss.v3i9.5627

Authors

Xiaoqin Shi

DOI:

https://doi.org/10.54691/fhss.v3i9.5627

Keywords:

ASS; Python; Cloud Server; Audio Data Processing; JSON.

Abstract

Automatic Speech Scoring (ASS) system developed on a basis of automatic speech recognition (ASR) technology is a powerful computer-assistant tool for oral test scoring. However, due to the limits of high equipment costs and high-tech operating costs of a local ASS, ASS cloud services have become the first choice of most oral English teachers and learners. The purpose of this paper is to develop and test modules in Python to preprocess the audio data, connect to the cloud server, and convert JSON data format into common Excel form. 1056 pieces of audio data were collected from test-takers’ read-aloud task of CEST-4 (College English Speaking Test band 4)) and six variables (i.e., “pronunciation”, “fluency”, “integrity”, “speed”, “duration”, and “overall”) were defined. After analyzing the data of the test results, it is found that the oral test score is mostly affected by the “pronunciation” and “integrity”, and the accuracy of pronunciation is the strongest predictor of oral performance. The modules and functions are helpful for teachers and students to use in daily oral test/practice, and these modules can also be employed in other second language oral test scored by ASS cloud sever, like oral Chinese test. Our results can provide reference and guidance for future oral research and teaching.

Downloads

Download data is not yet available.

References

Cucchiarini, C, Strik, H, and Boves, L (2000b). Quantitative assessment of second language learners’ fluency by means of automatic speech recognition technology. Journal of the Acoustical Society of America, 107(2) 989–999. DOI:10.1121/1.428279.

Derwing, TM, Munro, MJ, and Carbonaro, M (2000). Does popular speech recognition software work with ESL speech? TESOL Quarterly, 34(3) 592–603. DOI:10.2307/3587748.

Bernstein, J, Van Moere, A and Cheng, J (2010). Validating Automated Speaking Tests. Language Testing, 27(3) 355–377. DOI:10.1177/0265532210364404.

Evanini, K, Heilman, M, Wang, X and Blanchard, D (2015). Automated scoring for the TOEFL Junior comprehensive writing and speaking test. ETS Research Report Series, (1) 1–11. DOI:10.1002/ ets2.12052.

Hayashi,Y, Kondo, K and Ishii, Y (2023) Automated speech scoring of dialogue response by Japanese learners of English as a foreign language, Innovation in Language Learning and Teaching. DOI: 10.1080/17501229.2023.2217181.

Hirai, A, Kondo, Y .and Fujita, R. (2021). “Development of an Automated Speech Scoring System: A Comparison with Human Raters.” Language Education & Technology, 58 17–41. DOI:10.24539/let. 58.0_17.

Zechner, K and Loukina, A. (2020). “Automated Scoring of Extended Spontaneous Speech.” Handbook of Automated Scoring: Theory into Practice. Yan, D, Rupp, AA and Foltz, PW. London, Chapman and Hall/CRC. [Crossref].

Bamdev, P, Grover, MS, Singla, YK, and Vafaee, P, et al. (2023). Automated Speech Scoring System Under The Lens. International Journal of Artificial Intelligence in Education, 33 119–154. DOI:10. 1007/s40593-022-00291-5.

Li, X, Li XM, Chen SH, Ma SH and Xie, FF (2022) Neural-based automatic scoring model for Chinese-English interpretation with a multi-indicator assessment. Connection Science, 34(1) 1638-1653. DOI: 10.1080/09540091.2022.2078279.

Ockey, GJ, Gu, L and Keehner, M (2017). Web-based virtual environments for facilitating assessment of L2 oral communication ability. Language Assessment Quarterly, 14(4), 346–359. DOI:10.1080/ 15434303.2017.1400036.

Downey, R, Farhady, H, Present-Thomas, R, and Suzuki, M, et al. (2008). Evaluation of the usefulness of the Versant for English test: A response. Language Assessment Quarterly, 5(2) 160–167. DOI:10.1080/15434300801934744.

Chen, L, Zechner, K, Yoon, S, and Evanini, Y K, et al. (2018). Automated scoring of nonnative speech using the speechrater sm v. 5.0 engine. ETS Research Report Series, (1): 1–31. DOI:10.1002/ ets2. 12198.

Loukina, A, Zechner, K, Chen, L, and Heilman, M (2015). Feature selection for automated speech scoring. 10th Workshop on Innovative Use of NLP for Building Educational Applications. Stroudsburg, PA, Association for Computational Linguistics. [Crossref].

Yoon, S and Zechner, K (2017). Combining human and automated scores for the improved assessment of non-native speech. Speech Communication, 93: 43-52. DOI: 10.1016/j.specom.2017. 08.001.

Bernstein, J, Cohen, M, Murveit, H, Rtischev, D, et al. (1990) Automatic evaluation and training in English pronunciation. 1990 International Conference on Spoken Language Processing, Kobe, Japan, ProcICSLP-90. [Crossref].

Bernstein, J, Van Moere, A. and Cheng, J (2010). Validating Automated Speaking Tests. Language Testing, 27 (3) 355–377. DOI: 10.1177/0265532210364404.

Eskenazi, M (2009). An overview of spoken language technology for education. Speech Communication, 51(10) 834-844. DOI: 10.1016/j.specom.2009.04.005.

Evanini, K., Cogan, H. and Hakuta, K. (2017). Approaches to Automated Scoring of Speaking for k-12 English language proficiency Assessments. ETS Research Report Series, (1):1-11. DOI: 10.1002/ets 2.12147.

LeCun, Y, Bengio, Y and Hinton, G. (2015) Deep learning. Nature, 521 (7553) 436–444. DOI:10.10 38/nature14539.

Li, H (2018) Deep learning for natural language processing: advantages and challenges, National Science Review, 5 (1) 24–26. DOI: 10.1093/nsr/nwx110.

Zhao, W, Liu, X, Jing J and Xi, R (2022). Re-LSTM: A long short-term memory network text similarity algorithm based on weighted word embedding. Connection Science 34 (1) 2652-2670. DOI:10.10 80/09540091.2022.2140122.

Witt, SM and Yong S.J (2000) Phone-level pronunciation scoring and assessment for interactive language learning. Speech Communication, 30(2–3) 95-108. DOI:10.1016/S0167-6393(99)00044-8.

Zechner, K, Higgins, D, Xi, X and Williamson, DM. (2009). Automatic Scoring of non-Native Spontaneous Speech in Tests of Spoken English. Speech Communication, 51 (10): 883–895. DOI:10.1016/j.specom.

Pearson Education (2011). Versant TM English Test: Test Description & Validation Summary; Palo Alto; CA; Pearson Knowledge Technologies. DOI:10.1016/S0167-6393(99)00044-8.

Xi, X, Higgins, D, Zechner, K and Williamson, D (2012). A comparison of two scoring methods for an automated speech scoring system. Language Testing, 29(3) 371–394. DOI: 10.1177/02655322114 25673.

Jiang, JL and Chen D (2021). Study on ASS of Subjective Questions: Review, Reflection and Prospect J. Chinese Foreign Language, (06) 58-64. [Crossref].

Lou, K and Han, B (2014) Review and enlightenment of Ordinate and SpeechRater ASS system. Technology Enhanced Foreign Language Education, (04) 27-32. [Crossref].

Bernstein, J, Cheng, J (2007). Logic and validation of a fully automatic spoken English test. The Path of Speech Technologies in Computer Assisted Language Learning: From Research Toward Practice. Holland, V.M. and Fisher FP, Florence, Routledge. [Crossref].

Sun, H (2021). A review of auto-scoring of spoken English at home and abroad. Foreign Language Education in China, (02): 28-36 + 89-90. [Crossref].

Li, M, Yang, X, Feng, G and Wu, M, et al. (2008). Feasibility study and practice of large-scale college English oral test reading machine. Foreign Language world, (04) 88-95. [Crossref].

Gong, L, Liang, W, and Ding Y (2009). Feasibility analysis and Practice research of using machine marking for large-scale Oral English test and reading questions. Technology Enhanced Foreign Language Education, (2) 10-15. [Crossref].

Lv, M (2015), The exploration and practice of Intelligent assessment technology in large-scale oral English test marking. Chinese Examination, (10) 51-57. DOI:10.19360/j.cnki.11-3303/g4.2015.10. 009.[Crossref].