Interspeech 2007 Session ThD.O2: Language learning and assessment
Type
oral
Date
Thursday, August 30, 2007
Time
16:00 – 18:00
Room
Darwin
Chair
Maxine Eskenazi (Carnegie Mellon University)
ThD.O2‑1
16:00
A Text-free Approach to Assessing Nonnative Intonation
Joseph Tepperman, Signal Analysis and Interpretation Laboratory, USC
Abe Kazemzadeh, Signal Analysis and Interpretation Laboratory, USC
Shrikanth Narayanan, Signal Analysis and Interpretation Laboratory, USC
To compensate for the variability in native English intonation and the unpredictability of nonnative speech, we propose a new method of assessing nonnative intonation without any prior knowledge of the target text or phonetics. After recognition of tone events with HMMs and a bigram model of intonation, we define an utterance’s automatic intonation score as the mean of the posterior probabilities for all recognized tone segments. On the ISLE corpus of learners’ English, we find intonation scores generated by this technique have a 0.331 correlation with general pronunciation scores determined by native listeners. In comparison, the SRI Eduspeak system’s proposal for pronunciation scoring based on suprasegmental features derived from prior knowledge of the target text yields a 0.247 correlation with listener scores on a similar corpus. Because it is text-free, our approach could be used to assess intonation outside of a strictly educational application.
ThD.O2‑2
16:20
Automatic Generation of Cloze Items for Prepositions
John Lee, MIT Computer Science and Artificial Intelligence Laboratory
Stephanie Seneff, MIT Computer Science and Artificial Intelligence Laboratory
Fill-in-the-blank questions, or cloze items, are commonly used in language learning applications. The benefits of personalized items, tailored to the user's interest and proficiency, have motivated research on automatic generation of cloze items. This paper is concerned with generating cloze items for prepositions, whose usage often poses problems for non-native speakers of English. The quality of a cloze item depends on the choice of distractors. We propose two methods, based on collocations and on non-native English corpora, to generate distractors for prepositions. Both methods are found to be more successful in attracting users than a baseline that relies only on word frequency, a common criterion in past research.
ThD.O2‑3
16:40
Evaluating and Optimizing Japanese Tutor System Featuring Dynamic Question Generation and Interactive Guidance
Christopher Waple, Kyoto University
Hongcui Wang, Kyoto University
Tatsuya Kawahara, Kyoto University
Yasushi Tsubota, Kyoto University
Masatake Dantsuji, Kyoto University
We are developing a new CALL system to aid students learning Japanese as a second language. This system is designed to allow students to create their own sentences based on visual prompts, receiving feedback based on their mistakes. The questions are dynamically generated, resulting in a large variety of challenges. The students may choose to receive guidance in order to complete each task, selecting the level of help that best suits their needs. A scoring system is also incorporated, which awards a grade to students based on the errors made and hints used. The trial of the system has been conducted with a number of students, providing the statistics of actual errors and hint usages. With these data, we have trained the weights of the scoring system by taking into account the impact of each issue on the proficiency of the students. The validity of the estimated score is generally confirmed by predicting the proficiency of the students.
ThD.O2‑4
17:00
ASR-based pronunciation training: Scoring accuracy and pedagogical effectiveness of a system for Dutch L2 learners
Catia Cucchiarini, CLST, Department of Linguistics, Radboud University, Nijmegen, The Netherlands
Ambra Neri, CLST, Department of Linguistics, Radboud University, Nijmegen, The Netherlands
Febe de Wet, SU-CLaST, Stellenbosch University, South-Africa
Helmer Strik, CLST, Department of Linguistics, Radboud University, Nijmegen, The Netherlands
A system for providing Computer Assisted Pronunciation Training for Dutch was developed, Dutch-CAPT, which appeared to be effective in improving pronunciation quality of L2 learners of Dutch. In this paper we describe the architecture of the system paying particular attention to the rationale behind this system, to the performance of the error detection algorithm and its relationship to the pedagogical effectiveness of the corrective feedback provided.
ThD.O2‑5
17:20
A Bayesian Network Classifier for Word-level Reading Assessment
Joseph Tepperman, Signal Analysis and Interpretation Laboratory, USC
Matthew Black, Signal Analysis and Interpretation Laboratory, USC
Patti Price, PPrice Speech and Language Technology
Sungbok Lee, Signal Analysis and Interpretation Laboratory, USC
Abe Kazemzadeh, Signal Analysis and Interpretation Laboratory, USC
Matteo Gerosa, Signal Analysis and Interpretation Laboratory, USC
Margaret Heritage, Center for Research on Evaluation, Standards, and Student Testing, UCLA
Abeer Alwan, Speech Processing and Auditory Perception Laboratory, UCLA
Shrikanth Narayanan, Signal Analysis and Interpretation Laboratory, USC
To automatically assess young children's reading skills as demonstrated by isolated words read aloud, we propose a novel structure for a Bayesian Network classifier. Our network models the generative story among speech recognition-based features, treating pronunciation variants and reading mistakes as distinct but not independent cues to a qualitative perception of reading ability. This Bayesian approach allows us to estimate the probabilistic dependencies among many highly-correlated features, and to calculate soft decision scores based on the posterior probabilities for each class. With all proposed features, the best version of our network outperforms the C4.5 decision tree classifier by 17% and a Naive Bayes classifier by 8%, in terms of correlation with speaker-level reading scores on the Tball data set. This best correlation of 0.92 approaches the expert inter-evaluator correlation, 0.95.
