Like a “Swiss knife which cuts in two directions”: On the development and use of SpeechIndexer as a documentation and teaching tool

Abstract

SpeechIndexer has been developed for language documentation and learning at the ETH in Zurich. Its original goal was to help access the archive recordings of endangered Austronesian languages of Taiwan. In later years, it was more and more applied to preparing teaching materials not only for Austronesian languages, but to help organize and retrieve authentic speech materials of other modern languages. Starting from the precise indexing and retrieval of single speakers, it has grown into a small but complex tool which can deal with multiple speakers and indexing of overlapping speech segments. There have been several challenges: (1) it was necessary to find an optimal coding of the speech participants in recordings and a retrieval mechanism of participants' speech peculiarities and (2) we have to deal with overlaps in the speech of participants. Overlaps are unnoticed by the system, since the pause-finding algorithm looks for silence breaks for suggesting phrase units. A speech extract where multiple speakers are talking simultaneously is segmented into a single speech segment if there are no obvious pauses. A mechanism that filters out the various speakers' voices that can be individually indexed is the long-term goal. A way of marking speaker overlaps is the realistic short- or medium-term goal. Our presentation introduces the solution we developed for the encoding scheme of speech participants and the respective retrieval of their speech characteristics. Furthermore, we show the marking system we have devised for overlaps. Further disambiguation research is needed to find out the respective types of overlaps in individual speech acts. An indexing possibility is to be provided still, and we are further working on this problem. Finally, we demonstrate on some Formosan Indigenous Languages materials, where SpeechIndexer is used as a tool to build textbooks, how the multiple speakers' linear indexing is one more powerful asset to choose this software for concordancing speech corpora and integrating them into advanced learners' textbooks and multimedia materials. In teaching dialogues by authentic recordings (dialogues, discussions, drama, theater) selectively speaker's voices can be silence and the learner can practice, playing that part. Documentation and language teaching are kept at a distance by academic and educational institutions. We still believe, however that they are but two sides of the same coin, they all deal with the same authentic language materials, therefore the SpeechIndexer software can help by simultaneously satisfying both needs.