堀内 靖雄

Obese and overweight individuals are at high risk for chronic diseases such as sleep apnea and diabetes. Therefore, it is necessary to track eating behavior to determine the causes of obesity; however, it is time- and labor-intensive to follow the lives of specific individuals and observe their eating behavior. Thus, a method to automatically monitor eating behavior should be considered. As one approach to monitoring methods, we propose a method for convenient recognition of food category for food intake sounds recorded by microphones (below the ear microphone, throat microphone and acoustic microphone), which is less burdensome to the body and better from the viewpoint of privacy protection. Furthermore, a comparison of MFB and large-scale pre-trained speech models (wav2vec2.0, wavLM, and HuBERT) showed the effectiveness of large-scale pre-trained speech models in the food recognition task.

To enhance speaker verification for short utterances, we have developed a Same Speaker Identification Deep Neural Network (SSI-DNN). This network identifies whether two utterances are uttered by the same speaker with greater accuracy by focusing on the same texts. In this paper, we extend the detection target of the SSI-DNN from monosyllabic utterances to word utterances to improve the speaker recognition performance. Experimental results showed that the SSI-DNN trained on word utterances achieved an EER of 0.1% to 2.8%. These results indicated that the SSI-DNN outperformed the x-vector-based speaker verification method, which is a representative speaker verification method.

Hands-free control of shower settings, such as temperature, is highly desirable, enhancing user convenience when both hands are occupied or eyes are closed. In this paper, we propose a speaker-dependent, template-based isolated word recognition system using pre-trained large speech models (LSMs) to realize voice-activated shower control with a single microphone. Specifically, we examine the performance of 3 LSMs (wav2vec2.0, HuBERT, WavLM) as well as conventional MFCC as features. Additionally, we investigate speech enhancement using a Convolutional Recurrent Neural Network (CRN) to improve robustness against shower noise. Our experiments for recognizing 30 words with SNRs ranging from -5 dB to 20 dB demonstrate that HuBERT achieves the highest recognition accuracy (77.8 to 95.6%). CRN, on the other hand, improved recognition accuracy only under -5 dB conditions, but its accuracy was only 80.8%.

We had shown experimental results on prosody of languages characterized by real-time dialogue such as speech, sign language, finger braille and so on. These results were discussed along with various research results both from inside and outside Japan. Based on the results, we examined a structure that enabled real-time dialogue with a light mental burden. Furthermore, we will propose a model which makes real-time dialogue possible by elucidating information structures of various languages characterized by real-time dialogue. The model to be proposed can explain various phenomena in real-time dialogue.

テキスト独立な話者認識において，音声変動やセッション間変動は話者認識の精度に大きな影響を与える．本論文では，PCA変換を用いることにより，音声データのセッション間変動を削減することを提案する．提案手法を用いることにより，MFCCを用いる従来手法に比べ，誤認識率を42.6%削減でき，MFB-PCAに基づく手法に比べ，誤認識率を27.2%削減できた．

Sign language is a visual language in which main articulators are hands, torso, head, and face. For simultaneous interpreters of Japanese sign language (JSL) and spoken Japanese, it is very important to recognize not only the hands movement but also prosody such like head, eye, posture and facial expression. This is because prosody has grammatical rules for representing the case and modification relations in JSL. The goal of this study is to introduce an examination called MPR (Measurement of Prosody Recognition) and to demonstrate that it can be an indicator for the other general skills of interpreters. For this purpose, we conducted two experiments: the first studies the relationship between the interpreter's experience and the performance score on MPR (Experiment-1), and the second investigates the specific skill that can be estimated by MPR (Experiment-2). The data in Experiment-1 came from four interpreters who had more than 1-year experience as interpreters, and more four interpreters who had less than 1-year experience. The mean accuracy of MPR in the more experienced group was higher than that in the less experienced group. The data in Experiment-2 came from three high MPR interpreters and three low MPR interpreters. Two hearing subjects and three deaf subjects evaluated their skill in terms of the speech or sign interpretation skill, the reliability of interpretation, the expeditiousness, and the subjective sense of accomplishment for the ordering pizza task. The two experiments indicated a possibility that MPR could be useful for estimating if the interpreter is sufficiently experienced to interpret from sign language to spoken Japanese, and if they can work on the interpretation expeditiously without making the deaf or the hearing clients anxious. Finally we end this paper with suggestions for conclusions and future work.