|
Abstract : |
Multimodal interaction enables the user to employ different modalities such as voice, gesture and typing for communicating with a computer. This paper presents an analysis of the integration of multiple communication modalities within an interactive system. To do so, a software engineering perspective is adopted. First, the notion of "multimodal system " is clarified. We aim at proving that two main features of a multimodal system are the concurrency of processing and the fusion of input/output data. On the basis of these two features, we then propose a design space and a method for classifying multimodal systems. In the last section, we present a software architecture model of multimodal systems which supports these two salient properties: concurrency of processing and data fusion. Two multimodal systems developed in our team, VoicePaint and NoteBook, are used to illustrate the discussion. section, we present a software architecture model that supports the most salient properties of such systems: concurrent processing and data fusion. MULTIMODAL SYSTEM: A DEFINITION In the general sense, a multimodal system supports communication with the user through different modalities such as voice, gesture, and typing [3]. Literally, "multi " refers to "more than one " and the term "modal " may cover the notion of "modality " as well as that of "mode". Modality refers to the type of communication channel used to convey or acquire information. It also covers the way an idea is expressed or perceived, or the manner an action is performed [4]. Mode refers to a state that determines the way information is interpreted to extract or convey meaning., |
