University of Toronto,Victoria College, May 3 - 5, 2002 

Preliminary program

On May 3-5, 2002, a series of papers on topics relating to the multimodality of human communication will be discussed in a multidisciplinary conference to be held at Victoria College in the University of Toronto, Toronto (Canada). The tentative program of this series is found below with several position papers and the abstracts of the papers to be presented. This event is open to scholars and students who are interested in the issues to be discussed. This three-day conference is organized thanks to the support of the Social Sciences and Humanities Research Council of Canada, Victoria University, the Dean of Arts and Science and the Department of French of the University of Toronto, and which publishes this page and will be hosting the ensuing e-conference. The financial contribution of Lingnan University is also gratefully acknowledged. Other applications for support have been made and their support hopefully will be acknowledged in a further edition of this announcement. In September 2002, the papers which will have been discussed during the conference will be posted in their final form on a new redesigned website and a call for e-papers will be launched. It is expected that the contents of the initial position papers and their discussion will stimulate other specialists not only to respond and comment but also to develop criticisms and articulate alternative views on the problems which will have been tackled by the conference participants. Submissions will be peer-reviewed by an editorial committee which will be formed at the end of the conference and a selection of the submitted papers will be added to the site for a period of two years, thus allowing for meaningful interactions to take place over the website. The languages of this conference will be English and French.





Organizing Committee

Version en français

Rationale: During the last decade, a new field of research has emerged in the domain of human communication: the study of complex multimodal processes which form the basis of social interactions. A theoretical view which considered that the complexity of these interactions could be broken down into sets of parallel, self-contained modules, was progressively replaced by a new perspective that acknowledges the fundamentally integrated nature of these processes at their very source. Both through argumentation and empirical research, this approach generates models and theories which consider communicative interactions at the microlevel with a higher degree of resolution than was previously possible. This new knowledge points to a vast range of applications, notably in the management of interpersonal communication, education and human-computer interfacing.

There are several sources accounting for such a conceptual shift, among which some of the most important are: (i) advances in the study of gestures in relation to language (e.g., McNeill, D. (ed.) Gesture and Language, Cambridge University Press, 2000); (ii) new discoveries in the cognitive neurosciences (e.g., Cazzaniga, M.S. (ed.) The Cognitive Neurosciences, MIT Press, 1995; Rizzolatti,G., L.Fogassi & V.Gallese, "Neurophysiological mechanisms underlying the understanding and imitation of action". Nature Reviews: Neuroscience (September 2001,vol.2, no.9, 661-669); (iii) developments of multimodal interfaces between humans and computers that attempt to model real life interactions as well as creation of multimodal autonomous agents (e.g., Lee, J. (ed.) Intelligence and Multimodality in Multimedia Interfaces, AAAI Press, 1997). All these trends point to multimodality as a most promising focus of research. The knowledge produced by these programs is already having a profound impact on our understanding of communicative processes such as language acquisition, second language learning, intonation, symbolic interaction, alienation and acculturation, emotions, as well as on the creation of innovative softwares in education, medicine, business and entertainment, which must take stock of these new perspectives if they are to overcome the risk of knowledge fragmentation through the multiplication of disconnected expert systems.

The purpose of this conference is to create an interdisciplinary forum through which specialists in some of the most relevant domains will present position papers and interact with each other and with graduate students. As mentioned above, these face-to-face interactions will be expanded into a virtual symposium online through the posting of peer-reviewed articles.

Several considerations have prompted the organisation of this event. Two or more disciplines necessarily intersect when scientific attention focuses on actual, real life problems rather than artificially isolated questions raised in the context of a particular school, theory or paradigm. The latter is essential for the advancement of specialised knowledge but often lacks the comprehensiveness wich is required for truly addressing the complexity of social and cultural behaviours as well as technological developments. This is why periodic opportunities of interfacing with each other in a significant manner constitute for specialists from different disciplines a precious, indeed irreplaceable commodity.

This conference project emerged from the consideration of a set of practical problems: How to efficiently teach a second language? How to conceptualize and manage body movements and emotions in the process of communication, acculturation or cross-cultural interaction? How to produce human-friendly autonomous artificial agents? These three orders of questions have in common that they cannot be answered by relying on a single modality and its corresponding discipline since all these problems involve the multimodality of human communication. Their understanding requires broader theoretical perspectives rooted in contemporary scientific knowledge. It can be expected that this conference will achieve a modicum of epistemological synergy with a potential for long-term research development.

Venue: Victoria University, 73 Queen’s Park Crescent East, Toronto, Ontario, Canada  M5S 1K7

Emmanuel College (EM 119) and Northrop Frye Hall (NFH 205)

Registration: Admission free for graduate students. Others: $75.00 (including coffee, program, abstracts and printed position papers). There will be no advance registration. The registration desk will open May 3 at 1:00 pm, Emmanuel College room 119 (first floor). Cash or cheques only (Cheques should be made payable to Victoria University).

Organizing Committee:

  • Paul Bouissac (Professor Emeritus, French)
  • John Kennedy (Professor, Psychology)
  • Tsuyoshi Kida (Linguistics, Université de Provence)
  • Philippe Martin (Professor, French)
  • Yves Roberge (Professor, French)
  • Anne Urbancic (Senior Lecturer, Italian, Semiotics

Papers can be submitted in different formats: RTF, PDF, TXT, DOC or Html to




-The Multimodal Brain : How the Senses Combine in the Brain 
 Marcel Kinsbourne

-Optic and Haptic: the Picture
John Kennedy  

-Emotions and the Non-verbal Structuring of Relationships
 Keith Oatley

-Spoken and Nonverbal language in Early Modern Italy: Defining a Standard
Dario Brancato

 -Representing and Interpreting Intonation: A Diachronic View
Ivan Chow

-The Multimodal 'Language' of Cirque du Soleil: Crossing Borders
Jamie Skidmore 

-Speech-gesture: Grounded Imagery in acoustic Parameters
 Arika Okrent 

- Left- and Right-Brain Hemisphere Contributions                          to Speech-Gesture Production
Susan Duncan

- Meaning-making Across Semiotic Scales
Paul Thibault

- Cross-Cultural Understanding: The American Ghost
Monica Rector  

- The Multimodal Melody of the Text: A Model of the Text Generator
Michael W. Mair

- Multimodal Discourse: Gesture, Speech and Gaze
 Francis Quek

Philippe Martin
Intention, Inattention and Intonation 

- Outline, Tactile pictures, and Shape-from-Shadow Pictures
Juan Bai




- To be able to read all PDF documents you need to download a free program called Acrobat Reader 5.