Paul J. Thibault,
Dipartimento di Scienze del Linguaggio,
Università degli Studi di Venezia

Meaning-making Across Semiotic Scales: Exploring the complementarity of semiotic resources for connecting and integrating diverse space-time scales

Action and meaning are emergent properties of the time-dependent cross-couplings of a number of different systems on diverse temporal scales. Meaning emerges in and through the interaction of a number of different semiotic modalities and the physical-material world. In the real-time of the unfolding activity, various semiotic modalities (1) selectively map salient features of the material world to their own activity, (2) at the same time that they map selected features of other semiotic modalities to their own activity and (3) they map selected features of participants’ perceptual-motor activities to their own activity. It is the dynamic cross-coupling in time of a number of such heterogeneous systems that produces the meaning-making event. These include the various semiotic resources that may be deployed, objects, events, and so on in the material world, and the perceptual and motor activities of the participants. Action emerges in and through the cross-coupling and interaction of a heterogeneity of semiotic-discursive and physical-material systems along the lines outlined above. Genre-specific meaning-making activities and their development occur because the cross-coupling dynamics of all these systems create, in time, an internalised attractor space. It is in this space that participants’ experiences prior to the here-and-now event, the previous stages of the same unfolding event, and here-and-now responses to perceptual stimuli at any given moment all act in synergy to produce genre-specific semiotic performances and their resulting object-texts.

Meaning-making is, then, a dynamic process which is determined by a number of perspectives, all of which interact together to produce the occasion-specific activity. First, there is the perspective of what it is possible to mean in terms of the intrinsic characteristics and dynamics of any given semiotic resource system. Secondly, there is the sequential unfolding of events in real-time, the logogenetic selecting and deployment of semiotic options and their cross-coupling with specific external events, as well as changes in cross-coupling strengths in moments of transition from one phase of the activity to another. Thirdly, there are the sensori-motor reactions of participants to what they see, hear, feel, touch, grasp, point to, and so on, in their spatio-temporal purview at any given moment.

A theory of human social meaning-making needs to account for both linguistic systems and their modes of deployment along with the ways in which other non-linguistic semiotic resource systems are co-deployed either in concert with language or with other, non-linguistic, resources. Meaning-making is inherently multimodal in precisely this sense. This paper explores the thesis that meaning-making is centrally concerned with the cross-coupling and matching of bodily dynamics with the world in the course of jointly enacted and time-bound discursive activity. This means that the participants in time-bound discursive activity must reduce the degrees of freedom of both the external world and their bodily dynamics so as to achieve a fit between the two. However, this fit is best seen as a flexible, dynamic and adaptive process whereby heterogeneous elements and resources self-organise in what I call the ecosocial space-time in which interaction occurs. I shall propose that the cross-coupling of multimodal semiotic resources is the very engine of discursive activity. On this basis, I shall consider the following issues:

1.       the need to re-think the expression stratum of semiosis in relation to bodily dynamics and their role in meaning-making;

2.       the modelling of the cross-coupling dynamics that link both the material world and the bodies of interactants to the semiotic resources deployed; intra- and inter-semiotic cross-couplings and the text-context relationship;

I shall also argue that the multimodal meaning-making resources are integrated, rather than autonomous, systems of meaning-making resources. This suggests that beyond a certain point it is inappropriate to treat the various modalities of meaning making – viz. linguistic, gestural, pictorial, and other resources – as constitutively separable. Thus far, the tendency has been to describe the specific characteristics of these as separate systems. However, we also need to ask how various semiotic modalities act on and affect each other not only instantially but also systemically. Only in this way can be begin to understand the ways in which non-linguistic meaning-making resources affect participation - conscious and unconscious - in the pedagogical process.

The discussion will be illustrated through the analysis of a number of videotaped sequences of discursive interaction.