Information, Imitation, Communication: An Evolutionary

                                    Perspective on The Semiotics of Gestures 


                                      Paul Bouissac (University of Toronto)


            In spite of [...] difficulties of conscious analysis, we respond to gestures with extreme                         alertness, and, one might almost say, in accordance with an elaborate code that is                written nowhere, known by none, and understood by all”.

                                                Edward Sapir (1927) [quoted by Jacques Cosnier, 1977:2046]


1.Introduction: Gestures in folk-semiotics

            The semiotics of gestures has undoubtedly been one of the most productive domains of semiotic  research during the 20th century. It has also been one of the most popular, since examples taken from the gestural repertory have most often served to illustrate the meaning of semiotics. An easy way to convincingly introduce semiotics to a naive audience is indeed to present, for instance, a summary of Ekman and Friesen’s (1969) categorisation of hand movements. After a few demonstrations, the students think they know what semiotics is about and they discover that without knowing it they have been doing semiotics for a long time. With the addition of some examples from other cultures, comparative semiotics soon appears to be one of the simplest and most useful things to learn.

            This is not surprising if one realizes that theorizing about signs is historically rooted to a large extent in the consideration of gestures, understood as conventional movements of the arms, hands and head that carry distinctive social meanings. Indeed, gestures seem to have early captivated human imagination to the point that the emergence of the concept of sign may be traced back to a reflexive perception of these peculiar motions of the upper limbs which probably gained preeminence in human visuo-motor interactions as an outcome of bipedalism. Still today, “gesture” and “sign” are commonly used as synonyms, as, for instance, the term “sign language” shows. Over the centuries, the stylisation of expressive movements displayed by orators and actors and the efforts made towards normative descriptions of these movements for the purpose of training students in these communicative skills contributed to the development of technical languages referring to gestures. In addition, differences between the repertories of gestures observed by travellers in various countries brought into focus a vivid notion of cultural relativism whose articulation  remains a central part of the modern semiotics of gestures. As a result, the metalanguage itself of semiotics reflects these gestural semantic prototypes. Words like “index” and “symbol” have clear gestural origins. Consequently, applying semiotic models to the  analysis of gestures creates a strange epistemological situation in as much as the semiotic metalanguage is largely redundant with respect to its object which is also the source of its metaphoric origins. Hence, perhaps, the capacity of this metalanguage to refine with exquisite details the representation of gestures in analytical texts which have literary qualities (e.g., Birdwhistell 1970:50-54; Poyatos 1986). However, this metalanguage, which definitely enhances the precision of perception, seemingly failed to produce any scientifically interesting information. The semiotics of gestures yields mostly trivial knowledge, and, in spite of having been for a long time the focus of intense attention, remains in the form of a perpetual agenda. Adam Kendon’s state of the art report, “An Agenda for Gesture Studies” (1998), pointedly emphasizes the relative inconclusiveness of the research in this domain.

            The explicit models that have been put to use in such research probably bear a large part of the responsibility for this failure.  The various models that permeated and transformed traditional semiotic discourse in the 20th century came from fields alien to gestural behavior: the long-distance transmission of information, the functional structures of phonological systems, and philosophical taxonomies based on various systematizations of commonsensical phenomenology. These models produced respectively the notions of code, of distinctive feature and of referential categorisation. Extracted from their original epistemological context, these concepts lost their specific operationality and produced all-purpose attractive metaphors.  The generalisation of these models may not be adequate, but in their extrapolated forms they are sufficiently vague to loosely fit gestural data among others. Perceived through these models, many domains of human and animal activities became  semioticized to the point of giving rise to what could be called a folk-semiotics, similar to folk-psychology, in which gestures provide neat examples for all the virtual categories peddled by the proponents of the semiotic doctrine of universal communication (e.g., Danesi & Perron, 1999).


2. Evolving limbs and gestures.

              Perhaps the time has come to step out of the grid of these models and to attempt to ground gestural research on sounder foundations. Not that all the descriptive discourse produced during the last hundred years or so has been useless. It must be recognized that as long as the human brain was a mere black box in the diagrams that were designed to map the flow of human kinetic information there was little choice but to scrutinize the gestural output in the form of methodic representations. The models that were used have certainly been productive in this respect. Isolated gestures and sequences of gestures  have been represented through descriptions and photographs or schematic drawings for the purpose of categorical or functional identification. These are research artefacts which segment and freeze complex biological and social processes at a macro-level of phenomenological observations mostly operating through linguistic and narrative filters (Bouissac 1973). It is symptomatic that once film and video recording became available gestural research was concerned with the selection of  the most relevant “slices”, looking for significant “peaks” in otherwise continuous trajectories, for instance. These “peaks” and “patterns of peaks” were labelled by borrowing expressions from scientific or vernacular languages, in a manner that is not very different from the gestural nomenclatures that are found in various brands of Indian Yoga and Chinese Taoist body exercises like those performed in Tai Chi and Qi Gong. Rather than providing information on the dynamic interactions of agents through body and limb movements, these repertories document the representations of some aspects of body movements in discourse. These repertories are usually focused on individual bodies but are assessed from the point of view of an observer placed at a constant distance from the body observed.  The scientific interest of these collections is not so much the positivistic entreprise of collecting gesture-data in order to illustrate the diversity or similarity of patterned movements among cultures, but rather the possibility of raising tantalizing questions relating to these representations themselves. How is the observational distance determined? How are the representations framed? What is left out of the frame? How are the disembodiment and desocialisation of gestures achieved in these graphic representations? How do the epistemological assumptions of the investigators impact upon the production of the data itself? Curiously, it seems to be taken for granted that these questions are not particularly relevant when gestures are concerned. They should be, on the contrary, of utmost importance in as much as all direct investigations create a gestural  interaction of their own, and even various versions of hidden or “candid” camera imply focusing, framing and editing. The social construction of the investigative context includes an abundance of gestural interactions that can only arbitrarily be excluded from the picture.

            To put all this in perspective, it might be useful to go back in evolutionary time. Primate bipedalism is very recent, but limbs have a long, well documented history. Many organisms can live and reproduce without limbs and appendages. Clams have done very well over millions and millions years with two muscles that control the opening and closing of their shell. Sea cucumbers also manage quite well. However, several taxa have evolved limbs in various numbers and along various axes under a variety of selection pressures. Spiders, centipedes, crabs and octopuses, to name only a few, show different versions of  adaptation to challenging environments. Efficient locomotion pays off when food is sparsely distributed over a wide area and when mating requires locating the appropriate conspecifics at some distance. Adapted mobility is  no less vital  when fleeing  dangerous rivals or predators in environments that may present obstacles such as rough and irregular mineral surfaces, grassy and bushy lands, or arboreal entanglements. These environments selected effective limb designs that vary depending on how they are situated with respect to the body plan, how they are articulated, how their extremities interface with the terrains through which they move, and how their motricity is controlled by the brain interfaces with perceptual inputs. These limbs are often multifunctional and serve as signaling organs for threatening, bonding, or courting. At the same time, these often non-retractible extensions of the body mass are a liability as more body surface is exposed to wear and tear as well as to predatory capture. They can even be construed as “handicaps” when the signaling functions take precedence over other vital functions (Zahavi and Zahavi 1997). The trade-off between mobility and risk sets a new range of opportunities for natural selection to favor multifunctional limbs with features that are assets not only for catching preys and mates and for fending off aggressors, but also for technically modifying the environment in ways that improve the rate of survival by digging, weaving, stone walling, etc.  Hooves, claws, nails, articulated digits, suckers and “velcro” are examples of such multifunctional devices that hit, grasp, grip, catch and throw, and play a part in reproductive, offensive or defensive actions. The standard explanation for the signaling role of limbs is “exaptation” or chance pre-adaptation, that is, the fact that an organ can take up a function for which it was not selected but for which it happened to be adaptive with respect to a different set of environmental pressures. This theoretical view assumes that signaling evolved as a by-product of some organs which had evolved under other pressures than communicative ones.

            The tinkering of evolution is opportunistic but not necessarily economical and optimal in the long term. It often leads to dead-ends. Even the most obstinate Dr. Panglosses’ arguments are strained when trying to account for the high rate of dysfunctions and extinctions. If the study of gestures is implicitly based on the belief in a human “faculty” or “competence” to communicate visually through gestures and if the purpose of the inquiry is to uncover the system, let it be structuralist or generative, that makes such acts of communication possible, methods inspired by various kinds of linguistics will be considered appropriate and the phenomenological intuition of the “native gesturer” will be sufficient to sort out the details of the investigation. Gestures can then be construed as embodiments of information between intending and  understanding minds.  But if human limbs and gestures are understood in the biological continuum of the evolutionary process and in the context of primate ethology complicated by successive layers of cultural and possibly cross-specific imitations, they cannot be reduced to mere variables in a communicative exchange. Cosnier (1977), Fridlund (1994), Cosnier and Vaysse (1997), among others, have called the attention of researchers to more comprehensive and more credible models than the communication arc. Recent advances in the understanding of the brain substrates that are necessary parts of gestures make it possible to attempt the construction of  a more complete semiotic representation of interactive movements at a micro-level. The assessment of the evolutionary significance of these behaviors and an understanding of the origin of their cultural selection should help develop truly explanatory theories of gestures.

            The purpose of this paper is not to develop a full-fledged evolutionary theory of gestures -- something that will eventually require the convergence of several disciplines -- but to sample relevant information among the various domains of research that bear upon the evolution of limbs, the vital negotiations of space, the signaling potential of articulate movements and their neurological basis, and the form and functions of social dynamic behavior with a view to initiating a double epistemological movement: building, on the one  hand, a “bottom-up” process that consists of collecting data and attempting to integrate them in progressively more comprehensive models; elaborating, on the other hand, “top-down” conceptual constructions inspired by theoretical views on the evolution of social interaction and communication. The eventual congruence of both levels can be held as a motivating ideal for the semiotics of gestures. With this ultimate goal in mind, the next section of this paper will focus on recent discoveries on the micro-processes of space representation in relation to body and limb movements. It will be followed by a section devoted to a review of new developments in the understanding of imitation, an aspect of gestures that has so far been neglected in semiotic research in spite of its obvious importance. It is indeed essential that systematic efforts be made to bring interesting data from the sciences to the theoretical awareness of  semioticians.   





 3. The space of gestures: sensory maps, mirror neurons and motor schemata

            It is by pure convention that we say the body is in space and moves in space. The body is space and its movements are spatial as well as energetic transformations. The geometrical grids or matrices that have been constructed for the purpose of gesture representation and analysis are based on a sort of Euclidian fallacy, a geometrisation or weightless simulation of motricity which reduces movements to Cartesian coordinates and determines gestures with respect to the (successive) position(s) of articulate parts of the body, mostly the upper limbs, neck and torso. It is as if the three-dimensional body (in the abstract, geometric sense) were floating in a sort of immaterial ether as unobtrusive as the white sheet of paper on which  schematic figures are drawn. This epistemological position deserves the name of “theory” in the etymological sense, that is “visualisation” at a sufficiently remote distance for allowing an encompassing point of view while maintaining enough clarity to distinguish each relevant part. But if we recognize that the body is space, it is important to specify that the abstraction, or virtual reality, we call “space” is not represented in the brain as a unitary map.

            This virtual space that has been constructed by “theoretical” Euclidian geometry is homogeneous and conceptually sustained by the assumption that the brain processes spatial information with respect to a general all-purpose map. Actually, the brain holds numerous spatial maps (Rizzolatti et al. 1994) in various areas and with various unrelated functions. Sensory maps suggest that the body’s boundaries are formed by a multiplicity of interfaces within a heterogeneous medium defined by cutaneous, olfactory, auditory, gustatory and visual cues. For any organism the accurate monitoring of these combined moving boundaries is a prerequisite for survival: avoiding destructive contacts (collisions, absorbtions, displacements) and achieving protective and reproductive conjunctions (social bonding among conspecifics or interspecific mutualistic associations, copulation or external release and fertilisation of eggs). Predation, aggression, and reproduction require a precise mutual monitoring of movements and more particularly the discrimination between the physical movements of non-biological objects and biological movements that are interpreted as potential strategic moves and always demand counter-moves. The visual interface (for those organisms that have evolved such a monitoring system from photosensitive cutaneous surfaces to three-dimensional ocular vision) provides a marked advantage as it considerably extends an organism’s boundaries, as do sonar, electrical and magnetic discriminatory sensitivities in some other species. Human vision for instance discriminates biological motions with great precision. A number of empirical studies have shown that even young infants can accurately discriminate such motions (e.g., Fox and McDaniel 1982), indicating that this sensitivity is functional very early in the human visual apparatus and is plausibly wired in. From the earlier cinematic experimentations of Marey (1894) to recent, more sophisticated testing by Neri, Morrone and Burr (1998), there is ample evidence that the brain can recognize biological motions with minimal cues. Although most experiments have involved human motion, it is probable that this sensitivity extends to the motion of other organisms, and that, conversely, other visual species have evolved similar cross-specific sensitivity, if only in the context of predation.

            From the vantage point of civil society which is essentially a strict, self-imposed mutual monitoring and enforcing of spacing and motility control, it is not easy to realize how much tension, violence and frustration are involved in the constant policing of space, movements and gestures. Only occasional break-ins, push-overs, brawls, aggressions and stampedes as well as offensive olfactive invasions, offer glimpses of unregulated spacing negotiations and breakdowns of boundaries. The civilizing process (Elias 1982) develops an abundance of “manners” which for the most part are space and motion controling devices in the multisensorial sense evoked above. The regulation of olfactory, auditory and cutaneous (haptic, tactile) interfaces has the same spatial significance as the control of motricity through socialization.

            Many of the human brain’s spatial maps are located in the cortical areas involved in the control of movements (e.g., eye, head, arms, hands). Interestingly, Rizzolatti et al. (1997) have shown that Area F4 of the ventral premotor cortex of the monkey brain has a large proportion of neurons which are bimodal, that is, which respond to both visual three-dimensional stimuli and tactile stimuli applied to the face or arm. But still more striking is the fact that the visual receptive fields of F4 neurons “are circumscribed to the space around the tactile [receptive fields] as if the cutaneous space extended into the visual space adjacent to it [... and that] the visual [receptive fields] of F4 neurons remain anchored to the body regardless of the position of the eyes and of the body parts on which the tactile [receptive field] is located” (Rizzolati et al. 1997: 190). Graziano and Gross (1995) conclude from their experiments with primates that  “extrapersonal space”, that is, the visual space near the body that extends outward from the skin about 20 cm, is represented in the brain by bimodal (visual-tactile) neurons in at least three interconnected  somatotopic  maps in which the tactile and visual receptive fields are adjacent, and that these maps are centered on the limbs rather than on the head or trunk. When the arm is moved, the visual receptive field moves with it. This visual receptive field is confined in depth to a region within reach of the animal’s arm. The neurons fire when movements are detected  in the extrapersonal space, but with particular intensity when these movements are oriented toward the body parts concerned. Lesions in the corresponding areas in humans cause deficits such as optic ataxia (misjudging the location of stimuli for the purpose of reaching toward them) (Newcombe and Ratcliff 1989). Graziano and Gross tentatively sum up their results by stating that “the visual space near the animal is represented as if it were a gelatinous medium surrounding the body that deforms whenever the head rotates or the limbs move. Such a map would give the location of the visual stimulus with respect to the body surface in somatotopic coordinates” (1995:1031). But since the same neurons often have both sensory and motor activity, these areas are probably best described as sensory-motor interfaces. Similarly to the visual receptive fields, the motor response fields for arm movements appear to be arm-centered.  To the distinction between egocentric and allocentric maps that is usually accepted (e.g., Paillard 1991), it is therefore necessary to add a third kind of spatial representations in the cluster of brain maps: “[...] A type of egocentric representation, a body part-centered one rather than a head-centered one” (Graziano and Gross 1995:1032) that lends some measure of autonomy to the limbs. Francis Ponge’s “La main” (which metaphorises hands as two small animals tied at the end of the arms) provides a literary expression of this state of affairs in a way that reminds clinical descriptions of the effects of some brain pathologies.  

            Another important aspect of these investigations is the apparent fusing of the motor and visual neuronal responses they reveal. If indeed the cutaneous space extends into the visual space adjacent to it, and if, as Rizzolatti et al. suggest, “F4 contains a store of motor schemata for bringing the head or the arm toward specific spatial location” within this extended cutaneous space (1997:190), the spatial map as expressed by receptive fields is dynamic rather than static. This is demonstrated by the experiments of Graziano, Hu and Gross (1997) in which the stationary presence of a three-dimensional object in the peripersonal space of a monkey causes the firing of motor neurons even after the stimulus has been withdrawn in a manner which makes  the monkey “believe” that it is only masked. A possible explanation is that the discharge of neurons reflects a potential action, a motor schema. The visual space mapped in this area is also a motor space.  In another functional area (F5) that relates to object-to-hand movements, objects appear to be described more in motor than in visual terms. Rizzolatti et al. tentatively conclude that movements “carve out a working space” from undifferentiated visual information through the construction of motor-sensory neuronal interfaces.

            The cluster of research programs that has been introduced in this section is only a very small sample of  the results of an increasing number of such investigations in domains relating to space representation and motricity at neuro-functional levels. What kind of theoretical and methodological consequences can be drawn from these perspectives for the semiotic study of gestures?

             First, the idea of extrapersonal space is more than a mere methodological concept. All gestures are deformations of this secondary cutaneous interface. It would seem that the primary functions of reaching out toward the periphery of extrapersonal space can only be an adaptation that compensates for the vulnerability of a comparatively soft cutaneous envelope.  If limb projections into peripersonal space are linked to nutrition and reproduction, the appropriation of this space is vital. The only manner in which space can be secured is through occupation, or saturation by appropriate signaling or marking, mainly oriented toward conspecifics. Gesturing might primarily have evolved under the necessity of spacing, of marking the peripersonal boundaries by constructing a radial motor space not only centered on the head-trunk axis but also on limbs. The bilateral symmetry and facial orientation of hominids would then require  compensatory peripheral behavior through rotation for which proper spacing is needed. All these pressures would favor at the neuronal level the integration of visual, tactile and motor responses within a cluster of mapping. A similar integration is required for the mediated securing of peripersonal space through artefacts that extend the limbs or that are projected on targets. Aiming presupposes a similar cross-modal integration. However, all social mammals, and in particular the primates must control intra-group aggression and evolve what Franz de Waal (1989) has labelled “peacemaking” behavioral devices, which are all gestural. Many primate (including human) interactive gestures that are extended across peripersonal space seem to achieve a balance between spacing and peacemaking. To construe these human gestures as purely arbitrary, conventional, cultural in an abstract communication framework  is perhaps to miss the point in as much as the interacting bodies are not in space, emitting visual patterns that travel through space, but they are interfacing spaces that construct each other through mutual spatial carving and motor mapping. If this perspective is correct, relevant factors for the description and understanding of gestures would include population density, distribution of resources, level of material culture, degree of civility enforcement, nature and severity of the tensions within groups and between groups, and the like, keeping in mind that collective memory as well as the force of inertia can carry over many generations, through vertical imitation, patterned interactive behavior that have lost their historical (evolutionary) adaptive value. The etiology of gestures can only emerge from a multidisciplinary approach that should encompass evolutionary biology, neuro-ethology, archeology, physical and cultural anthropology, and social history.

            Secondly, it should not be assumed as a starting point that gesturing is communicative in the sense suggested by functional models such as Buehler’s and Jakobson’s which still permeate semiotic research in paralinguistic and nonverbal communication.  The notion of gestures as complementary and supplementary communication in the context of verbal face-to-face interactions must be revised in view of accumulating evidence that the universality of this well described phenomenon (e.g., Feyereisen and de Lannoy 1991) is not learned but appears spontaneously in blind speakers.  Iverson and Goldin-Meadow (1998) for instance conclude that gestures in talk do not depend on either a model (imitation) or an observer (seeing listener) but appear to be integral to the speaking process itself. For some, gestures reflect the cognitive processes that underlie speaking (McNeill 1992, McNeill and Levy 1993); for others, gesturing facilitates this process (Krauss et al. 1991, 1995, 1996; Rauscher et al. 1996). These important findings do not preclude the possibility that the gestural output may be eventually controled or molded by cultural input (through imitation and schooling), but they definitely indicate that  other methods of approach than the ones provided by the  traditional semiotic models are required.  


4.  Imitation, interpretation and communication

            The theoretical and empirical study of gestures undertaken under the aegis of semiotics has been construed as a special case of communication. Gestures have been understood as signs and accordingly classified following a variety of encoding parameters. This communicative bias probably originates in the long-recognized existence of sign languages and in the early treatises of rhetoric that endeavored to normalize the gesturing of orators and preachers. Later, paralinguistics and kinesics have focused  attention on the part played by extralinguistic bodily phenomena in the delivery of speech in interactive situations. Eventually, a privatively defined category was created by communication theorists and semioticians under the name of “nonverbal communication”. This later domain lumped together a great variety of “patterned behaviors” and, with the possible exception of the chaotic agitation caused by the Tourette syndrome or by epilepsy, the idea that all body movements communicate in one way or another has imposed itself on researchers. As a result of this bias, probably reinforced by the ideology of a society that saw itself as living in “the age of communication”, two important domains relevant to gestures and body movements in general have been neglected: interpretation and imitation. Interpretation has been reduced to a complementary pole in the communication model as the function of the decoder. Imitation as a process has received very little attention in semiotics except in discussions of iconicity, in which mimesis appears as a lower form of semiosis exemplified by a minor performing genre (mime). In addition, the notion of imitation carries with it the semantic stigma of “not being genuine” or of “lacking originality” or even of not being fully human as the term “aping” suggests, a liability in Western academic and popular cultures that value individualism and reward creativity and innovations. Naturally, whether or not imitation is a fashionable concept is irrelevant to its actual function as a  biological and social process. The trend is now to use the term “copying” in ethological investigations of imitative behavior, e.g., Dugatkin (1996) who often adds “imitate” between parentheses when he uses the verb “copy”.                   Epistemological avenues other than systematic applications of communication models to whatever moves are possible, starting with an approach that would consider the very construction of communication models as an interpretative strategy. This later view is implied by Gazzaniga (1998) in the context of the fast developing evolutionary cognitive neurosciences. Perceptual and cognitive systems that were shaped by natural selection as surely as our size, shape and color, operate mostly without our being aware of, let alone remembering, these processes which enable our survival in the particular environment that sustains us. Narrative  models that appear to originate in the left brain (not only because they are narrated) provide “the string that ties events together and makes actions or moods appear to be directed, meaningful and purposeful” (133). Communication models have little predictive value. They always apply to the past. They purport to explain how it has worked. It is a well-known fact that successful communication consultants are great improvisers and risk takers, able to catch opportunities they had not planned and always ready to take credit for any bit of sheer luck that falls upon them, being never short of interpretative supply “to explain” their occasional successes.

            Interpretation is often perceived as the realm of hermeneutics that interfaces with the semiotics of communication. It appears as more open to speculative considerations and philosophical openness. However, evolutionary neuropsychology now begins focusing on the process of interpretation, its brain basis and its dysfunctions (e.g., Hauser and Carey 1998: 83-89; Gazzaniga 1998: 138-148). The attribution of intentionality is particularly scrutinized in the context of developmental psychology. How much “intention” is involved in imitation, how much interpretation is involved in the ascribing of intentionality to imitation, are typical problems that emerge from these new empirical inquiries.

            Imitation was the object of considerable scientific attention about a century ago. Tarde (1903), McDougall (1908), Guillaume (1926) bear witness to this interest. Davis (1973) and Wyrwicka (1996) point to these early preoccupations that have been revived in the context of new approaches to connectionist linguistics (e.g., Christiansen 1995), observational learning development (e.g., Wyrwicka 1996) and cultural evolution (e.g., Sperber 1996). The notion of imitation  receives a surprising content in the perspective of memetic epidemiology applied to human cultures (e.g., Blackmore 1999) as well as to animal communication, notably to birdsongs (Lynch 1996).

            Those involved in the semiotic study of gestures cannot remain indifferent to these epistemological developments. While some categories of gestures are the objects of precise and deliberate training, most of the gestural repertoires that can be observed simply “spread” both vertically and horizontally in populations, often in association with social rituals and artefacts. The conceptual border between communication and contagion is blurred. The transfer of knowledge (“knowing how” as well as “knowing that”) most often results from an imitation process that greatly exceeds and sometimes subverts the explicit communicative process. The “Symposium on Imitation in Animals and Artifacts”, that was organized by the Department of Informatics at the University of Edinburgh in April 1999, underlined  that imitation is one of the most important mechanisms whereby knowledge is transferred between agents, let them be biological, computational or robotic autonomous systems. Yet, the study of imitation has lacked a rigorous foundation and no major interdisciplinary publication is available to bridge the many disciplines in which imitation appears to be crucial.

            Serious advances could be achieved in the theoretical understanding of gestures if we could develop a knowledge of how imitation does occur and what is its evolutionary significance.

The first question is very complex. As Iacoboni et al. (1999) state: “Imitation has a central role in human development and learning of motor, communicative and social skills. However, the neural basis of imitation and its functional mechanisms are poorly understood. Data from patients with brain lesions suggest that frontal and parietal regions may be critical for human imitation but do not provide insights on the mechanisms underlying it”. The first step in this inquiry is to realize that imitation is probably not a unitary phenomenon, but covers a range of domain-specific mechanisms. It has been demonstrated that some facial and manual gestures are imitated by neonates from a few hours to a few days after they are born. Spontaneous imitation of vocal or other sounds is also observed in very young infants. Researchers have hypothesized a “resonance” mechanism that interfaces visual and motor neuro-processing. Gallese et al. (1996) and Rizzolatti et al. (1996) have shown that an area in the premotor cortex of macaque monkeys (F5) contains neurons which fire both when monkeys perform an action and when they see another individual perform the same action. Experiments by Iacoboni et al. (1999) have revealed several stages in motor imitation processes that involve neuronal activities in the right parietal lobe (where kinesthetic copy of the movement is formed) and in the left inferior frontal lobe (where activation patterns correspond to the processing of visuo-motor information). The latter appears to interface with the understanding of an action goal (e.g., lifting the finger) while the former determines the kinesthetic details of the movement with respect to proprioceptive information concerning limb position. Iacoboni et al. (ibid.) raise an interesting semiotic question in the conclusion of their article that provides evidence for the direct matching mechanism of imitation: “ [...] How an individual may preserve the sense of self during action observation, given the shared motor representation between the ‘actor’ of the movement and the ‘imitator’”? 

            Following their recordings of the firing patterns of 532 neurons in the premotor cortex of macaque monkeys, Gallese et al. (1996) provided evidence of what they called mirror neurons in which they tentatively located the basic mechanisms of imitation. “[...] mirror neurons form a system for matching observation and execution of motor actions” (1996:606). They hypothesized that a matching system similar to that of mirror neurons exists in humans: “This imitation process could be based on an observation / execution matching mechanism [... That would] extract the essential elements describing the agent of the action (arm, hand, face, etc.) and code them directly on specific sets of neurons with motor properties like those of the monkeys’ F5 motor vocabulary” (606). Mataric and Pomplun (1998), whose research bears upon the connection between visual perception and motor control in humans, suggest that “people analyze human arm movements largely by tracking the hand or the end-point, even if the movement is performed with the entire arm”, and, when imitating, they “use internal innate and learned models of movements, possibly in the form of motor primitives to recreate the details of whole arm posture and movement from end-point trajectories” (1998: 191-202). They emphasize that, in spite of its core importance in human dynamic behavior, imitation per se is rarely the object of scientific comprehensive inquiries and continues to be addressed by different research communities.

            The detailed knowledge of how gestures are spontaneously imitated during the process of socialization and acculturation would shed light on a human universal behavior that is taken for granted but little understood. Once this is achieved, the question of the evolutionary significance of imitation can be addressed in more precise terms. Notably, it would help to bring into focus another dimension of gestures that still remains imperfectly explained: their temporal nature. Investigating imitative gestures in real time, as the process actually takes place, will show that the body is time as much as it is space. A multiplicity of biological clocks are presently being uncovered in the phylogenic and ontogenic make-up of the various mechanisms regulating organisms’ development over time through a complex entanglement of biological rhythms. Timing, tempo and punctuation, and their neurological basis, are as important for the understanding of motor behavior as the information processing of visual trajectories and motricity on the brain level. The study of flocking, mobbing, pack hunting, fighting, courting, etc. shows that the temporal coordination and orchestration of movements must have evolved simultaneously with agility, mobility and sociality. As research on the behavior of teleost fishes indicates, “schooling” -- that is, the synchronized and polarized swimming exhibited by groups (or “shoals”) of  fishes which remain together for social reasons --  is a vital strategy that relies on the micro-temporal coordination of collective movements (Pitcher and Parrish 1993). But this is true of any social species. The peripheral body space is at times a most busy traffic area which must smoothly interface with other peripheral body spaces if motion is not to become a lethal liability, as stampedes and collisions often dramatically demonstrate. When the semiotic  study of gestures shifts its focus from individual motor patterns to interactive meta-patterns and overcomes the constraints of the communication bias of its methods, biological time and its dynamic morphologies will become theoretically foregrounded rather than being treated as a mere parameter. 


5. Conclusion: Towards an evolutionary framework for the study of gestures  

            There seems to be a general agreement that a comprehensive theory of gestures is still lacking (Kendon 1998). The definition of the object itself remains controversial as well as the determination of what should count as relevant data for the semiotic study of gestures. Some tend to restrict the scope of the inquiry to phenomenological descriptions of stereotypic movements that carry conventional meanings within a cultural area (e.g., Posner et al. 1998). They are guided in their inquiry by linguistic and rhetorical communication models. Others want to extend the observation and recording to a much more inclusive domain that encompasses a wider range of interactive movements and rely on a multi-scale approach and cross-specific comparisons (e.g., Chance 1988; Mair 1986; Poggi 2001).

            The most serious advances to date appear to have been achieved in the domain of gestures associated with speech, not only gestures substituting for speech, but also gestures that form a close interface with language (e.g., McNeill 2000). The relationship between gestures and syntax, for example, is much better understood now than it was when the category  “paralinguistic” was coined in order to cover whatever escaped the reductive linguistic definitions of the earlier part of the 20th century. Progress in the functional descriptions of sign languages has contributed to focus the attention of researchers toward holistic perceptions of embodied linguistic interactions (Stokoe 2000). Such approaches are now generally accepted except by those who still cling to their principled quest for a “language organ” (Chomsky 1998). Great advances have also been made in the knowledge of the development of gestures and their perception in neonates both in humans and their closer primate relatives (e.g., Trevarthen 1986, Schmuckler 1993).

            As indicated in the two previous sections, the abundant research that bears upon motricity, the muscular and skeletal dynamics of the limbs (see review by Dickinson et al. 2000), clinical neurology dealing with motor impairments (e.g., Goodwin 2000), rhythmicity associated with biological clocks (e.g., Grillner 1985; Pastor & Artieda 1996; Schechter 1996), and the anatomical and functional investigation of the motor and visual representations in monkeys’ brains (e.g., Prut & Fetz 1999) remains scattered in a huge number of scientific publications. There is a growing need for semioticians to engage in meta-analyses of these publications, which often address the issue of meaning in their own idiosyncratic terms. Semioticians interested in gestures must expand the scope of their data and the depth of their knowledge in order to propose fully scientifically-informed theoretical views with the hope that these views could impact significantly upon a broad range of disciplines.

            While the knowledge of brain processes associated with dynamic behaviors has considerably advanced, the results of this research do not amount yet  to a theory of gestures. They often provide additional  puzzling evidence that becomes part of the problem rather than part of the solution. Nevertheless, these data allow for much more precise formulations of the problems, and their clustering within a semiotically-informed context often outlines unexpected but promising configurations.

            Once gestures are fully understood at the micro-level of their muscular and neurological processes and in view of the general physical and mechanical laws of their environment (e.g., Marteniuk, Mackenzie & Baba 1984, MacKenzie 1985), the issue of explaining why and how they evolved remains untouched. Artificial simulations and task improvements achieved through sophisticated servo-mechanisms can lead to insights into the energetic and ergonomic strategies of moving organisms. But  the “how” does not provide definite knowledge regarding the “why” although it is a prerequisite for developing explanations which, in turn, can shed precious light upon obscure aspects of gestures. As has been suggested in the third section of this paper, the full understanding of gestures must include an explanation of the evolution of limbs. But this is a very complex story that shows all sorts of appendages and limbs budding forth under a variety of environmental pressures and, at times, regressing to purely vestigial organs, like, for instance, the hindlegs of some species of snakes whose function appears now to be limited to facilitating copulation (Lewis 1989; Preuschoft & Chivers 1993; Duboule 1994). The hand is not the exclusive privilege of primates. It has also evolved in other lineages with similar multi-functions such as grasping and gripping, carrying and discarding, caring and courting, feeding and grooming, attacking and defending, etc.  If we want to understand human gestures we must conceive of them in the evolutionary framework that provides the best explanatory theory of behavior to date, albeit not without problems. Some semioticians may find it expedient, and less challenging, to skip the multiscale complexity of human semiotic behavior by relying on philosophical systems that propose all-purpose treatments of any problems with a stock of abstract notions borrowed from information theory, logic and linguistics. A recent example of this trend is the publication of a triple issue of Semiotica under the title Biosemiotica 2 (Hoffmeyer & Emmeche 1999). Most of this issue’s contents is placed under the aegis of Jakob von Uexküll (1864-1944), a notorious anti-Darwinian whose approach seems to blend well with some interpretations of C.S. Peirce’s philosophy. Perhaps, in the current state of human knowledge, skipping evolutionary considerations for the study of behavior is misguided and can only lead to epistemological dead ends (Lorenz 1981).

            While the phenomenological investigation of gestures, mostly during the past one hundred years, has yielded interesting results in the form of descriptions and categorizations, these nomenclatures are overwhelmingly restricted to the spatial (visual) dimension, the communicative functions, and the individual sphere. A first strategic move could now be to shift the focus of scientific attention to the  temporal dimension, the interactive and performative functions, and the social or collective relevance of gestures. The second strategic move should be then to identify the adaptive frameworks which would allow for the integration of all these aspects and levels into a complex whole, keeping in mind that evolution, as it is understood now, is not a rational, economical and optimal process, but an opportunistic and wasteful tinkering that works by chance rather than by design since natural selection simply eliminates organisms which carry ill-adaptive phenotypic traits with respect to particular variations of their environment. 

            Such an approach would necessarily foreground performativity, interactivity, rhythm and coordination, imitation and, more generally, sociality. This means that the focus of the inquiry should not be fragmentary individual-centered movements but patterns of interactive movements. This would probably lead to the discovery that gestures form patterns of patterns that “inform” social intersticial space. These meta-patterns could be heuristically mapped into four kinds of behavioral areas: (i) defensive/aggressive (including dominance, bluffing, intimidation, submission); (ii) cooperative (comprising complementary movements of two or more agents toward a common goal, including the above); (iii) reproductive (not only as courtship rituals but also as strategies toward fending off rivals); (iv) parasitic (that is, behavioral algorithms which exploit the copying capacity of organisms, and spread among populations while “informing” the above basic meta-patterns). Obviously, such an inquiry would require both an inclusive theoretical framework and advanced recording and analytical technologies. In the meantime, the intuitive and speculative construction of meta-pattern models relevant to the evolutionary understanding of interactive postures and limb movements can be an invaluable incentive toward such a goal.           



Annett, M, (1985) Left, Right, Hand and Brain: The Right Shift Theory. Cambridge MA:                             Lawrence Erlbaum & Associates.

Blackmore, S. (1999) The Meme Machine. Oxford: Oxford University Press.

Boesch, C. (1991) Handedness in Wild Chimpanzees. International Journal of Primatology, 12,                   541-568.

Birdwhistell, R. (1970) Kinesics and Context: Essay on Body Motion Communication.                                 Philadelphia: University of Pennsylvania Press.

Bouissac, P. (1973) La mesure des gestes: prolégomènes à la sémiotique gestuelle.= Approaches                  to Semiotics 3. The Hague: Mouton.

Bulwer, J. (1974) [1644] Chirologia: or, The Natural Language of the Hands, and Chironomia:                or, The Art of Manual Rhetoric. Edited by J.W. Cleary. Carbondale IL: Southern Illinois                         University Press.

Burr, D.C. (1981) Temporal summation over moving images by the human visual system.                Proceedings of the Royal Society of London B 211, 321-339

Chance, M.R.A. (ed.) (1988) Social Fabrics of the Mind. Hillsdale, NJ: Lawrence Erlbaum                        Associates

Chomsky, N. (1998) Minimalist Inquiries: The Framework. (=MIT Working Papers in                                  Linguistics). Cambridge, MA: MIT Department of Linguistics

Cosnier, J. (1977) Spécificité de l’attitude éthologique dans l’étude du comportement humain.                         Psychologie Médicale 9, 11, 2025-2030.

Cosnier, J. (1977) Communication non verbale et langage. Psychologie Médicale 9, 11,                               2033-2049.

Cosnier, J. & Vaysse, J. (1997) Sémiotique des gestes communicatifs. Nouveaux Actes                           Sémiotiques  52-54, 7-28

Christel, M. (1993) Grasping techniques and hand preferences in Hominoidea. In Hands of                      Primates. Preuschoft,H & Chivers, D.J. (eds.). Vienna / New York: Springer-Verlag,                         91-108.

Danesi, M. & Perron, P. (1999) Analyzing Cultures: An Introduction and Handbook.                              Bloomington: Indiana University Press

Davis, J.M. (1973) Imitation: A Review and Critique. In Bateson, P.P.G. & Klopfer, P.H., (eds.)               Perspectives in Ethology. New York: Plenum, 43-72

De Waal, F. (1989) Peacemaking Among Primates. Cambridge MA; Harvard University Press

Dickinson, M.H. et al.(2000) How animals move: an integrative view. Science 288 (7 April                 2000), 100-106

Di Pellegrino, G., Fadiga, L., Fogassi, L., Gallese, V. A & Rizzolatti, G. (1992) Understanding             motor events: a neurophysiological study. Experimental Brain Research 91, 176-180

Duboule, D. (1994) How to make a limb? Science 266 (28 October 1994), 575-576

Dugatkin, L.A. (1996) Interface between culturally based preferences and genetic preferences :                 Female mate choice in Poecilia reticulata. Proceedings of the National Academy of                 Sciences. U.S.A. 93, 2770-2773

Eibl-Eibesfeldt, I. (1975) Ethology: The Biology of Behavior. Trans. By E. Klinghammer. New                 York: Holt, Rinehart & Wilson.

Ekman, P. & Friesen, W.V. (1969) The repertoire of nonverbal behavior: categories, origins,                         usage and coding. Semiotica 1, 1, 49-98.

Elias, N. (1982) Power and Civility. Trans. by E. Jephcott. New York: Panthoen Books

Feyereisen, P. & de Lannoy, J.-D. (1991) Gestures and Speech: Psychological Investigations.                         New York: Cambridge University Press.

Fox, R. & McDaniel, C. (1982) The perception of biological motion by human infants. Science

            218, 486-487

Fridlund, A.J. (1994) Human Facial Expression: An Evolutionary View. San Diego CA:                                Academic Press.

Gallese, V., Fadiga, L., Fogassi, L. & Rizzolatti, G. (1996) Action recognition in the premotor                 cortex. Brain 119, 593-609

Gazzaniga, M. (1998) The Mind’s Past. Berkeley CA: University of California Press

Golani, I. (1976) Homeostatic motor processes in mammalian interactions: a choreography of                     display. In Perspectives in Ethology. Bateson, P.P.G. & Klopfer, P.H. (eds.). New York:               Plenum, 69-134

Goodwin, C. (2000) Gesture, aphasia and interaction. In Language and Gesture. D. McNeill (ed.)             Cambridge: Cambridge University Press, 84-98

Graziano, M. S. A. & Gross, C. G. (1995) The representation of extrapersonal space: a possible                        role for bimodal visual-tactile neurons. In The Cognitive Neurosciences, M. Gazzaniga               (ed.) Cambridge MA: MIT Press, 1021-1034

Grillner, S. (1985) Neurobiological bases of rhythmic motor acts in vertebrates. Science, 228 (12                   April 1985), 143-149

Guillaume, P. (1971) [1926] Imitation in Children, trans. by Halperin, E.P.Chicago: Chicago                        University Press.

Gutmann, M.F. (1993) The constructional preconditions of the basic organization of the tetrapod                  limb. In Hand of Primates. Preuschoft, H. & Chivers, D.J. (eds.). Vienna / New York:                   Springer-Verlag, 309-321.

Heyes, C.M. & Galef jr., B.G, eds. (1996) Social Learning in Animals: The Roots of Culture,                       New York: Academic Press.

Hoffmeyer, J. & Emmeche, C. (eds.) (1999) Biosemiotics 2. Semiotica 127 (2/4)

Iacoboni, M. et al. (1999) Cortical mechanisms of human imitation. Science 286 (24 December             1999), 2526-2528

Iverson, J.M. & Goldin-Meadow, S. (1998) Why people gesture when they speak. Nature 396                  (19 November 1998), 228

Kendon, A. (1998) An agenda for gesture studies. The Semiotic Review of Books, 7,3, 9-12

Krauss, R.M., Morrel-Samuels, P. & Colasante, C. (1991) Do conversational hand gestures                communicate? Journal of Personality and Social Psychology, 61, 743-754

Krauss, R.M., Dushay, R.A., Chen, Y. & Rauscher F. (1995) The communicative value of                        conversational hand gestures. Journal of Experimental Social Psychology, 31, 533-552

Krauss, R.M., Chen, Y. & Chawla, P. (1996) Nonverbal behavior and nonverbal communication:             what do conversational hand gestures tell us? In Advances in Experimental Social                          Psychology, 28, (M.P.Zanna, ed.), 389-450

Landsmeer, J.M.F. (1993) Evolution and the hand. In Hands of Primates. Preuschoft, H. &                     Chivers, D.J. (eds.). Vienna / New York: Springer-Verlag, 323-333

Lewis, O.J. (1989) Functional Morphology of the Evolving Hand and Foot. London: Clarendon

Lorenz, K. (1981) The Foundation of Ethology. New York: Springer-Verlag

Lynch, A. (1996) The population memetics of birdsongs. In Ecology and Evolution of Acoustic                      Communication in Birds.  Kroodsma, D.E. & Miller, E.H (eds.). Ithaca: Cornell                         University Press, 181-197

Mair, M.W. (1986) The eye in the control of attention. In The Biological Foundations of                                 Gestures: Motor and Semiotic Aspects. Nespoulous, J.-L., Perron, P. & Lecours, A.R.                 (eds). Hillsdale, NJ: Lawrence  Erlbaum Associates, 123-146

Marey, E.-J. (1894) Le Mouvement. Paris:Masson

Marteniuk,R.G., MacKenzie, C.L. & Baba, D.M. (1984) Bimanual movement control:                                     information processing and interaction effect. The Quarterly Journal of Experimental             Psychology 36 A, 335-365

Mataric, M.J. & Pomplun, M. (1998) Fixation behavior in observation and imitation of human             movement. Cognitive Brain Research 7, 191-202

Mather, G. & Murdoch, L. (1994) Gender discrimination in biological motion display based on                      dynamic cues. Proceedings of the Royal Society of London. B 259, 273-279

Mather, G., Radford, K. & West, S. (1992) Low-level visual processing of biological motion.                         Proceedings of the Royal Society of London B 249, 149-155

MacKenzie, C.L. (1985) Structural constraints on timing in human finger movements. In                         Differing Perspectives in Motor Learning, Memory, and Control. Goodman, D., Wilberg,             R.B. & Franks, I.M. (eds.). Amsterdam: Elsevier Science Publishers B.V., 157-173

McDougall, W. (1908) An Introduction to Social Psychology. London: Methuen

McNeill, D. (1992) Hand and Mind: What Gesture Reveals about Thought. Chicago: University                     of Chicago Press

McNeill, D. (ed.) (2000) Language and Gesture. Cambridge: Cambridge University Press.

McNeill, D & Levy, E.T. (1993) Cohesion and gesture. Discourse Processes 16, 363-386. Meltzoff, A.N. (1996) The human infant as imitative generalist: a 20-year progress report on                      infant imitation with implications for comparative psychology. In Social Learning in                  Animals: The Roots of Culture, Heyes, C.M. & Calef jr. B.G. (eds.) New York: Academic             Press, 347-366

Meltzoff, A.N. And Moore, M.K. (1977) Imitation of facial and manual gestures by human                      neonates. Science 198, 75-78

Napier, J.R. (1993) Hands. Revised by R.H. Tuttle. Princeton NJ: Princeton University Press.

Neri, P., Morrone, M.C. & Burr, D.C. (1998) Seeing biological motion. Nature 395 (29 October                       1998) 894-896.

Newcombe, F. & Ratcliff, G. (1989) Disorders of visuo-spatial analysis. In Handbook of                           Neuropsychology, Vol 2 , F. Boller & G. Grafman (eds.) New York: Elsevier, 333-356

Pastor, M.A. & Artieda, J. (eds.) (1996) Time, Internal Clocks and Movement. Amsterdam:                  Elsevier Science / North-Holland

Peters, D.S. (1985) Mechanical constraints canalizing the evolutionary transformation of tetrapod             limb. Acta Biotheoretica 34, 157-164.

Pitcher, T.J. & Parrish, J.K. (1993) Functions of shoaling behaviour in teleosts. In Behaviour of               Teleost Fishes. (Second Edition). Pitcher, T.J. (ed.). London: Chapman & Hall, 363-439

Poggi, I. (2001) The lexicon of the conductor’s face. In Language, Vision and Music. McKevitt,                     P. (ed.) Amsterdam: John Benjamins.

Ponge, F. (19  ) Première ébauche d’une main. Paris: Gallimard, 116-118

Posner, R., Krueger, R., Noll, T. & Serenari, M. (1998)  Stinkelfinger / Berliner Lexikon der                 Alltagsgesten Fassung 2.  Berlin: Arbeitsstelle fuer Semiotik, TU Berlin

Poyatos, F. (1986) Nonverbal categories as personal and cultural identifiers: a model for social                         interaction research. In Iconicity: Essays on the Nature of Culture. P. Bouissac, Posner,                         R. & Herzfeld, M. (eds.). Tuebingen: Stauffenburg Verlag, 469-525

Preuschoft, H. & Chivers, D.J. (eds.) (1993) Hands of Primates. Vienna / New York:                           Springer-Verlag.

Prut, Y. & Fetz, E.E. (1999) Primate spinal interneurons show pre-movement instructed delay                         activity. Nature 401 (7 October 1999), 590-594.

Rauscher, F.H., Krauss, R.M. & Chen, Y. (1996)             Psychol. Sci 70, 226-231

Rizzolatti, G., Fadiga, L., Fogassi, L. & Gallese, V. (1997) The space around us. Science 277 (11                        July 1997) 190-191

Rizzolati, G. & Arbib, M. (1998) Language within our grasp. Trends in Neurosciences 21,                   188-194

Sapir, E. (1927) The unconscious patterning of behavior in society. In The Unconscious: A                      Symposium. Dummer, E.S. (ed.). New York: Knopf, 114-142.

Schechter, B. (1996) How the brain gets rhythm. Science 274 (18 October 1996) 339-340

Schmuckler, M.A. (1993) Perception-action coupling in infancy. In The Development of                                 Coordination in Infancy = Advances in Psychology Series. Savelsburgh, G.J.P. (ed.).                 Amsterdam: Elsevier

Sperber, D. (1996) Explaining Culture: A Naturalistic Approach. Oxford: Blackwell

Stokoe, W.C. (2000) Gesture to sign (language). Language and Gesture. D. McNeill (ed.).               Cambridge: Cambridge University Press, 388-399

Tarde, G. (1903) [1890] The Laws of Imitation, trans. by Parsons, E.C.. New York: Henry Holt

Trevarthen, C. (1986) Form, significance and psychological potential of hand gestures in infants.             In The Biological Foundations of Gestures: Motor and Semiotic Aspects. Nespoulous,                 J.-L., Perron, P. & Lecours, A.R. (eds.). Hillsdale, NJ: Lawrence Erlbaum Associates,                   149-202

Winters, J.M. & Crago, P.E. (eds) (2000) Biomechanics and Neural Control of Posture and                 Movement. New York: Springer-Verlag

Wyrwicka, W. (1996) Imitation in Human and Animal Behavior. New Brunswick/London:                  Transaction Publishers.

Zahavi, A. & Zahavi, A. (1997) The Handicap Principle: A Missing Piece of Darwin’s Puzzle.                         New York: Oxford University Press.