Performative Technography of Video Game Cultures: How to Analyse Body Practices in and with Media

Author: Nino Ferrin (FU Berlin)

* * *

Research for this publication was supported by the Marie Curie International Research Staff Exchange Scheme within the 7th European Community Framework Program.This publication is an open access, free of copyright, deliverable of the DIGIT-M-ED project (DELIVERABLE NR.: 4.4; WORK PACKAGE NR.: 4). Please cite this guide as following: Ferrin, N. (2013) Performative technography of media game cultures: How to analyse body practices in and with media. Berlin: DIGIT-M-ED.

* * *

A network theory approach to the field of newer media is to be understood as a methodological framework to research about a set of different practices that declares technology and media as part of the everyday life and even conceptualizes a “symmetrical anthropology” of man and things. In those networks the hybridity of practices forces the investigator to take into account that all activities are based on underlying histories of the actors and their environments. In this setting the paradigm of the performativity of culture focuses on transformative processes that cannot be planned or controlled. Also intention and emergence are no longer to be thought as binary antipodes, but as basement for the creation of new social realities. That leads to the point, that network and also performativity theory enable an actor in his/her agency and at the same time limits him/her. The different principles are on the one hand, that Bruno Latours network metaphor brings together things and man, whereas the performativity paradigm stresses the actions of man in double constitution of speech and action, cognitive and body knowledge and so.

The discussion is also represented in current approaches to theories of practice (Schatzki, Reckwitz). This point of view enables a metatheoretical understanding of the performative network based integration of media in the actors’ worlds and vice versa. Also it points to the stabilization of the world through the generation of knowledge in techno-social interactions.

This also counts for the ambivalent media practices one can oversee in everyday life situations, where for example technical devices enable an actor to intentionally gather knowledge or to deal with ludic spaces, but at the same time emerge unforeseen interactions or side effects. By implying those concepts in a methodological approach, one can transform the concept of Ethnography to Technography and focus the data of everyday life practices, such as videotaped situations, where young people deal or play within digital arrangements or games. In a last step the reconstruction should be confronted to educational anthropology to refer to concepts of the body, the imagination and the senses.

The following example focuses on body practices in and with media. In this context, gestures have the status of reciprocal and, in principle, open processes of body-related meaning production. In the context of a praxeological approach to research, it is first necessary to investigate the empirical data taken from participative and video-supported observation of different game practices with the game console “Wii” from Nintendo with regard to observable, thus implicit as well as intentional movements. The identification of the reciprocal gestures of various actors is put into practice as a becoming-significant movement in the process of the two-way allocation of meaning. The specificity of the medial field is subsequently revealed as a technical-situative interpretive force, as a special case of socialization. It therefore seems appropriate to make a few introductory remarks on the technical features in order to gradually formulate the consequences for an empirical investigation of gestures in and with media.

In contrast to the mouse, joystick or keyboard, the Wii console is mainly characterized by a new kind of handling, which is best seen in the innovative controller. In this controller – the Wii Remote – sensors have been implemented that transmit its position in the room through its location/direction in relation to the infrared camera in the device and two additional sensors near to the screen, as well as registering changes of movement. These data are simultaneously recoded into analogous movements on the screen. This already shows how meaning is produced by the technical recording, conversion and playback of data. The console defines itself as a controlling body, since it actively re-shapes the player’s positions and changing direction by carrying out the allocation of meaning within the medial framework, whereby the medial image on the screen can be given the quality of a synchronicity in relation to the parallel body movements. In other words, movements carried out in real space are staged almost simultaneously on the screen. This is implemented by the just mentioned “Wii Remote”, which is marketed by the manufacturer as follows:

“Wireless and motion-sensitive, the Wii Remote offers an intuitive, natural way to play games. The ergonomic controller plays into the conventional motions you make every day. When you hit a drum, swing a tennis racket or swing a bat, why should you have to press a button to replicate these things? With the Wii Remote, you can use everyday motions to help make you the center of the game. Nintendo designed the Wii Remote to be the most multifaceted gaming device ever. It can be a sword in one game and then a steering wheel for racing games. It’s your paintbrush, your golf club, your aeroplane, but most of all, it’s your key to unlocking a world of fun you’ve never imagined.”

Hence, the controller can operate as a tennis racket, which is held, like a real tennis racket, in the hand. The head of the racket, which the controller lacks, has to be imagined, while the swing of the racket is “carried out” with the controller. The special nature of this game practice is primarily the heightened physical activity during play.

The following empirical scene shows the qualities of these image-generated gestures by demonstrating the power of body images over movements in real space. It culminates in the question of control: who controls whom or what?

The video images are taken from a family event to celebrate a christening. The game console is set up in a large living room with enough space for a screen for the video projection, a number of rows of chairs and a generous playing area. The controller just described, which has been set for “boxing”, is linked with another controller and held in both hands. In this way, the movements can be transmitted to the console. This converts the data into digital movements on the screen. The screen is split into two fields with a boxer in each, both with their backs to the respective players. Each boxer fights his respective adversary in front, so that the players have their avatar in front of them without mirror reversal. The console simulates the sounds of punches as well as the noises of the crowd. Every unblocked punch is accompanied by a ping.

In the video sequence, the two opponents – sparring partners in the true sense of the word due to the doubling effect – find themselves, after the completion of two bouts, in a wild fight at the beginning of the third round. Both players strike each other with punches. It is worth noting that the younger player operates at a very high tempo and without trying to block punches, while the older player, more in the style of a real boxer, uses his fists to protect his head and places his punches in a more considered way. In the course of the fight, it is not possible to make out any feints, which must be due to the technical requirements, which as a result of the rate of punches are not able to assimilate this strategic factor in the representation. On the other hand, the turning of the upper body with arms angled forward, as evasive maneuver, offers a kind of reaction to the flurry of punches. Both opponents use this evasion many times during the fight. When, as the fight develops, one of the avatar boxers – in this case the one of the older boy – falls to the ground and is counted by the referee, he bends his arms in hectic movements towards his upper body and brings them down again to a right angle. After the referee has counted to eight, the player relaxes and gets back into position – the avatar stands up (again). In the meantime, his rival waits in front of the screen and scratches his face with his hand.

As the fight restarts with a ring of the bell, the older player takes up a defensive position, leans forward with the upper body and is again covered with punches from his opponent. Only after a few seconds, however, the latter’s avatar also goes to the ground and must gather his forces via arm movements – which means try to stand up. Meanwhile, his opponent waits, as he had himself a few seconds earlier, by breathing in deeply and smoothing out his shirt while shifting his weight from one foot to the other. As the fight starts again, he punches the air three times, but after only a short period he lands on the floor again, and is counted out. This brings the official part of the fight to a close. The number ten finally signifies the conclusion of the fight. It is also the reason for his adversary’s jubilation, which is expressed in a gesture of triumph: the younger boxer stretches both arms vertically into the air. The loser, on the other hand, turns away from the screen with his hands knocking against his legs and immediately gives the controller to a seated viewer. At the same time, the audience starts applauding and calling out words of appreciation.

What does this scene show us about the connection between body and image? Isn’t it the movement that functions as a link between the avatar on the screen and the body, isn’t this the source of the circulation of the allocation of meaning? What is the relationship that occurs between apparatus, player and image?

The data registering the location and direction of the controller is transmitted to the source of interpretation, the console. On the screen, the movements are represented as significant gestures, since they demand a reaction from the actants (cf. Mead 1973). At least, this is suggested by the almost synchronous image playback as well as the attitude of all present, since they exclusively focus on the screen, thereby promoting it to a social space. This transition point also reveals the permanent circulation between the movements of the body and those on the screen through the reciprocal relation between apparatus and body (the being-moved of the Wii Remote is a human-technical hybridisation) as well as from the resulting screen gestures back to the body movements. Here, the self-referentiality of the performative practice can be recognised, since, through the inherent link, symbolic operations outside and gestures within the image on the screen effect the constitution of meaning. It also makes clear, however, how the significant gesture first achieves visualisation through the power of the technical apparatus. Only movements converted by the console are emitted as image. However, for everyone present – audience and players – it is precisely due to the non-visible computing of the movements, that the actual cause for a hit or a successful evasive manoeuvre is withdrawn. Less strongly executed movements are also converted into a pre-programmed punch, which can have a no less relevant result in the context of the game.

The character and control of the action depend on the framework of the technical application; intention and planning are at the discretion of media. The reciprocal relation of the real adversaries, who are standing next to each other, happens only rarely. Instead, the opponent first appears on the screen, in the place where the avatar meets its adversary. Nevertheless, the players are assigned competence by the fact that the effect (of the punch) obtains meaning through a computing process.

However, resistances are also revealed here: the older of the two boxers is superior to the younger in his method as well as his strength. Through the adaptation of the game’s internal rules, those of the technically-virtually possible, however, a ‘badly’ executed punch (in sports terms) can also achieve a techno-social competence. But this can also lead to a misjudging of one’s own abilities. This is fed by competence attributed by oneself or others to master movements that obtain and suggest meaning only through their digital conversion, for example that a tennis forehand is successful. Through the intersection of two overlapping fields, the game appears to be more than just an image – simultaneously also a bodily accomplishment, whereby the learning and appropriation of new patterns of movement is achieved in the shortest time. The reference to the growing tendency of feedback of new game practices to physical performances thus opens up an ambivalent process of evaluation. The imagined ability of boxing in a no-mans-land as shadow boxing against a phantom adversary, a phantasmatic body in virtual space, creates potential for reflection about the relationship between body and body image and shows that the body should not be conceived as unitary and reducible to an image. On the other hand, the viewing of one’s own medial gestures also suggests the fantasy physical of practices’ possession as available body.

Finally, the acquisition of new game practices relates to a further point of medial specificity. The handling of the Wii Remote reveals the connection between body and object in the medium of the gesture. This technology is the powerful instance of the regulation of physical practices in whose framework the virtual body can be shaped. Gestures emerge and acquire significance. Virtual bodies act among themselves as well as with the player, and thereby bring about physical experiences and forms of knowledge that can develop medial as well as sporting relevance. The reciprocal condition of the body activities inside and outside of media can be summarily described in a shift of terms. If in boxing, the talk is still of inter-subjectivity (with Mead), in the medial learning situation, an inter-objectivity (with Latour) can be assumed. Here subject and object are transformed into a collective network and henceforth operate as actants, which results in a kind of action network. The circular self-causation and self-determination of the actants – that is, controller and player thought of as interrelated – therefore results at the point of contact between hand and controller as well as in the virtual body, which in this sense can also be conceived as networked actor.

The playfully acquired media competence as recognition – attributed by oneself or others – of ability in the use of technology can best be described as a process of body formation which is experienced through the constant self-referentiality of the circulation of meaning between image and body and the integration in medial structures. Such experiences are found within game forms offered by new technologies, which is why they cannot be substituted by other (sports) practices. These experiences produce orientations, which have an action-guiding character, since the formation of body-related self-images are also developed through media-related practices and offer an imaginary space for individual as well as social interpretations and projections, and thus new sources of self-learning.

In the example presented, a mediacy of body movements is revealed, which, as gestures, only receive a specific density and complexity in the process of the practical event. In the computer generated body/image interaction, a technical device is activated in which the movements carried out are interpreted and formalized through computing processes. Conveyed here are, however, specific actions and reactions between the real game adversaries, which are meaningful for the participants themselves. For the participants, the body movements make “sense” in relation to the technically represented movements of the adversary.