Homepage » Acting (on) the Text: the Case of New Media (6. 2014) » Introduction 6

Issue 6


Acting (on) the Text: the Case of New Media  

Eve Kalyva and Christos Physentzides 

A text is a process of sharing, the shared creation of meaning.
M. A. K. Halliday, “So you say ‘pass’…thank you three muchly”

Interactivity is a key feature of new media. Applications such as digital storytelling, visual art installations, computer games and social networking sites support new ways of exchanging information. These are characterised by flexibility in data manipulation in relation to generating, structuring, storing and sharing data. Another characteristic of new media technologies is multimodality. Multimodality describes the varied or simultaneous engagement via different modes of communication such as visual, textual, audio, somatic-kinetic etc. Neither interactivity nor multimodality is exclusive to new media; rather, one should speak of different degrees to which new media support these. A more recent term for talking about a particular effect or strategy of new media applications is remediation: re-doing and extending production from old to new media or across the media; as well as shifting information exchange between media. Thinking along these three terms addresses the cultural significance of new media in a specific manner: the shifts across one medium to another, across one domain, spatio-temporal continuum, cluster of information or focal point to another. Apart from allowing one to consider the advantages and limitations of the medium itself, such new media shifts can generate socio-political critique across the networks of production and dissemination of information.

Turning to the text, new media offer new ways of thinking about, producing and engaging with texts. They equally do so in relation to images and sounds and other forms of input, so let us use “text” in an extended sense of the word to describe a container of information with variable materialities and margins; better, a site of production not only of contents, but also of practices and theorisations. The benefit of addressing the textual rather than the visually, audio or otherwise conveyed is that it makes it easier to project the analysis to production (in any case, someone has to programme the machine) and to theory. This line of enquiry follows Roland Barthes (1977), when he called for opening up the notion of the “text” as a methodological field, plural, paradoxical, caught up in a process of filiation and restored in language; Jacques Derrida’s (1976) discussion of textuality in relation to writing, chains of signification and theory; and the analysis of social discourse and communication (Halliday 1994). Thus, the text can be understood as a dynamic site wherein power is enacted, identities constructed, interpersonal relationships formulated and social behaviours regulated.

As such, the case of new media incites us to engage with cultural production in wider terms. Authors become more involved in material and formal production; likewise, readers become users, rather than recipients, and often co-authors. Reconsidering the relation between author and reader in less singular and more interactive terms extends to rethinking other binaries, often concealed, such as between producer and consumer, and between (geo-political) source centre and periphery. By the same token, one should also address the role of the participating audience, agency, responsibility and the shaping of corresponding communities. Texts can act–they actively shape behaviours and preconceptions, embody concepts and enable critical reflection. Equally, meaning-making is a social process; how is, then, the shared creation of meaning structured in multimodal environments? Here, it is important to scrutinise the different materialities and remediations of expression and communication. Engaging with the text can be understood as a material and social practice capable of creating new dynamic spaces. With reference to the relation between inscription and its conveying medium, and between participating agents, this practice can open up new, accessible and more democratic spaces, and lead to a new kind of collective: from collective production to the enhancing and shaping of communities, and to collective action. This issue of Synthesis seeks to examine how new media applications modulate signification, shape interpersonal relationships and corresponding communities within which they operate; moreover, how they function as carriers and generators of world-views and ideologies. Attention is given to the valorisation of cultural production and exchange with reference to the communal and the collective; the relation between private and public, the individual and the community, the body and the other; and new models of co-existence, and of social and political action.

Narrative, visual and meta-spaces

In literature and the visual arts, applications such as hypertexts, blogs, interactive platforms and mixed media installations offer multiple narrative levels and multimodal (textual, visual, sonic, tactile) interplay. Users can explore a matrix of narrative possibilities in a dynamic unfolding (Landow 1992). For Katherine N. Hayles (2001), the narrative takes shape as a network of possibilities rather than a sequence of events; a narrative space characterised by complexity and fluidity wherein different subjectivities are reconfigured. Crucially, this narrative space supports fragmentation and can therefore resist a totalising view or singular experience; as such, shifts between agents and modality of engagement can act critically. In this regard, the concept of the open text becomes particularly important. Open-endness can be understood in terms of access, form, collaborative production and dissemination, as well as a means to test and challenge the hierarchical structures and value systems that are placed upon singularity and copy-right in the intellectual rights and art market alike.

Samira Nadkarni’s “‘All the moments in our lives occupy the same space’: Tracing the Space of Memory in Tim Wright’s In Search of Oldton” offers an insightful discussion. Interactive and multi-layered, In Search of Oldton (2004) combines digital story-telling with various material supports and remediated outputs (blog, website, maps, playing cards, radio play). It asks participants to contribute to the reconstruction of a past and lost home town, and engages with issues of memory, recollection and the communal. Past and present, memory and fiction, the material and the virtual shift and interlock in an expansive cartography where, as Nadkarni argues, Wright remains the architect and the primary author since the audience’s contributions are remodelled and narrative paths selected; nevertheless, authorship is dispersed. In this anachronistic process, a pre-digital past is to be brought to recollection and digitally reconstructed. While this past cannot be accessed, its shared search becomes re-located and re-mediated in the multimodal space and place of Oldton, Nadkarni observes. Moreover, and supported by the project’s interactive platform, the above process becomes collective. In the place of a fictive, imaginative town rise new networks of connections across a community of users who project not only personal experiences in relation to their own lives, but also in relation to how they envisage inter-personal, family and community relationships to be. Oldton and its remediations, then, acquire shared meanings as they grow and proliferate, and allow to map patterns of social interaction beyond the community of its users.

Interactivity and programmability alter the writing mechanisms in new media; by extension, they also affect associated metaphors for memory, archivability and embodiment. This is the focal point of Athina Markopoulou’s article, “How to Undo Things with Codes: New Writing Mechanisms and the Un/archivabe Dis/appearing Text,” where she discusses William Gibson’s Agrippa, a Book of the Dead (1992) and Garry Hill’s Writing Corpora (2011). These two projects employ ephemeral texts in order to involve their audiences in processes of inscription and destruction; in doing so, they also critically engage with associated ideas about preservation and embodiment, but also with the cultural values and significance placed upon them. Gibson’s text, discussing the magic power of proper names, is presented in a format that causes it to disappear upon first view. Efforts have been made to recover lost data and archivise the work, in a process that alters and remakes the object at hand. The fear of loss exposes a collector’s fetishism, Markopoulou argues. It also confesses the spectatorial desire of a privileged, possessive and voyeuristic gaze. Is Agrippa the book and something else, then?, the author asks. In Hill’s interactive installation, the visitor’s movements cause scattered letters to form words as long as one remains still. This conceptualises an intermediary metaphorical space between the body and inscription that, Markopoulou suggests, conditions the ways our bodies are experienced. Here, participation and agency (of the artist as well as of the audience) are suspended, locked in a dynamic relationship between inscription and destruction, meaning, reflection and action.

The fleetingness of information takes central stage in the collaboration between the author Michael Joyce and the visual artist Alexandra Grant. Tatiani Rapatzikou, in “Language Amplifications and Visualisations in Michael Joyce’s was and Alexandra Grant’s babel,” explores the interconnections and interrelations that can emerge by the shifts between languages, codes and mediums of expression. Such a  collaborative practice can enhance the field of experience spatially and conceptually, but also seems capable of postponing the singularity of the author’s voice. Joyce’s text is a postmodern novel in a printed medium that marks a departure from his well-known work with hypertext. Still, it becomes useful to examine how networks of information may rise when older media are pushed to the limits and, in this case, brought into dialogue with a visual platform. For Rapatzikou, such networks of information can be understood as landscapes that can constantly expand and alternate via the input readers provide, as they move or cruise through them. True for all media, one should consider how they structure the way we perceive and process information, but also how they model the way we approach them. Reconceptualisations about the medium, its form and contents, and about the role of the author and the user are part of the artistic enquiry (the case par excellence is Stéphane Mallarmé’s 1897 Un coup de dés jamais n’abolira le hasard); to this end, technological advancements offer new interactive, multimodal and immersive means.

The above discussion raises several questions regarding the relation between the shifts across different spaces, places and bodies, mediation and actuality, and the ontological extensions of physicality vis-à-vis the virtual. A case at hand is Lynn Book’s “DErangements [a study for UnReading],” a visual, textual and sonic bricolage enabled by new media platforms, including this journal’s. In Book’s work, words are animated, float across the screen, appear and disappear over images, photographic stills and video clips. In turn, images slide, fade out, are superimposed onto themselves and over text and sound, split up and burn. The sound, equally shifting between background and foreground, is a multi-layered mixture of voices, whispers and screams, clutter and sounds of wilderness. Rhythm helps to guide and at the same time disperse this accumulation of means and meanings that articulates a fleeting embodiment. The search for a body becomes the search for identity, the right to call upon and mediate a body through history and competing cultural and hierarchical positionings of the self, but also of the spectator, the artist, the object and the message. Here, the crafty employment of available media offers different experiences, different readings and modes of engagement. Johanna Drucker (2013) asks to consider the enacted and event-based character of digital activity. Re-examining traditions of critical theory and several frameworks in the digital humanities, Drucker rejects the rhetoric of an immaterial and dis-embodied digital plane and argues that it is materiality that provokes performance: “Performative materiality emphasizes the production of a work as an interpretative event.”  

Agency and interaction in computer games

Any discussion around new media should be cautious of presumptions about the ‘immateriality’ of the digital, not only in technological but also in societal and political terms. Likewise, one should scrutinise the choices offered to new media users in relation to older media, which are often too readily celebrated as newly-found freedoms supported by participation and interactivity. Here, the case of computer games is a particularly good testing ground. As a cultural product, the main industry for games is entertainment even though significant research has been developed in relation to serious games. There are different multimodal and interactive channels that can achieve immersion in the act of playing a game (the gaming experience), and the differences and new capabilities of games in relation to visual input, participation and narrative structures have been compared and contrasted to cinema, theatre and literary studies. Two particular focal points are agency, which relates to the responsibility for the choices that a game’s interactive set-up manipulates; and the creation of an interactive site with reference to the cultural significance of games as carriers of ideologies and world-views.

A computer game can allow a self-reflective experience, even critical in wider terms (of the industry, society, the role of the media in mediating reality etc) by negotiating gameplay and-story telling, and by giving and taking choices. Kyle Eveleth examines how game mechanics allow for critical reflection on participation, and suggests the concept of agency decay in his article “When Players Feel Helpless: Agentic Decay and Participation in Narrative Games.” Games combine different aesthetic, narrative and ludic elements. The player’s participation is ergodic and dynamic, Eveleth explains, and partakes several interpretive practices such as exploration, configuration and transversing the game’s textual elements. But what happens when a game sabotages the gaming experience and the player’s expectations and ethical beliefs? Eveleth compares the narrative and participatory aspects of two games, Braid (2008) and Actual Sunlight (2012), that seem to be moving to the opposite ends. While the former allows world exploration without rush, the latter is instructive from the onset and offers no choices or variability. By manipulating these two modes of experience, the ludic and the narrative, games can prompt critical reflection on participation and the responsibility for one’s actions, Eveleth elucidates, introducing doubt to the ability of a player to act without being coerced in participatory media.

The ethical and political extensions of how choice is structured in games discusses Stephen Joyce in “The Rapture at the World’s End: Non-optional Choice and Libertarian Idealism in New Media.” Joyce examines the idea of cognitive dissonance, of breaks in the circle of narrative and agency built up by the game and manipulated across the narrative and ludic experience. In the case of Bioshock (2007), Joyce illustrates how the strategic use and withdrawal of agency becomes an additional means to convey, and critically engage, the game’s world setting: the debate between traditional conservatism and radical liberalism (rather than socialism/capitalism) with particular reference to social and market control, and morality at an individual level. These characterise contemporary American society where their corresponding political parties, while being conceptually opposed, share governance. As a result, Joyce maintains, Bioshock constructs a nexus between two arguments, a political one and one in relation to the medium. Rather than contrasting gameplay to narrative, one should see how these work together to set up a particular type of experience where ideologies and world-views are not only reflected but also enacted through the player’s choices. It is a world, aesthetically crafted and materialised in participation, where the player is caught up. At the end, there are no innocent choices.

Social sites and the mediation of the communal

The final section in this issue discusses social networking sites. Returning to a question raised earlier regarding where the community of users might be located, the following two examples–the international hospitality network Couchsurfing and the experimental Proyecto Venus (2000-06) in Argentina–are two cases where new media platforms facilitate and reconfigure communication and exchange, while primarily advancing the coming together and the creation of interpersonal relationships offline.

Trust is fundamental to a hospitality site. Alexander Ronzhyn and Eugenia Kuznetsova, in “Conveying the Message of trust through written texts in Hospitality Social Network CouchSurfing.org,” investigate how trust is shaped, articulated and conveyed across this online platform. Apart from personalised profiles, images and friendship links that the site supports, Ronzhyn and Kuznetsova examine the language used in the public references left between surfers and hosts. They compose primary research data from the site’s top destinations, Paris, London and Istanbul, and a part-of-speech analysis of the use of adjectives, nouns and verbs. Interestingly, as Ronzhyn and Kuznetsova’s analysis reveals, rather than articulated as such, trust is preferably conveyed indirectly through an elevated use of adjectives. In addition, people tend to give weight on the activities and values that the corresponding parties share, as well as portray trust by metaphors of family and home.

In a community of users where the element of opportunism weights the text (more and better references increase the chance of one being solicited to host or surf), patterns of expression rise in terms of describing someone or presenting one’s self, as they do in relation to expectations of behaviour and social conduct. For its part Proyecto Venus, where an equally interactive online platform facilitated interpersonal communication and event organisation, sought to allow desire to be articulated and materialise, breaking, or disabling, the constraints and limitations that social practices conventionally face. The project can be described as a multidisciplinary laboratory of social praxis, searching for new forms of living together based on collaboration and the exchange of products and services. As the artist and sociologist Roberto Jacoby who created the project explains in his interview, Proyecto Venus was not conceived as an alternative or escapist society modelled by interactive media; rather, it was fundamentally orientated towards alternative and more direct ways of coming together, co-creation and social interaction. In addition, the project had its own currency, the venus. In a period of economic and socio-political crisis not dissimilar to the one that Greece faces today, this experience allowed concrete moments of socialisation and self-management, and the means to imagine a new type of society and ways to achieve this. Most importantly, it paradigmatically sought to redefine the norms and evaluation systems of social and cultural production. Members were prompt to evaluate, and value, their products and activities and by extension to re-assess social responsibility and actions at a collective and personal level. In doing so, Proyecto Venus reconfigured and actualised a communal, public and political space.


Agency describes a critical capacity, or the capacity to act in the world in more specific terms; it can also be used to describe unconscious behaviour, apart from purposeful activity, in which case one has to examine, and challenge, the corresponding degrees of awareness. Agency can be of an individual or a class, and is central in discussions of struggle in sociological, economic and cultural terms (class, race, gender etc). Therefore, processes that destabilise agency–the presumed facility to act and the assumed responsibility for one’s actions–can generate positive criticism. The above discussion has explored shifts in practices and paradigms in relation to the author, the reader, the audience, the producer and the consumer; and has engaged with multimodality, interactivity, social behaviour and cultural products as carriers of world-views. New media create new, and help conceptualise anew, spaces and meta-spaces of activities, interpersonal relations and criticism. Such spaces function in complex ways; they materialise, embody and generate their contents the same time they structure and mediate them.

The user may be invited to act, but there are degrees of constraints in the structuration of experience. Apart from the evident technological limitations and the creator’s preference, one should critically examine how such limitations are often concealed. In other words, what is the price of a presumed free choice and celebrated participation in a neo-liberal social context? Activities that seek to problematise this are often met with resistance on behalf of the audience who, when all is said and done, knows best how to be a spectator and a good customer. Generating suspicion, therefore, destabilising agency and conditioning participation become critical strategies to raise awareness regarding the role and function of the producer, the user and the medium. New media platforms can specifically implicate the experience of the participatory act itself in choices already made. In doing so, they can advance new modes of and critical frameworks for cultural production and social mobilisation.

Works Cited

Derrida, Jacques. Of Grammatology. Trans. Gayatri Spivak. Baltimore: John Hopkins UP, 1976. Print.

Drucker, Johanna. “Performative Materiality and Theoretical Approaches to Interface.” Digital Humanities Quarterly 7:1 (2013): n. pag. Web.

Barthes, Roland. “From Work to Text.” Image-Music-Text. Trans. Stephen Heath. London: Fontana Press, 1977. 155-164. Print.

Halliday, M. A. K. “So you say ‘pass’…thank you three muchly.” What’s Going on Here? Complementary Studies of Professional Talk. Ed. A. D. Grimshaw. Norwood, NJ: Ablex, 1994. 175-229. Print.

Hayles, N. Katherine. “The Transformation of Narrative and the Materiality of the Hypertext.” Narrative  9.1 (2001): 21-39. Print.

Landow, George. Hypertext 2.0: The Convergence of Contemporary Critical Theory and Technology. Baltimore: Johns Hopkins UP, 1992. Print.

Ryan, Marie-Laure, “Will New Media produce New Narratives?” Narrative Across Media: The Languages of Storytelling. Ed. Marie-Laure Ryan. Lincoln, NE: U of Nebraska P, 2004. 337-359. Print.