1. Introduction
Transmedia traversals: marketing meaning and identity
Jay Lemke University of Michigan, USA
In
the study of multimodality, we seek to understand not simply how multiple sign systems or semiotic resources ( The
study of of signs and symbols and their use or
interpretation)
are co-deployed (correspondingly positioned) to make new meanings, but also how the ways we do
this are shaped by the role of media in our society. In this paper I am going to sketch out a theoretical
model for the relationships
among media, identities,
and the political economy
of new globalizing markets
(growth to a global or worldwide scale). I will illustrate this model
by reference to two new and changing
social phenomena: the development of transmedia, (Storytelling across
multiple forms of media, with each element making distinctive contributions to
a user's understanding ) intertextual
networks that cross the boundaries of genres and media, and the emergence of traver- sals, our
ways of making meanings
and living our lives across the boundaries between sites and institutions. I will propose
that commercial transmedia promote systems of differentiated (Identify differences between ) and
hierarchized (arranged in a hierarchy.) identities,
identity markets (the development, promotion and extension of a product), which we
learn to re-appropri- ate
and re-organize as members of social networks.
Multimodal
media make meaning by intersecting the semiotic resources of language, visual display, sound
and music, cinematic movement, material artifacts, and abstract animation.
If we consider action
itself to be a semiotic system (Lemke, 1984; Martinec, 1998, 2001), then
in the case of interactive
multimedia, we may imagine a dynamic semiotic cycle in which users interpret what is being
displayed to them across
many media, initiate or respond with action meaningful to them, and then interpret the reaction of
the media system
(Aarseth, 1997). We can analyze such multimodal processes and systems in terms
of a very simple cross-contextualization
principle I have proposed (Lemke, 1998), building on the work of
Michael Halliday (1978) and others in the functional linguistics and social
semiotics traditions (e.g. Martin, 1992; Kress, van Leeuwen, 1996, 2001; Thibault,
2000).
- Semiotics, also called semiotic studies are the study of signs and sign processes (semiosis), indication,
designation, likeness, analogy,
metaphor, symbolism, signification,
and communication.
Halliday
had originally proposed in his analysis of language as a semiotic resource that
every utterance in
principle makes meaning along three simultaneous axes: presenting a state of affairs (ideational meaning), - (conceptional:
being of the nature of a notion or concept) taking a stance toward content and interlocutors ( A person who takes part in a dialogue or
conversation) (interpersonal-attitudinal
meaning)
( relationship attitudes), and defining informational and textual wholes and continuities (textual-cohesive meaning) (causing cohesion). Kress and
van Leeuwen and others (e.g. O’Toole, 1994) have shown that visual semiotics in drawings,
diagrams, painting, and architecture could be similarly analyzed, and my
own work on scientific publications, political cartoons, and NASA websites
(Lemke, 2002; see also http://www- personal.umich.edu/~jaylemke/papers/polcart.htm) established that the same scheme applied
when linguistic text
and other media were
integrated in various genres.
576
577
Jay Lemke
In
each genre there was a
different sense of what constituted ‘the whole work’ (EG: if a sign of a bloody heart was on the front cover horror film you would interoperate it as a sign of foreboding & death & fear, whereas if it was on the front of a hospital pamphlet you would read it as scientific and see it as a tool to aid ones medical understanding) and therefore which signs were to be interpreted in
relation to which other ones. In a political cartoon, often enough, the
humor is apparent neither from the image alone nor from the captioning text by
itself, but only from their juxtaposition in the whole cartoon. – as it creates a satirical,
darkly comic effect by positioning the opposite statements together to form a
powerful whole.
(Facebook Schmacebook - private eye - satirical magazine)
We interpret the presented visual content
differently because we align its meaning with that of the presented content of
the accompanying text (and vice versa, in multiple possible ways). (the combination alters the overall perspective)
-
Smiley Politicians would look wholesome & honest apart from the caption
states the contrary?
FOR EXAMPLE :
We
simultaneously (instantly)
do this for their respective signs (Belonging
or relating separately to each of two or more people or things – separate but
related to the same whole picture - the x2 signs - in this case & photograph & the caption) indicating interpersonal
(communication - between the two) intimacy or distance (happy politician & horrific caption = distances the audience from the piece) and also for various evaluations
(conclusive ideas & opinions on the piece) of the presented content such as its importance or desirability (how much in the end they want to read it or buy into this present idea.)
Finally,
we connect one part of the
image with another because this is suggested by the text, and we make new cohesive (creating a whole) ties
(analyze) within the text because of semantic (Relating to meaning in language or logic)cues provided by the image (explore it further
ourselves). What we do in some genres
with text and image, we do in others with the addition of musical or other
auditory signs - videos, voice clips etc (van Leeuwen, 1999), dynamic movements as in animation or cinema (Martinec, 1998, 2001), and even our own motions and actions in clicking the video- recorder remote or moving the computer
mouse. (so to understand the meaning of the symbols on a website you would have to physically interact & explore through movement as supposed to reading an article & thinking about its meaning & what the photograph accompanying it implies).
Semiotics = The study of signs & the sign processes :
Indication - A sign or piece of information that indicates something
Designation - The action of choosing a place for a special purpose or giving it a special status
Likeness - The fact or quality of being alike; resemblance
Analogy - A comparison between two things, typically on the basis of their structure and for the purpose of explanation or clarification.
Metaphor - A figure of speech in which a word or phrase is applied to an object or action to which it is not literally applicable.
Symbolism - The use of symbols to represent ideas or qualities
Signification - The representation or conveying of meaning.
Communication - The imparting or exchanging of information or news.
Semiotics are closely related to the field of linguistics, however, semiotics also study non-linguistic sign systems. Semiotics are usually divided into three branches:
Semantics: The relationship between signs and the things to which they refer; their denotata, or meaning (what they denote : e.g. a rose = a flower - connotes love)
Syntactics: The relationship among signs in formal structures (the branch of semiotics which deals with the set properties of languages and systems of symbols.)
Pragmatics: Relation between signs and the effects they have on the people who use them (of or pertaining to a practical point of view or practical considerations.)
The
space of semiotic meanings is multiplicative (nature of multiplication): it
expands combinatorially (the state of being combined) in possibilities, only to
contract the more intently
to some nexus (A connection or series
of connections linking two or more things )of
instantial (represent by instance) meaning in each actual multimodal sign or
‘text’. The
meaning of each lexical (relating to the words
of a language )sign can
be interpreted, from its range of potential meanings, not merely by its syntagmatic (related as members of a syntagma – a
syntactic string of words that forms a part of some larger syntactic
unit -SYNTAX.)
lexical context and its
paradigmatic (for relating to a grammatical paradigm - A
typical example or pattern of something; a mode)set of (not-deployed) lexical alternatives, but together with
its visual or musical co-text (and
absent co-texts, the signs-not-chosen). So many more combinations of signs from multiple modalities are possible in
multimedia, and though many fewer are typical, nonetheless each conjoint multimodal sign-complex is one
choice out of a much larger universe of possibilities, and therefore
indexes one much more specific, specified meaning when it occurs.
COMBINING THESE PHENOMENONS:
Traversals, transmedia, identity markets,
and mediation by social networks.
The aim is to explore the changing political
economy of the sign (Baudrillard, 1988). In the fantasies of Power,
and often enough in practical fact, the appeal of a transmedia mythos (its
characters, dramas, values, and proffered identifications) invites prospective
consumers (and voters) to form
strong identifications with its key elements, and above all with the implied principles of
differentiation and hierarchization (referring to hierarchy - A system or organisation in which people or groups are ranked one above the other according to status or authority) of identities which will work to
create and expand markets
defined by particular identity types. (opening up to new audiences).
These may be, as Bourdieu (1984)
has presented not just markets
and dispositional (A disposition is a
habit, a preparation, a state of readiness, or a tendency to act in a specified
way )preferences for films or magazines,
but also for news media, politicians, rhetorics (persuasive / impressive ), and policies.
Into
this perfectly regulated world of transmedia and identity markets comes a somewhat unexpected
phenomenon: internet-mediated, broadly accessible, large-scale?? - everyone has access & availability - high culture & popular culture merge - signs.
Social
networks in which consumers
discuss, critique, cannibalize, and re-appropriate the content of transmedia
franchises, while adding to the total transmedia complex their own productions (anyone can add to what something means?). Consumers are now also producers or at
least participants in networks that are creating interpretations of transmedia
potentially (and in such complex meaning spaces, almost inevitably) at odds
with those of the official producers and managers of the franchise.
These
expanding and mutually interacting networks now operate on an extended complex, combining old and new official additions to the
franchise with the large contribution of fan production, re-interpretation, and
critique. What is judged authentic to
the mythos edges out of the control of the official producers and into the
control of the consensus of expert consumers. Newcomers turn to the
communities for social reciprocity (The practice of exchanging things with others for mutual benefit) and a sense of genuine participation, and they find
far more interpretive assistance than comes from the producers. They do so just
as the franchises are growing beyond the bounds of individual interpretive
capacity, while still retaining, and perhaps even strengthening their appeal.
Our
identities arise in large part, and certainly insofar as they can contribute to
social solidarities (a union of
interests or purposes or sympathies among members of a group), though our identifications with beliefs, values, ideals, groups and communities, and increasingly,
under the influence of mass media and consumer marketing, with our choices in consumer markets
and our preferences in media. We are a society in which consumer values and the ideals of life
depicted in marketing messages have few effective competitors on a mass scale.
For most young people today, neither
politics, nor religion, nor education offer very appealing identifications.
They are too transparently corrupt, bankrupt. Consumer materialism steps in to define success, to define popularity,
to define what is worth having and doing. Along with goods and services, it sells us values, and more implicitly
the principles of social differentiation
and hierarchization that favor its interests.
Now,
however, there is a new counter-weight. Online
communities and social networks are not merely sites for exchanges of
information. They are also sites for experimentation
with identities. Whether we try on identification with a dangerous
killer, a soldier for justice, a good-hearted thief, a wicked wizard, a
commander of battalions, a builder of cities, or a leader of civilization
(all pervasive themes of video and computer games), we find it convenient and
perhaps necessary to also participate in communities where we figure out how to play these roles more
successfully, where we discuss our exploits with peers and amateur experts,
and where we develop identities both for
our fantasy personas and for our community personas.
We
identify not just with characters or goals, but with
interpretations of those characters and goals. Our interpretations are increasingly
shaped by our participation
in social networks,
and the dominant interpretations
available in those networks are products not just of the official media,
but also of the history of productions by fans
of the extended, more total
transmedia complex. As we find ourselves more and more identifying with and
playing out the
consequences of alternative roles and interpretations, inevitable conflicts and
contradictions will arise, and inevitable spaces of alternative
possibilities will emerge. No one will control the identity consequences of
transmedia, when our experience of them is mediated by large, diverse social
networks.
So there is in this sense a double multiplication: each semiotic modality multiplied by each of the others in use, and each axis of meaningfulness of each modality multiplied by each other axis, both within that modality, and across all the modalities. This awesome potential accounts for the ways in which multimedia can be confusing, overwhelming, and amazing. This is the full sign-potential of our cultural traditions, never fully realised in the past because of technological and resource limitations. From the Eleusinian mysteries, to the cathedral High Mass, to Wagnerian Gesamtkunstwerk, to sound-film, color television, music video, and now three-dimensional, interactive, immersive digital virtual worlds, we have been exploring the possibilities of multimodality for centuries. Possibilities both for the better and for the worse.
All of these media forms and their genres have become instruments of power and privilege. Ideologies which have naturalised beliefs, attitudes, and values congenial (Pleasant because of a personality, qualities, or interests that are similar to one's own) to those in power have been embedded in and promulgated (Promote or make widely known ) historically through all these multimedia. When mass-circulated media moved from folk-tradition to centralized production and distribution (CHINESE WHISPERS TO - TEXT, IMAGES, SOUNDS, 3D INTERPRETATIONS, CONTACT, CARTOON) , control of the ideological content of media became an available tool of power (MANIPULATION - through appeal). The art of the cathedrals, the pamphleteers (A writer of pamphlets, esp. ones of a political and controversial nature) and newspapers printed texts, illustrated magazines, phonograph recordings, radio broadcasts, and on to the newer media which dominate the landscape of signs today have all been used for such purposes.
We are, however, large complex societies and our elites are not monolithic (Constituting or acting as a single, often rigid, uniform whole). There are fundamentalist Christian television broadcasts, films, and now computer games (e.g. Eternal Forces, http://www.eternalforces.com/ ) in the US and other nations, and in many places also fundamentalist (Forming a necessary base or core; of central importance) - (Fundamentalism is the demand for a strict adherence to specific theological doctrines usually understood as a reaction against Modernist theology, primarily to promote continuity and accuracy) Muslim media, - among the most influential media resisting, at times, the dominant ideologies and political economy of global capitalism. Most media, however, are coming under the ownership of multinational corporations which control the channels of mass distribution (CONGLOMERATES). Among these media I include not only publishing, film, music, and games, but also their less explicitly ideological but equally semiotic cultural partners: fashion, food, and consumer merchandise of all kinds (Barthes, 1967, 1977; van Leeuwen, 1999; van Leeuwen & Caldas-Coulthard, 2002). The marketing of all these signifying artifacts positions them as indexical (of or relating to or serving as an index) signs of particular meanings, identities, and statuses within market ideologies.
(modality - A particular mode in which something exists or is experienced or expressed)
The meanings we make are a product not only of our
immediate needs but also of the modes of social organization in which we
participate. We fill out forms, give job-talks, write essays, and make
small-talk because we participate in larger- and smaller-scale social
institutions from the nation-state to the family and the business office. Within
these settings we deploy the resources appropriate to various more- and less-
prescribed written and spoken
genres as our immediate needs and longer-term ambitions dictate. What is true of
the meanings made specifically with language is true more generally of the
meanings made with every form of human action: each act participates in local
constructions of meaning on shorter timescales at the same time that it also
participates in the systematic networks of interdependent activities that
sustain institutions and societies over much larger distances and longer times.
- So the interpretations of signs is relative to the :
CONTEXT & CONSUMER (AUDIENCE) & PLACE / PURPOSE.
- interpretations of semantics are not set and one image or word could
connote a variety of things to different types of people.
-Applied : This is how in web design one would calculate the way that
users would interoperate the specific sight & its usability, as what can be interoperated simply for one individual may be entirely different for another, and the individual break down of the contemporary multi-modal sight not only feeds all audiences but is subject to the specific audience of that set genre (broadens the genres & creates different rhetoric links for different individuals), and to a wide variety of individuals, exploring the ethos of the group through a vast spectacle of media types in order to link all of these together & produce a map of their musical attitude.
- WEB DESIGN : For example - Eyetracking visualizations show that users mostly read Web pages in an F-shaped pattern
two horizontal stripes
followed by a vertical stripe.
- Users start by reading in a horizontal movement, along the upper part of the content area. This initial action forms the F's top bar.
- Then, users move down the page a bit and then read across in a second horizontal movement which covers a much shorter area than the previous movement. This extra element forms the F's lower bar.
- Lastly, users scan the content's left side in a vertical movement . Usually a fairly slow and systematic scan that appears as a solid stripe on an eyetracking heatmap. Other times users move faster, creating a spottier heatmap (This last element forms the F's stem)
- Occasionally it has been found that users will instead read across a third part of the content, = resembling an E shape or they'll only read across once, which makes the pattern appear as an inverted L
( this study was carried out my Jakob Nielsen)
A conceptual structure for understanding & designing complicated cultural user experiences in terms of multiple trajectories (A path / taken course). Trajectories capture the idea of establishing a Logical and consistent journey through an extended experience which combines a variety of different spaces, times, roles and interfaces ( a point where x2 things interact).
| The framework identifies the different kinds of transitions and traversals (taking a zigzag path - of the beaten track - un structured - complicated) that may occur along a trajectory and shows how diverging, converging (combining) and crossing trajectories describe important aspects of multi-user experiences and reflect the tension between pre-scripted narrative and interactivity. (what the author or maker has posed as 'the answer' and the users interpretation. |
In 2006, Oliver Reichenstein wrote 'Web Design is 95% Typography'. Typography can not only guide a viewer along your journey, but it create a personal link, as it is an expression of the person who creatively designed it, something which is extremely key when thinking about a band or artists sight, when your key aim is to market & sell them as a product & also create a level of intimacy with fans. For 99% of all web design , typography and white space is going to be your underlying foundation of the entire site. 
