Korsakow’s Palette (clouds, connectors and corridors).

As you work up, remake, bend, stretch, scratch, scrape, rework, manipulate, wend, tear, rip, cut, glue, paste, sketch, and otherwise think through the making in the doing of a Korsakow film it is worth thinking about what the terms of this making consist of. For example if I were filming something the terms of my making might include:

  • exposure
  • frame rate
  • composition and framing
  • movement (of camera, of subject, of lens via zooming)
  • exposure contrast
  • lighting
  • depth of field
  • focus

These are the formal things I can use to make with, my palette if you like.

In a Korsakow film, a part from the actual video clips (which would of course include the list above), when I am designing the work I have:

  • thumbnails
  • videos
  • sound
  • colour
  • background (colour, photos, sounds)
  • text
  • size
  • clouds
  • corridors

For each of these the other terms generally apply. For example for the thumbnails that are used for navigation I can think about if they have sound, text, colour. What size should they be? This is the formal language of stuff I have to work with, and their various combinations – keeping in mind I can have different interfaces in the one project.

In addition, via keywords, I have clouds, connectors, and corridors. Clouds are clusters of dense interconnection. Connectors are those nodes that bridge between two or more clouds. Corridors are passages that I want to insist upon, for instance (and most commonly) the opening screen of many a Korsakow project. Clouds I take from Mark Bernstein’s ‘cycles’ (in his “The Patterns of Hypertext”), connectors I think I just grabbed then since the alliteration was nice, corridors I remember from something Anja Rau wrote way back in the early days describing parts of Mark Amerkia’s Grammatron.

Tags: ,

Reprise: Interpreting Forward

Yesterday I wrote:

In new media we interpret forwards not backwards. Eg in a Korsakow film we see the current clip and what comes next. We always wonder not “why do I see this clip after the one before” but “what is the relation between this clip and those thumbnails?”. So we interpret forwards here, this is the reverse of film interpretation which always can only look backwards because you can never see what comes next. This is, surprisingly, not considered in much work in this field where the hermeneutics of this media is treated as the same as in film.

I wrote that while sitting in the little cafe at the National Film and Sound Archive on my way out of the Visible Evidence conference to get a taxi to get the plane to get on back home. So bit hasty.

I should strike out new media and replace it “interactive documentary” or perhaps “network documentary”, or even “networked nonfiction”. And a screenshot to show what I mean.

57 Reveries

So in this example the video clip of the black cockatoos we understand primarily not by what we saw before, but in terms of the thumbnails that are now available. These thumbnails are the future horizon of the film, where it might go next. In this particular case this decision is the users, but in relation to interpreting the work, that is its hermeneutics, we also understand the black cockatoo video by the relations we hypothesis between it and a small dam outside, some mountains in mist, a couple of raves, feeding a king parrot, a finch, an echidna, the same clip, a woman’s shoulder, and a match. If nothing else we might speculate there is an outdoors and nature, if not out and out ornithological relation going on. We might also realise that the text under the clip (“I didn’t want to stand up because I knew the wind from her dreams would blow me down. (Silverleaves, January 2010.)”) might also offer connection to the woman’s shoulder.

We can see that to understand the current video here we do so by wondering about the relations it has to what we see as the field that it is already visually situated within, we cast forward, and of course we actualise this by choosing one of the nine available. In a traditional film we can never see what is next. We can speculate, and the film can meet or not our speculations and hypothesises, but in general we always interpret backwards. “That’s why that happened.”

This also means the thumbnails here do a lot of hermeneutic work as it is the patterns that form here that can most strongly make the deep structure of the interactive work visible and understandable to the viewer. If they realise this is where they need to look (most don’t), the work becomes much more intelligible, and interesting. The future horizon of the film is made literally visible and available to us, in a ‘normal’ film it would be like seeing the future the film ahead as small images across the bottom of the screen as it played (that’d be an interesting experiment, no?), and we choose it. Try it, you’ll soon see you spend less time remembering I saw that clip and now this one than what is the relation of this current clip to those other ones it is explicitly declared to be related to.

I also discuss this as a part of the ‘sketch film‘ task I get students to first make in Korsakow over on the post industrial video site I wrote.

Tags: , , , ,

Textobjectext Part One

TinderboxScreenSnapz001.jpg

The Textobjectext symposium was a well run event. Very well organised, small group in attendance but diverse and interesting mix of presenters all worrying about objects, materiality, affect and practice. Excellent catering by way of a gift to those that participated.

My own ‘paper’ was sketched and presented in Tinderbox as a hypertext. Let’s be blunt. This was because I did not have enough time to let the work get crafted enough, and using Tinderbox made the writing faster, more agile, while also doing a lot of work in the presentation to make present some of the questions I was wanting to explore. But it was a first version, and I lost all my contextual threads while waiting to present as I got quite engrossed by the prior papers (one made easily intelligible some really interesting things in maths, the other did a nice job of thinking about invasive species and what that language is all about), so when it came to my turn I was befuddled!

As a result I lost the sense of what it was I specifically wanted to get to. I knew I wanted to draw a line between writing and thinking to the rule of n-1 (Deleuze and Guattari’s rhizomatics), Bergson’s sensory motor schema and perception (where perception is always a reduction in relation to what is), and Harman’s stuff about the way objects withdraw themselves to approach each other and hypertextual writing. That line is pretty straightforward, and I briefly sketched it out, but then was left with wondering, OK, and? It felt like a reprise of mid 90s hypertext theory where I was substituting new materialism and objects for ‘poststructuralism’ and saying that writing hypertextually (so much of the theoretical work around hypertext looks at its relation to the literary/narrative, or to reading, but very little has explored the experience of writing hypertext in itself) allowed the objectness of the ideas to be maintained since each object in the Tinderbox work kept its facets available to other facets all the time. But I was clumsy, and even worse, inelegant.

Thankfully the questions and the conversation threw up some very nice ideas and I think provided more context for what I have been struggling to express and explore. I was trying to situate hypertextual writing as a specific form of humanities academic research practice that allows the thickness of research to be closer to the surface and experience of writing, all the way through writing and editing, and that within a post humanities context this is to participate in a larger assemblage of parts where the writer is a relay amongst other parts, and where the nodes within the hypertext retain their, their what?

I will return to it, but after Christmas, as Visible Evidence is looming large and I don’t want to trip over myself a second time.

Tags: , , ,

Thinking about Parts and Wholes

I use quite a lot of hypertext readings when I teach interactive video. I also refer to this stuff a lot in my research. This is partly because my academic history went something like media and cinema studies, general and comparative literature, then hypertext, then a return via hypertext to film and video. It isn’t like I don’t ‘have’ cinema theory, just that when it comes to thinking about interactive video online I find most of it pretty lame.

I think this is because cinema theory approaches the problems of new media from the wrong end. Cinema approaches a lot of this work like it is trying to understand audience identification, visual pleasure, genre or realist narrative and continuity, most of which either don’t apply or are very different in small screen interactive work. Hypertext theory, at least of the humanities variety, comes from a different theoretical view as the literary, perhaps oddly, is able to accommodate difference and collage much more easily than much cinema theory. This might be because most cinema theories try to answer the question of how can something that is made up of clearly discontinuous parts be experienced as continuous and whole. This gets answered using theories of montage, realism, psychological and psychoanalytic identification, phenomenological intentionality, and so on. Whereas literary theory has never really had to answer that question, it really hasn’t had to account for the problem of wholes from parts, and so can more easily think, theoretically, about things that have always been experienced as whole that are now fractured.

Tags: ,

Relational Media

Extracted from a current work in progress:

Relations in multilinear work matter, indeed multilinear media could more accurately be described as ‘relational media’. Multilinearity is premised on allowing basic units of a work (what early hypertext theorists, partly following the example of Barthes’, have usually labelled as lexias or nodes, and what Thalhofer in his Korsakow interactive video system refers to, awkwardly, as the “Smallest Narrative Unit” or SNU) to be able to be connected, arrived at, and followed differently, where I might find a particular node from any of several others.

Relations also matter in linear media where connections between segments is affected by editing, though here once these edits are made these relations become fixed – this is perhaps the key distinction between hard and soft media. A consequence of this fixity is that content is often thought to matter more than sequence and a result of this is that many practitioners new to these forms try to make these ‘relations between’ singular rather than plural. In this case work becomes serially constrained so that multiple relations becomes this and then this, so that the problem posed by relation is reduced to one of navigation and architecture and becomes assigned to extra diegetic elements such as menu commands or navigational buttons to read the work. The significance of relation and plurality in multilinear works is simply avoided by recourse to such instrumentalism.

Relations matter because they are always multiple. Things, whether they are ideas, video sequences, or sentences always have an immensity of relations that they are situated within and by, whether realised or not, and things only come to be known to the extent that we make and actualise these relations. In my video work relation and multiplicity has been explored through variations between video windows, for example in video diptychs. These works are more interested in the simultaneous formal relations between coterminous video sequences than the ersatz multilinearity that results from serially arranged varying sequences of this and then this and then this. This latter case remains formally close to the traditional cinematographic timeline, where the task of the work is to assemble a story from a variety of parts, but the story remains a linear narrative. By pushing narrative aside and allowing videos to play together, to relate to each other programmatically, thematically, associatively and even disjunctively, is to make more forceful propositions in relation to softvideo than simply trying to solve the problem of narrative sequence. In softvideo and its relational poetics narrative is not the minimal condition or state but only one mode that could be performed. Narrative here is no longer sovereign, even though many ‘database’ defined projects privilege narrative as given. Such privileging of narrative risks being a critical and theoretical cul de sac. In such a model narrative is the privileged term, a theoretical inside, and database is merely a technical apparatus constructed around narrative as the problem. In my work I begin from different premises where simultaneity, rather than seriality, is an effort to make concrete how soft practices are about relations, that they are determined by relation and that soft systems are engines and assemblages that enable multiple relations.

Tags: , , ,

Violence of Text

This is a collaborative project undertaken with honours students in Media and Communication Design. A research symposium (“I Link Therefore I Am”) was held in 2002, the students were given the brief to document the symposium and to create a publication that worked from the premise of academic ‘writing’ being invented then, so what would an academic network native thing look like? Violence of Text is what they created, it is impressive.

Original Citation:
Miles, Adrian, Dion Tuckwell, Erica Watson, Amelia Chappelow, James Taylor,Shaw Cunningham, Reuben Stanton (eds.). “The Violence of Text.” Kairos 8.1 (2003): http://english.ttu.edu/kairos/8.1/binder2.html?coverweb/vot/index.html

The Violence of Text

Tags: , ,

Draft: Korsakow and Affect (2)

Another section of a book chapter, this comes out of the earlier part I posted the other day on Bergson, Affect and Korsakow. Here I take the movement image and the sensory motor schema and use it to think about Korsakow films:

This tripartite series of noticing, deciding and doing is underwritten by an economy of movement and action that ranges from the purely autonomous (for example homeostasis) through to the relatively free (reading a menu to decide which meal to eat). There is a perception, then an action, which may be automatic or calculated. This economy of activity, of action and of doing, is Bergon’s sensory motor schema. In this system affect becomes the remainder where action is not adequate to a perception. For example, I see a snake and I jump in fear. While I am now away from the snake and understand that I am safe I still feel anxiety, stress, tension, fear and relief, all at once. The jump, even where it has happened without thought – and it might have been an impressively large jump too – is not adequate to my perception of the risk and danger, has not equalled it, and so this remainder with no where to go as an action resolves into affect.

We can see that a K–Film is even more strongly inscribed within this sensory motor schema than cinema because the structure of noticing, deciding, and doing is fundamental to the organisation of a Korsakow film and interactive work in general. This is literal, as in a Korsakow film a user views video and at some moment during this they make an explicit decision which requires the motor action of a mouse click on an icon or button within the interface which causes something to happen. They perceive, decide, and then act, and the system repeats.

In doing this we shift from being viewers or readers to users because we become Bergson’s ‘living image’, that is, literally the gap between action and reaction that forms the movement image via the sensory motor schema. In cinema this gap is overcome in movement through montage which corrals these varieties of images into relations that become fixed in their order and occurrence upon the screen. This indeterminacy is resolved in the movement image because the film will do something, itself. However, in a Korsakow film there is always this ongoing site of indetermination located in the user who necessarily becomes an affective relay between perception and action, watching and clicking.

As a consequence systems such as Korsakow are strongly aligned to what I characterise as ‘affective narratives’. Stories that enlarge the moments and possibilities around a situation, event or milieu. Between a seemingly simple proposition or scenario and its implications and understanding. This ‘enlarging’ in a Korsakow film is achieved through including and allowing for multiple points of view, polyvocality and even simply because an affordance of online media is the ability, in concert with combinatory systems, to utilise as much footage as desired (since you are now liberated from having to choose amongst original footage to make your work fit a strict duration). The function of such a combinatory engine (in Korsakow’s case through the use of keywords) is to produce a multiplicity of relations between clips and sequences. For example, in Thalhofer’s documentary practice using his Korsakow system each film is orientated around a simple and open question or problem. In such cases the work is not didactic in the sense of making a specific or directed argument but offers up a field of views through interview, stories, asides and observations and through its use of keywords then constructs an architecture of associations that allows for the connections between its parts to remain loose and fluid. These associations are affective as there is always this interval or zone of indetermination between any current sequence and those that become available. In Thalhofer’s films, even though they are documentary, they align themselves towards the affective through the openness of their associative architecture and as a consequence of this the user must listen to the work rather than merely navigate.

Tags: , , ,

Time, Rather than Space?

I’ve never understood why people think things like Korsakow or, even hypertext systems, are spatial. The literature and criticism abounds with spatial metaphors. Why? (Yes, there is spatial hypertext which deliberately spatialises relations, but that’s a special case, though perhaps also problematic.) The relations are temporal, not spatial, things are near to each other in time, not space. I think this matters, a lot, as it means the way we conceptualise these things is, possibly, deeply structurally wrong.

Tags: , ,

From viewers to users, four propositions for digital specific documentary

This is the second abstract I submitted for consideration for this particular panel at Visible Evidence (Canberra, December), the panel has been proposed by Kate Nash and is themed around “Exploring Documentary Interactivity“.

Interactivity within documentary poses several problems in relation to both the making of documentary itself, and the perception of conception of documentary audience. In relation to making documentary the elastic nature of online media in terms of storage, distribution, and delivery means that the key editorial and directorial decisions around what is retained, and what is then ‘on the floor’ can be removed. In other words it is relatively trivial to build online systems for documentary practice that allow the maker to make available the majority of their material. However, this causes two significant problems. The first is that the role of selection, of editorial intervention and, basically, authorship, is often experienced and thought to be diminished precisely as a result of this technological liberation – if most of the material can now be made available, what then is my role as a documentary director? The second problem that of course follows is that we know that most audiences, most of the time, do not and will not want to see all of the footage available, so how then can a work be structured to allow audiences to ‘scale’ their attention in relation to the work?

The simplest answer to this problem is to recognise that viewers are now users which means that as makers we a) do not ‘own’ the space of distribution and reception (it is a personal screen where the user generally does several things, in several programs, at the same time), b) do not own their attention (they have not given up their time to attend a screening or to watch something on a box), c) can be expected to return to a work where there is a reason to do so (it is no longer a single view economy), and d) that granularity and porousness are fundamental attributes of network specific media and practices.

In this paper each of these principles will be elaborated upon in relation to interactivity documentary to identify and critique aporias in existing practice and to describe possible future practices and models.

Tags: , , , , ,

Edit Versus Writing

So I hit that point in my writing where I couldn’t figure out how to end, or quite what had to happen next. If anything. That is when I know it is time to step free form drafting, print it all out and start editing. Stage one is the sketch. Stage two, where I need it on pages, is the sculpting of the form. Stage three is the polishing. This chapter, which is about affect, interactive video and I guess what I think is the poorly titled ‘database cinema’ has been good to write since it is for a multilinear electronic format (Scalar) so it can be granular, open, and porous to its own meanderings.

Tags: ,