(English version by Douglas Lee Arnold)
introduction
precedents and the present environment
the imagination
imagination and dynamic ideography – close concepts
the imagination and video & cinema – what is changing
imagination and interactivity – the end of response stimulation
the author/creator and the reader/spectator/public – categories in disuse
poiesis – an ongoing experiment
the imagination site – a mental extension that constitutes itself
notes
The idea of writing this essay1 has matured over the last four years, after long years of creative activity in the production of digitally-generated works in CD Rom and Internet. Over 15 years of work immersed in this media, I saw paradigms and concepts be transformed practically year after year. Now my interest is concentrated on the emergence of a new communicative environment, imbued with a new art.
In spite of working in a digital environment, during this period I continued producing art in other media. Since 1985 I have dedicated myself to literature, having published 8 printed poetry and 2 prose works. I have also dedicated myself to producing audiovisuals, documentaries and authorial works shown on television and copied on video tape.
My first contact with computers was in 1987, when I started making video-poetry programs. Personal computers were beginning to take hold and become a reality. Soon after, at the start of the 90’s, multimedia arose with the dissemination of computers with graphic processors, CD Rom readers and support for audio and video. That was when I started my career as director and scriptwriter of multimedia projects.
Several previous texts I had written followed the evolution of the ideas that led me to writing this essay. In 1987 I wrote The Kinetic Board, in 1994, Video-poetry and Poetry and Technology, in 1996, it was Multimedia, Imagination and Zen Poetry, in 1998, Multi-poetry, A Temporary Name While Waiting for a New One to Show Up was published, and, finally, in 2000, Digital Video and Interactive Video2.
The query that led me to write this text is related with the name multimedia3, given that it is a new form of multi-sensorial and interactive communication in which known media are integrated and managed by the computer, which is transformed into a communications vehicle. I think that the name favored the multiplicity of media in detriment to its integration. I even came to invent the term intersense to refer to these interactive and multi-sensorial software, but the name multimedia continued, increasingly universal in use and more and more non-specific.
This media fragmentation without something that integrated media into a new orderly system had repercussions in my personal life and in those of the digital artists that I know. What was I: poet, writer, director, programmer, coordinator, scriptwriter, photographer, or musician? Until today, the artists that work in digital media are classified in a strangely ample, non-specific manner. Artists and works in digital media, multimedia artists, creators in new media – in some way, the intrinsic aspects of art and communication that arose with technological evolution have still not been done justice, and that is what I intend to do.
There is a concept that defines my present situation more precisely: I am an imaginator, a proposer and maintainer of imaginations. So why not say it and show why: That is what I propose – to weave arguments that follow the arising of a new form of expression, supported by digital means and more specifically a new form of art that has taken hold in this new technological and communication matrix.
I consider this essay to be an inaugural text. For this reason there is ample space for the incorporation of new ideas and concepts to this first insight, besides new developments that I still have in mind. I consider the text incomplete and open to incorporations and exegesis. It is turned public in order to initiate a conversation about this subject and not to exhaust it.
II – Precedents and the present environment
We live in an important moment in the history of communication through computers. If we briefly take a look at the interfaces4 of personal computers, we find an evolution that came from the command line in text to the graphic interfaces of today, based on the desktop5 metaphor. This evolution, nevertheless, has come to a point where it is necessary to adapt the new communicative functions that the computers have come to have, especially from the 1990’s onward with the onset of multimedia and the expansion of the Internet.
When the computer stopped being an instrument of work and became a communication vehicle, its function changed considerably. So much so that today, a good part of technological research of hardware seeks to resolve this question: wireless networks, large flat screen monitors, high-speed networks, connectivity with cell phones and palmtops, storage systems for video, etc.
From the point of view of software, the need for change has also arisen. The interface with operational systems that simulate the desktop has its days numbered. Today, a new generation of users utilize icons for tools in software, whose icons no longer refer to the original object, which is unknown to them. It is with the popularization of the computer that people wound up substituting tradition work tools. To exemplify, how many web designers that use design programs are familiar with aerography? Nevertheless, in the interface of the programs there is an icon that refers to the design of this tool, unknown to this generation.
As time goes on, this situation will certainly get worse. That is why options to the desktop are being researched. A new interface of the future is already being talked about, the immersive one, where the user interacts with the information in a multi-dimensional environment. Several advanced systems of virtual reality6 have already been developed and their technologies have been incorporated into personal computers, especially in environments for games and authoring software for multimedia.
With the universalization of the Internet we became familiar with the metaphor of the electronic page, composed with text, pictures and graphic elements, with some breaking of linearity through connections between documents made by means of hypertexts. Video and audio are ultimately accessed in windows that simulate a screen. This metaphor, a representation of pages in printed media in a computer, has great difficulty dealing with media elements that need to have temporality, such as video, audio and animation. But this was the first time it was demonstrated that the computer could be used to communicate and was not just for our work.
Simultaneously, multimedia authoring systems brought us the metaphor of the theater and movies to computer interfaces by introducing the timeline. Think of media objects as actors, the screen as a stage, place actors on the stage, distributed along a timeline and program their interactions. That is how much software works that is used in multimedia authoring.
Even more recently, in order to deal with the growing volume in video and audio rich content, disseminated by the Internet, CD Rom’s and DVD’s, the metaphor of the media console has grown in importance, a representation of a home appliance on-screen remote control, somewhat like the panel of a car or an airplane cockpit. It is a simplistic metaphor where the user activates, deactivates, sequences and makes adjustments to the videos and audio segments. The audiovisual information is now in a window surrounded by control instruments.
To get to the heart of the matter, one needs to emphasize the fundamental role of the Internet in transforming the computer into a communication vehicle. Global interconnectivity and the possibility of interaction between myriad production and consumption information channels has become a reality. It now advances with the speed of physical communication media, which has allow immediate exchanges of large quantities of data, vital to the applications with intense use of audiovisual media and systems that use real-time interaction.
Additionally, the Internet is fundamental in the dissemination of the concept of hypertexts, a term coined by Vannevar Bush in 19457. Instead of a system handling and archiving information based on hierarchical and classifying structures in traditional data banks, information in hypertexts is constituted of knots, agglomerates of information – clusters, caches, that, through associatively established connections, are able to establish networks by diverse manners of affinities, much in the way we establish our mental processes. A word in a text remits to another text that remits to another.
Later on, with the dissemination of computational support resources to other media besides text, such as the capability of graphic processing, photo and design exhibition, video and audio, the term was amplified to hypermedia, where not only texts are associated to others, but any other media element refers to another one through these associative connections. Information caches become documents composed with the presence of several media elements.
As we will see further ahead, the imagination effects a new metaphor that abandons the metaphors used in present-day software. This does not mean that many of the characteristics used in these present interfaces will not be incorporated with a new meaning. It is safe to say that the concept of hypermedia will be incorporated, because what is intended is to give it an environment that is more adequate for its full development.
As time goes on, immersed in this environment of transformation, it became clear to me that a new form of expression, a new art, a new communication vehicle was arising. I needed to find an expression that would help me to better understand what was happening and that would help me in my job of creating new works expressed in this manner.
To arrive at this concept, I embarked from a personal difficulty I had in expressing my creative activity and from my perception of the evolution of the communication interfaces and software in computers. I thought that multimedia privileged diversity instead of “uni-meaning”, and I was quite impressed with all-to-common incompetence in elaborating software. A good part of software developers resorted to the overlapping use of non-interrelated media among them, only for the purpose of being considered as multimedia software.
I always have been a critic of “pornographic dictatorship” imposed on them. Do programs have to use many media to be multimedia works? A lone voice would not be considered so, even if it were the best way to express an idea. Indeed, it is not always that the excess of media is an adequate way of an expression.
Everything continued to evolve in one direction: ourselves, as we think, as we manipulated information in the most sophisticated system ever created for it. The term hypertext itself gives off the idea. It is no wonder that the classic text by Vannevar Bush is called As We May Think.
Reviewing my experience, delving deeper into is said to be the most recent research in neuroscience, re-reading authors that deal with the evolution of digital means, I considerably expanded by comprehension about the latest developments in this area. I sought to rethink everything I had learned from a kind of point zero. In fact, I went through a “de-categorisizing/re-categorisizing” situation, where one finds new terminology that is more useful and adequate to what was going on in my mind and deeds.
The word that appeared to me one day was imagination. It could have been thought, imaginetics, mental art, etc. In Chinese, the symbol that represents theses concepts is the same. But I chose imagination. What is important is the definition I will trace for this form of communication. What is important is to delineate some of its characteristics from the first creative experiments with this proposal and arrive at the characteristics of the new art that is appropriated from this universe at our disposal.
The definition of imagination as a communication media is an evolution of the definition of multimedia. Imagination is a multi-sensorial and interactive communication form, mediated by computational systems, such as multimedia, but it advances in the search for a new metaphor that abandons the desktop metaphor, the graphic page metaphor, the geographic metaphor, the timeline metaphor and the console metaphor, used in present-day software. Imagination is the fruit of the incorporation of languages and media into a new language and media and not their utilization and juxtaposition.
Imagination is based on the creation of computational environments, here called collaborative mental extensions that are proposed and maintained by a community. These environments, in turn, are also interactive with other connected communities in networks.
In building them, I used metaphors that directly relate to the functioning of the brain. The building blocks of these environments are called images. Also, certain mutant states of articulation among them through connections and diagramming are called conscious and unconscious mental states. In the end we have a software environment which as a whole resembles a live informational organism, which I call Imagination.
An imagination is like a movie, a book, a theater play, a painting.
This environment contains images in permanent flow and connection with one another. Such process occurs perceptibly and also imperceptively through interactions and other actions foreseen in written software code. Using the mental metaphor, in a certain time we will have access only to a part of the images that we bring into focus, in a process similar to that of our consciousness, while others remain unperceived, however in interactions with the whole.
The following is a description of neuroscientist Antônio Damásio about the functioning of our mind and how this can be transported with some limitations and adaptations to the computer environment the I am proposing. 8
“The process we call mind… is a continuous flow of images, many of which reveal themselves as logically interconnected. The flow moves to the forefront of time, slowly or quickly, in an orderly or disarrayed manner and, sometimes, advances not only as a sequence, but in several [sequences]. At other times, the sequences compete, converge or diverge and, sometimes, overlay themselves. Thought is an acceptable word to translate such an image flow.”
The imaginations proposed, maintained and interacting with communities are, in fact, collaborative mental extensions in computational environments where groups express and read their ideas in the form of interactive images that construct/deconstruct like a living organism. The imaginations evidently are also interactively related to one another, exchanging images, mental states, communities, etc.
Before going on to develop the idea, I would like to define a little better the term image, in the sense I use it here. To this end, I once again refer to neuroscientist António Damásio, because it is now very important to definitely disconnect the term from vision, image here has a more ample sense and that will be useful later on.
“The term image does not only refer to a vision… By the term Image I want to denote mental patterns with a formed structure with the currency of each one of the sensorial modes. which include several senses: visual, hearing, smell, taste and somato-sensorial . which includes several senses: touch, muscular, temperature, pain, visceral and vestibular. The word image does not only refer to visual images and does not just refer to static objects. When I use the term image, I always want to mean a mental image. I do not utilize the word image to refer to standard neural activities that can be found through present methods of neuroscience in the sensorial cortices when they are active.”
The descriptions of how images are produced in the human mind, also from António Damásio, will help us to comprehend how images can organize themselves in the imaginative media, in a computer environment.
“Images are constructed when we occupy ourselves with objects, from outside the brain to inside, from people and places to toothaches; or when we reconstruct objects from memory, from the inside to the outside. The production of images never stops while we are awake and is maintained even during part of our sleep, as can be demonstrated whenever we dream.”
“Images are the currency of the mind, the words that I am using to transmit these ideas are formed in the first place as images of phonemes and morphemes – audio and visual images, or somato-sensorial, and it is only later on that they were placed on this page in written form. In the same manner, these now printed words in front of the eyes of the reader are processed initially in activating other non-verbal images, with which the concepts that correspond to my words can be mentally exhibited.”
“The images can be conscious or unconscious. Nevertheless, we should note that not all images that the brain constructs become conscious. There is an enormous disproportion between the great number of images that are constantly generated and that compete with one another, and the window, which is relatively small, through which the images become conscious – the window through which the images are accompanied by sensation, imagetic as well, from which we are to learn from. In other words, speaking metaphorically, there really exists an underworld below the conscious mind, and this underworld has several levels. One of these levels is made up of images that attention was not paid to, another level is made up of neural patterns that perceive all the images, whether these eventually become conscious or not. Another level is further related to neural machinery needed to maintain the archives of neural patterns in the memory, the type of neural machinery that encompasses implicit dispositions, innate and acquired”
The imagination is a possible space-time, a present agora with certain duration of past and future, where flows occur and ordering of images are organized into clusters, Benjaminian9 constellations, in re-signified languages. An imagination begins by taking off from a proposal that aligns initial conditions of publication of information and an interaction model of the communities correlated to it, especially that of the sponsor community and its interactive communities.
When we begin we don’t know where we are, what we will be. The work starts in this manner, establishes a tempo for itself and constructs itself. It does not lack this or that, nor is it totally published, nor totally in preliminary form. It is it can be for the moment, and many things are not there because a time has not yet been established for them to be done. The time of an imagination makes up part of its syntax. Its evolutionary patterns are defined by active communities.
The imagination takes recourse in the concept of version, such as with software programs, which start up one day and are permanently under change afterwards, to the measure that implementations arise, new forms, possibilities, technological advances, etc. Works are never finished after they are published, and this is a substantial change.
As we have already seen, they acquire what I call organicity, characteristics that share similarities with biological models, as well as with the human brain. They are in permanent mutation, are born, die, grow, regenerate, duplicate themselves, remember, forget, create connections, etc.
Neuroscientist António Damásio tells us in his book Descartes’ Mistake: 10
“… the operation of the neural circuitry depends on the pattern of existing connections… experience models the circuitry design. Additionally, in some systems more than in others, the synaptic potentials can be altered over the life of an individual to reflect different experiences of the organism and, as a result of the brain circuit design, also continue to alter themselves. The circuits are not only receptive to the results of the first experience, but are repeatedly flexible and susceptible to being modified by continuous experiences.
“… Some circuits are remodeled innumerous times over the life of an individual, in accordance with the alterations the organism undergoes, while others remain predominantly stable and form the ‘spine’ of the notions that we construct about the internal and external world.”
The narrative here is not just predefined; the communities can mount their sequencing, the imaginations and focal planes, the consciences and “unconsiences”. The perceptive focus of objects placed in the time-space continuum, following their affinities, causalities and their multi-sequencing, make up part of the work’s syntax, as is also the exchange of theses consciences and “unconsiences” between users in the communities as well.
Images pulse, communicate with themselves through several sensorial means, re-make themselves haphazardly, inside-out, explore new syntax combinations, random, non-linear paths. The videos become interactive and multi-flowing (mutlistreaming videos), with alignments and non-alignments in relation to several simultaneous, coordinated or random timelines. Audios configure themselves associated to texts, phonemes, incidental sounds, articulated in musical language. Soundscapes11 are created in a mental environment.
This new form of expression and communication abolishes the already tenuous frontiers between diverse language forms in a digital environment, the frontiers between creator and his publics, the frontiers between languages, styles, etc. Ultimately, it is no longer appropriate to say that we have languages and multiple entries and exits in these simultaneously acting environments. Printed text, software programming, photography, cinema, video, bi and tri-dimensional animation, theater, painting, etc. here become de-categorized from their original meaning as media or language.
The new language arises from the integration of theses languages, these media from man’s senses. What is proposed is the construction of a new language incorporating these elements. What is proposed is something simpler and more powerful: the construction of communication models based on how we feel and think, on how we search, articulate, internalize and externalize what we receive through our senses.
The imagination structures itself as a work that communicates via multi-sensorial digital entries and exits and is made up of a digital database with mutant syntax based on programming languages that order the sensorial elements of the work. This new syntax allows, among other things, the creators and the reading publics of the work to permanently interfere in the work. These categories also undergo profound modifications and tend to disappear, as we shall see later on.
Time is treated like flows. We have no single vector of linear time. The notion of simultaneity is strongly recalled. One can expect a timeline in the work that concludes other flows to the measure that the work is constructed. However, each object, each moment has its own mutant, interactive timeline with their communities and users that also mold them to their time. Parallel flows occur in an instant of time; temporal vectors can be inverted. These new possibilities open up doors for us to interactive multi-flows, especially the audiovisual ones and cause us to abandon the concept of single, unidirectional time that rules the present languages of the cinema and video.
Nevertheless, it is necessary to avoid the idea that, in order to conceive of imagination works, everything must interact and nothing can be linear. In our mind many narratives exist that demand linearity; there are images that should not be very interactive. When the euphoria of non-linear interactive media is passed, the time to know how to use these elements of language in a better-balanced manner is at hand. Linearity and other sequencing forms or otherwise, interactions or otherwise, when, how and whether or not to use them, between which elements, etc. are decisions that make up part of the new syntax which is delineating itself.
I now see time as an action component of imagination. Time corresponds to movement. And imagination is capable of dealing with the widest range of temporal dimensions. Those that our eyes and ears perceive and also those perceptible only by comparison made by the memory of distinct images. There are animations that are fruit of movements whose duration between states is greater than our capacity to perceive their variation of light or the immediate sound propagation. Think about a growing child in a vegetable garden. Most of the movements have imperceptible durations to our audiovisual sensorial organs without memory’s interference. These movements, or these tempos, that are not dealt with in present day kinetic media, such as cinema or video, that privilege instantaneous perception, are also now capable of being treated in imagination works.
An imagination is not there to be watched: it is there to be in. They are “presenced” works. The notion of stage or screen, or of page, that remits to the concept of projection or mirroring dislocates itself to the idea of an immersed presence of information. This is placed as an accessible and available out-of-body option, capable of its own relationships, established at the act of proposing or at the interactive developments of the work during its maintenance. I should emphasize that the movements or interactions of the imagination work continue occurring, even in the absence of an onlooker.
The space where the collaborative mental extension is housed does not have to be bi or tri-dimensional, nor even quadri-dimensional, when time is included. The informational space can be abstract or realistic and can be constructed in several forms. What is sought out is immersion. One does not attempt to increasingly represent reality, rather, to extend our mind more and more.
The environments do not need to be rooms of marble with 3-D reflection. The architecture of imagination works is free from representation of reality, similar to what occurs today with Hypersurfaces12 – a new area of architecture that studies the fusion of physical space and media, and experiences with off-screen multi-media in robots and general appliances. An imagination space does not even require a computer screen. Its architecture is part of its syntax.
Virtual reality, as it is proposed, adjusts itself to the expression of some ideas, such as screenworks, reconstruction of physical environments, etc. However, a large range of possible applications exists where the desired results would be the de-identification with reality, or the a-identification. And even when we think, for example, of a remote manipulation console of a machine, certainly the environment will have components close to the reality of a tractor operation, but will have other components overlaid and interacting that will facilitate, modify and adapt this operation. And this will be a new reality that is no longer the initial one that served as a basis for the construction of the metaphor.
There is still the application of translation environments of sensorial entries and exits outside of our scale of perception, such as, for example, the translation of light and sound, e-rays and other energy spectrums. This type of environment that will deal with the amplification translated from somatic human perception does not need to have a realistic conception. The conceptions will be connected to the objectives that one plans to achieve.
A good book transports its reader to become immersed in an environment that perhaps in a hyper-realistic one does not. The presencing of the imagination occurs as of a fluidification and identification between the interactive community and the information and syntaxes stored in the collaborative mental extension. The comprehension, familiarity and utility of imagination for the ends proposed is what will count.
We are standing before a new horizon of exploration of these new perspectives, initially with introductory experiences in several points of intersection: visual-interactive, visual-sound, sound-interactive, linear-non-linear, ordered and random. We will have the limitation of an incipient language, still poorly developed, of the present software and hardware interface and of slow speed of most of the internet connected networks. However, what exists nowadays is more than enough to implement practical projects using these ideas.
IV – Imagination and Dynamic Ideography – close concepts
The concept of dynamic ideography13 proposed by Levi is a concept I consider very close to the one I am proposing and, in a certain way, overlaid. According to him, “Dynamic ideography is a way of writing that is demanded by contemporary technical support. It functions in following the principal of figurative and animated representation of mental models, especially by replicating phonetic language on a visual plane, like the alphabet.”
“Among so many uses of dynamic ideography for scientific research, education, training and group communication, leisure and diversion, who knows, maybe a still unknown art form is to be born from a new cinema-language.”
As we can see, Levi proposes a new form of communication constructed in computer that manipulated kinetic images, thus creating a new kinetic-visual alphabet. He proposes an artificial imagination.
He comes close to what I am proposing here, discussing in a brilliant fashion several characteristics of this new language, naturally similar to many that I propose here. Nevertheless, his concept is too restricted to the substitution of a textual, static language by visual-kinetic language in computer environments. This dichotomy between text and visual, static and kinetic, permanently recalled in comparisons between the advantages of the visual system in relation to the textual, impeded him to deal with the question in a wider form, as I intend to do here.
What is sought after is a more ample, integrating plane. If we shall deal with kinetic images, making representations of mental models, then we have an integrated set of language and sense. More that becoming this or that vehicle of preference for expression of all ideas, thoughts and feelings, we can comprehend that we have all these languages and sensorial entries-exits at our disposal to store and communicate, similar to what occurs in our mind. We are not dealing with one or another means. We are dealing with all forms. It is evident that each image will demand an appropriate combination of forming blocks, whether they come from phonetic, textual, visual, smell, etc. And this dosage and this amalgam will become the important elements in the development of a new imaginative language.
So, I would say that the writing of contemporary technical support may function according to the principal of metaphorical representation of mental models, replicating textual, spoken or written language, kinetic or not and a visual and audio language on an imagetic plane, not just a visual one. On a plane where images are multi-sensorial caches integrated into a new language.
I cannot fail to mention in this brief exposition to include explicit mention of constructive blocks of images temporarily outside of present-day computer environments, such as touch, smell, self-awareness, etc. We do not yet have computer interfaces disseminated from and for these senses. Some of these things already exist in experimental environments and virtual reality. Videogames have already demonstrated that it is a question of time before we are also working with these elements in our everyday encounters with PC’s.
V – The Imagination and Video & Cinema – What is changing?
It’s funny… it’s as if we have been waiting too long for PC’s to begin to support visual and sound image in movement. They already can and we are dumbfounded by it all.
A multimedia video application began to occupy the window, usually set off by a button, preferentially a button with a movie icon. Only now computers have more resources for processing and videos can be larger, even occupying the entire screen. Additionally, it is possible to play several videos simultaneously. It doesn’t make sense to play videos in windows, as if something apart from the rest of the screen.
Kinetic audiovisual content in imagination is part of the interface, of a much larger set of language elements that interact in an even larger space than the screen. The video interacts with information in other media and now can become the central piece of the interface, in what I call video-anchoring. Video-anchoring refers to the video’s positioning as the central element of the interface, a role that up till now has been assumed by text and static graphic screens. The visual and sound image in movement, which is integrated and interactive, has a temporality that ends up impacting the relation of the information with people and with the software design.
Video making up part of the interface also makes it become kinetic. When I concluded my first project using video-anchoring, in the CD Rom Discovering Brazil 200014, the idea clearly came to me the information is no longer frozen in time, waiting for us: it is contained in the imaginetic flows that make up the application and in its relation with the public.
The inclusion of interface temporality is very interesting, because the video must play, and in this sense, it helps to breach the present structure of the menus and tools, which are always available for immediate access. We can accelerate routes, move forward, backward, to the side, jump, etc., but in the interface we are always confronted with the need to wait some time to arrive or leave from and to some place. The information moves with the flows.
And, as syntax and the information are not necessarily only in the video, a space for video-illustration, audio-illustration, video-anchoring and the crossing of several visual images and sound flows with other elements on the screen, such as texts, photos, drawings, animation and diagrams, menus, consoles, etc arises.
Little has been done when dealing with non-linear video production and with the possibility of on-line editing, such as changing cameras and block sequencing. These videos create the need for new procedures, both at the time of capturing and when editing scenes, as well as when managing exhibition. We further have to think of “leafing through” videos, like moving them backwards and forwards or to the side. And I really think the idea is antiquated and not very practical to transform the screen into a replica of a conventional editing station or into a remote control like today’s media consoles.
Thanks to the possibility of programmable movements via software, we are at a new frontier in the visual arts field. Although they deal directly with movement, the cinema and video, by the fact that they work with takes in seconds, chained in sequences of minutes and hours, require the continuous presence of the spectator in the showing room. They are not able to explore in real time some of the fundamental movements of life, which are perceptible only through the comparison of “static” states through memory.
Imaginations can move themselves, transforming themselves at speeds the eye cannot follow instantaneously. They have a “life” of their own, independently of the presence of a spectator. They are modifying themselves and presenting themselves to whomever is connected at a given time. Thus, we have an opportunity to explore time and movement concepts heretofore unimaginable in terms of artistic representation: we can explore the dominion of imperceptible movements of catastrophic or gradual duration, infinitely small or large. Programming that lasts for hours, days, seasons, years. It is possible to conceive of works that have life and that, when information is erased from the computer, may die, like landscapes that lose their leaves in the winter. The work is in movement, whether it is perceived or not by whom interacts with it at a given moment.
The imagination also frees us from the single timeline of cinema and video, where a video channel and another audio channel (further divided into 2 or 4) follow synchronized timelines. At the same instant, in the same imagination timeline, or in an individual timeline of some image, the elements of video and audio can be however many there are and be aligned among themselves or not. Furthermore, these alignments or un-alignments can be programmed in the syntax of the software. In fact, we are dealing with audiovisual entries and exits in a new manner. The images of our mental extension have new and interdependent autonomy that impact in their audiovisual emanations.
Here, the concept of sound landscape can be extended to mental landscapes, including visual elements also. We have to take the concept of being in seriously and no longer consider people as spectators the see and hear what someone recorded for them. The imagination deals with environments, universes and not with stages, screens and audiences.
After all, what changes in an environment where the sources of a visual image and audio emission have autonomy and interdependence among themselves?
Let us think of a room in the physical real world with a TV on, a child on the sofa watching it, a person waxing and polishing the floor and another person reading a book. The person reading listens to the polishing machine making noise, hears his heart beating – if he is paying attention, listens to the child’s sounds, the book is being handled and the sound the TV is making. One may see the book page, the TV, the child or the person waxing or close one’s eyes and see nothing.
Let us then use this room as a metaphor in an environment we will create, from something that can be methaphorized in our collaborative mental extension. In an imagination the waxing machine can be disconnected or even moved outside the environment, we can focus on the heartbeats or not, we can show the images on the TV, the waxing machine, the child, the book, of what the person who is reading is about to think, simultaneously or otherwise. And these elements are all kinetic and also interactive among each other. This means that the TV interacts with the child who can change channels and with this change the images it produces, the waxing machine interacts with the person moving it, the book interacts with the person reading…
In the cinema or in video, we watch that which is recorded in this environment, with a reading that was done from the edition of the creators of the work. In an imagination, we watch visual and sound impressions resultant from interactions of the elements in the environment and among them and you and among you, them and other members of the community. I have no doubt that audiovisual language is about to be transformed enormously.
VI – Imagination and Interactivity – the end of response stimulation
What has interactivity been in most Internet and CD Rom software? By clicking a button, giving the user immediate access to the information he desires and allow him to customize the software by a resource called “skin”. This notion of interactivity, which in fact makes the user increasingly passive, is not dissimilar to a simplified model of a reproduction of interactivity in the fashion of Skinner’s stimulus-response model. And what is worse, it is imbedded with the false impression of interactivity, since the connection between information contained in the software and the user have already been previously defined.
One of the first measures in the design of interactivity of an imagination is to throw out the idea that the click of a mouse, following by navigation or execution of a task, is the preponderant interaction of a digital work. We need to liberate ourselves from the curse of the button, a materializer of a compulsive and instantaneous desire.
In an imagination, the design of interactivity is fundamental. It is necessary to define how it occurs among images, itself and the communities, and the relations that we can create among the communities. It is time to seek out new forms of interaction, in navigation and in the relationship between the users of software and the information. It is necessary to include interaction with time – create maturity cycles, catastrophic movements, gradual movements. Interaction with random events. Interaction with negatives – reality molded around negatives. Associative interaction, crossed interactions or those based on potential.
And nothing is better that to seek ideas of mental metaphors. After all, the neurons guard connections with even hundreds of thousands of other neurons simultaneously, forming groups in the shape of networks, exchanging information, dealing with potential changes, creating circuits to generate short term information, undergoing modifications to deal with long term information, carrying sensorial information and translating and connecting feelings and thoughts.
As simple as they may be, the new interactivity models developed from these new premises will be infinitely richer that what is available today.
VII – The Author/Creator and the Reader/Spectator/Public – Categories in disuse
Because of everything said previously, it no longer make sense to speak of author and spectator when dealing with imagination works. They are open works by their intrinsic nature. On the other hand, it wouldn’t make sense to de-categorize people without re-categorizing them when interacting with this re-contoured work.
So, let’s talk about them. An imagination needs minimal conditions to begin to exist and, later on, to continue expanding. To begin with, it is necessary that someone propose the work. The proposer, person or group, is the one that starts of the process of creation of the basic initial conditions of the imagination work. An already proposed work will be signed up in a computer system’s bits where it is accessed by the communities.
The public, reader or spectator, I call communities because this word synthesizes the new position of the public, which, besides receiving information, interacts with the works, where its influences and participates in its configuration, in feeding it data and in its development. Each community interacts with the others and among themselves and with the work. It is important to note that the same person can take part in distinct communities and that these communities do not have rigid limits.
A specific community basic for the existence of a work is the maintaining community. A work with maintainers continues existing. Like any living organism, an imagination needs financial resources to remain filed in databanks and maintain connectivity. Like any organism, the work tends towards entropy and chaos if it does not periodically receive content maintenance, software, etc. The maintaining community is also fundamental for the maintenance of the work. And finally, because of its mutant, kinetic and interactive nature, an imagination needs constant implementation to make sense of its existence. An imagination without maintainers tends to die due to unfeasibility, making it chaotic and inoperative.
The other communities have varying levels of access and capability of changing the work, defined over the development of the work. In function of their interests and characteristics, these are the communities that read and edit the work; without them, it would also “die”, this time, because it would be forgotten. In the case of not making sense, an imagination becomes one more of the billions of uninteresting documents today available on the net.
What have been called open or interactive works are works where the author abdicates his control over content of the work in favor of the reader, which also does not have a defined form, since it is a mutant work in progress. That is how imagination works are. Nevertheless, the distinction between authors and readers does not matter – nor does the loss of control over the work by the community of maintainers and proposers.
The imagination organizes itself like a network, where the system in some way needs to validate its users. After this validation, the user qualifies oneself to frequent the work, obtaining permissions that go from one possibility of interference in some sections to some people and communities to interference in all sections, affecting all groups. What one desires is to create premises of relationship that allow for shared creativity, without leaving the work drift into a completely chaotic state.
Using our mental metaphor, it is desirable to have much change and interrelation in parts of the work, but that other parts have a slower pace and more restricted access. To the contrary, it would tend to be completely disfigured. A work that succumbs entirely to the control of whomever uses its does not communicate. If there is some type of “id” place there, talking with users, the work will go away.
On the other hand, it also does not make sense to create works with rigid controls in an environment resistant to mutation, change of state, etc. The relation among possibilities of access, edition, additions and deletions of images and environments make up part of the new syntax that arises. Each imagination presupposes its optimum level interactively, also mutant, of levels of access to the communities.
Imagination has an editorial trait to the measure that the communities and individuals may modify aspects of content, diagramming, syntax, etc. in the work. This characteristic brings to light an important interaction that had been lost with the appearance of the press and later on of mass communication media. With these media, a condition that limited the works was created as of the process of publication.
Imagination works can be reproduced in large quantities without ceasing to be plastic, enough for them to be modified by whoever uses them. It is important to emphasize that this retrieval brings with it novelty allowed by technology: the computer can register the versions of the work while it is being modified. The registry of the mutation can be preserved.
VIII – Poesis – An ongoing experiment
Having made clear that what is in course is the development of a new art form with its own language, which incorporates and modifies others, but which is still incipient, it is necessary to experiment and create new amalgams of language elements that will become consolidated later on as elements of imaginative language. To help cement the structure of the new language, I looked at the existing ones for that which had more affinity with the new one, with the unmapped one, to supply syntactical operational models to be used as the backbone of an imagination language.
I opted for poetic language. I proposed to myself that I use the term in its widest meaning, not only that restricted to literary art. It is because that, in spite of the consensus that poetry is an art of writing, I observe the term frequently being used in connection with other arts. Such a poetic image… the poetry of a musical phrase…. of a character… The term is also used in many everyday situations without a direct connection with text where the term is used with a connotation that makes us believe it is more of a “mental state.” It is this amplified meaning connected to a sensitive, creative “mental state” and its syntactical operations, including not only those of text, but others as well – imaginetic ones – to which it adapts itself.
Over the last few years I have sought to give poetic language the status of a language that is an articulator of languages in the creation of a new imagination language. I tried this path because I consider that poetry is an extremely suggestive language form. It is inaugural in the sense that it recombines language codes with extreme ease. This faculty of rupture and divergence, this initiating strength, always makes the poetic text reach the imagination with great intensity to whoever has access to it. This has resulted in the use of poetry in almost all the sacred processes of the past and in the processes of rebellion and contemporary social advances.
I do not say that poetry cannot be reproductive and conformist; it is like that, too, when the author or reader wishes it to be. However, I sustain that poetry indisputably has a very large potential for innovation embedded in its formal construction characteristics. The facility that it has to incorporate visual and audio elements of language, the facility for recombination, translation and re-codifying languages and fusion of images diverse origins is part of its essence.
As a language form, poetry easily invokes mental images, leading to rearrangements and internal re-creations, giving life to the imaginary element, bringing it new codes and freedom of composition. The opening up of possible language combinations, the silence between the images, the formation of semantic sets, all of this contributes in a decisive way for this characteristic to be the base for the creation of a new language.
Why not use this potential to vigorously articulate the integration of existing languages into a new creative art form and circulation of imaginations?
IX – The Imagination Site – A mental extension that constitutes itself
The Ciclope Imagination Site is a work that was inspired as a laboratory for the development of the concepts laid forth here. It would not make sense to be elaborating these rationalizations if there were no environments to be practiced in, creating a desirable exchange between thought and action. The Site is a collaborative mental extension that has existed in my mind since May, 2000, after years of experiments in the construction of sites dealing with digital art. Afterwards it was launched in June 2002 on the Internet at www.ciclope.art.br with versions in CD Rom and intranet. Since then, a digital environment has been under construction with permanently mutating organic characteristics.
The site experiments with the concepts of adapted imaginations to the limits of present day networks, especially on the Internet and to the present state of hardware and software of home computers. However, taking into consideration that the work is alive and is under ongoing development and is also evolving with the media where it manifests itself, it will be easy to understand it in its present state and see its potential in the near future. As a matter of comfort, the site is fictitious. It is through fiction that I can realize the task of scrutinizing reality with greater levity. In a world where the mind is still confined, controlled and many times untouchable, fiction frees us from the dilemmas of truth and lies and projects us into other spheres of wider, more fluid understanding.
The site is of an indefinite duration and is updated monthly to facilitate the visiting routine of the communities it is forming. Its functioning is intuitive and its interface contemplates several new tools to deal with the imagination in digital media, taking into consideration prevailing restrictions. The objects on the screen are draggable and drag themselves, are amplifiable, rotate, become transparent, duplicate themselves, disappear, can be manipulated and edited. These objects are called images and form amalgams of audiovisual, textual and graphic elements indistinguishably among themselves, as I discussed in this essay.
Video and audio illustrations, multi-linear sequencing, multi-flows, language fusion, mental states, focus, consciousness and unconsciousness, worlds in movement – there, these concepts that we have gotten to know are articulated in a series of experiments of interactivity and graphic organization in constant evolution.
1 To finalize this essay I had critical input and suggestions from Maria Aparecida Moura, Matheus Braga and Luciana Tonelli.
2 The texts that point to the trajectory of the ideas exposed here can be found in www.ciclope.art.br/memoria/textos (portuguese).
3 Multimedia: although the term is also used to describe works of art and shows where two or more languages and expressive forms co-exist, here I use the word to denominate a new form of multi-sensorial and interactive communication in which known media are integrated and managed by the computer that is transformed into a communication vehicle.
4 The interface realized translation operations, establishing contacts among heterogeneous media and works as a ticket operator. A human-machine interface is a set of programs and apparatus that allow for communication between an IT system and its human users through entry-exits means.
5 The metaphor Desktop was developed by Xerox and later on gave origin to the Macintosh and Windows operating systems.
6 Virtual Reality refers to technologies or ambientations that provide realistic leads to some or all senses, sufficiently enough so that the participant suspends his disbelief.
7 Vannevar Bush – As We May Think, Atlantic Monthly, 176. v.1p.101-108, July 1945.
8 António R. Damásio – O Sentimento de Si – Publicações Europa América ltda – Portugal – 2000.
9 Walter Benjamim – Magia e Técnica, Arte e Política – Editora Brasiliense – São Paulo, 1994. Origem do Drama Barroco Alemão – Editora Brasiliense, São Paulo, 1984. Para Ler Benjamim, Flávio R. Kothe, Francisco Alvim Editora, 1976. Benjamin presents the concept of Constellation, in opposition to the concept of System. Constellation indicates a relationship with dialectic characteristics where the parts acquire a more complete sense in the whole they are constituted in without subordination. His considerations about the role of the narrator were useful in the construction of digital works narratives, serving as counterpoint to the pure and simple rupture of linearity/temporality and the trend for the creation of mosaics with unarticulated elements, seen in much present-day software.
10 António R. Damásio – O Erro de Decartes – Companhia das Letras, 1994.
11 Murray Schaffer O Ouvido Pensante – Editora Universidade Estadual Paulista, 1991.
12 To read more about Hypersurfaces, I recommend reading of HyperSurface Architecture (Vol 68, 5-6/99, profile 133) and HyperSurface Architecture II (Vol 69, 9-10/99, profile 141), published as part of the series Architectural Design (AD) by Academy Editions, John Wiley & Sons., London. In them theses by Stephen Perrella are exposed, one of the main researchers in the área. The two title of the magazines are available through www.amazon.com. The text Cyberarchitecture: virtual architecture beyond real space metaphor, by Ana Paula Baltazar dos Santos, The Bartlett, University College London, UK is another good research source.
13 Pierre Lévi – A Ideografia Dinâmica – rumo a uma imaginação artificial? – Edições Loyola, São Paulo, 1998.
14 Descovering Brazil – an interactive video interface – presentation text of the CD ROM Descobrindo o Brasil 2000, at Prix Moebius -International Multimedia Festival, at George Pompidou Center, Paris, 2000.