Altering Human Sensorium

By Andrey Mir on September 26, 2016 • ( 1 )

Artificial flavours, augmented senses, immersive media, augmented reality, virtual reality

By shaping the media environment, media are able to tune the human sensorium according to their “bias”. Equipped with ideasthesia/synesthesia, the sensorium follows the environment. In its turn, thanks to neuroplasticity (and ideasthesia/synesthesia), the sensorium is able to adapt humans to any media environment. Media always probe the sensorium; the sensorium always adjusts in order to unfold all capacity of media, and reach their limits and their demand for a new experience. This interplay between the sensorium and media lies in the foundation of media evolution. In the process of adaptation, for the sake of better experience, the sensorium sooner or later employs all capacity of any new media. <…>

The represented reality of literate media has already freed human beings from physical reality, yet just symbolically, in the human imagination. The induced reality of digital media can capture humans without any use of their imagination, literally, as a surrounding environment.

On its way from the given to the represented and then to the induced, media evolution has to modify the sensorium, first on the foundation of natural “likeness”, then, in some other way, under its own laws. In this context, we can search for some indications of enhancing “natural” senses and then of transcending them (as this metaphor was used by Ray Kurzweil in the title of his book The Singularity Is Near: When Humans Transcend Biology).

Here is a possible list of such improvements of the sensorium by technologies. The list is not complete, but is indicative.

Signs of altering of the human sensorium

Artificial flavours
Augmented senses
Immersive media
Augmented reality
Virtual reality

Signs (or predictions) of transcending of the human sensorium

Angelism and dismissal of gravity
Navigation in the digital space: from physical to social dimensions
Social dimension: the sense of others
Social dimension: the thirst for response

Artificial Tastes, Artificial Smells, Artificial Sounds

Technologies in culinary work, perfumery, and music have aimed to evoke enhanced sensations. Any attempts to improve natural sensation, in fact, have been leading to the creation of artificial substitutes. Certain logic can be found in such a tendency. Natural tastes, smells, or sounds are too regular, too indistinguishable for distinctive sensory experience (that most often can be described as pleasure) to be had. Strange as it may seem, purification of sensations has always had to do with artificial stimuli.

The history of nutrition gives us a good example. People are capable of processing food before digesting. The ability to cook is one of the traits that differentiate humans from animals. Throughout the entire course of history, by purifying the taste sensation, humankind has been trying to obtain flavours that do not exist in nature. Flavour additives and enhancers also make food cheaper and more storable. But initially, they aimed to make food tastier. Historically, salt and sugar, along with a huge variety of spices and condiments, played precisely the same role as contemporary artificial flavours: to improve and enrich the taste qualities of food, simultaneously having made it, in fact, unnatural.

Media ecologists should pay particular attention to the phenomenon of artificial flavours. “Old” and “new” flavours invisibly reshape the environment pretty much in the same way that media do. For example, a marketing trick with “tomato-flavoured potato chips” aims to recall the natural taste of tomatoes as something valued. It makes sense for those familiar with the original tomatoes’ taste, but makes no sense for the many children who have simply not been made familiar with the taste of real tomatoes. Moreover, if they happened to try a real tomato, they would recognize its taste only because they are familiar with an artificial tomato flavour. The taste enhancer absolutely detaches the reality given to us in induced sensation from “the reality given to us in sensation”. Only one question remains: why do we still need tomatoes? The enhanced taste still relates to the natural environment, but with decreasing necessity.

The same analysis can be applied to the millennial efforts of people to purify, enrich, and enhance smells. Fragrances of all sorts and fresheners of all sorts aim to improve the perception of surroundings. They act absolutely similarly to artificial flavours: being, in essence, unnatural, they fake some natural properties and eventually withdraw the human sensorium from the natural environment into the “better”, induced environment.

The development of the “use” of other senses can be analyzed in the same media-ecological way. For example, all smartphones are designed to produce a clicking sound when taking a photo. This sound obviously imitates the noise of the mechanical shutter in the old types of cameras. The clicking smartphone is a “mechanically flavoured” digital device. However, today’s majority of smartphone users have never used a camera with an actual mechanical shutter. For them, this sound means nothing except the sound of a smartphone taking a picture. This is another example of the continuing detachment of our sensorium from a “natural” environment.

In a certain meaning, similar to artificial flavours, music and poetry have been developed to purify specific human sensations. In this case, it is the sensation of the others, experienced via sounds. Primitive rhythms were used to coordinate people’s locations in space and people’s collective efforts in time in the era of hunting-gathering. Rhythm lies in the foundation of group cohesion. It is not for nothing that McLuhan, when speaking about the capacity of radio to reverse humans from individualism to collectivism, compared radio to the “tribal drum” [7] (Chapter Radio: “The Tribal Drum”).

Nowadays, precisely like artificial flavours, most sounds produced by people and sensations induced by these sounds have little to do with the natural environment. People now live in a constantly collapsing audile space, whose implosion shapes a sound cocoon around everyone. The state of alienation experienced by an individual with ears corked by earphones makes this audile cocoon almost visible. Earphones drastically increase the amount of time spent by one in the induced sound environment, which aims both to alienate and to please. Another significant trait: while detaching people from the physical surroundings, the audile cocoons attach their inhabitants to one another in the induced reality of music, radio, and phone conversations. The reality of an individual cocooned by earphones is a space that is physically individual but virtually shared.

It is quite safe to say that humankind has always been seeking ways to induce better sensations. The contemporary trends of consumption of organic or natural goods reflect some fears and some resentment, but in general, the induced has always been perceived as something more valuable (enhanced) than the given. Such speculation may be concluded with the thesis that our entire civilization is the movement from the natural to the artificial, which means from the given to the induced. This movement was drastically boosted by the introduction of electricity, which promised to become the main supplier of sensation.

Augmented Sensorium: Artificial Senses

During World War II, the Soviet neurolinguist Alexander Luria was the head of a neurosurgery evacuation hospital. He treated hundreds of brain-injured soldiers. In particular, he was working to invent a method of rehabilitating patients with dynamic aphasia, who were unable to deliver utterances sequentially. Luria forced them to pick up cards sequentially, which through exercise gradually restored their speech ability [19]. This method shows how a verbal function that is lost because of injury to one brain region can be compensated for and then restored by the training of another brain region, which is thought to be initially responsible for physical, not verbal, activity.

Luria’s invention shows that disrupted brain abilities can be compensated for by the activity in other brain regions. The same is applied to “disrupted” senses. This is the gift of neuroplasticity represented, at the level of the sensorium, by synesthesia. As Nicholas Carr put it,

Thanks to the ready adaptability of neurons, the senses of hearing and touch can grow sharper to mitigate the effects of the loss of sight. Similar alterations happen in the brains of people who go deaf: their other senses strengthen to help make up for the loss of hearing. The area in the brain that processes peripheral vision, for example, grows larger, enabling them to see what they once would have heard [18] (p. 25).

Today, gadget developers try to exploit the phenomenon of synesthesia in order to help people with disabilities. For example, for visually impaired people, a device has been developed that can transmit the spectrum of colours and lighting around a person, along with spatial orientation, into the mouth cavity, by means of a lollipop-shaped device, and using slight electric stimulation [20].

Slight electric stimulation can be used not only for the compensation of impaired senses but also for inducing senses that we are not certain are or were inherent to human beings. German scientists have developed a new device, the feel-space belt, which allows the wearer to feel the Earth’s magnetic field and be oriented in the four winds, just like birds and bats are [21].

In another case, a Spanish avant-garde artist, Moon Ribas, has gotten a subdermal cybernetic implant that allows her “to feel” every earthquake on Earth in real time. In fact, the implanted device just receives “data from a custom iPhone app that aggregates seismic activity from geological monitors around the world. She describes the physical sensation as akin to having a phone vibrate in your pocket. The stronger the quake, the stronger the vibration”, the report says. Ribas’s new ability is called “the seismic sense” [22]. Ribas uses it to perform a dance piece called “Waiting for Earthquakes”. She stands on stage and waits for an earthquake to happen. Then she dances according to the vibrations she feels.

So far, such manipulations with the sensorium do not amount to a truly new sense. The feel-space belt just transforms the magnetic currents into vibrations that the body can easily perceive; the seismic implant does the same. In reality, the devices produce just a cognitive effect induced by the physical impact on receptors of the “old” sense, which is tactility. It is safe to say that this transition of meaning of one “sense” via the other sense is symbolical. It requires time and effort to recognize and learn the “content” of the signals, while the real, natural senses are immediate for perception, as they require no symbolic interpretation.

More interestingly, these experiments allegedly restore to humans the senses of the magnetic field and seismic activity that are presumably inherent to biological beings. These sensorium augmentations just improve human physiology (the report calls Ribas’ new seismic-feeling ability “a superpower”).

However, electricity allows pushing the boundaries of the human sensorium, or even an exceeding of them. An electronic bracelet called the Pavlok punishes the wearer with a slight electric shock (they call it a “zap”) in case the wearer passes a deadline, or smokes when having pledged to quit, or breaks some other rules established by themselves (so far just by themselves). The device is designed to facilitate the fight against bad habits [23].

In fact, these slight irritants induce a fear of punishment that fosters a sense of guilt. Maybe this can be described as a new type of synesthesia, something opposite to ideasthesia, because in this case, the sensory stimuli evoke the cognitive experience. For now, the punishing bracelet has been being programmed by the owner for certain displays of bad behaviour to get a negative reaction. Becoming more sophisticated, such a device could take upon itself more responsibility in making decisions on what is bad and good for a human, finally ending up in violation of Asimov’s First Law; the subjugation of humans for the sake of their well-being is one of the alleged scenarios of the rebellion of the machines.

The electrical extension of the sensorium cannot but will go further. Moscow engineer Vlad Zaitcev has inserted a payment chip under his skin to pay the subway fare. He also was reported to be planning to insert a bank card chip into his other hand [24]. Zaitcev has become one of the hundreds of today’s real cyborgs [25]. Sooner or later, the development of payment implants has to bring to bionic people the sensory perception of a bank balance. Heating or vibration could indicate the state of the account, similarly to what the feel-space belt does. Then, thanks to synesthesia or because of the development of cognitive interfaces, people may learn direct, not just symbolical ways of experiencing their financial state (or whatever will exist in place of finances).

The acquisition of this financial sense would well correspond to the logic of media evolution. It is the same for others, for now unknown senses, which still have to appear in order to further extend the sensorium in the digital environment.

Immersive Media

In August 2015, the beer company Stella Artois constructed “a big white dome” named “Sensorium” in downtown Toronto. Here is a description of the project:

A multi-course dining experience with beer and food pairings where each dish will be inspired by one of the five senses—sight, sound, taste, touch & aroma. Within our sensorial dome, guests will be immersed in a 360 degree experience, surrounded by video and interactive elements that will engage and amplify all of the senses throughout the night [26].

Brands and entertainers seek to immerse consumers in the experience of artificial reality entirely. Today, 4D and 5D movie theatres (even 7D movie theatres exist) offer an almost full package of sensorium stimulation. “Spectators” are being shaken, touched, blown with hot or flavoured air, poured on or sprayed with water, and moved down and up according to what is going on onscreen. Artfully combined and synchronized, these impacts together create the effect of being present in an imaginary world created by the movie. The effect of presence or co-sensation—this is what arts, or literature, or movies, or media have always been seeking to achieve.

However, 3D, 7D—or speaking precisely, 5-Senses (5S) simulation—is just an exercise in shifting human perception from the real world to an artificial one. This exercise is in an interim stage, which nevertheless shows the direction of media evolution. The 5S simulation still uses the physical stimulation of nerve endings. It is still as biological as in the real world, even though the reality of 5S immersive media is artificially induced. Observing other oncoming digital media technologies, we can say that the time is coming for the stimulation of nerve “beginnings”, not just endings.

Augmented Reality

Improvement of the “natural” sensations, followed by an augmenting of the sensorium with new electrically induced senses, logically leads to the development of the phenomenon of augmented reality. From enhancing and augmenting senses, media evolution has to move toward enhancing and augmenting surroundings. The trend is obvious—the creation of new capacity of the body is not enough, since the creation of the entire world in the digital space has become affordable. Instead of the representation of reality in the human mind, media evolution leads to the representation of the human mind in the induced reality. Video games and social media have paved the way. They insert “a representative” of the user into the reality of the game or social interaction. The augmented reality technologies facilitate this process at the level of the sensorium. As Wikipedia puts it,

Augmented reality is a live direct or indirect view of a physical, real-world environment whose elements are augmented (or supplemented) by computer-generated sensory input such as sound, video, graphics or GPS data [27].

For a McLuhanist, augmented reality can be seen as the further extension of the central nervous system, but with a new, significant trait. For the first time in the entire evolution of sensations’ mediation, the improvement has neither been done on the side of human, nor been attached to a certain sense. With augmented reality technologies, the improvement entirely occurs “on the side” of the reality (or at least somewhere between the sensorium and reality). McLuhan’s extension of the human body starts transcending the bounds of the body and transiting into the surroundings. Everything described before has related to the augmented sensorium; now it comes to augmentation of reality itself.

According to McLuhan, Innis, and other media determinists (even if they rejected this title), any media (better to say mediums) are able to shape an environment, but they do this just metaphorically or via some physiological, social, or cultural impacts. With augmented reality, media starts shaping an environment literally, directly, and immediately. At today’s stage, this is just the addition of some induced objects or data into the picture of the surroundings. The next stage of media evolution, the technologies of virtual reality, combine the immersive media idea of full sensational immersion with the augmented reality idea of digitally shaped reality.

Virtual Reality

Media evolution leads us to gradual resettling from the physical world to the “best” one, which is the virtual one; from the given, through the represented, to the induced. Along the way, media evolution sentences us to be entirely immersed in this new environment with all our five (or more) senses, just as we have existed in the real world, until now.

In the present day, the most advanced technologies that can implement these ideas are technologies of virtual reality. They are already capable of resettling us into the induced world without any real-world “earthing”. As Wikipedia puts it,

Virtual reality, also known as immersive multimedia or computer-simulated reality, is a computer technology that replicates an environment, real or imagined, and simulates a user’s physical presence and environment in a way that allows the user to interact with it. Virtual realities artificially create sensory experience, which can include sight, touch, hearing, and smell [28].

Interestingly, classical dictionaries fail to define fast-emerging phenomena of this kind, relinquishing this function to Wikipedia. People who develop technologies also hardly worry about solid definitions. But even Wikipedia, that tremendous enterprise of collective thought, is not able to cope with the nuances of newly arising technologies. Thus, Wikipedia tries to present the concepts of virtual reality and immersive multimedia as synonymic, which is obviously not the case. Immersive media (or multimedia), such as the Stella Artois sensorium dome or 5D movie theatres, clearly differ from such technologies as virtual reality headsets. To distinguish immersive media from virtual reality, it may be said that the immersive media technologies create the induced reality for the human body, while the virtual reality technologies create the induced reality for the human mind. Indeed, immersive media together with all previous technologies of enhancing sensations induce new sensations of reality by stimulating nerve endings, while virtual reality induces an altered reality by stimulating nerve “beginnings” (almost; the full effect will come into play after a cognitive interface is developed as part of the achievement of a direct mind-machine wiring).

Virtual reality is most often used for play or training. Both of these sorts of activities are aimed at simulating a new reality for which humans should be prepared. In a more abstract and philosophical sense, the virtual reality technologies offer humans training for resettlement into an induced world.