Creating an embodied spectatorship of sound

This interview with Professor Tony Myatt, University of Surrey, UK, continues the series conducted by artist Jana Winderen and Executive Director and curator at IHME Helsinki Paula Toppila. The idea for a series of online interviews came up when, at an early stage of our collaboration in 2019, Jana was thinking about staging a seminar to raise awareness of the current state of the Baltic Sea and other water systems. Instead of flying people in from abroad we decided to share their expertise in text form on IHME Helsinki’s website.

Tony Myatt is a sound artist, engineer and academic who specialises in spatial audio production – the creation of three-dimensional sound projections for sound-installation art, film and live-audio performances. He is Head of the Department of Music and Media at the University of Surrey. Tony’s research and teaching focus on contemporary aesthetics in electronic and computer music; spatial sound reproduction and recording; the composition and performance of computer music and contemporary audio art. For more than ten years, Tony has worked closely with artist Jana Winderen on the creation and presentation of her spatial audio installations and performances. Jana and Tony are currently collaborating on IHME Helsinki Commission 2020 Listening Through the Dead Zones.

Between engineering, science and art

Paula Toppila (PT): What was it that made you study sound and also to work with sound in a variety of ways as you do today?

Tony Myatt (TM): This is going to sound contrived, but it isn’t [laughs]. I grew up in a house where there was quite a lot of music, I studied music, I have three degrees in music and in related areas.

In my early teenage years I started to experiment a little bit with electronics and listening to sound in different ways. I was completely struck by what happened when you sat in the right place between two stereo speakers. I thought this effect was just magical. The sound comes from thin air, there is a position when the head is in exactly the right place where the realism of the sound emanating from nothing is absolutely convincing. And so even back then I had an interest in electronic music, and contemporary music, and I had this sort of technical excitement around sound reproduction.

One of the first things I ever experimented with in my early teenage years, was a technique called Hafler sound, where you can take a loudspeaker and attach it in a particular way to the stereo-amplifier speaker outlets and place the loudspeaker behind you. It presents a difference signal, the difference between the two front loudspeaker channels, which will often be the sounds from the environment of the recording, reverberations; the things that are the difference between left and right loudspeaker channels.

Using techniques like this you instantly step away from the idea that the presentation of a linear stereo image is the only possible way to reproduce sound. So I’ve always had this interest to pursue spatial audio, and I have been fortunate to have an academic career that has allowed me to do that.

PT: Please, continue about your work in the academic context.

TM: I sit somewhere between being an engineer and a scientist, and also being an artist. Now I’m a Professor at the University of Surrey Department of Music and Media, which educates sound artists and sound recordists who work across these fields – have a background in engineering and in the artistic domain. Our sound-recording course is a very successful programme, whose graduates contribute to the creation of music and support many international artists in the commercial music industry around the world. But my interest for a very long time has been to work with spatial audio and the spatial presentation of sound, which is significantly an engineering and scientific question.

I began working with full Ambisonics, which has now become very popular and has been adopted by Facebook, Google, YouTube, Apple and many other companies, because it is highly appropriate for virtual and augmented-reality audio presentation of spatial sound. I began working on this in the late 1980s.

The Ambisonic system was developed in the UK by a mathematician called Michael Gerzon in the early 70s. It’s a very long-standing technology, but it was a thought and an idea that emerged way before the technology capable of fully exploiting it. It lay dormant for a while and I started doing research when I moved to the University of York in the late 80’s. I was very fortunate, as there was a guy who worked in York called Dave Malham, who has been a pillar of the Ambisonics research community for a very long time. I spent more than twenty years there working and developing different types and approaches to the use of Ambisonics.

What became apparent during that time, and also relates to the work Jana and I do, was that there were very important issues relating to human spatial audio perception, and how people understand and relate to spatial audio; improving surround sound wasn’t simply a technological question.

The perception of space as we understand it through sound is a very important part of our engagement with the world of sounds – real-world sounds and those of wildlife, wildlife habitats and our environment. I worked with a PhD student, Peter Lennox, on researching what was known about spatial perception at the time, and as most of the research in the field prior to that time had been conducted in the visual domain, and we had to start drawing parallels with the audio.

We discovered a lot of things about phenomenon, about J.J. Gibson‘s work on the ecology of perception and important concepts like the idea that we humans like to explore things to understand our spatial environment, that we investigate things as a natural mode of perception.

So I’ve spent now quite a period of time, particularly working with Jana and other artists like Chris Watson, to investigate how we can make spatial audio that appeals to our spatial perception mechanisms, is explorable, and has a sense of an engagement that’s different from most people’s experience of reproduced sound. In fact we try to make soundfields as best we can to place the listener in the environment that the recordings were made in.

Rich and diverse underwater sound world

TM: My input to projects like this is quite varied. On the one hand, I’m very interested in the technology and the audio reproduction, but particularly in presenting people with things they haven’t heard before.

I think one of the real downsides about surround sound – because everybody believes it’s a solved problem and a mature field; they go to the cinema and see the badges on the door or on DVDs which say presentations are in Dolby Surround Sound formats. But even with Dolby Atmos, the newest cinema sound standard, the technology doesn’t aim to present a continuously surrounding and enveloping system, because it’s not designed for people to “explore” perceptually, as it has to be screen-focused.

So using contemporary digital-audio workstations and audio tools it’s still difficult for content producers to address some of the elements I mentioned, about spatial audio perception, which I think are crucially important within the type of spatial-presentation methods I employ.

This means that the type of spatial audio work Jana and I create is extremely rare, and there is almost nowhere else where one can experience this type of sound technologies and phenomena of reproduced sound, apart from art galleries, specially designed venues, such as architectural pavilions, and through some sound installations.

Jana’s work primarily focuses on sounds in subaquatic environments. I think it can be quite difficult for people to really understand how sound works underwater, because the barrier of the water surface really means that in most people’s experience, for example, sound doesn’t come out of the sea. We can’t walk up to the sea and hear all of the fish and the animals who live there communicating with each other; they are contained, and operate within a space which is perceptually very distant from our own.

As my wonderful colleague Chris Watson regularly says, we do live with the legacy of Jacques Cousteau’s statements, where he began every episode of his ground-breaking underwater TV documentary series on marine life by saying “I’m now entering the silent undersea world”. It wasn’t that the environment he was exploring was silent, simply that he couldn’t hear it because his ears evolved to work in air and not water.

When we start to record with hydrophones, and as some who fish know from using oars as listening devices, it becomes apparent that the undersea world is an incredibly sound-rich environment, and of course water is a great medium for sound, so sound can travel over huge distances.

One of the critical ecological experiences underwater recording and presentation brings is that through making works like this one, we are able to give people an experience of the richness and diversity of underwater sound. By transducing sound through hydrophones and then replaying it over various loudspeaker configurations, we can give people an experience of underwater sound environments, particularly in spatial terms which cannot be heard in any other way by humans.

Jana and I have done installations where we’ve used sound polluters, too, boats and other things, which have become part of the works ecological motivation, presented in stark contrast to the natural habitats that other animals inhabit. Being able to do that in a way that has some sort of unique spatial presence also means that what Maurice Merleau-Ponty described as a potential for embodied spectatorship can exist; listeners can engage with the thing they are observing, and ideas are communicated not through text, images or written materials “about the ocean”, but through listeners own experience and engagement with the phenomenology of the work itself.

We control lighting to remove distractions, and we present as high-quality, multifaceted, multi-dimensional spatial sound reproduction as we possibly can – which is a combination of many years’ experience working with different types of spatial audio technology and materials. Jana sometimes works with two hydrophones in the water to capture sound, I keep saying: “Use more.”

Jana Winderen (JW): Four, I use four. Two on my feet and two on my hands. [laughs] We have been working together in the whole process and also in the production in the studio, because I work with a speaker setup here. We would early on walk to the place and go on a site visit, and you send decoders to me to work on and simulate the situation in the studio, and then in your studio.

On spatial presentation of sound

TM: Yes, one of the important things is that spatial information is captured when you record sound. This is very important for us to be able to reproduce sound in a convincing way in spatial terms. A regular microphone will take my voice, the sound that comes from my mouth, the sound that comes from my chest, the reflections from the walls around me, and my environment, and reduce it to one point in space at the end of the microphone. The notion that we might take that one pinpoint of sound, put it back into a speaker system, and expect a convincing spatial sound reproduction is expecting a lot.

Retaining spatial information from the very outset, and having discussions about how we might approach this, is a big part of how we work together. We collaborate from the conception of a work, to think about how we will capture and reproduce materials, how these might be best presented in the exhibition space. This dialogue is very important.

Jana mostly gathers sound alone, but we understand the environment we’re recording in; the environment that people will find themselves in when they hear the work. I usually write software that allows Jana to work in her studio to create a loudspeaker and decoding set-up that’s very specifically designed for the space where we’ll ultimately exhibit the work. We collaborate throughout that process and discuss things, and then ultimately we always spend time on site making sure that the balance in the mix and the production is suitable for the environment. That’s often related to things like the acoustics of the room or ambient noise levels, but there are many things that impact on that, I think.

There were some very interesting examples of how we collaborate when we did Rising tide in Oslo recently. Jana had made some field recordings with a Sound Field microphone. She captured a whole spatial scene of tremendous crashing, moving waves, a fantastic recording. Obviously you can’t be in the crashing waves with all the technology and equipment required to record them, you have to be above them, somewhere, somehow.

It was really apparent when we played this recording back on a big 360-degree sound system, more so than in other places where you have played that material I think, that it just didn’t work, and sounded very strange. To combat this, I devised some audio-signal processing methods in software, to take this sound and seat it lower in the room, twist it around, and reorientate its audio perspective, so that listeners had the correct perceptual cues to allow them to perceive the sounds as if they were surrounded by waves.

Some of the exhibition reviewers commented that there was this real sense of these waves moving across the floor and people having encounters with them. Which of course takes us back to that sort of embodied spectatorship idea, and that’s what we’re trying to do. So our collaborations can be a phenomenally complex and varied process, and I think that’s why it takes two of us to realise these works, because they require quite a lot of different skills to support the entire process associated with the creation of a work.

Creating a holistic soundscape for Helsinki

JW: Yeah, I’m just thinking about moving towards IHME Helsinki Commission 2020 and this is again a very new speaker setup we are working with now. There, people will be overlooking the sea. I went on a site visit some time ago to listen and told you about the experience of the space and its acoustics.

Because of Covid, you have not been able to be there ahead of time, but we will spend a lot of time there again in August to work the piece into the environment and the local acoustics of this very particular space.

TM: I think you’re right, every space we work in presents its own challenges and problems and interests. We always try to respond to the site with the work. The works are always mixed on site, but actually all the technologies, and in some cases, as Jana said, sometimes some of the content, is altered in response to the space. In Helsinki it’s a very interesting site because it’s a rowing stadium, and it’s outdoors and it’s overlooking a sea bay, and potentially a site where some of the sounds used in the work occur.

We looked quite carefully at photographs, images and drawings and so on to work out a loudspeaker array design, which would actually utilise the rowing stadium almost like it was still a rowing stadium, so that people could sit and use the space in the way that it was intended by the architects – but to have a spatial audio engagement with underwater sound, not to watch rowing races. We’ve designed it so that listeners will experience the rectangular loudspeaker array of fourteen loudspeakers, which corresponds to the width of the bay that you can see from the stadium.

In all of the things I do with spatial audio production and the way in which I project audio from loudspeaker arrays, in my ideal scenario, nobody should ever sense that the sound is coming from a particular loudspeaker. All of the loudspeakers are designed to act together, to create a seamless soundfield. In Helsinki we’re going to create a seamless plane of sound in front of listeners. The sounds do move, but it will be a holistic soundscape, like a large sound canvas presented in front of the whole stadium.

PT: That sounds amazing, I cannot wait to experience the piece in August and hear our audience feedback. My last question has to do with the relationship between art and science as you are in that interesting position of working in between these two disciplines, how do you see this relationship?

TM: There are always interesting discussions around that, as if these were separate things. But I think in many cases they can’t be separated, they are the same thing. I think some scientists might describe their exploration and experimentation as a form of artistic practice, too. Leaps of faith, inspiration, are common to both approaches.

I think Arnold Schoenberg was right in his assessment of the impact engineers could have on the creative process, when he first encountered what he referred to as Tonmeister sound engineers recording his music. He said that the engineers and technicians around him could have as big an input on the work per se as he had as the composer of it. Because they could absolutely control how it sounded and was heard by audiences. Therefore, having an artistic sensibility as a recording engineer or spatial audio producer is a vitally important thing.

Since I began working in spatial audio, I have seen quite radical developments in surround sound and technologies. Michael Gerzon’s ideas about Ambisonics and Higher-Order Ambisonics are no longer impractical and limited by early-1970s audio technologies. From the late 80s, where there was just about a sufficient amount of disk space and computing power to start to explore multichannel spatial audio, to the point we’re at now, where we’re moving into virtual realities and hundreds and hundreds of audio channels stored on laptops and processed live in sound installations, is an unimaginable leap in technology.

If you’re working in the digital domain as an artist, you have an engagement with the technology in some way. I’m not saying that it’s vital to be a computer programmer or digital-audio engineer – many, many artists are not interested in technology per se, and really focus on what it does – but digital technology is the medium we work through, and so people like me often have a natural interest in understanding both audio technologies and the physics and acoustic of sound; and why wouldn’t that also be an inspiration for new contemporary art practices.