The road to immersive audio visual art has been a wavy process, quite literally.

In the fall of 2023 I had finally done it, reached a point of complete burn out. I was sitting there unsure of my day job as an Architect, unsure if my music blog was going in the direction I had hoped, and also had ended a year long relationship that sort of crashed and burned out of nowhere. Looking for a silver lining I found that there was a real opportunity to make a change, to turn things around and really go for my dream.

Being someone who has many different interests, whether that be Art, Architecture, Music, Live Performance, Product Design or Psychology, finding a career that encompasses most of them is always challenging. Maybe you can identify with this, sometimes one field of interest can be quite limiting.

The moment when everything became clear to me was through a moment of inspiration. After watching Dave & Gabe’s Touching Sound: Building Interactive Music Installations video on Youtube, where they break down their approach to developing musical structures and environments. Little did I know there was a career where I can use all my skills, experiences and passions in one area of interest. I thought, “you’re telling me I can use Rhino 3D, Grasshopper, Blender, Ableton and Touch Designer all for one project?”, the answer was YES!

As I embark on this new journey that I set out within the new media art world, it is important that I start with understanding WHY I am doing this, and the process of doing so, illustrate to you some of the valuable principles and knowledge from my experience in Architecture and Music Production over the years.

After all, starting with a strong WHY provides a solid foundation to overcome any HOW.

5 Architectural Principles for Immersive Audio-Visual Art

1. Use of Space

One of key skills during Architectural planning and exploration of design development, use of space and spatial study explore the various options for maximum impact of your concept. Architects are the masters of designing environments and look at the process and spatial qualities from varying depths of scale. This is a super useful transferable skill in the world of new media art and immersive experiences, as we can design our environments and produce a strategy for user experience.

Later in the process we can set out the arrangements for speakers, screens and projection map surfaces to align with the stage, or main centerpiece of the exhibition. The nature of having accurate CAD models to work from allows use to precise and produce quality installations that allows the user to be fully immersed without unwanted defects or distractions in the mix.

In Rafael Lozano Hemmer’s work at Gray Area we see lighting arrangement in a parametric topographical form that would most likely be model in a software similar to Grasshopper as the geometric form would need to be incredibly precise when setting out these mathematical non-Euclidean geometries.

2. Materiality

Materials are like the colour palette of an Architect. They can dramatically transform the look, feel and acoustics of a space. They are an incredibly important factor in the design process and should be carefully considered at the earliest possible stage of design concept.

Reflective, translucent or textured surfaces can be used to manipulate the behaviour of light and sound within a space. Aesthetics are not the only thing that changes our perception of a moment, as you can imagine a concrete space to have a very different feel to a wooden room!

However aesthetics do play an important role. For example light itself can be powerful material, just take a look at James Turrell's work where it is clear why he has been coined as ‘the master of light’, as he plays with colour, diffusions and contrasting exposure in a pure and minimalist way, presenting light as the only medium.

3. Projection Mapping

How can we project a rectangle onto a triangle? Project mapping allows us to widen the range of objects that we can project our visuals on to in the real world. This allows freedom to project onto irregular shapes or even turn regular shapes into the irregular using optical illusions.

Projection mapping is essential for the transformation of spaces, and to ensure that your artwork is projected onto canvas or object in the correct proportions. The environment can be modelling in architectural 3D software to ensure that the geometry is correct before installation.

4. Acoustics

We touched on this earlier with materiality, as one of the key elements of good architectural design is acoustics. Not just in materiality but the angle, diffusion and density of the surfaces plays a role in ensuring the space is calibrated for the right levels of reverberation and sound design.

The use of existing buildings like churches can be utilised for echo and reverb, however acoustic treatment should be installed to dial in the return tail end of the reverberation, to make the sound more clear.

5. Scale and Interaction

Psychology is also an encompassing element of architectural design, demonstrated with the user interaction and experience of the space. Putting ourselves in the mind of the user is crucial to understand what experience will best impact the user, this can be done with touch-sensitive walls or motion detecting sensors to really immerse the user.

Scale is another factor that plays on the optics and psychology of the user, as installation can be monumental in providing an intense or intimate experience to hit a more personal note.

I’m glad my Architectural skills will not go a miss in this new journey that I will embark on.

3 Music-Production Principles for Immersive Audio-Visual Art

Music has always been a passion of mine. Some of my fondest memories are of playing shows and jamming with friends. My music journey began when my friend approached me one day between classes and said “Hey, you played bass that one time in class, do you want to be in our band?”. I didn’t even have a guitar, but I said “Yes, definitely”.

The next day I went to PMT, the music store in our local town, and bought a bass guitar and joined the band. I then later immersed myself in guitar, vocals, piano and later years experiencing using synthesiser. I always loved the creative nature of live performance and energy from the crowd.

This interest in music production and live performances mixes well with the immersive audio visual art world, especially because they are in a sense live exhibitions with crowd interaction. I want to bring these skills to the immersive world to bring people together at these exhibitions.

Touch Designer has a built in feature that allows connectivity with Ableton Live, which is perfect to bring these two worlds together. We will be looking at the various principles of music production, theory and sound design that can be used in the immersive experience and audio-visual art world, in the aim to highlight the importance of this field of interest.

1. Spatial Sound Design

Panning, EQ, Pitch and positioning of speakers can all impact the 3D soundscape to ensure the user is fully immersed in the exhibition.

With panning you can play around with the width and breadth of a soundscape, as each frequency will have more space to fill. The problem of having too much centered to the default can lead to frequencies being too muddy. This is the first step to creating a more 3D sounding atmosphere.

Using EQ to muffle or bring out bands of sound can have dramatic differences in audio experience as unwanted frequencies can be dialled in or out. It’s amazing how much EQ can transform the sound from something unusable to something pleasurable with simple EQ..

The detuning of sounds can emulate the natural physics of sound in proximity. Think of a car whizzing past where the frequency of the sound when closer will be higher than when in the distance, thus giving off different pitches at different distances, this is known as the Doppler Effect.

2. Texture

Sound, much like smell, has an ability to take you to a place in your memory. It’s the textures and imperfections within a sample that gives it distinct character and sense of place. Pure music notes are unfortunately boring and can often be played down out by wave lengths that have varying degrees of harmonics to take up and fill out the auditory space. This makes it extremely useful when contrasting and comparing different elements of sound design to produce a composition that works.

Noise can be represented in image and sound, this can provide us with opportunities to pair auditory candies with visual delights.

3. Musical Mapping

Understanding fundamental elements of music theory or music performance is essential for immersive experiences. Artists may want certain musical keys represented in different colours or contrasting shapes for different elements of the song. How the beat, melody and frequencies all work together harmoniously can dramatically improve the experience for the user, as the visuals and the music should be one synchronised resulting entity.

Ableton Live is a powerful DAW (digital audio workstation), and these all can be connected to TouchDesginer through CHOPs and set up. They can be set up using MIDI channels where each note can be edited before or after, retaining all the note information – powerful, yet CPU heavy; or you can set up the simple audio channels and then read the information in TouchDesigner. This is a much lighter, efficient and reliable set up, however it comes with the drawback of not being able access nearly as much information.

What Are My Aims For This Website?

New media art can be a powerful tool for getting different people who wouldn’t otherwise connect with each other closer together. The interactive nature of new media art and immersive installations has the power to engage people using technology as the medium.

We are all so connected to our phones and “plugged into the matrix” if you will. I want to create new media art that doesn’t just focus people onto the artwork. It should be a tool for the community and engagement. A back seat driver of networking, rather than just a focal point of sensory delight.

Additionally this website provides me with a platform to help other aspiring audio-visual creatives to learn as I discover new and intriguing techniques in the new media art world. Whether you are just starting out using TouchDesigner, Ableton, Python and more, we can grow a community with like minded people that can help each other create amazing installations and experiences for our visitors.

Finally through growing this site I would like to turn this home experiment into a fully functioning brick and mortar studio, with a like minded enthusiastic powerhouse of a team, to bring a fresh perspective into the new media art world, and be up there among the great new media art studios like teamLabs and MeowWolf.

Let’s bring people closer together through the serendipity of human interaction, with the injection of technology as a medium to create a meaningful experience.

Why I Became an Immersive Audio-Visual Artist