A transcript of Episode 204 of UX Podcast. James Royal-Lawson and Danwei Tran Lucani talk to Aaron Day and discuss Designing with sound.

Transcript

[Music]

JAMES: Hello, I’m James Royal-Lawson.

DANWEI: And I’m Danwei Tran Luciani.

JAMES: This is UX Podcast. We’re in Stockholm, Sweden. And you’re listening in 182 countries from Indonesia to Oman.

DANWEI: Today we’re going to be talking about sound and designing with sound.

JAMES: It’s something that we’ve had on our list of topics for over five years and at last we have a great excuse to talk about as Amber Case and Aaron Day have written a book about it.

DANWEI: It’s called Designing with Sound: Fundamentals for Products and Services and it’s published through O’Reilly.

JAMES: Aaron Day, one of the authors, is an auditory experience designer and has spent the past two decades designing sound experiences and interfaces for a whole string of international clients.

DANWEI: And Aaron is out guest today.

JAMES: Can you give us an insight into the practical side? How do we go about designing sound or even designing with sound?

AARON: Well, I think the first question to ask yourself; is this a problem that needs you to add sound to fix it? I divide my work into two categories. I just give them the names additive and subtractive sound design. And calling something subtractive sound design can be a little confusing because there’s things called subtractive synthesis. But it’s just terms that I use the last 20 years so that I knew what I was doing when I was doing it. And to that point, before you add sound to a system or a product, as yourself, does this need to make more noise that it already is? If it doesn’t are there ways that I can reduce the noise or the sounds that it makes to improve the user experience.

JAMES: So by that you mean it would be adaptive sound, I guess?

AARON: Well, no, not even that. It could be – so you have a product that has a fan in it and cheaper fans spin faster and they make more noise. I don’t know what the room you’re in sound like now but you may have a couple, one or two PCs in there or something or you may have some device that needs to be cooled. And that fan noise stacks up. And maybe a way that you could – if you’re working on some sort of product and it has a fan would be to put a better quality fan in with bigger blades or lower spin or a lower RPM and that will reduce the centre frequency of the noise. So instead of going [verbal description] or maybe not even at all, not at all.

And that’s what I mean by starting by looking at ways that you can take away sound from a produce or an experience to let the other sounds come through. Now, there are, of course, times when if you need a phone to ring then you do need to add sound to it. But I try to start with how can we make it quieter, how can we take the signal away? Because as it stands, we’re being bombarded from morning to night with things trying to get our attention.

DANWEI: I was just thinking about the general design principle, less is more. I guess it applies to sound as well since sound is as well another design material that should be treated as such.

AARON: Yes, it works for me.

JAMES: Yeah, and the whole thing of getting a clean baseline, I guess.  I don’t mean baseline as in a lower sound but just like kind of starting with a simpler sound stage to put something else on top.

AARON: Back to this grabbing your attention, there are so many things around us trying to grab our attention. Each time out attention is taken away you have to spend time getting your focus back. I don’t want to sound too dramatic about it but I think this has a negative impact on our mental health. So as designer, I’ve tried to start with how can we take sound away and also to make it not bad, that is, make the result of the design be something that unless it’s supposed to annoy people – like a fire alarm or something – to make sure that whatever it is that you’re building or you’re adding to this product isn’t going to increase any stressful experiences that the person might have any more than they have to.

Sorry – I’ve lived in Germany for 18 years now and I’m illiterate in two languages to pardon me for wondering.

JAMES: It’s okay, Aaron. I’ve lived in Sweden for 18 years so I know exactly what you mean. At times my language in English and Sweden just kind of evaporates. But how do you judge how good or bad a sound is?

AARON: That’s a $60,000 question, isn’t it? It depends. So I’ll give you an example of one of the problems that I struggle with to this day and it’s a presentation to decision-maker problem. So the impact on sound on your experience, if it’s something that you hear 10,000. If it’s your washing machine, if it’s some sort of alert that you get over a chat application or whatever. If you have to hear that all the time you’re going to become acutely attuned to noticing that.

Which means that it’s not going to have to be as loud or as colorful as it might be if you were trying to convince someone that this was the right way to go or the right sound to choose. So what happens is early on in my career – I started off by wanting to make things quiet, and clean, and smooth, and as reduced as possible, I would march into the client’s office with my ill-fitting tie and they’d say, “All right, let’s hear it.” And I’m like, “Woah, you mean now?” And they’re like, “Yes, yes, yes, just play it on your laptop.” And I’m like, okay. And it would be, maybe, a noisy glass-walled conference room or something.

And then I’d play some little “mewww” that would come out of the speakers. And they’re like, “That’s it? We paid a bunch of money for that.” No, but imagine you hear it 50 times during day. And they’re like, “It’s not loud enough.”

So it’s a big problem of judging what is and isn’t good especially if the real effect won’t be known overtime. That said, there are a few things that you can do to make sure that you have some level of quality. Of course, you don’t want the sound to be unnaturally distorted, it has to fit within the performance envelop of the play out hardware. That is, if you’ve got a little tiny speaker, don’t expect it to make low frequency sounds.

JAMES: Right.

AARON: And I would say some of the most common words that client’s would use to describe their wishes for a sound would be warm and low. And those are two things that most mobile hardware or inexpensive consumer electronics – they don’t do that or they didn’t do that up until recently. And things have changed because the play out hardware has changed, the quality has become better, it’s certainly become louder.

I’m sorry to give you wondering answer to your quick question, but it depends. And sometimes you have to end up giving people something that you know isn’t going to work and let them discover that themselves.

DANWEI: I know one approach in finding out what works and what doesn’t work in the situation when it comes to the design is to experiment a lot. And early, which I know from your book is something that you encourage people to do.

AARON: Absolutely.

DANWEI: How do you go about experimenting with sound? What kind of tools do you use? When it comes to graphical design, a lot of designers are familiar with pen and pencil or digital sketching tools. But how do you sketch with sound?

AARON: I like to make funny noises with my mouth. If I’m starting to make something or if I – and I just discovered this, it wasn’t intentional, but early on, some of my first big projects were in Korea. And I gave some presentations in meetings and I was trying to describe the client as what it would sound like. And so I made noises like – you know, it would be kind of a harmonic thing that would go up and it’d go [verbal description] and then I’d make some other [verbal description] clicking noises or whatever. And then at the end of the meeting my host came out to me and said, “You know, I don’t know if anyone really understood what you were you saying because you were talking way to fast but they really liked the noises that you made. So we’re going to continue with this project.”

I learned two things there. One is to speak slower if you’re presenting to people who aren’t native English speakers or native speakers in whatever language you’re speaking. And two, yes, that it’s good to be – that’s a way of starting, it is a way of starting. Just use your mouth to make noises or bang on things with your hands. That pushes the concept into the physical world. And I think it’s a nice place to start if you can because an OS X or a Windows interface, you’re already at an obstruction, you’re already away from – I mean, I’m 50 years old so I’m half analog and half digital. But for me, I’m moving away from me as soon as I open up an interface.

So I think by starting with an idea before you open up some sort of digital tool, even if that idea is just you pounding two rocks together, I like to start there.

JAMES: And I love the fact that making prototypes with your mouth is as simple as picking up a pen and paper or a white board marker and sketching something in the wall.

AARON: Sketching, actually, another way is I will draw X and Y axis and then draw amplitude – so X and Y axis and then draw amplitude envelopes overtime. And there’s certainly a long history of this in electronic composition if you look at early electronic scores. They look like triangles and rectangles all stretched over the X-axis of a graph or Y is indicating amplitude and X is indicating time. And that’s one way to start. In particular, if your goal is to create a language, then you can already start buy looking at rhythm and timing.

So there are different standards. For example – I forget what the ISO is but there’s a hospital standard that requires alerts to be in groups of three, I think. And so, yeah, drawing pictures and making noises first, that’s a great way to start. And the second would be, maybe grabbing any kind of musical instrument or something that’s designed to make sound and then go to the personal computer and open up software. But I don’t like to start with software if I don’t have to.

JAMES: But speaking of when to start, when is a suitable point to get involved with the sound design?

AARON: These are the hobgoblins of my life for the last 20 years. So I started doing this in 1998 and general up until 2010 or so whenever I would be brought in on a project, it would be after everything else had been decided. If it was just a software product, the software was done and then someone will remember, “Oh yeah, we need to add some sound for this.” If it was a hardware product, all the specification for the play out hardware had been established.

So any sort of changes or improvements or features you might have wanted to include that would benefit the audible experience of that product were already set and stone. It’s good to come in as early as possible or at least have it on the project’s radar. Again, there’s so many different kind of products and services you can be building. And of course, you’re limited by budget and so on and so forth. But if this is some sort of sound that is going to be part of the brand – that is part of the real experience of the product, then you want to start thinking about it as early as possible so that you don’t box yourself in like, “Oh wow, we should have put in a bigger speaker or a better amplifier.” Or whatever.

But now it’s too late. So it’s nice to get in as early as possible, that’s changing. Most of the projects I’m on now I get approached fairly early on and it makes the process a lot easier.

JAMES: I guess that’s not really a surprising answer, really. Because the sound cannot be extracted from the experience. It’s as much a part of the experience as touch and as sight.

AARON: Yes, you’d be surprised. I think sound is the younger brother that’s kept out of sight in a lot of project, or at least it’s felt like that until the last minute. But again, as the size of the graphical user interface decreases – this is a drawing, I don’t know if we put it in the book, but as the size of the GUI increase, the importance of the AUI, audio user interface, increases to the point where when one disappears the other one becomes all you have. And you see this now in voice user interfaces. And there’s a great book about that, I forget the name of the person who wrote it, but it’s on O’Reilly, there’s another book on designing voice user interfaces. And they addressed this quite a bit.

DANWEI: I’m a bit curious since you’ve been working with so many different kinds of industry. Which domain do you think is the one pushing the boundaries when it comes to sound design?

AARON: It’s hard to say. So game audio is certainly in terms of its use of procedural audio or algorithmically-generated real-time audio, game audio is way out in front of everybody else but they also have – whether it’s someone sitting in front of a PC or someone wearing headphones, they have the luxury of lots and lots of cycle to burn on a computer and a known play out environment either headphones or some PC speakers. And so you could say that makes their job easy but at the same time they’re creating moods and words and aesthetics, and have been for quite some time. So I’d say game audio is way off the front.

I’m doing a lot of work for mobility transportation automotive and, specifically, I’m working with a company called Brüel & Kjær, they’re a Danish sound and vibration company. And it’s fun because I’m discovering that there’s – this is sort of like I’m going back to school right now. This sort of work is great because I’m working with all these engineers who are NVH; noise, vibration and harshness engineers.  And not many people know, but the automotive industry has employed people concerned and focused on the way automobiles sound and feel. There’s a lot of resources that go into that. And it’s not just the engine, no, it’s everything about the car.

So they, besides gain audio and, of course, any audio-specific companies, like people who make microphones or film, I think transportation mobility automotive is probably one of the leaders in that space.

DANWEI: Yes, it does not really surprise me. I was thinking about the automobile industry because from the little I know, there’s a lot of sound design going on to give the right feeling of the brand, like, how it should sound like when you close the door or open the hatch, the clicks and everything of the mechanicals, from what I understand.

And I’m thinking that also that it adds and extra layer now when there’s electric cars that may not make these mechanic vocal sound but they’re digitally added to it. And I’m a little bit curious what your take is on that – should an electric car sound like an old-fashioned traditional motor car that goes on fuel or should it have its own distinct personality that’s purely digital?

AARON: That’s a question now, isn’t it? So in IC, internal combustion engine, it does more than just make a recognizable noise. And of course, the amount of engineering know-how and design love that goes into designing the way engine sounds is so much more than, I think, people would realize. I have met the people who do this and they’re artists, they’re passionate artists. And when you take that away, now only have you taken away the huge component of the time-based aspect of the brand but also you’ve taken away a masking component.

So if you remove that IC engine, then there’s all sorts of noises that automobiles make that you don’t hear and you don’t really want to hear if that engine is gone. So for example, there’s some pumps that supply the windshield wiper cleaner fluid and they go, [wiki, wiki, wiki, wiki, wiki, wiki, wiki].

JAMES: Yes, it’s a much more scratchy noise you get from that pump if you do hear it.

AARON: Yes. And maybe you’ve bought a really nice car and it’s an electric car and it costs you a bunch of money and then you hear this [wiki, wiki, wiki, wiki, wiki, wiki, wiki]. Or you hear little bits of plastic in the car whose rattling before may have been masks, but now you can hear. How do we put that back in the car into the experience but without making it sound contrived? Because it’s my strong opinion. Although, there are plenty of cars out there right now, you may even be driving one where they’re adding engine orders to an existing engine.

So you can have a four-cylinder car, you can add some more engine orders to it and make it sound really tough. But if you don’t have an IC engine then I think it’s like audio skeuomorphism. You remember when Apple came out with a little notepad and there was, like, torn paper?

DANWEI: Yes.

JAMES: Yes.

AARON: And I’m not buying it. I’m not buying it at all. And it cheapens the experience. I submit that anyone who says they know what the future of electronic sound for EV is lying at this point because nobody really knows.

DANWEI: I think it’s going to be super exciting to see what your professional comes up with because, it’s like you say, I feel like now in the stage where the graphical user interface was one day when they were just trying to emulate the physical world in a physical space. And now I think it’s going to be right through in the sound to figure out how the pure authentic digital sound should sound like, I feel like.

JAMES: Yeah. And I think, eventually, if you think about – thankfully we don’t have typewriter sound effects when we use our keyboards on our computers. So at some point, you let go of the previous technology, so you no longer feel attached to the sound that was once there.

AARON: The benchmark that I’m proposing to my clients is, let’s design such that if we take the sound away, people complain. That’s a way to start, it’s not the only but that’s a way to go into it because you can – think about the sound effects for spaceships on TV, it’s usually a lot, kind of, tuned noise – starship enterprise or whatever. And you don’t notice that sort of atmospheric sound during the program or the film you’re watching. But if you take it away, then the game is up, the jokes I up.

So the same in EVs, I think it’s going to be a combination of quiet, ambient sound that is designed in such a way to give you critical information when you need it and maybe to bring back some of the excitement that was missing after you took away the IC engine. Because the IC engine also supplies a lot of vibration in the car, and it’s the vibration that gives you the excitement.

Recently I was at some manufacturer’s getting to take some rides and some fun sport cars. And, although, I was listening for the sound, I was really trying to pay attention to what the vibration was doing. And it’s incredible, it sends your pulse up, it sends your cortisol or your adrenaline, like, all these happy juices are going through you. And it’s not just the speed, it really is the way the whole thing vibrates and courses around you. And that’s something we’re going to put back in as well.

Or if you want that kind of experience. Because if it’s not there and you have lou1d sound in a car but you don’t have any of that vibration coursing through it, it’s half the sandwich – I don’t know my metaphor there. But anyway, it’s half the sorry.

JAMES: Thanks a lot, Aaron, for helping us understand sound better and giving us a better chance of designing it.

AARON: You’re welcome.

JAMES: I think it’s really, really to think about not only designing sound but designing the lack of sound. But what’s also just as fascinating was something we got into after we stopped recording. You work with – or research sketching.

DANWEI: Yes, that’s right. And I’m very curious about how to facilitate sketching when traditional tools like pen and paper may not be sufficient to explore all the different options that you may want to explore before picking out promising design concepts. And to sketch with sound is just another way of exploring different alternatives of sound before you pick one that might be more promising that you want to iterate on and refine.

JAMES: So sketching a sound could be like – what Aaron did when we were chatting to him. He was just making noises and kind of just sketching a sound without using an instrument.

DANWEI: Yes. So sketching is you do whatever it takes to express your idea whether it’s to use your mouth or whether it’s to wave your arms around and do some dance or draw something on a paper. Whatever that’s easy, fast and not expensive to do, to express as many options as possible to be able to explore.

JAMES: And Aaron himself, I wrote down a quote from him. He said at the end, “Successful design happens outside of language.”

DANWEI: Very nice.

JAMES: Yes, very nice. Which I think is a wonderful little phrase. And I love how through talking to Aaron we’ve seen how sound is visual in a way. If you relate it to how we work as UXers or visual designers with digital medium that sound is no difference in many ways.

DANWEI: No, that’s just another element to the whole experience. And it plays a very important role in the whole personality of the product or service that we encounter.

JAMES: Links related to this episode are on UXPodcast.com and we also send them out as part of our backstage email which you can sign up or at UXPodcast.com/backstage.

DANWEI: And something else you can listen to is Episode 122, Calm Technology featuring Amber Case. Thank you for listening and remember to keep moving.

JAMES: See you on the other side.

[Music]

DANWEI: Knock-knock.

JAMES: Who’s there?

DANWEI: Art

JAMES: Art who?

DANWEI: R2D2!

This is a transcript of a conversation between James Royal-Lawson and Danwei Tran Lucani recorded in February 2019 and published as Episode 204 of UX Podcast. 

Transcript kindly provided by Qualtranscribe.