Computer voice
Season Two, Episode Eleven

[Music]

Per Axbom
Hello, I’m Per Axbom.

James Royal-Lawson
And I’m James Royal-Lawson.

Per Axbom
This is UX podcast. We’re in Stockholm, Sweden and you’re listening to us all over the world from here in Sweden to Switzerland.

James Royal-Lawson
Joining us today is Meena Kothandaraman. Meena is an Experience Strategist at the human centred research practice, Twig and Fish.

Per Axbom
As well as her consulting practice, Meena also lectures, at the Bentley University human factors and information design graduate program.

James Royal-Lawson
In our conversation with Meena today about research, we dig into the question of how to ask the right questions.

Per Axbom
Today’s interview was recorded at UXLx, which is a conference that is held every May in Lisbon, Portugal. Tickets are on sale now and sell out every edition.

James Royal-Lawson
So, on the podcast, it’s a situation we regularly find ourselves in, is that we are supposed to come up with questions before the interviews. But sometimes we don’t. Which means we have to come up with questions on the fly. And during these podcasts, we often find ourselves talking about subjects that maybe we’ve never thought about before. Or maybe we haven’t thought about for a while. Okay, this is a podcast context. But this is also something from a research concept- context. And there is that pressure to come up with the right question.

Meena Kothandaraman
Absolutely.

James Royal-Lawson
You feel that?

Meena Kothandaraman
Yeah, absolutely. There’s, there’s no doubt when you are in the asking position, if that’s fair to say, when you’re the person asking the questions, you think a lot about the wording, the choice of wording. Does it actually convey what you want it to convey, but you have a bit of a trump card, because you know, ultimately what you want to go after. And you know, how you’re going to sort of craft a path to get there.

James Royal-Lawson
Right.

Meena Kothandaraman
For the participant, they’re not quite as lucky because there’s this thing called bias. And if you tell them things in advance, then sometimes people go, “Well, wait a minute, you’re biassing them.” So it’s a little harder for the participant, I would say because it’s hard for them to actually project that arc that you’re going toward, and sort of meet you at the end in the right place. And then often wonder, “Did I do a good enough job? Did I actually answer the question?”

Whereas for, of course, the person asking the question is, “Did I ask the question that I wanted to?” Because sometimes, it’s sort of like when you play Pictionary. You know, you’re probably both familiar with Pictionary. And you, you see the word is chair, and you start to go, Okay, you draw something. And I don’t know, if somebody starts talking about cows, and then they suddenly get stuck with farm animals. And then they go flying off into farm animals. And then you’re just trying to pull them back to wooden furniture, and there’s no hope. So it’s one of those moments, exactly. It’s one of those moments where you really pray that you don’t, you can very carefully sort of, it’s almost like I visualise it as putting little garden pebbles on your path and go, you’re going to just follow this trail with me. And if they can follow you all the way to the end. Bravo.

James Royal-Lawson
Yeah, cuz you don’t want them getting distracted by the cow chair. Too early anyway. I mean, it can come later on.

Per Axbom
But there was something you said there. I mean, there’s so much self doubt, both within the interviewer and the interviewee.

Meena Kothandaraman
Yes.

Per Axbom
So there was a dance going on,

Meena Kothandaraman
Yes.

Per Axbom
beneath, in, like, a lower level, which we don’t really address.

Meena Kothandaraman
Absolutely. And the thing about the interviewer is that they are, they are dwelling in this content. So if it’s about the podcast, for example, you have thought about the podcast, you think about it, you are the people who are curating that experience, but as the person who is answering the question. I didn’t wake up yesterday thinking about your podcast, with no disrespect.

Per Axbom
Oh no!

Meena Kothandaraman
I know. Sorry. I woke up thinking about the fantastic breakfast. I was! So sorry. But you know what I mean, it’s that kind of, my brain goes elsewhere because I’m not actually going to think about your world immediately. So if you are a financial company, and you go “Tell me how you invest”, I’m gonna go, “Wow. Okay, investments… Like when was last time I invested?” and “What did I, what did I do?” and, it’s recalling those moments that is incredibly difficult, where you want to sound, you want to articulate your story in a way that you have all the pieces together. You don’t want to overlook something. And how often is it that when we answer a question, we’re like, “Oh, and I forgot to say this!”, and you want to add it in, but then did you add it in in a way that it placed itself properly in your story? It’s very hard on both sides.

And I really feel for when when people think that conducting research is merely the ask of a question. There’s so much more to it. And we want people, we want to give people a chance to articulate their thoughts as well and sort of reflect on them. So if I, if I asked you how your morning commute is, and you just go, “It’s gnarly.” There’s more to that. I mean, even our prior conversation right now about sound, and how sound fills a space and how do we consume sound? It’s more than just “Well, I love listening to podcasts.” What is it about the podcast? What is it about the music that you play in the background? What is it about the words or the intonation in your voice? James? Like, what is it about those details that we really want to understand and give that person, that person who’s answering our questions, a moment of reflection to really step into, and go “Huh, I didn’t realise actually do like silence in the background. It never occurred to me, actually, you know what, that’s a good point you raised.” When people can arrive at that moment. That’s when you know, you’ve sort of, you know, hit the magic.

Per Axbom
Or do you even just like the podcast, because you happen to be listening to it while you’re passing through this beautiful space in nature.

Meena Kothandaraman
Precisely. So is it, is it intrinsic? Is it extrinsic? We want to unpack those details, like, my personal, my personal pet peeve, as a qualitative researcher is, people are sort of missing the point of doing qualitative research. Because often what they want to do is quantify the results. Five of seven people said they were happy. You know, two of sevens said that they were sad. Some said they were neutral. And it’s very, very difficult to understand that. What do you do with that? Because ultimately, whatever we do provide, as qualitative research output, needs to either inspire or inform the design team accordingly. But hopefully, the qualitative side really is meant to inspire. Really it gets a lot of excitement, because you’re hearing that story from somebody who’s giving you a very well rounded account of something that happened.

Per Axbom
The two out of seven? Well, that’s so interesting, because does that mean that the two people out of seven don’t matter because they’re the minority, whereas the qualitative research could tell you, they’re maybe the most important people.

Meena Kothandaraman
The thing about that, that I find is really interesting, I’m putting my cup down so slowly, is that when we look at the reasoning for qualitative research, a lot of times it is to inspire. What we want to do is then, if we’re looking, if somebody is a designer, and they’re like, “How are we going to design this?”, what they’re looking for in that moment is inspiration. We don’t need to tell them how to design because they are designers. I think designers are brilliant people. But what we need to do is give them a landscape, sort of a variety of options to consider or not even options, that’s not the right word.

It’s a variety of considerations. So that if I go, “Okay, well, here’s how emotions sort of pan across”, they can then go, “Wow, we’ve done these, but we haven’t done these.” And it starts to offer them sort of sparks, if that’s fair to say, because that’s what I really think research output gives you is a spark of creativity, it’s like it prompts you in a certain direction. That gets you excited about “Wow, we didn’t consider this” or “Oh my gosh, here’s the gap in this experience.” And now I can go and solve it. And if you like research interactions when we are in that moment of gathering data from a participant, if we don’t describe that space or describe that gap that somebody has, in depth, or even to the point where we feel they’ve really explored that aperture, like the wider aperture. I think we really lose a lot of richness to the data.

James Royal-Lawson
Yeah, there’s a richness and also you narrow the spectrum, I guess as well.

Meena Kothandaraman
Absolutely.

James Royal-Lawson
And present, you almost use the word options and then back to promotions. But I mean, I understand what you mean because it’s that spectrum of research insights that,

Meena Kothandaraman
Absolutely.

James Royal-Lawson
“Yes, we got these ones. And maybe these are the most common ones that we can group this into a pile.” That’s common.

Meena Kothandaraman
Yes.

James Royal-Lawson
But then we’ve got all of this.

Meena Kothandaraman
Yeah. And so even the person who gives us information who seemingly looks like an outlier, we want to consider that because maybe that outlier, again, reveals something to us of importance, like, why did that person actually abandon a particular solution? What was it about that? And what were the points that they raised? Because in our research interactions, we broadened the discussion. What were they able to tell us that makes us go “Hmm”, because the one thing I will also stress is, you know, qualitative research output, people like to quantify it. People also like to directly ask their participants what they want, which I also think, is an error. They are not the designers. The design team is sitting behind us with researchers. I always think of them as sitting right there listening to everything we’re doing. There, it makes no sense to go, “So, you know, James, tell me what you would like? Per, what would you like?” You don’t know in that moment. You’re not thinking about it as deeply. So if you just answer the question for the sheer sake of answering the question, you might be giving me an off the cuff response.

Per Axbom
Because you want it to be a good response

Meena Kothandaraman
Precisely. Oh, my God. Yes. That’s another loaded one. But yes, precisely. Right. So, so then you give an answer and then what happens? We play that moment out. I look at you and I go, “Oh, well, you know, you weren’t a good respondent.” and “that made no sense.” And I just, I lose it in that moment. And then I come back, and I talk to my team and go “Per wasn’t really a good respondent. Right? And that’s really not fair. It’s not fair to you. But it’s not fair to the process. And that actually, unfortunately, gives qualitative research a bit of a bad rap.

Per Axbom
Oh, that’s a good point. And I think, what I struggle with is that a lot of people in UX lack the type of curiosity that you’re describing right now.

Meena Kothandaraman
Yeah.

Per Axbom
Which I think, I mean, it’s supposed to be core to what we do- that curiosity, where you actually, you are open to finding out stuff that you would never have thought of finding out?

Meena Kothandaraman
Yep, absolutely, I’m going to say something maybe controversial, that you might get some feedback on. I do not advocate, especially with- So let’s not take a step back here. When we look at research, there are multiple types of research. This was the workshop that I ran at the conference here, that was all about understanding the different scopes that you go under, different lens of inquiry that you go into when you’re doing research. There’s sort of the wide open, when you’re really trying to learn about people, much more like the discovery in the exploratory research. And then there’s more the validated of side of things. Validation, generally, was what started this inquiry. It started with validation. And people still to this day, try to shove as much as they can into validation.

And the challenge is, is that when we look at this, we realise that validation only services us so much. So with validated research, what happens is people sort of go at it, and they start to, start to just make everything, again, quite quantified. What we need with the exploratory and the discovery side, is that curiosity. We need to want to ask lots of questions, we need to want to go in completely unbiased, sort of with no charter in place, we do want to guide the conversation. So it’s not like we want to be haphazard. But we don’t want to go in with any loaded assumptions, if that’s fair to say. What we want to actually notice with the overlap, is that when we have design researchers, we have designers and researchers in one role, they’re exactly what you just said, is they lose a little bit of the curiosity because half of their brain is tilted toward solution.

When I listen to what you have to say, aiming for solution, I’m actually sort of jumping ahead of you. I try to get ahead of you or I’m not even actually fully absorbing your story. And really likes like I said, probing towards the ends. I almost like to think of it as, you know, when you put your glove on your hand, you stick your hand in, sometimes you’re very rushed about it, you just stick your hand in and it’s functional. Right? It works. You can sort of pick up something, but you know, when you really push your hand in and goes all the way to the tips, and you’ve sort of explored every nook and cranny of that glove? That’s what I believe researchers who are core researchers really seek to do. They have that element of curiosity, they have that interest in trying to get to the ends of every detail, with no intent to solution in that moment. So I know when I say this, sometimes I’ve said this at conferences before and people come up and say, “Oh I disagree with you.” And I’m fine with that. I’m perfectly fine with that. But I don’t believe one person should do it all because there is a skill set.

Per Axbom
Yeah.

Meena Kothandaraman
That is different for each.

Per Axbom
You also have to go in with a lack of pride, because what I see is that people go in and they try to actually find data that supports what they already believe.

Meena Kothandaraman
Precisely. Yeah, almost with an agenda in play.

Per Axbom
Yeah.

James Royal-Lawson
And that ties in, but exploration and validation, as a similar thing when I do work with analytics. But again, it’s just another type of research, but that curiosity, to kind of explore and find things compared to finding the answer to something. As in, “We did this, we need to validate it’s better.” That comes up in both quantitative and qualitative.

Meena Kothandaraman
Absolutely. And with quantitative, and you know, and people, for some reason, again in history have pitted qualitative against quantitative and I’ve never quite understood it. They just service to different parts of the story. Like quantitative you want the volume, you want that velocity of data that’s coming through and how quickly and how rapidly things might be changed. You can see shifts and patterns so well. It services a very important moment in time. But if you want to, again, describe the space, you can start with markers that are quantitative, but ultimately, you will have to go back to quantitative because you need that depth.

And that’s where I think I, again, it seems to be, I almost feel like it’s a bit of a joke, like people are like, “Oh, well, you mustn’t like quantitative data.” I’m like, “No, I love it!” Same with all data equally. But it’s just that both service a particular moment. But that curiosity, I think, I do- It’s funny, when I do talk to fellow researchers who are just researchers and are not, like, our job is to empower the design team to shine, to do as much as they can- these are already brilliant people. They just need to be able to anchor themselves in whatever it is that we’re putting out there. But I feel like that curiosity element is waning a little bit. And I wish people would be, you know, if you’re happy, describe happy to me, like, are you happy because the weather is beautiful? Are you happy because you’ve had your morning coffee? and you have no allergies? Like, what are you happy about? It’s not just take, well, happy as a box. I’ve checked it and moved on.

Meena Kothandaraman
Exactly. And even the words we use, like the word family, the word vacation.

Meena Kothandaraman
Yes.

Per Axbom
They mean different things to different people.

Meena Kothandaraman
Yes.

Per Axbom
So that means that I cannot just be content with the words that they’re using back at me. And I think, to understand them, I need to probe and understand “What does family mean for you?”

Meena Kothandaraman
Absolutely, absolutely. Because that’s, it’s a beautiful, beautiful thing when you get to do that. And be able to describe it in ways that you can actually create meaning to that word, because we use so many words aimlessly. So especially when we are trying to support designers. You know, if we as a group, let’s say we are an organisation, we say, you know, “How can we create?” You know, I use this example, yesterday, we, in my workshop, where we were talking about affordability of housing in the city of Boston. Right now we’re in the middle of a housing crisis. That was the background of the story.

And what I told people is if you ask a designer, “How can we create affordable housing?” If you do not have an aligned understanding of that word, affordable- If we do not, as a team, have an understanding of, how do we as a team define affordable? How do we describe it? But then how do our residents or people who are potential buyers, how did they describe affordable? If we cannot make those two ends meet, then I have no understanding how I can go and validate recreated affordable housing. Yes or No? I have no metric that I can measure against. Therefore, it’s a bit hollow. That’s where I think we get ourselves into trouble because, again controversial maybe, people are just trying to speed to the end. They’re not taking the time, and it doesn’t take a lot of time. But we have to just stop a little bit and go, “Are we doing? Are we getting what we need here? Are we asking what we need here?” And it sort of refers us back to our initial conversation of making sure that we can get people to articulate their thoughts and expand that space and describe that space so that we do get what we need to sort of define these useless terms.

Per Axbom
Exactly.

James Royal-Lawson
I did have one more follow up question. So you mentioned that, about curiosity waning, you feel like- So, why do you think curiosity is waning? And is curiosity one of the things that you can awaken in people? Can you learn curiosity? You develop your own?

Meena Kothandaraman
I do have an answer for this question. I think curiosity is waning, because curiosity takes time.

James Royal-Lawson
Yes, look into that.

Meena Kothandaraman
I think you need time. If I ask you a question today, and you give me an answer, and I do everything I can to get as rich an answer is possible, I will guarantee you, you will come back tomorrow, with even more. Because, you’ve had a little bit of time to think about it. Now, we all know, in our work worlds, time is not something that we get afforded. So people are again, constantly trying to rush forward. So for example, especially with qualitative research, if you are doing, you know, one hour of a session with someone you should be, the rough metric is you should be spending 2x of that time really thinking about their data. Teams want it in 1/10 of the time back. Researchers, I will say, and again, these are all quite contentious statements I’m putting out here, but this is honestly my true belief.

Researchers have not done a good job of revealing the effort that it takes to actually do that work. Our job as researchers is not to just say, “I have a question that I want to ask you and you have to answer it”, we have to shepherd the team through a moment of reflection, where they really think deeply about something, they think deeply about a story, how somebody describes that space, how we pulled together everybody’s data, to then create structures to describe that space. These are all things that take just a little bit of time. I’m not taking months and months of time, but you definitely need extra hours in the week. You cannot simply just turn around, because that’s why it has come to this moment of, “Well, if you want a quick turnaround, I’ll just count numbers.” But that doesn’t tell you anything. If two out of seven people said that they felt that affordability is an issue in the city of Boston, what does that tell you as a designer? I’m going to advocate “Nothing”. It’s going to literally tell you nothing. You’re going to go, “Great.” And then you’re going to go back to what you’re doing. And therefore they render qualitative research a bit useless. It’s just more work that we had to sit in on and listen to, and I don’t blame them.

So it’s part of, part of it is our ability to reveal a true process to go through that effort, not only with our teams, but also to expose that effort with the participant. Getting somebody to open up about something, maybe it’s personal, maybe it’s medical, tell me about your journey as you went through an illness or a condition in a hospital. That is not something somebody can just simply, you know, spitfire an answer to. It doesn’t work that way. We lose that human understanding there. So I feel so, it’s such a warm feeling when you make a connection with someone. And in that moment, if you can get them to really think about things and go, you know, “I never thought about it this way.” Or if they actually don’t run away from your session, and they actually sit with you for five extra minutes and go, you know, “You said something earlier, I said something earlier. We have this thing in between us called an activity and we’ve documented together. I can see patterns that I’m sharing with you beyond what you can see, I can also see them.” It’s so, it’s so, it’s a really magical moment.

Per Axbom
Make time and pursue the magic.

Meena Kothandaraman
Yeah. Oh my god, yeah.

Per Axbom
Thank you so much.

Meena Kothandaraman
Absolutely. Thank you so much.

Per Axbom
I thought that this was a really beautiful conversation. And it made me reflect a lot on how I conduct my work and how open I am to different things happening during an interview. Whether I asked questions or not. All the things that are, as she says, below the surface, and there are so many things to uncover. But often you have, like, a time limit for your research or your interview when you want to at least get some value out of it, then you have some sense of “I think this should be the value I get out of this interview.” So you’re constantly thinking about the interview, rather than being inside the interview and being with the person. It’s really, really challenging to do this right.

James Royal-Lawson
I mean, that’s something that I reflect on. Every time we do one of these shows, Per, is that, you know, we listen back to the interviews that we’ve done with our guests

Per Axbom
Yeah.

James Royal-Lawson
And then we record these bits. And sometimes, I have no memory at all of the interview.

Per Axbom
No, exactly. The takeaways from listening back to it are completely different from doing it.

James Royal-Lawson
Yeah. Because the, what inspires me, what leads my train of thoughts and questioning during the interviews that we conduct with guests, isn’t always the same as the reflections and the realisations I gain from listening back to me and you asking the questions sometime later.

Per Axbom
Exactly. Because you and I aren’t even the same people anymore. Because time has passed. We’re in a different context, we’re in a different state of mind. And the realisation of how much that actually influences your perception. And your interpretation of what someone else says, is hugely important to understand what a difficult process it is to synthesize knowledge and insights from your conversation with another person.

James Royal-Lawson
Yeah, and this is us reflecting on being the questioner and being the researcher and analysing, synthesising what we’ve we’ve gathered. But, like we heard from Meena, the challenge is also on the other side of this. As the person put in the spotlight, to answer the right question now, in this one hour interview, whatever, half an hour interview you’ve got with them. That’s, that’s daunting.

Per Axbom
Exactly. I noted down this quote from her, “Half of their brain is tilted towards solution.” I love that explanation of it, because that is what is happening. So often, when you’re in that situation you’re thinking about, “Well, if they say that, how could I potentially solve that for them?” Rather than staying with them and figuring out “Did I even understand what they said?” I have this exercise I do when I teach, where I show pictures of different individuals in different situations. And it does show the words family, vacation, and a nice day. And the first picture is of a young girl. And the second picture is of a man in a third world country on a bicycle and he’s selling fish. And the third picture is of a Porsche parked outside a luxury house. And so, the concept of family and vacation and having a nice day are completely different. And you have to realise well, they are completely different for everyone. So it’s so easy to jump to the assumption that “Yes, I understood what you just said.” But it’s very rare that you understand another person’s situation in that way. And there’s always more to discover, you always have to be more curious if you really want to get to the bottom of what they feel and what they experience.

James Royal-Lawson
Absolutely. But it also, I think, leads on to, well, Meena in the conversation, she says, gives the example about, like, I think it was investment, “Tell me how you invest”, and “When did I last invest?”, you have to think about it. And then you can’t spontaneously deliver all of the information you want. Maybe, like you say it’s more complex, it’s complex, and you recall, and your memory is complex around it. Meena, in the talk she did at the conference when we interviewed her- She was discussing the, or she gives advice on how we can use artefacts, I guess, in interview situations as a tool, rather than continue to hunt for that right question and put the interview participant in the spotlight to feel like they have to perform, they have to deliver, to our question. And there was some really, really nice, I mean, I’m terrible at this, I’m really, I’m really bad at remembering the value of using artefacts. And Meena reminded me, or reminded us all in the in the talk, about how the artefact doesn’t have to be, you know, an object in, you know, traditional sense of an artefact, it could be that you’re getting the participant to create an empathy map together.

Per Axbom
Exactly.

James Royal-Lawson
When doing the interview, instead of you asking, “Tell me about the last time you did x.”

Per Axbom
Right. You just need a piece of paper, really.

James Royal-Lawson
Yeah. Or it could be that you’re telling them- oh, you maybe use a worksheet to help them structure the conversation or the dialogue better. So instead of, like, going in at them and saying, “How would you describe experience X and compare it to experience Y that other people might have.” You know, you can use templates in your work to get this down on paper and help people explore their thinking. And, I guess, extract the research in a more organic and natural way than when you just shine the spotlight on them.

Per Axbom
But it also makes it more inclusive and accessible because not everyone is able to articulate and explain and answer questions. And now, by doing that, what you just said, you take away the pressure. You don’t, you’re not answering questions anymore. You’re exploring something together. So there’s also, the power imbalance becomes less because you’re doing something together on a piece of paper instead.

James Royal-Lawson
Well now I’ve gone back to thinking about the cow-chair!

Per Axbom
Oh, yeah!

James Royal-Lawson
Don’t go to the pictionary. Yes. Pictionary. Because if we’re talking about using templates and writing and drawing and so on, then it could quite easily end up in the cow-chair!

Per Axbom
It can, get there, of course. Right. So it’s the same biases occur there as well. What you think they’re drawing may not be what they’re drawing. That’s excellent, James. I love that.

James Royal-Lawson
But, you know, the point isn’t them to fill it in. They’re not filling in for completion sake. They’re, they’re filling these templates in as a vehicle for the conversation.

Per Axbom
Exactly. Love it. Do we have any recommended listening, James?

James Royal-Lawson
Yeah, recommended listening. All right. I’m going to, I’m going to throw us back quite a long way now. Seven years ago, actually, to when we talked to Dan Brown, in Episode 170, of season 1

Per Axbom
Oh, was that a time when he was in Stockholm?

James Royal-Lawson
That’s correct. Yeah, it was EuroIA here in Stockholm, and we had a chat to him about discovery. So the discovery phase of projects and the, doing the research upfront for these kind of things and how you work with them.

Per Axbom
Excellent.

James Royal-Lawson
I think it’d be good to listen to.

Per Axbom
Good find.

Per Axbom
If you want us two, as part of your next conference event,

James Royal-Lawson
You could have one of us if you really wanted.

Per Axbom
Oh, no.

James Royal-Lawson
No?

Per Axbom
The other one will be a shadow tag. We’re offering workshops, talks, courses to inspire and help you grow as individuals, teams, and organizations. And all you have to do is get in touch by emailing hej@UXpodcast.com. Remember to keep moving.

James Royal-Lawson
See you on the other side.

[Music]

James Royal-Lawson
Per, what did the pebble say to the stream?

Per Axbom
I don’t know James, what did the pebble say to the stream?

James Royal-Lawson
Water slide!