The Conversation Arc

The Conversation Arc is a way to structure the user study sessions. As you’re planning your interview guide, you are thinking through the whole conversation. You want to think about collecting all the data we’ve identified previously, but also how do you do this in a smooth, natural way. You want to think about setting it up, how to transition from one part to the next – how do you organise things so that it flows in a logical way and you can bring the interviewee along in the process.

The user study interview should form a natural conversational arc, starting from the point the person arrives at your location, or when you meet up, until the point where you hand them their gift and they depart.

Part 1: Build Rapport and trust

At the beginning of the conversation, you’re building rapport and trust. You want to invest some time and energy into this process. If you jump too far ahead (for example, jump straight to showing the product prototype) before they know what is going on, what you want from them, what the ground rules are – you’re not going to get good data out of the process. It’s the same as any natural conversation when you first meet someone, you don’t start by asking really deep and personal questions.

Rapport and trust-building start from the first contact – from when the receptionist answers the door, or they are greeted when entering the building. You start by having a bit of chit-chat about the weather, was it easy to find the place? how was the traffic? do you work near here? have you done this before? Basically small-talk. You want to make the person feel at ease as quickly as possible, make them feel comfortable, and ensure they know exactly what is going on.

Part 2: Introduction

During the introduction, you set some ground rules, let them know what you are doing and what you expect from them, and what they can expect from you. Refer to the interview guide for an example of an introduction.

Part 3: Discovery

It’s good to start the interview process with the discovery part, as this provides you context for the next part (usability). It will help you interpret how they respond to the product, as you get to understand a bit more about them and their experience. It also allows you to customise some of the tasks that you will ask them to do in the usability part – which makes the process more relevant to them, and seem more natural. The purpose of the discovery part to open up and broaden the scope of the conversation. You will go outside of the specific product you want to ask about and understand the user’s larger life context.

The template includes a number of examples of both discovery and usability questions. Pay attention not only to what the questions are about but also to how they are posed.

Part 4: Usability

In the usability part of the interview, you want to give the user some tasks to perform. If you refer back to the interview guide, it provides a template on how to set these scenarios up. You want to prompt users to travel to points of interest. Note that you want to send them into a general direction and you should expect them to “bump into” the functionality, but not exactly direct them what to do e.g. don’t say “click that button” but rather position it as “now I want you to try and do xxxx”.

You start with a broader more general task and observe the user as they work their way through your product. Once they get to the end of the task, and they stop – if they have missed anything you wanted them to do on the first go, you can ask them to complete some of those smaller level tasks. So, you’re providing them with the freedom to explore the product as they wish at first, and then coming back to complete any steps or additional functionality after the fact.

Remember, you should not say any of the words that can be seen on the UI. You should never use any words that can be on the screen – for example, use the name of a field label, button, or anything that is visible to the user. You want to the user to find their way naturally without being prompted by you.

Part 5: Debrief

Once you have worked through all the usability tasks, and as you get towards the end of the interview, you want to get a general overview of the user’s perspective. Summarise what they saw, get their overall takeaway from the session. How do the various pieces compare to each other? Think about the pros and cons of various parts of the product – what did they like and not like? How does this feel to use? How could this be better?

At this part of the interview, you’re also starting to send some subtle signals that you’re getting to the end of the process, and this allows for a gentle and natural way to wind down and conclude the conversation.

In the “Example Usability Questions Template” – there are a number of example questions that you can work from.

Remember, very importantly, you do not want to lead the witness – by embedding the answer in the question. You want to ask open-ended questions, and allow the interviewee to respond as broadly and naturally as possible. So for example, don’t say something like “did you think that xxx was great?”, but rather “what did you think of xxx?”.

Part 6: Cool Down

At this part of the interview, you’re starting to signal that you are getting to the end of the process. You can use various techniques, like a change in your voice, or physical posture. You can put down your pen or your notepad or your forms/questionnaires, move things out of the way. You start shifting into “thank you” mode. Say “this has been really great, the feedback has been incredibly useful”, “I really appreciate you coming in to do this”.

What is also useful here, is to summarise a couple of the key points the interviewee raised in a very generic way. “It’s super interesting to understand a little more about how you ……”. You may find that more valuable feedback comes out at this point.

Part 7: Close

At this point, the interview is over. You conclude by thanking them for their participation, you hand over the incentive (which is another strong signal that the interview is over), and you will be walking back to the reception/exit area with them.

Tip: Don’t stop recording until the person has completely left the building. Sometimes another conversation will pop up right at the end, and there would possibly be some more useful information there.

Preparation and Logistics

The question is – how exactly do you talk to these people? How do you create an environment for each participant that is clean, neutral, and somewhat innocuous? If you look at the Interview schedule template, there are a number of items to work through. Let’s just focus on the time around the interview:

8 hours and counting

At the start of the day, as you prepare, you want to get your head, body and physical presence, your location and your team into “interview mode”. Here are some useful tips:

  • Don’t wear perfume or cologne.
  • Don’t wear overt branding of your company or any brand or label – dress neutral.
  • Send a reminder with a schedule to your team and notetakers about observing sessions.
  • Prepare the hardware (computer, mobile devices) you’ll use for testing.
    • Make the computer or devices look neutral and generic.
    • Hide all icons on the desktop, any distractions or visible bookmarks.
    • Clean the keyboard and screen.
    • Install and test prototypes, presentations, wireframes
    • Shortcuts: Add bookmarks or desktop shortcuts for prototypes, sites, etc.
    • Set the computer screen and settings to match target users’ preferred setting
    • Set an innocuous home page for the browser (e.g. local weather or news).
    • Clear the browser history, cookies, cache.
    • Minimize the browser or prototypes.
    • Start and test the screen recorder and/or set up the video camera.
    • Start and test screen-sharing software for observers.
    • Run through all the prototypes under actual test conditions. Then reset them.
  • Print copies of NDAs and interview guides.
  • Clean up the room – erase whiteboards etc.

10 minutes and counting

Right before the start of the interview, you want to be 100% focussed – remember sometimes interviewees will arrive early too.

  • Post a “Do not disturb” sign on the door of the interview room.
  • Turn off your cell phone.
  • Turn off all notifications from all devices powered on – e.g. computer, test devices.
  • Have a glass of water and take a bathroom break.
  • Freshen your breath – chew mints or gum.
  • Get into character: curious, open, objective. Smile! Try to see the world through the participants’ eyes.

The below template contains a more complete user studies schedule, starting from 1 week out, until the next day.

Template – DIY user studies – schedule

User studies are weird

Remember, this is a somewhat weird and unnatural situation. You have a person who’s come for an interview, they don’t quite know what you want, or what they are doing there. They possibly feel self-conscious, and you’re sitting awkwardly to the side or behind them, watching them struggle through a prototype. You don’t know each other at all, you’re spending a few minutes together, and it is unlikely that you will ever meet again in future.

Also, it is a one-way engagement, you’re asking all the questions and the participant is working through it. They are not learning anything about you, but you’re learning a lot about them. And.. you’re going to pay them for this. It’s just a really odd situation in general.

So note, that the things covered previously – about the conversation arc, and making participants feel comfortable is really important, and will vastly improve the quality of the data that you will collect. Also, it will be more fun for both of you – being awkward isn’t fun for anyone.

It could be helpful to consider yourself as a “host”. Someone is coming to visit and spend some time with you. You’re there to make them feel comfortable as much as possible.

Get into character – the “Researcher Persona”

Before the interviewee arrives, you need to get into character, by the time they get there, you should be 100% in character. You’re the host, receiving your guest. You’re smiling and friendly. Pro tip: smiling changes the sound of your voice and how you express yourself. You should smile whenever you’re doing interviews – including when you’re doing telephonic interviews.

You’re grateful that they’ve made the time available to come and see you and you’re very happy to see them.

You’re going to make them feel welcome, comfortable, and take care of them through their visit.

You’re there to learn from them, and you’re really interested in them.

Everything they say is super interesting, and your body language and facial expressions will reflect that.

At the same time, you need to remain neutral. Curious, open, encouraging but neutral – without any strongly held opinions, and without leading them. In other words, if they provide you with feedback or an answer, you will find it interesting, but you will not have an opinion if the response is good or bad either way. You should be saying “uh-huh” rather than “that’s good!”. There are no good or bad answers, all answers are good.

Be very careful not to get defensive. Be aware of your own attachments to your product, or certain outcomes. You don’t want to explain or defend any position. Divorce your emotions from the thing that you’re discussing. Feedback about your product is not feedback about you as a person.

Also, make sure that you are not pitching the product to the person. You’re not selling the product, you’re not there to enumerate the value proposition, get excited about the features or anything like that. If you’re describing anything, you’re doing so in a fairly neutral manner.

Interviewing tips

Here are a couple of additional interview tips:

  • Ask follow-up questions – “why?”. Also, “silence” is a powerful follow-up
  • When in doubt, ask for clarification
  • Answer questions with questions
  • Keep it personal and concrete – avoid hypotheticals
  • Don’t pitch the product
  • Watch the time (surreptitiously)
  • Be careful when testing your own designs – note that you don’t get attached to the product, or the outcome
  • Watch non-verbal cues (yours and theirs). Facial expressions, body language, tone. Are they comfortable? Tentative? Nervous? Bored?

Closing the study

Recording information

  • Watch and listen
  • Don’t filter or judge
  • Take users seriously
  • Take notes
  • Periodically note the time
  • After the session note top issues
  • Don’t do anything rash (like fix bugs)
  • Summarise, prioritise at the end of the user study