How to discover user insights without in-person interviews

2019 Code for Canada fellow Andrea Hill makes the case for remote interviews as part of a robust user research practice.

Earlier, I wrote about some of the surprises I’d encountered while conducting user research within the Government of Canada. Today, I’m going to delve into how we ensured we engaged with our constituents outside the local National Capital Region.

Prior to my team starting our fellowship in June 2019, our host agency had conducted a cross-country tour and spoken to over 750 public servants. But we knew that one round of interviews wasn’t enough. Our team wanted to be sure we were continuing to engage with users, without taking on additional costs or the overhead of more travel.

Identifying people to talk to

Let’s be honest, it’s always easiest to talk to people with whom we share similarities. Location, language, abilities. But you stand to learn much more when you explicitly put in the effort to move outside your bubble. Our focus was on including people outside our geographic region.

1. They may have unique needs we were oblivious to
2. They aren’t asked for input as frequently, and therefore may be even more interested in being involved

In our case, we were working on learning more about government travel. While the majority of travel of public servants is between a few major hubs, there are obviously many other trips as well. We assumed that people from other locations may have different vendors or considerations when they travel.

But if even if you don’t have a reason to believe some people may have different needs, you should endeavour to practice inclusive design. This means casting as wide a net as possible — even stretching into what may initially seem impossible — so you are not unintentionally excluding people / missing out on certain critical information.

As an aside, I recently got a preview of a session on “Building Inclusive Services Starts With Research” led by Julianna Rowsell and Hillary Lorimer of the Canadian Digital Service. It was a great reminder that my efforts are still leaving a lot of people out of my recruitment efforts.

But we all have to start somewhere. I’ll simply call out my own lack of attention to connecting with non-English-speakers and users of assistive technology in our recruitment.

After quite a few interviews with people who sit at desks all day within the National Capital Region, we started to be more intentional in reaching out to others. Who were the Government of Canada employees working remotely, or “from the Regions?”

We also sought out employees who travelled to fulfill their mandate in ways other than “attending meetings in-person.” We ended up speaking with several participants who were both based in Western Canada and travelled frequently in the field. As suspected, they provided us with a wealth of information we didn’t get from speaking to public servants who held desk jobs similar to our own.

Defaulting to remote interviews

It might seem easiest to speak to people in person, but I pushed our team early on to conduct all our interviews using conferencing technology. If we always used a video conferencing platform, we could have consistent processes and procedures. We would have a level playing field regardless of where the person was, and erase any bias we may have towards trying to recruit local people ‘because it’s easier’.

Disclaimer here: I was a remote user experience researcher way back in 2012, and later spent three and a half years working for a web conferencing provider. I’m extremely comfortable with this technology and the user benefits.

Video conferencing meant we could connect with people at a time that was convenient for all. We didn’t have to block out a lot of travel time before or after sessions, or find available meeting rooms and worry about secure access into buildings. We didn’t have to worry about ensuring they were going to be at a certain location at a given time, and coordinating multiple visits back-to-back. If something came up, we simply moved the meeting without worrying about rebooking flights. We recognized that our participants were helping us by taking their time to speak with us, so we wanted it to be as painless for them as possible.

Scheduling using Calendly

Speaking of making it painless for participants, we used a web-based platform called Calendly for scheduling. Willing participants could select a time that worked for them via the web interface, and then also reschedule if something came up without worrying about a back and forth email chain.

When we showed off Calendly to a group of public servants across Canada, they were pleasantly surprised that the times were displayed in their own timezone. No need for them to do the conversion back to Eastern time (where we are located).

To be clear, I use Calendly for all scheduling, regardless whether it’s a remote session or not. Ideally, Calendly can keep in sync with your calendar to update your availability, but I wasn’t able to connect it to my Government of Canada email address (Outlook, not O365). Instead, I integrated Calendly with my (Google-based) Codefor.ca email account and calendar, and then did the manual sync with my GC calendar.

For the interviews, I used Zoom video conferencing. As I mentioned earlier, I’ve been in this industry for awhile, and Zoom was a tough competitor. After the company I worked for was acquired and our product shut down, Zoom was my obvious choice. Based on the fact that it seems to be the platform of choice for meetings happening in North America now in March 2020 (during the Coronavirus pandemic), I’d say many people feel the same 😃

One of the biggest benefits to using conferencing software to connect with participants is the ability to record the session. My poor colleagues tried to take notes in real time for our first few in-person sessions. They quickly learned the value of recordings:

1. Catch actual verbatims…
2. in context…
3. along with intonation and gestures…
4. and without getting hand-cramps!

Now, storing recordings can be tricky to manage. Please consult your local ATIP (Access to Information and Privacy) folks, and be careful about data storage, especially on cloud servers.

Just because you’re recording the interviews doesn’t mean that the content is now simply readily available (and understandable) to everyone. No one wants — or has the time — to sit through hours of interviews. You still have to do the key activities of synthesis and interpretation. Recordings just mean you have a way to “check your work” — referring back to a source recording to be sure your recollection of someone’s response was accurate.

OK, now I worry I’m starting to get into more general best practices in research repositories and data storage, so I’m going to pause on that topic. This post is really about how you can use technology to conduct user experience research so you’re not limited to in-person consults.

However, I will mention one more digital tool that our team used for synthesis. MURAL is an online whiteboard where we could store and organize our insights. This meant we could share artifacts digitally, and didn’t have to all be in the same room to organize our thoughts. We also didn’t have to worry about someone taking a picture of a whiteboard, and then trying to locate that photo later!

MURAL can take cells from a spreadsheet and create individual “sticky notes” for each, which makes entering them a breeze (now, if only there were a way to use some type of metadata to specify different colours at the creation phase.. feature request!)

You can do this

There will always be certain benefits to human connection and observing people in their natural setting. But when it comes to things like usability testing or relatively straightforward interviews, it’s much better to do research than to use perceived barriers as an excuse to exclude.

So, the next time you find your user research efforts restricted by physical proximity, I challenge you to try to conduct the interview remotely. Unless you or your participants are in a location with no internet access (and there are fewer and fewer of those places), I bet you can make it happen!