Like any job that needs to be done, carrying out research requires you to find the right tool for the right job. 

This week we’re looking at research methods.

Read time: 3 mins

tl;dr: There are certain research methods that are right for certain jobs – eg. surveys for measuring and interviews for deep-diving into a topic. But – importantly – there is a wide range of alternatives and newer methods you may wish to consider.

I’m going to give you a breakdown of the main methods I use and why.

Tool selection

Before getting into that, let’s take a look at how you select your method of research – how you pick the right tool for the job.

The fundamental decision you need to make is between quantitative and qualitative methods of data collection.

But let’s cut the jargon – this essentially boils down to whether you need to…

1. Measure, assess the size of something, take a broad sweep of a topic

2. Go deep into a topic, looking for nuance and details

In an ideal world, you may decide you need a little of both. There is also a spring of new platforms which suggest they can do both at the same time – ‘quali-quant’ solutions. I am a little dubious of this.

Let’s start with measurement, the obvious tool being a survey.

Surveys

• Good at measuring or estimating the size of something – eg. the size of the problem you’re trying to solve or assessing your brand awareness

• Provide a size and level of data which can give you confidence to make big decisions – particularly true when you need to influence those on the c-suite

• Everyone answers exactly the same questions, same wording in the same order which provides consistency in the data

Surveys are less good at providing you with depth of information or unearthing the nuances within certain topics.

For that you need something like…

Customer interview
– aka user interview, depth interview, semi-structured interview

Very ‘of the moment’ in the startup world, lots of advice out there urging founders and their teams to set up interviews with customers and potential customers to collect vital feedback.

I can only applaud this – as long as it’s the right approach for the given situation.

Some features of interviews…

• One-on-one time with a participant to be more exploratory with your questioning

• Typically follow a ‘guide’ with a set of questions/topics as well as allowing the interview to be more free-flowing conversation, allowing for things to come up that you hadn’t previously thought of

So very useful, BUT…

• Interviews don’t allow you to measure the size of your market opportunity

• They lack any discussion or dynamic that can be fostered within a group

• They can be very time consuming

Group discussion – aka focus group

Less talked about and have come in for some criticism due to a tendency for ‘group think’. 

But this form of data collection can be incredibly useful for…

• Developing ideas – with a level of discussion that you won’t be able to generate in a one-to-one interview

• Ideas and thoughts to be generated which would otherwise be lost

• A way to carry out research with more people, more quickly

New methods

There’s a much longer list of research methods than I’m covering here, many of which are new or hybrids of the old and the new. Some of the new ways I’ve been working over the last few years…

Online group studies

Typically, the group discussion has been lifted and shifted to an online environment such as Zoom. Group size will need to be reduced from typically an 8-person focus group to a 4-person online to make it more manageable.

But this ‘lift and shift’ approach doesn’t allow for the benefits of online research. I’ve found that a more ‘deconstructed’ focus group that  doesn’t happen live, in a 90-minute window but which might take place over a series of days or weeks works incredibly well.

Participants take part in their own time, in the comfort of their own environment and are given a set of activities, questions and tasks to complete eg.  posting their thoughts on a topic, through to providing a screen recording of them using a website etc.

I find the benefits of this type of data collection are…

• A level of honesty you might not be able to foster as easily with a traditional in-person group

• The ability to iterate – to use the learning from day 1 to influence the questions and activities set on day 2 etc

• A wider variety of activities which creates a higher level of engagement

Online listening

In theory, the ability to listen in to conversations that are happening online around topics you are interested in is super exciting.

In practice it’s a little more difficult in that (at the time of writing) certain platforms will not give access to their data and, despite advances in machine learning, you do need to spend an inordinate amount of time cleaning and fine tuning your search queries to be certain what you’re picking up is relevant.

It’s the future though and definitely worth considering.

This brings us onto the idea of observing behaviour vs asking questions (about behaviour).

Observation vs asking questions

In certain circumstances the value of observing people will deliver huge insights over asking them for their opinion. Tapping into ‘system 1’ thinking (the fast, automatic, unconscious, and emotional response) is much more likely by observing behaviour. 

Example – years ago I was working in TV broadcasting at the time when new tech allowed audiences to skip ads. It was an existential crisis in commercial TV. However, through observational research, we were able to see (to prove) that whilst people claimed never to watch ads (thanks to their ad-skipping tech) in reality – by observing them – there were clear examples of not only them watching TV across the ad break but in some cases, rewinding back to a favourite ad to watch it again.

Observational research

Aka ethnography has in the past been very time consuming, therefore very expensive. Recording people going about their daily lives and then sifting through the footage is prohibitively expensive for most businesses, let alone startups.

But technology has allowed platforms to spring up to offer a more hybrid approach to ethnography, largely through the use of smartphones. Observing behaviour from a UX perspective is also much easier to achieve these days through new tech platforms.

I could carry on writing about this topic for ages so if you’d like to chat through more about research methods and tools, you know where to find me.

Book a chat with me at meetgideon.com

Thanks for reading