Tag: user research

Anatomy of a good sticky note

Leisa Reichelt

Originally published on the GDS User Research Blog

For many of our teams, the ‘unit of data’ in qualitative user research is the sticky note – or, more precisely, units of information captured onto sticky notes. People often think this is amusing, given we’re always talking about digital by default.

Using sticky notes to analyse research helps us take a fast yet thoughtful and collaborative approach to analysis, and in a way that no digital tool I’ve come across allows. There’s just something about moving bits of paper around…

As a result of making and analysing so many sticky notes, we’ve become a bit opinionated about what a ‘good’ user research data sticky note looks like.

There are good, bad and downright useless ways of recording data onto sticky notes.

Here’s what we’ve found works best:

  1. Capture verbatim quotes or direct observations. Don’t make analysis notes on the sticky note. Capture exactly what you’ve seen or heard the participant say. Let the sticky note become the voice of the user.
  2. One data point per sticky note. No bullet points. During analysis, you need to be able to independently move quotes and observations around, so don’t group them on one note.
  3. Code sticky notes with a participant ID. Mark each sticky note with a code that will help you identify which participant said or did the thing you noted. An easy way to do this is to number participants in the order you see them. Our sticky note example shows a number 5 – this note refers to the 5th participant in the study.
  4. Use colour thoughtfully. Colour coding can be very useful when you’re doing analysis, so think about how to use it meaningfully. We capture all observations on one colour sticky note, then use a different colour to indicate findings and another colour for actions. Some people like to use different colour sticky notes to indicate things that went well and things that were problematic. Other people use colour to indicate different types of participants, for example, ‘confident online’ and ‘not confident online’.
  5. Thick pens. Match the thickness of your pen to the size of your sticky note. Never use a biro. We usually use 3×5 inch sticky notes and a pen with 0.5mm width.
  6. Encourage observers to write in capital letters. Although it’s slower, it makes everyone’s handwriting more legible, which is important when you’re doing analysis.
  7. Decent quality sticky notes. There’s nothing more annoying than having your sticky notes rain from the walls while you’re trying to do analysis. When it comes to sticky notes, the glue really matters and you get what you pay for.

Follow these guidelines, and you’ll be set for a great research analysis session.

Sample size and confidence: how to get your team to trust qualitative research

Leisa Reichelt

Originally published on the GDS User Research Blog

If you’re doing qualitative design research, don’t worry about sample size. Sample size and statistical significance don’t matter*.

The only thing that matters is how confident your team is about the next decision they need to make** 

You might get that confidence from watching 2 people.

You might need to watch 20.

You might need to watch 20, then check your hypothesis at a larger scale with a survey or analytics.

It all depends on the confidence of your team.

There are only 2 things you need to do to start building confidence in your next decision:

  1. Start watching end users using the thing you’re making. (Do research in every iteration.)***
  2. Make sure enough of your team observe the research. (Research is a team sport, help your team get their exposure hours.)

As soon as you start doing this, you’ll be so busy working out what to do with everything you’re learning, you’ll have no time to worry about statistics.

You’ll know what is broken and what is not, and you’ll know what you need to spend more time learning about.

It’s as simple as that.

Footnotes

* Obviously it’s better to see a lot of people over very few

We recommend you see a handful in each iteration – keep iterating, and you’ll gradually get a larger and broader sample. Also, the ‘Is 5 people enough?’ question often discussed by usability people is irrelevant when you’re seeing 5 people in every iteration.

** Your next decision might be…

  • How do we fix the question in this form so that people can answer it better?
  • Is this service meeting the user needs?
  • What is the next most important thing we should prioritise in the backlog?
  • Does this work better than what we’ve currently got live?

*** Who do you watch?

It’s tempting to launch into a complicated segmentation exercise. Don’t do that. There will be some obvious segments – just start with those. Start talking to people and the segmentation that really matters will emerge from those discussions.

The important segments will sometimes be things you couldn’t have guessed at before you started meeting people. Start talking to the easiest people. Pick the people who are most likely to be able to (and want to) use your service and test with them. If your service doesn’t work for them, it won’t work for anybody.

As you begin to solve problems, make it harder for yourself by inviting people who are going to be more difficult – those with edge case and the complex user needs and those with lower (or no) digital literacy.

What user researchers do when they’re not researching

Leisa Reichelt

Originally published on the GDS User Research Blog

A lot of teams aren’t used to having a user researcher embedded full time. When we suggest that they should, we’re often asked:

But what will the researcher do when they’re not conducting research?

We’ve discovered that our user researchers are only actually researching about 30% of their time.

Whilst it is our job to understand end users and how they interact with our service, we think our most important job is this: to make sure that everyone in our team understands end users with the same empathy, accuracy and depth as we do.

Here are some of the things that embedded user researchers do when they’re not researching:

We work with the team to improve the service

  • Analyse research with the team, understand the key insights and work out what actions to take as a result.
  • Work with designers, content designers and developers to make prototypes for testing something new. We sit with the designer and developer so we can chip in and answer questions about what’s being made and why.
  • Spend time with the business analyst or product manager to prioritise features and stories. We also work through questions thrown up, as a result of research, around user stories and acceptance criteria.

We communicate what we know

  • Give research ‘playbacks’ and share key research insights with the project team and stakeholders.
  • Compile video clips from the research so those who couldn’t attend can see real users for themselves.
  • Write blog posts on our research findings.
  • Prepare and deliver presentations, demos and show-and-tells.
  • Make things to post on walls  to keep people thinking about our users, their needs and the challenges we need to overcome to make a great service for them.

We prepare for research

  • Organise upcoming research participants by either writing screeners for agencies or contacting potential participants directly.
  • Work with the design and development teams to work out future user research needs and to create research hypotheses.

We look at site analytics to…

  • See how, and if, we can use analytics to quantify findings we’ve observed qualitatively.
  • Spot patterns in the data and identify future research needs.

These activities are essential in making sure that the time and effort put into user research isn’t wasted. Research is only valuable if your project team understands it and acts on it. These activities help make that happen.

At least 3 days a week

We recommend that user researchers are embedded in a team at least 3 days a week. In fact, most researchers find that just 3 days a week leaves them stretched for time in getting everything done.

It’s a different approach to research than most are used to, but in agile projects, we’re finding it’s the only way that works.

User research not just usability

Leisa Reichelt

Originally published on the GDS User Research Blog

Our user researchers work with project teams throughout a service lifecycle. They’re not just testing usability, they’re researching with users and feeding insights back to the team all the time.

Here are some of the things our researchers bring to a project team:

  • Understanding user needs: identifying them, framing and prioritising them correctly.
  • Informing the service design: helping build a picture of the end-to-end service across channels – both current and ideal service states.
  • Guiding the product strategy: helping product managers understand which stories (and features) are most important and how best to prioritise and group them to deliver value to end users.
  • Providing feedback to policy design: some things about policy can only be learned when they’re being implemented. User research offers opportunities to gather feedback useful in shaping policy design.
  • Helping the entire team maintain an empathetic view of their end users: it’s easy to fall back into stereotypes, assumptions or self-referential design. We help the team make decisions based on an understanding of end users.
  • Improving usability of the digital service: we make sure that the digital service is designed so that it’s as easy as possible to understand and use.
  • Understanding assisted digital needs: we help service managers understand challenges faced by users who require assistance in using digital services and help explore the best ways to support those challenges.
  • Identifying and exploring opportunities to promote channel shift to digital: we help service managers identify moments in the service that are opportunistic for encouraging people to choose digital. We also help experiment in making channel shift more effective.

When research doesn’t happen right

If you’ve had a bad experience with research in the past, chances are research wasn’t happening at the right time or in the right way.

Front load research too much and you might end up with rich insights about your customer base and what they want, but you can also end up with lots of great ideas that aren’t technically or commercially feasible.

Leave research until the end of the project and you run the risk of applying ‘lipstick to a pig’, as in, making something really usable but which no one wants to use because it’s not actually satisfying a user need.

Get real value from user research

By including user research throughout your project, you can make sure that you’re always getting actionable insights, learning things that help you make informed day-to-day decisions about your project, as well as continuing to grow a strategic understanding of your service and your customers.

That’s how you get real value from user research.