Tag: user research

It’s user research, not user testing

Leisa Reichelt

Originally published on the GDS User Research Blog

At least a dozen times a week I find myself saying to someone:

We call it user research not user testing. We test our design, our words and our ideas. We don’t test our users.

It’s a little thing, some might call it pedantry, but we think it matters.

The way we talk about things shows the way we think about things. It’s important to remember that people come to participate in user research to help us learn whether the current approach we’re taking to service design is meeting user needs or not.

It’s us being tested, not our users, and that’s a good thing.

It’s how we learn and continue to improve the quality of work we do, whether we’re designers, developers, researchers, product owners, security people or policy makers.

If people can’t use the thing we’ve made, that’s a reflection on us, not our users.

This is particularly true for us. We’re people who work in government and have a responsibility to make things work for everyone who wants to do things online, no matter their level of digital literacy.

You can, if you like, call it usability testing. That’s okay, if the thing you’re doing is testing the usability of the interface design. Most often though, it’s user research: a mixture of usability testing and more generally trying to better understand our end users so we can make better services for them.

5 ways to help user research work better in agile

Leisa Reichelt

Originally published on the GDS User Research Blog

User research doesn’t have the best reputation for fitting well into agile project teams. Agile requires speed – getting useful insights to an agile team quickly can be difficult to achieve.

There are 5 things that have helped us make user research more effective in agile:

  1. Dedicate a user researcher to each project team
  2. Do research in every sprint
  3. Evaluate the now/explore the future
  4. Gather both tactical and strategic insights every sprint
  5. Communicate research to your team in many ways

It’s taken us a while to settle on those 5. I’m going to explain what we used to do compared to what we do now.

1. Dedicate a user researcher to each project team

We used to treat user research like an internal consultancy. Teams would brief a researcher to do a piece of work and the researcher would either run the research themselves or procure the work from an agency.

In those days, a 4-week turn around was considered fast for user research. By the time the research came back, chances were high that the team had moved on a sprint (or more) and the research was irrelevant.

Also, because the researcher knew little about the project when they were called, there was no guarantee they were researching the right thing.

What we do now

We now aim that researchers have at least 3 days each week sitting with the team.

This lets us design better research projects and we’re able to learn useful things at the right time for the project’s needs. We can do more than just usability test interface elements and, because we’re now sitting with the team, we can make sure that designers and product owners are able to make practical use of what we’ve learned.

2. Do research in every sprint

We think that agile sprints are one of the best things to have happened to user research.

When a user researcher joins a project team, the first thing they do is make sure a round of research happens in every sprint.

We mostly do lab-based, 1-on-1 depth sessions – it’s the easiest way for everyone in the team to observe the research and get exposure hours. We use other research methods in addition to this regular ‘heartbeat’ of research.

User research is a team sport – an important team mantra.

The wins

Researching in every sprint helps the team maintain a learning attitude throughout the project. This is the best and most valuable outcome of integrating researchers into agile teams.

It also allows us to experiment at low cost with a range of different approaches to service design and we’re continually learning about our audience – what’s important to them and how best to design the service to their needs.

3. Evaluate the now/explore the future

We don’t worry about how many sprints we should be ahead of the developers – we believe researchers need to be all over the service all of the time or at least traversing the ‘now’ and the ‘future’ on a very regular basis.

Evaluating the now

The role that people seem most familiar with in digital teams is evaluative.

Evaluation asks, through usability testing, whether the thing we’ve designed is doing a good job of meeting user needs and is easy to use or not.

Exploring the future

Regular experimental research (using a light-weight prototyping environment) gives the team the opportunity to explore a range of different approaches to design and content and to understand whether the effort to bring those things to reality would be worthwhile.

This is critical in the earliest project phases (alpha and beta) but also supports the continuous improvement required by the Digital by Default Service Standard.

4. Gather both tactical and strategic insights every sprint

Agile research doesn’t restrict you to ‘tactical’ insight only.

Research in each sprint should be both tactical (answering design hypotheses) and strategic (understanding the audience and what’s significant to them in the context of the service).

The trick is to make sure you’re asking strategic questions regularly then making time to update your models and communicate them back to the team so they’re useful.

Our researchers create models, personas, journeys etc. to capture strategic insights, which they update and share with the team as they learn.

… which forms a nice segue to the last point.

5. Communicate research to your team in many ways

Different teams have come up with different ways of communicating key strategic insights to their team, from posters to slide decks.

We don’t produce research reports in agile teams. They take too long to deliver and, when you’e involved in day-to-day design decisions and prioritisation of the backlog, they’re not necessary.

We have another mantra:

Don’t do research for researchers, do it for the team.

Communication is our way of delivering on that.

Sit with the team

Communication happens because the researcher is co-located with the team and can represent the end user when the sprints are being planned and stories are being prioritised.

Communication also happens when the researcher is sitting next to the designer while the interface is being designed – they can provide feedback then and there based on what they know from research.

Present and share

Researchers present at regular show-and-tells (where we show videos from research) and find opportunities to ‘play back’ the research to team members who weren’t able to attend.

We also share the results of strategic research on walls – personas, journey maps and interfaces annotated with research findings.

In conclusion

These 5 points have made a big difference to the effectiveness of user research in our agile teams. We hope you’ll find them useful too.

How we do user research in agile teams

Leisa Reichelt

Originally published on the GDS Blog

Getting user research into agile teams in a way that is timely, relevant and actionable is a challenge that teams the world over are tackling. Working effectively in agile has recently been the driver of some fairly significant changes to the way our researchers work at GDS.

In the early days of GDS, the User Research team worked across the different projects here, as well as helping support the exemplar transactions. We did a mix of in house research, as well as managing usability testing and other research that was outsourced to specialist companies.

Using this model we found it difficult to provide insight to teams quickly enough, and researchers were spread across projects to the extent that they were never really a part of the team shaping the design or developing real depth of expertise in a particular product or its audience. This was less than ideal for both project teams and researchers.

We’ve been iterating how we do research in the same way we iterate our product design, and learned that the following techniques seem to integrate good research into agile teams more successfully.

Dedicated researchers for each team

Rather than a team of researchers taking research briefs from lots of project teams, each project team has a dedicated researcher working closely with the team of designers, developers, content designers and product owners.

This allows the team to be closer to the product design and adopt a more ‘experimental’ approach  hypothesising about what design or content approaches might work and designing ways to measure what is more or less successful.

Test designs at least every fortnight

The ‘two week rule’. We don’t design anything for more than two weeks without watching real end users interacting with it. This means that most teams are out in the field or in the research laboratory at least every fortnight, putting our design and content in front of potential end users.

Everyone in the team should take part

We believe that research is a team sport and encourage all team members to observe research sessions for at least 2 hours ever 6 weeks. There is good evidence that this helps teams create better products. And having dedicated researchers in our teams makes it easier for everyone on the team to get regular opportunities to watch people use the product they’ve designed.

A varied toolkit

It’s easy for teams to get comfortable with a small set of research methods and to use those for everything. An experimental mindset requires that we are always looking for better ways and a more varied research toolkit to help us get a richer and more accurate understanding of our users and their needs.

We use a mix of qualitative and quantitative methods and work closely with our web analytics colleagues to design studies that allow us to better understand how people respond to interface design and content using A/B testing and detailed path analysis in early prototypes.

See it through from analysis to action

Getting interesting insights from research isn’t hard. Getting those insights into the design of products can be surprisingly tricky. We analyse our research collaboratively and make sure we extract both findings and actions from each study. Findings build our understanding and go into our research library. Actions go into the project backlog, into sprint plans and therefore into the product design.

We don’t do powerpoint presentations or detailed reports and we use the time we’ve saved from that to work more closely with the team.

Sharing what we learn

As we do more research in more of our teams, there is a lot to gain by sharing our findings. We’re experimenting with ways to capture and share research across teams and even across departments, so that research becomes a real valuable and reusable asset for government.

We’re iterating how we work in the same way we’re iterating our projects. As we find better ways to work within teams and within the agile framework, we are better able to help teams build empathy with users and shape products that are focused on their needs.