Make research actionable
Research with people is most valuable when it leads to shared understanding. Analysis, synthesis, and sharing help us reflect on the data we’ve collected and determine a course of action that involves the broader team.
From data to insights
Articulating insights can involve different activities, but it’s primarily a time for everyone on the project team to work together to begin to map out larger patterns and themes.
At 18F this generally involves:
Making meaning always happens in relation to the research questions and/or problem statements identified during research planning. Some questions you might ask:
Multiple user groups might require multiple solutions
The public is made up of people with different needs and might require different designs. Most products are set to solve things for their majority user. Accommodate different contexts, access methods, languages, and user cases. If you are designing one path to a government service, who might be left out? Consider:
- What are the tradeoffs around prioritizing the “majority” of users over “edge users?”
- How might cognitive load and tech savviness imply different needs from a website or require different navigation paths through a service or product?
- How might work schedules, access to transportation or physical ability impact when and for how long someone might be able to use the in person component of a service?
Involving the team
Finding patterns in research is a great time to involve partners and team members, especially people who were not involved in conducting the research. A collaborative approach helps to:
When inviting partners, think about how exposed they were to the research process and set clear expectations on how you would like them to engage. Be mindful of power dynamics and your ethical obligation to respect the people who participated in your research. For example, you might want to be careful when involving a CIO in analyzing data you collected while interviewing their employees.
Before conducting shared analysis or synthesis, make sure that any quote could be attributable to multiple participants so no one person can be identified as the person that said it. This is especially important if your stakeholders know your participants or you are working with people from underserved communities who might be negatively impacted if they can be identified. For example, you might want to be careful when involving a CIO in analyzing data you collected while interviewing their employees.
Making meaning out of research can involve any number of different methods [18F design methods]. The most basic method is writing a summary of what you found. Write a brief summary of the data you collected that, when read together with your research plan, documents the research itself: the questions you started with, how you went about answering them, what you learned, and why it matters. Aim for true and useful, rather than comprehensive.
Outside of writing a summary, you should choose your methods based on the desired outputs and outcomes of your research. For example:
Whichever method you use, introduce it to your partner as well so they can build their research skills.
Any approved tool can be used. We’ve listed some of our approved, commonly used tools.
Insights into action
Turning insights into action helps us answer and communicate:
- What can we do to address the issues we identified?
- What impact do we think our changes will make?
- What do we need to do next?
Artifacts that we use to do this include:
- Research findings presentation template
- Design hypotheses [18F design methods]
- Design principles [18F design methods]
- Personas [18F design methods]
- Mental models [18F design methods]
- User scenarios [18F design methods]
- User stories
- Storyboards [18F design methods]
- Journey maps [18F design methods]
- Service blueprints
- Prototypes [18F design methods]
- Other creative outputs
These artifacts are tools for communicating the answers to your research questions and prioritizing opportunities for action. When choosing which artifacts to make, consider your audience and think about what will best create a shared understanding of your findings. For example, imagine you are describing an experience that is often a challenge for a specific group of users. In this case, personas accompanied by a journey map may be more effective than a set of design principles.
Making decisions based on the research
After each round of research, the whole team should identify how the research findings change the work planned for the next sprint or for future service design efforts. After discovery research, this could include prioritized areas for further exploration or prototyping. For a new or existing product, this could include new bugs identified, new features to explore, or a different design focus.
Artifacts are most useful when you act on them. For example, a current-state service blueprint could highlight the lengthy amount of time it takes for the public to receive certain benefits, while also highlighting the constraints that staff experience processing applications. What do you do next with these learnings? Here are some activities your team could do to act on these opportunity areas:
- Develop user stories: A user story is one or more sentences in the language of the user that captures what a user needs to accomplish. You can write it like this: “As a [X], I need [Y] so that I (can) [benefit].” Based on the learnings from the service blueprint, you can create user stories for the public and staff that can inspire recommendations. These user stories can be incorporated into sprint planning at the ticket level, where you can also link back to the service blueprint that inspired these user stories.
- Conduct a prioritization exercise: Your team may have a variety of ideas to reduce the time it takes to deliver benefits for both the public and staff. You can set up a two-by-two matrix on a virtual or physical whiteboard, with “level of effort” on the x-axis and “level of impact” on the y-axis. With input from your team’s product owner and any teams impacted by these ideas, like a representative from the staff processing benefit applications, you can plot the ideas from your service blueprint on this matrix. During this exercise, it is important to consider how proposed changes to the product or service could impact the staff delivering the service, which is why it is important to have a representative from the staff participate. Ideas with high impact and low effort could make sense to prioritize first, while ideas with high impact and high effort could be revisited later.
- Incorporate recommendations into the product roadmap: Based on the results of your prioritization exercise, you can slot recommendations into your roadmap, using a “Soon/next/later” framework.
- Keep your metrics up to date: Review what you are measuring over time and evolve them if new ways of measuring impact become more useful. What is important to measure may change over time or when better data becomes available.
Communicating your proposals is critical to seeing them come to life; even the most beautiful artifact loses its power if it’s left on a shelf. Here are some good guideposts for sharing and socializing research in government.
Before you share
At this point it is a good practice to delete any recordings from the sessions to further protect the privacy of your participants (and be sure to ask that anyone you shared recordings with to do the same).
We always try to make sure we are on the same page as the partner in protecting PII before publishing!
Understanding your audience
Our research often gets shared far beyond the project team. Depending on the project, stakeholders that care about our research could include:
Sharing research is all about storytelling. When packaging research, think about what role the audience plays in that story and what the outcomes will be for them. Being clear about who the audience is, what they need to understand, and how they might best understand it, helps us communicate our findings more effectively.
- Be careful of the conclusions you draw from any one study. Be honest to yourself and your partners about what you can and cannot conclude based on your research. Do not overstate your findings.
- If data will be shared outside of its original purpose don’t remove the context around it. Recontextualised data doesn’t not honor the people who provided the data.
Presenting the work
We most commonly share our research findings via presentations. These presentations can vary widely based on the audience. Here are a few presentation-building tips:
- Utilize our 18F-branded templates found on the 18F brand site to maintain consistency and save time. You may want to use this GSA-only: Research findings presentation template as a starting point.
- The presentation deck should tell a compelling story and be easy to read. Make sure to include enough content so those not able to attend the presentation can view the deck later and understand what you’re aiming to communicate. Refer to this presentation on GSA-only: How to design a better deck for additional pointers and guidance.
- Check out the GSA-only: Project resources folder for reusable content and templates; browse project artifacts from previous 18F engagements on GitHub for inspiration; or view the Design wiki on GitHub for additional examples.
- Feel free to include references or links to further reading at the end of your presentation.
Avoiding stereotypes and generalizations
Avoid stereotyping and generalizing in how you present findings, especially when talking about underserved communities and historically marginalized groups. Critically look at the quotes and information collected to determine if a quote may reinforce stereotypes.
Keep in mind that a researcher’s job doesn’t end with a formal presentation. Insights from a research sprint should be used to inform decisions for the rest of a project and to revisit on an ongoing basis. Always look for lightweight ways to share research and advocate for the voice of users.
One of the ways we do this at 18F is sharing a “weekly ship” of what the team has worked on each week. This could be an opportunity to share a compelling quote or pain point uncovered between rounds of research synthesis.
Sharing data with your participants
Data collected from research participants of specific groups should be available to them to use in ways they seek fit. While it’s not always relevant or practical to shareback primary data collected with all research participants, it’s crucial to the efforts of Tribal data sovereignty. Indigenous data sovereignty is the right of a nation to govern the collection, ownership, and application of its own data.
Tribal Nations have not historically had complete control or autonomy over their data and knowledge. In order to fulfill Federal trust and treaty responsibilities to Tribal Nations, federal employees should promote ethical research and enhance the research capacity of Tribal governments by providing people who participate in research with the opportunity to have the data collected from them.
Publishing on GitHub
While it is not mandatory, we often use GitHub to manage our projects. GitHub also allows us to keep the public up to date on our research findings. It is also a useful tool for communicating with potential vendors who may work on our projects in the future. Publishing findings can help vendors and the public better understand the problem a partner is trying to solve and how they are approaching it. It also signals that the partner wants to work with a vendor who is also willing to work in the open.
Handing off research responsibly
Research handoffs occur when wrapping up a round of research or an engagement, when vendors onboard, or when staffing arrangements change. Managing handoffs thoughtfully can help future researchers get up and running faster. At this point, ask:
In addition to the above considerations, providing an introduction or tour of the work is a helpful way to transfer knowledge to new team members or vendors. Consider writing a GitHub README document which new team members can read and refer to for project background, context, history, and quick links to important files or resources.