Recruiting customers with feedback panels

Background

Ever wondered how the federal government, cities, towns and other groups get a .gov domain for their sites? The Cybersecurity and Infrastructure Security Agency (CISA) manages all these domains across the web. Similar to .com, .org, or .us, .gov is a ‘top-level domain’, or TLD. Enterprises use a TLD to register a domain name (often simply called a domain) for use in their online services, like a website or email. CISA sponsors the .gov TLD and makes it available solely to U.S.-based government organizations and publicly controlled entities.

We worked with our partner at CISA to build a minimally viable product (MVP) and new website for the .gov registrar. As part of the work, we planned multiple rounds of testing with a mix of participants to give us feedback. We also wanted to collect feedback from current customers as part of our research, so we suggested CISA build a customer panel.

A customer panel is a group of selected customers who voluntarily provide feedback, opinions, and suggestions about the registrar and the possible future state. This connection with real customers gave us an opportunity to understand what matters for people who use the registrar now. It’s often a faster method of finding participants “from the inside out” of a site or service as opposed to a broader, more resource intensive recruitment process “from the outside in.”

Some of the benefits of building a customer panel:

• Direct contact with current users via site feedback as opposed to longer, external recruiting cycles
• Feedback from users who understand the service
• Identifying areas for immediate improvement (fixes, accessibility updates, and more)
• Identifying new features for a product’s roadmap (net new features and long-term changes)
• Building relationships with current customers for future feedback sessions and programs

Approach

Current users of the service receive an email notifying them when their request has been approved. CISA added a request for feedback to that email, asking users if they would participate in a brief interview to share their thoughts on the present and possible future state of the service. We screened participants to understand their needs and goals, and asked if they would be interested in seeing the potential future state of the site. Those who agreed went into our recruiting pool to test the MVP.

PRA implications

Asking participants if they’d like to provide feedback does not necessarily initiate the need for the Paperwork Reduction Act (PRA). If the call for feedback is open-ended and asks for no information to schedule a session other than a name or email address, it falls under the Social Media Guidance memo from 2010.

From the memo Social Media Guidance, dated 4/7/2010:

Feedback requests.

Under existing OMB policy, agency uses of general or undifferentiated “suggestion boxes” are not covered by the PRA. Similarly, an agency does not trigger the PRA’s requirements when it posts its email address or uses an application for brainstorming or idea generating on its website to enable the public to submit feedback. However, if an agency requests information from respondents beyond name and email or mailing address (e.g., age, sex, race/ethnicity, employment, or citizenship status), this request is covered by the PRA because it seeks information beyond what is “necessary” for self-identification of the respondent.

Outcomes

We conducted two rounds of tests on our MVP of get.gov and the new application process. Participants recruited through the current site were especially crucial in giving feedback to the team, helping us to measure value added against the current site.

Thanks to the panel, we were able to:

• Update copy to reflect the language and terms used by the public
• Make usability updates to the design system and form functionality
• Generate ideas for future features around domain management and updates

Key points when considering a customer panel

If you are considering building a customer panel, here are some tips:

• Make sure your call to action and feedback mechanism adheres to PRA guidelines.
• Clearly define the purpose of your panel — what type of feedback sessions or methods will you use?
• Identify the criteria of your ideal panelists — are there segments or demographics you need feedback from?
• Set expectations on when you expect to reach out to them. Are you running testing now, or will you send surveys in a month?
• Use the information you gather to improve your products and services — turn feedback into copy, functionality, or service design updates.

Other resources

• OMB’s Social Media Guidance
• User Interviews