In-Survey Screening

Rebuilding audience reliability by streamlining how respondents qualify for surveys

Adding Screener Blocks to the survey creation flow reduced time-to-value, boosted survey fill rates, and set the foundation for third-party panel integration: turning a crisis in data quality into a scalable design pattern.

TL;DR

01 Context

Suzy researchers need to ensure surveys reach the right audiences within a target 3-day window. Currently, they are constrained to Suzy’s proprietary panel and depend on quarterly screeners through Crowdtap, where member retention and engagement are declining.

By mid-2024, Suzy’s Crowdtap panel—its proprietary and primary audience source—was faltering. Overzealous quality controls led to legitimate users being banned, survey fill rates dropping below targets, and Client Success scrambling to patch gaps via third-party vendors. The platform’s dependence on a single audience threatened speed, quality, and client trust.

02 Task

Redesign the survey flow to make audience targeting more flexible and reliable—enabling users to apply screeners to any audience type, internal or external. Scope included redesigning the Questions page, logic and quota dialogs, and unifying all audience controls under one modular, scalable pattern.

03 Process

I audited prior research and internal feedback to reframe the problem: instead of optimizing for Crowdtap performance, focus on ensuring any user could reach their desired audience, regardless of source.

After stakeholder workshops and testing multiple interaction models (“New Block” vs. “Collapse & Expand”), we developed a hybrid UI that integrated screener blocks, quotas, and qualifying logic into one cohesive flow. The final design introduced:

  • Reusable screener blocks within the Questions page

  • Consistent logic dialogs and modular containers

  • Streamlined quota setup via modal for better focus and scalability

04 Impact

  • 500+ surveys used screener blocks in the first 7 days

  • 1,500+ surveys salvaged, protecting ≈$4.95M in revenue

  • 66% of surveys completed within 3 days (+16% from target)

  • Client activation improved, saving an estimated $1.14M annually

Role

Lead UX Designer

Team

UX/UI
Product Manager
FE/BE Engineers
Data Innovation Team

Platform

B2B SaaS

Overview

Suzy is a market research SaaS platform that enables enterprises to independently launch studies with agency-level rigor. It is known for speed, data quality, and DIY-friendly interface.

    • Used by researchers, marketers, and innovation teams

    • 400+ brands use Suzy in their workflow

Crowdtap is a B2C survey rewards platform connecting real U.S. consumers with brands through surveys and other research methods.

Suzy’s business problem

  • Without speed or data quality, Suzy fails to deliver its value to our clients, regardless of the product design.

  • The current workflow of quarterly screeners toward a retargetable panel significantly delayed user activation and slowing time to value.

Crowdtap’s audience problem

  • Measures to improve response quality tanked time-to-fill, one of Suzy’s core value props, while also banning legitimate Crowdtap users.

  • Efforts to grow and engage audience to meet client demand have been futile.

Suzy users’ problems

  • Survey data wasn’t trustworthy due to bad quality data: in May 2024, audience quality attacks further eroded client trust and led to churn, especially when 25% of responses were spam.

  • TTF crashed as a result of Crowdtap’s quality control measures: only 21% of surveys had a 3-day fill rate (benchmark is 50%).

  • Launching quarterly screeners created periodic friction for clients: not only did they have to run this additional survey once per quarter, but they also had to proactively ensure each quarter’s screener would work for their research goals.

Platform limitations

  • Legacy architecture of the Suzy platform was not built to accept data from 3rd party providers.

  • Implementing a sustainable long-term solution will take time, but clients needed their speed and data quality 3 days ago.

Client Journey (Before)

  • In the absence of an in-platform onboarding experience, new clients normally get onboarded through a series of calls with their client success managers—even the DIY users. Also, a resource portal called Suzy Academy provides short courses and video tutorials on how to use the platform. 

  • Field qualifying questions to the entire Crowdtap panel to pre-identify relevant participants that fit your target audience. Screeners are created on Suzy and each brand can launch a new screener every quarterly.

    TTV: 2 weeks–6 months

  • When a client is ready for a project, they program the survey and launch on Suzy. 

    Suzy delivers the survey to the pre-identified Crowdtap audience. Crowdtap users answer surveys in exchange for points they can redeem for gift cards.

    Target Time-to-Fill: 50% of surveys should fill in 3 days

  • Clients review the data set to conduct analysis and extrapolate findings. Advanced researchers use our in-platform Data Explorer tool. A summary of findings is generated via OpenAI and can be exported along with results data.

    Users can continue to target the same audience based on their responses to a survey, or launch a new survey altogether to their broader target audience.

Our Challenge

For several years, we assumed Crowdtap was a key differentiator for our platform, setting us apart from other market research platforms that had no proprietary audience of their own. 

Following a cross-functional effort of competitive research, client feedback via advisory boards, and internal interviews with commercial teams, we learned it wasn’t—but speed and quality were. Without these, the platform loses all value to its users.

Our dependence on Crowdtap became more of a weakness than a unique value-add. We had to make the platform audience-agnostic, which left us with this problem statement:

How might we ensure any user can reach their target audience in order to conduct research that meets their needs?

Research & Prioritization

Surveys, Audits, and Interviews

Despite having a high-level problem statement that posed an existential risk to our platform’s viability, we needed a clearer understanding of our end users’ behaviors and needs to help us formulate a cohesive product strategy. 

A ChatGPT-supported, quasi-affinity mapping analysis of Jira tickets, AHA feature requests and Slack messages uncovered fairly consistent themes. Sustained client frustration with the platform’s quarterly screener requirement and declining survey fill rates dominated the array of pain points. 

Meanwhile, our internal interviews with cross-functional stakeholders also revealed the direct connection between our Audience teams’ efforts to combat spam and bot respondents and our users’ core need to run complete surveys within a short window. The “direct connection” was a direct conflict of interest. 

If we wanted Suzy to be audience-agnostic, we needed to identify modules within the current experience that relied on a Crowdtap audience. I also conducted a comprehensive audit of our to identify these opportunities. 

Stakeholder Workshops

I conducted virtual stakeholder workshops that aimed to build rapport, empathy, and give folks from other departments a voice in the design process. This was an opportunity for cross-functional teams, who likely have conflicting interests, to gather for an open and judgment-free exchange of ideas. 

Following a first round of pure ideation, I took the ideas and created three prototypes of potential interaction patterns/experiences. The second workshop was structured like a recorded focus group session to encourage an open and collaborative exchange of ideas among multiple key stakeholders.

While this research method has its limitations and may play into stakeholders’ biases, it’s an efficient way to get 5+ folks into the same virtual room and foster discussion. 

Main Constraints

We had a fairly small pod consisting of a designer, a PM, one backend, two frontend, and one QA engineer. We had a behemoth of a problem to solve, and needed to build out short- and long-term solutions simultaneously.

I still cannot believe we only had one backend engineer on this entire initiative. 

Prioritization & Tradeoffs

Instead of creating solutions based solely on feature requests and hypotheticals, we made sure to focus on the goals of our end-users, which in turn also aligned with the goals of our organization to restore speed and data quality. 

Tradeoffs were based on:

  • What was going to allow us to make good on the core value prop? (time to fill)

  • What will allow us to scale well and allow us to grow in the ways we want?

Solutions & Iterations

Rearranging the workflow

All our survey creation wizards feature an “Audience” step; its primary purpose was to set targeting criteria for Crowdtap respondents based on their demographics and previous survey responses—we called this the “Previous Targeting” participants. In an ideal world, this step would be scrapped altogether and we could fully streamline survey creation. But, like I said, legacy infrastructure. One backend engineer.

In-survey Screener Blocks

We introduced an in-survey Screener Question block so researchers can find qualified participants in real time. This major change untethered the survey creation flow from the Crowdtap audience piece and sets the Suzy up for an audience-agnostic future.

Because I had intentionally built quota logic as modular components, updating the user flows and adding a “screener logic” function required less “net-new” UI. The time savings from wireframing and prototyping created room for internal usability testing, speedier delivery, and a generous QA period.

We removed the “Previous Targeting” feature completely, since retargeting participants of quarterly screeners would no longer be a part of our offering. In addition, we moved Quota settings to the next page (the main questionnaire builder) so users could add quotas based on screener responses, regardless of the audience source.

Audience top-ups for short-term relief

First, we added an admin-only Link Generator widget to all Suzy surveys so that our Field Ops admins could grab these URLs and connect a survey to our chosen third-party audience vendor. As an internal tool, only a handful of employees had access to this feature, so we were able to design, test, and ship this as a quick-turn solution. However, this spiraled into a workflow that was error-prone and hard to manage. Our admins needed a tighter workflow.

We were able to quickly follow up with an audience top-up flow that allowed admins to connect to one of two vendors via API without leaving the Suzy platform. Our Crowdtap demographic points were mapped to the vendors, and our admins were able to perform this task more easily. But this was a band-aid approach that did not get to the root of the problem we wanted to solve.

Ultimately, we still needed a scalable, sustainable long-term solution that would fully eliminate our clients’ dependence on the Crowdtap audience. 

GTM messaging

Building toward an audience-agnostic platform also means training CSMs and clients to unlearn some of our platform’s mental models around key flows in the product experience.

I collaborated with our Product Marketing team to define a GTM strategy that gradually introduces current users to upcoming platform updates. Together, we devised a three-part plan to gradually roll out the user-facing updates with contextually relevant messaging.

Designs that didn’t ship

  • Reusable screeners

  • Updated UI components for cards and accordions

  • Sample size calculator on the Audience page

Key Features

  • Field Ops Support

    We rapidly shipped internal tools in the short-term to iteratively support our field operations managers who had to manually perform audience top-ups on surveys that were taking long to fill.

  • Expanded Question Types

    While Screeners previously only consisted of multiple choice questions, updated screener blocks would support more question types such as text open ends, scale/rank, and custom grid.

  • "Auto-Assign" Groups

    We increased the maximum number of Auto-Assigns from 1 to 3, enabling our more advanced researchers to create respondent segments for future analysis.

  • Screener Logic

    Working off components and patterns we built for the Data Explorer initiative, I designed a Screener Logic builder that enabled users to define the qualifying logic for their survey.

  • Quota Logic

    Moving respondent quotas from the Audience to the Questions page required minor updates to the Quota Logic UI. We changed this feature from a full page to a modal for consistency with other in-survey logic builders and updated how users select quota presets.

  • Quota Prioritization

    If a respondent qualifies for more than one quota “bucket”, they can be assigned to their group based on user-configured prioritization. I collaborated very closely with our front-end designer and QA team to define use cases, edge cases, and interaction patterns to create an intuitive and flexible experience.

Results & Impact

Impact on speed, quality, and usage

  • Speed → three-day fill rates jump to 66% 

      • ✅+45% from October 2024

      • ✅+16% from target

  • Audience top-up usage → 1,500+ client projects serviced

  • Screener block usage (launched Aug 2025) → 500+ surveys launched

Impact on the business

  • Retained clients for platform rigor and workflow vs proprietary panel

  • Created future growth opportunities via saved screeners, audience marketplaces, self-serve global research, and Suzy for individual users

  • Suzy is now able to handle data from third-party panel providers

  • Crowdtap audience quality no longer poses an existential threat to Suzy and can be sunset smoothly

Impact on client experience

  • Eliminated unnecessary friction and reduce time-to-value for new user activation

  • Created a repeatable GTM messaging rollout for new feature releases

  • Empowered user adaptability to platform updates without increasing cognitive load

Key Learnings

  1. Design with scalability in mind early, and you’ll thank yourself later. Building modular systems is key to speeding up handoff without sacrificing users’ mental models.

  2. Keeping your assumptions in check should be an ongoing, system-wide exercise. A “key differentiator” today can be your single point of failure down the line.

  3. Having more than one plan creates room for multiple wins across company functions. Our solutions for an existential product threat also improved activation as a bonus!

•

LAST WORDS

• LAST WORDS

This project underscored how fragile a “key differentiator” can become in the face of external inflection periods. What once set Suzy apart had become its weakest link. By designing modular systems and keeping long-term scalability in mind, we not only solved an existential product risk but also improved activation and reduced friction along the way.

It reminded me that designing for resilience means building pathways for growth, not just patching immediate problems. Sometimes the most impactful design work isn’t about what gets added, but what gets untangled.

Next

AI Screener Tool