Tips for creating custom in-product surveys
As a researcher for almost 15 years, I've designed, fielded, and analyzed thousands of surveys. And as UserLeap's Head of User Research, I've helped customers launch countless micro-surveys on UserLeap. I've learned that while most basic survey best practices are consistent no matter how you field a survey, in-product surveys are a unique case for which some additional rules apply.
Unlike with an email survey, in which users respond at their convenience, in-product surveys ask users to stop what they are doing and pay attention to something else. For this reason, your primary goal when crafting an in-product survey, aside from getting your key questions answered, is to make it easier for your users to take your survey than to dismiss it.
Following the rules below will help you achieve this goal, and in turn, you will maximize your response rate, minimize disruption to your users, and obtain high quality data for your critical business questions.
Rule #1. Make it easy to understand
The first step in making your survey easy to take is making sure your questions (especially the first one) are easy to read and instantly understandable. If it is difficult to interpret or overly wordy, visitors are likely to dismiss it and move on. I recommend aiming for a third grade reading level, although I admit achieving this is harder than it seems. At UserLeap, we like to run our survey questions through the Hemingway app to get a gauge on the difficulty and adjust as needed before launching a new survey.
Rule #2. Make it easy to answer
In addition to being able to quickly understand your question, it's critical to make questions easy to answer. This means minimizing usage of high-effort questions--particularly as the first question in your survey.
High effort question include:
- Open ends, which require users to construct a thoughtful response
- Sensitive questions, such as personal or financial information, that some users may feel uncomfortable answering
- Questions that ask users to imagine a scenario that may not apply to them
Now, it's worth noting that I suggested minimizing high-effort questions, not eliminating them. Open ends, for example, provide exceptionally valuable insight and we absolutely recommend using them where appropriate. However, it's important to be thoughtful about how many to include in your survey and when to surface them, as open-ended questions typically receive lower response rates than closed questions.
In particular, it's best not to start your survey with an open-ended question. While it's certainly possible to do so and still obtain responses, it's often more effective to lead with a simple rating scale or multiple-choice question that requires minimal effort and gets your "foot in the door." Users are then more likely to feel committed and follow through on your second question, even if it's an open end. The same approach would apply to other high-effort questions, such as sensitive questions.
Rule #3. Make it Relevant
When conducting in-product surveys, it's important to ask the right question at the right time. That means asking questions that are relevant to what users are doing at their stage of the journey, or on the specific page URL where you're reaching them. For example, you should try to avoid asking users who have been active for months about their onboarding experience or asking users on an account settings page how they feel about your search experience. Instead, questions about onboarding should ideally be targeted to users who are currently in the onboarding process or have just completed it. And if you're interested in the search experience, the best place to target users is on the search results page.
Rule #4. Ask your most important questions first
For in-product surveys, it’s crucial to get the information you want the most right away, in the first 1-2 questions. This is because, generally speaking, response rates decline with each additional question, and this is particularly true for open-ended questions. It's often better to launch 2 separate, shorter surveys than one longer survey with critical questions at the end. They will receive less notice and have a substantially lower rate of response.
Rule #5. Keep it short
The response rate declines for each additional question you ask, so choose your questions wisely. While you can include up to 5 questions in your surveys, we recommend no more than 3 questions for in-product surveys. If you need to ask individual users more than 3 questions, it's probably better suited to an email survey.