2. Remember to test only one variation at a time for it to be a true A/B test: The testing stage will require two variations of your app screen – the current version and your re-designed version. When creating these variations, using an A/B testing platform will make it easy to design, run, and monitor your tests.
3. Run the experiment: Time to test your results. Set up your app to randomly show your original set-up to half of your users (i.e., the “control group”) and the second variation to the other 50% (i.e., the “experimental group”). By using a control group, you’re collecting baseline data to compare against your results. Without it, you can’t tell the difference between the response to your new designs or other variables, like seasonal chance.
4. Make a decision: Once the experiment is done, it’s time to crunch the data. First thing to do is to revisit your initial goal and hypothesis, and make that all-important final calls on whether the new variation is worth changing. Don’t be too hasty to lock in a new look. If the changes are significant, it’s smart to run the experiment over several time periods to ensure the results aren’t due to seasonality, or other variables.
As you continue to run more tests, remember that even with helpful tools, testing takes time and resources. Don’t waste time testing elements that won’t significantly impact your goal. Use app analytics data to help uncover spots in your app with a lot of opportunity and potential (think: screens with high traffic, high engagement, or large user drop off, for example). A good idea might be to have a devoted team member spend 25% of their time on monitoring analytics, identifying ad optimization ideas, and testing them.