Recently, Supersonic, Unity’s mobile game distribution solution, released two important updates, including a new A/B testing page and a revamped “All Games” page. This update is designed to empower developers to fully understand the key metrics of their games and make more informed release decisions.
The new A/B testing page will revolutionize the traditional game testing process by providing more accurate and statistically significant insights before developers decide how to move forward with their games; while the revamped All Games page will Allow developers to quickly view the relationship between game KPIs (such as CPI, game time, retention rate) and have a clearer understanding of what needs to be paid attention to and optimized.
Brand new A/B testing page
This update introduces an innovative A/B testing method designed to improve the error that outliers and small test groups can cause on test results, so that the wider test situation is taken into account when analyzing and displaying test results. to improve test accuracy.
Supersonic from Unity’s A/B test calculator can provide targeted results to help developers understand the audience size required for a specific test, taking into account target KPIs, error rates and number of variants. And other factors. Additionally, it estimates the number of days required to reach statistical significance and the number of users reaching preset milestones. Compared with most A/B testing tools currently on the market, Supersonic from Unity provides more in-depth and comprehensive testing information.
Developers can also see the statistical significance of test results through the new A/B testing page. Rather than simply getting average results, developers can see the full range of results. Not only does this give them a clear picture of where the majority of results are, it also allows them to make more accurate decisions by excluding outliers.
In addition, the A/B test page can not only display the statistical significance of the test optimization target KPI, but also examine the impact of the test on other indicators at the same time. Developers can gain a complete understanding of the overall performance of their tests and use the data to make informed decisions about how to move forward.
This upgrade of A/B testing tools by Supersonic from Unity will have a profound impact on the entire industry. In practice, a positive test result will be ignored because the main KPI shows no statistical change. However, this may occur because the audience size is not large enough, or the knock-on effect of the test has a positive impact on other equally important KPIs. For example, a non-statistical change in ARPU could result in a significant increase in D7 retention and playtime, which would require adjustments to the test, a result that might otherwise go unnoticed.
By presenting the full effect of an A/B test and the statistical significance of the results, developers can see the effect of all metrics before deciding which test to move forward with. This unique advantage makes this A/B testing feature a leader in the market.
Newly revamped “All games” page
In the revised “All games” page, developers can create custom views and manage all their games. They can choose to view games that are in a specific testing phase (such as Initial Testing, Iteration, or Advanced Testing) and customize the KPI view based on the KPI’s importance. There are currently nearly 30 KPIs for developers to choose from, including CPI, game duration, retention rate, APPU (average game duration per user) and other indicators, and can be segmented by channel and group.
By aggregating important data into a single view, developers can instantly gain an overview of all their games. Based on this highly transparent game performance data, developers and the Supersonic from Unity team will be able to make faster decisions about which games need attention and which channels to choose for promotion.
