2015 is known as an adjustment to the mobile game industry in China. With so many new phone producers, the amount of new devices as well as compatibility with the newest versions of Google Android has become a major concern. Quality has also become a necessary measure of a successful game’s life cycle as saturation of market competition has increased this year.
Mobile testing experts TestBird has compiled their study of the compatibility issues in mobile gaming for the beginning of the year and released “China’s Mobile Game Compatibility Testing for the First Half of 2015”, their first English language white paper.
Statistical data from the white paper on the entire first half of 2015 was developed using TestBird’s automated compatibility testing among 13,172 games from the beginning of the year. Of the games tested, TestBird found a total of 869,152 compatibility issues, which means that on average, each game has 65 compatibility issues, compared to last year figure at 52. The rise in 2015 is due to developers facing more severe adaptation pressure of new devices and Android Lollipop.
54% of compatibility issues due to game forced stopping and frozen screens
According to our studies, compatibility problems occur mostly with an app crashing to the home screen (referred to as a Game Forced Stop) and a game becoming frozen as the highest probability of an occurrence of major compatibility issue. Even more worrisome is that game forced stopping has increased from last year from 37.2% to 39% of major compatibility issues while game freezing fell to 15.5% percent from 18.5% last year.
Another key problem is startup failures have rose quickly from 4.4% last year to 12.2% this year. UI issues have dropped dramatically to 6.4% due to developers working more towards better UI performance this year.
Mainstream brands have less compatibility issues, urgent attention with Android 5.0 or later
TestBird has acquired 2,120 devices (covering 98% of the Chinese mobile market and over 90% abroad), and tested 13,172 games using these devices. When we compared the number of compatibility issues with various brands, we found mainstream brand compatibility was overall better than non-mainstream brands.
Compatibility issues were lowest with Xiaomi and Samsung brands with each having a failure rate for tested games of 9% and 10% respectively. Non-mainstream brands had the largest failure rate being as high as 21%, over two times as poorly as the leading brands.
By comparison with 2014 data, the report also found that due to the large number of new devices in 2015 moving to Android Lollipop, compatibility issues increased. Many developers have a large understanding of Android KitKat and therefore a drop in compatibility issues is expected. This trend of issues with new versions of Android may continue with Marshmallow being released this year. Lower end Android versions compatibility issues have increased as game developers move towards better compatibility with newer phones that are becoming available in the market using newer versions of Android.
CPU issues with Spreadstrum, while MediaTek and Qualcomm rate highly
With the performance of different CPUs in compatibility, compatibility issues arise with Spreadtrum in a troublingly large proportion. The best performance during our reporting were with MediaTek, Samsung, and HiSilicon CPUs. Further study will be reviewed for our full report.
Game Engine Performance and Usage in China
In our reporting of game engines, we have found most developers are currently using cocos2D-X, an open source game engine, and Unity 3D for their game engines as other popular engines like HTML5, Unreal Development Kit, and Flash Air have yet to be popular with developers.
During our testing, those who use their own customized engine continue to have a lot of compatibility issues, with a compatibility failure rates as high as 27%. More mainstream game engines, Unreal Developers Kit, Unity 3D, HTML5, and cocos2D-X, perform at similar levels of 19% – 23%.
Distribution of genres in Chinese mobile games
Game compatibility varies between genres
In the white paper we discuss games as they are divided into 12 genres, namely: Collectible card games, strategy, role-playing, tower defense, endless runner, flight shooting, elimination games (ex. Super Collapse 3), chess & poker, fighting, sports, virtual breeding, and puzzle games.
The report includes CPU usage, memory usage, average data consumption, IO wait rates, and the average temperature of the phone for each genre. It is recommended to read the full report to learn more about these issues individually.
For your own copy of the white paper, click here.