For many years now, almost no review of smartphones has been complete without performance testing, so it is not surprising that benchmarks designed for this task are popular not only among enthusiasts, but also among the most ordinary users who may not even know what hardware they are running on. their mobile device.
The results of benchmarks are usually simple and clear – in many cases, the results are presented in the form of a total number of points, according to which, as expected, one should draw conclusions about the performance of the device. But is it really that simple, and do benchmarks always reflect the real picture, allowing you to evaluate the power of a smartphone and its competitors? And are there any tricks that smartphone manufacturers can go to? Let’s try to figure this out.
How did benchmarks appear?
There is an opinion that benchmarks appeared on mobile devices because smartphones have been trying to copy the functionality of a PC since the appearance of the first model, and on computers, performance assessment has always been relevant due to the large amount of hardware available on the market, the ability to change individual components , as well as overclocking the processor and memory. On smartphones, even in 2020, for now, you have to limit yourself to overclocking some processors, installing custom firmware and obtaining superuser rights , and even then not on all devices, and modular devices have not yet taken root.
For a better understanding of where the first mobile benchmarks appeared, let’s delve a little into history. The first smartphone that was released back in 1992 and began to be sold in large quantities is considered to be IBM Simon – this is even despite the fact that at that time the term smartphone did not yet exist. However, the IBM Simon already had a touch screen, email software, and several pre-installed games.
But the world heard the first mention of the term Smartphone at the presentation of the Ericsson R380s operating on the Symbian 5.1 operating system.
It happened in 2000, at the same time when the OS for Windows Mobile communicators was released. It was the emergence of Symbian and Windows Mobile over time that gave users the opportunity to easily install many third-party applications, among which were benchmarks, of which more than a dozen have accumulated during the existence of operating systems. However, their performance was not too focused on when choosing a device, although performance tests were used by some users.
The lack of popularity of benchmarks both on Symbian and, to a lesser extent, on Windows Mobile can be explained by the fact that smartphones and communicators during the heyday of these two operating systems were less in demand than now, but there were also fewer manufacturers, as well as processor versions. After the appearance of Android OS in 2008, the situation changed radically – mobile software began to gain unprecedented popularity, which was also greatly facilitated by IOS and the first Iphone, which appeared a year earlier. Smartphones have become more convenient, and every year their performance has grown noticeably, so it is not surprising that many tests have been created to assess their power, some of which have already become outdated and have finally lost their relevance. For example, Will anyone remember the earlier Quadrant or Epic Citadel tests? Hardly, but at one time a large number of smartphone reviewers were guided by them.
What do benchmarks test?
The testing methodology is different for all benchmarks, so in no case can you compare indicators from different applications, just as it would not be entirely correct to compare the results obtained on different operating systems (more precisely, given the market features for 2020, on IOS and Android). The most popular parameters for testing are listed below:
Processor performance.
Testing two-dimensional and three-dimensional graphics.
The performance of internal (user) and RAM.
The speed of reading and writing from a memory card.
Performance when working with databases.
Other tests – throttling, heating, autonomy, video and audio playback, and more.
The results can be displayed both in conditional points (or, as they sometimes jokingly say, in parrots ), and in megaflops, gigaflops, megabits, megabytes, in the number of requests per second (QPS), in frames per second (FPS) and in other units of measurement.
What’s wrong with benchmarks?
From time to time, news appears on the network that well-known smartphone manufacturers are winding up ratings in benchmarks, and sometimes even processor manufacturers are accused of fraud . The main complaint is that in benchmarks, given the short testing time, the smartphone starts to work at the limit of its capabilities, while when solving other tasks, the actual performance decreases.
But can such a feature really be considered a hoax? Opinions among users are divided on this issue, since on the one hand it seems that they are trying to make smartphones more attractive in the eyes of buyers, and on the other hand, not everyone sees something bad in this, and it is not at all necessary that the hardware always work in full strength. Few tasks require this, and even in this mode, the smartphone cannot work for a long time – this will inevitably lead to overheating and increased battery consumption.
Go ahead. Smartphones that perform roughly the same in benchmarks will not necessarily perform the same in everyday tasks – for example, Unisoc SC9863 and Snapdragon 439 processors . The first one in Antutu eighth version scores more than 100,000 points, and the second – an average of 90,000. But in games, and in other tasks, the Qualcomm processor outperforms its competitor by several times, showing a noticeably higher number of frames per second and better results in all aspects of work . Another oddity is that in the seventh Antutu, the Snapdragon 439 chip still scores 5000 points more, that is, apparently, other counting algorithms are already used. But in the end, the conclusion follows that quantity does not always translate into quality.
And here another feature of benchmarks is revealed – this is the frequent release of new versions of applications in which performance results can differ significantly from those in previous versions. This creates some confusion, because for an objective comparison of smartphones, you need to run the tests in the same versions.
With benchmarks that test the operating time of the device, everything is also not so simple. It would seem that you can simply set 50 or 100% brightness on all smartphones, and thus compare the performance. But the peculiarity is that the maximum brightness level for all devices is different. When testing the display, other features may be found, such as dimming or brightening, depending on the information being displayed or the brightness of the ambient light. Therefore, without special equipment that would measure the brightness and allow you to set the same backlight level on all smartphones, the tests will not be objective.
The removal from the Google Play store , perhaps, of the most popular Antutu benchmark, whose developers have been repeatedly accused of violating privacy policies and fraud, also looks strange . But this is rather an isolated case, which still does not say anything.
So is it worth trusting benchmarks?
Good results in benchmarks do not yet guarantee that a smartphone will delight you with its work, but in most cases such software will help you understand how powerful a smartphone can be, and in the absence of other data, this will be useful information. But is this enough? If you just need a powerful device, then yes, especially if it gives out several hundred thousand points in Antutu.
Despite the various scandals associated with benchmarks, everything is not so simple with this software – some tests help to find out if a smartphone is prone to throttling (a significant decrease in performance under load), and they also display memory speed, battery temperature and other useful data, so put an end to all benchmarks is not worth it. Some of them will certainly be in demand for a very long time.
List of the most popular benchmarks
Antutu is a benchmark for testing all the main parameters.
Geekbench 5 is a comprehensive performance test in single-core and multi-core modes. In the future, a runtime test may be added, as was the case with Geekbench 4.
Androbench – detailed memory testing.
3DMark is an evaluation of the performance of the graphics and CPU.
PCMark is a performance and battery life test for smartphones and tablets.
CPU Throttling Test – Full CPU load to detect performance degradation. On new powerful smartphone models, throttling may not be detected, probably due to the fact that the application has not been updated since 2018 and does not take into account the features of some devices.
