Without massive speed and network performance data, operators cannot optimize customer experience or their investments.
Network statistics and drive-test data are how mobile operators measure the performance of their networks. Drive-tests especially are considered to be the best way to measure end-user mobile experience. And while it’s true that drive-testing enables very effective simulation of different services and detailed performance measurement, it’s still just that — a simulation. The only way to measure mobile user experience is to measure it on the point of consumption — the mobile device.
Current crowdsourcing vendors claim to provide just that. Mobile experience data collected from a large number of end-users, representing the real mobile experience. But despite this promise, crowdsourcing has remained a low-value tool for informing top management and giving marketing something to work with. Here’s why.
First, the data from user-initiated speed tests (like OOKLA’s SpeedTest) has a big measurement bias and a very low number of samples. Speed tests are run under a certain type of situations. Either the network is really good or bad, or the user has a new subscription and/or a phone. When compared to real maximum speeds end-users receive during normal usage, the results are very different.
And the data from background monitoring doesn’t deliver either. Companies like Tutela embed their technology into third-party apps and collect network performance data in the background. While this enables them to reach a lot of users, the sampling rates are typically very low and provide only a high-level view (country, city-level) of the performance. And these measurements are based on artificial traffic and not real usage.
As a result, the current crowdsourcing data is not used for network planning, optimization, or troubleshooting teams.
A great example of the confusion with current crowdsourcing is that in January 2021, different crowdsourcing companies announced AT&T (OOKLA), T-Mobile (OpenSignal) and Verizon (RootMetrics) as winners of some sort of “best” network in the US. So, is crowdsourcing doomed to remain just a marketing tool to make claims about being the best network when the results favour you? Luckily not.
1000x more speed data vs. traditional technologies
With next-generation crowdsourcing technology, also known as hybrid technology, operators can capture detailed and massive speed and performance data from real mobile usage. And by detailed we mean 1000x more speed data vs. traditional technologies. Hybrid technology is able to deliver massive speed and congestion data, by combining passive speed monitoring (delivers massive speed data) together with active latency measurements (used to detect when maximum speeds are reached).
By embedding this technology into their own applications (self-care and streaming apps) operators can easily collect performance data from 10–15% of their subscribers. This translates into millions of speed measurements every day.
Due to natural churn in their customer base, they can also collect significant competitor data as their apps will be running on devices with competitor subscriptions. In fact, more data they could buy from any crowdsourcing player.
This level of detail makes this type of crowdsourcing data actionable by technical teams. They can identify issues impacting real users, identify root causes, and improve. This is what crowdsourcing mobile network performance should look like: operators collecting performance data from their networks and using it to improve network performance — not to brag about it!
