Large scale “speedtest” experimentation in Mobile Broadband Networks

Abstract

Characterizing and evaluating the performance of Mobile Broadband (MBB) networks is a vital need for today's societies. Testbed-based measurements are of great significance in this context, since they allow for controlled and longitudinal experimentation. In this work, we focus on “speed” as an important Quality of Service (QoS) indicator for MBB networks, and work with MONROE-Nettest, an open source speedtest tool running as an Experiment as a Service (EaaS) on the Measuring Mobile Broadband Networks in Europe (MONROE) testbed. We conduct an extensive longitudinal measurement campaign spanning 2 countries over 2 years, and provide our experiment results together with rich metadata as an open dataset. We characterize this open dataset in detail, as well as derive insights from it regarding the impact of network context, spatio-temporal effects, roaming, and mobility on network performance. We describe our experiences about conducting speedtest measurements in MBB, and discuss the challenges associated with large scale testbed experimentation in operational MBB networks. Tackling one of the said challenges further, we introduce the notion of adaptive speedtest duration, and leverage a Machine Learning (ML) based algorithm to provide a proof-of-concept implementation called “Speedtest++”. Finally, we describe the lessons we have learned, as well as provide an overall discussion of how open datasets can support MBB research, and comment on open challenges, in the hope that these can serve as discussion points for future work. © 2020 Elsevier B.V.