Efficient distance join query processing in distributed spatial data management systems
Ημερομηνία
2020Γλώσσα
en
Λέξη-κλειδί
Επιτομή
Due to the ubiquitous use of spatial data applications and the large amounts of such data these applications use, the processing of large-scale distance joins in distributed systems is becoming increasingly popular. Distance Join Queries (DJQs) are important and frequently used operations in numerous applications, including data mining, multimedia and spatial databases. DJQs (e.g., k Nearest Neighbor Join Query, k Closest Pair Query, ε Distance Join Query, etc.) are costly operations, since they involve both the join and distance-based search, and performing DJQs efficiently is a challenging task. Recent Big Data developments have motivated the emergence of novel technologies for distributed processing of large-scale spatial data in clusters of computers, leading to Distributed Spatial Data Management Systems (DSDMSs). Distributed cluster-based computing systems can be classified as Hadoop-based or Spark-based systems. Based on this classification, in this paper, we compare two of the most recent and leading DSDMSs, SpatialHadoop and LocationSpark, by evaluating the performance of several existing and newly proposed parallel and distributed DJQ algorithms under various settings with large spatial real-world datasets. A general conclusion arising from the execution of the distributed DJQ algorithms studied is that, while SpatialHadoop is a robust and efficient system when large spatial datasets are joined (since it is built on top of the mature Hadoop platform), LocationSpark is the clear winner in total execution time efficiency when medium spatial datasets are combined (due to in-memory processing provided by Spark). However, LocationSpark requires higher memory allocation when large spatial datasets are involved in DJQs (even more so when k and ε are large). Finally, this detailed performance study has demonstrated that the new distributed DJQ algorithms we have proposed are efficient, robust and scalable with respect to different parameters, such as dataset sizes, k, ε and number of computing nodes. © 2019 Elsevier Inc.
Collections
Related items
Showing items related by title, author, creator and subject.
-
Use of Wild Bird Surveillance, Human Case Data and GIS Spatial Analysis for Predicting Spatial Distributions of West Nile Virus in Greece
Valiakos, G.; Papaspyropoulos, K.; Giannakopoulos, A.; Birtsas, P.; Tsiodras, S.; Hutchings, M. R.; Spyrou, V.; Pervanidou, D.; Athanasiou, L. V.; Papadopoulos, N.; Tsokana, C.; Baka, A.; Manolakou, K.; Chatzopoulos, D.; Artois, M.; Yon, L.; Hannant, D.; Petrovska, L.; Hadjichristodoulou, C.; Billinis, C. (2014)West Nile Virus (WNV) is the causative agent of a vector-borne, zoonotic disease with a worldwide distribution. Recent expansion and introduction of WNV into new areas, including southern Europe, has been associated with ... -
Effects of sub-anesthetic doses of ketamine on rats' spatial and non-spatial recognition memory
Pitsikas, N.; Boultadakis, A.; Sakellaridis, N. (2008)There are experimental evidences indicating that the non-competitive N-methyl-D-aspartate (NMDA) receptor antagonist ketamine impairs cognition and produces a series of schizophrenia-like symptoms in rodents (hyperactivity, ... -
Pre-training administration of anesthetic ketamine differentially affects rats' spatial and non-spatial recognition memory
Pitsikas, N.; Boultadakis, A. (2009)There is poor experimental evidence concerning the effects of anesthetic doses of the non-competitive NMDA receptor antagonist ketamine on rodents' memory abilities. The present study was designed to investigate a) the ...