Child pages
  • Abstract: Multiple Geospatial Data Integration In A Distributed Environment, Y1
Skip to end of metadata
Go to start of metadata

In this Research Allocation Request, we request an allocation of 341,440 SUs on Bridges and 15TB storage space on Bridges Pylon to continue the research initiated through our current Start- up Allocation. Our project aims to develop a high functionality data storage and exploration system for integrated three-dimensional (3D) urban remote sensing data. The system will continue to be built in a distributed computing environment based on the Hadoop ecosystem. The resource being requested will be used to facilitate the development of the system, as well as to evaluate the its performance and scalability. The request includes (1) a 5-node Hadoop cluster and a virtual machine running uninterruptedly for the duration of the allocation on PSC’s Bridges Cluster; (2) 4 types of Hadoop clusters with varying sizes from 10 to 40 nodes on Bridges for intensive performance and scalability testing, which will be reserved for several short periods of time; (3) a storage space on Bridges Pylon, and (4) staff support via ECSS to assist in system configuration, monitoring, and tuning. The computing resources at PSC’s Bridges, available to us through the Start-up allocation, have been essential to our research and have allowed us to evaluate multiple design concepts towards a distributed, scalable database system for management of discrete 3D point clouds and full waveform LiDAR data. In addition to the database prototypes, the current XSEDE allocation has enabled the development of a scalable, high-performance distributed computing solution for simulating solar irradiance from the world’s densest airborne LiDAR dataset collected by our group in 2015. The Start-up allocation also allowed us to evaluate the suitability and limitations of the types of resource offered by XSEDE. With this Research Allocation Request, we are looking forward to obtaining the necessary computing resources to further develop the spatio-temporal database system in order to accommodate other types of urban sensing data and to foster a new generation of highly efficient, scalable spatial data analysis algorithms.

  • No labels