Task: HPC processing and data storage for VT LiDAR
High Performance Computing (HPC) processing and data storage for cloud optimization. - Generate Cloud Optimized Geotiffs (COGs) of 2023 LiDAR products - Workflow should take standard rasters deliverables as input - Statewide and countywide - COGs of all VCGI’s LiDAR and Imagery holdings - COGs at the county level by year - COGs at the county level by vintage - COGS at the statewide level by vintage - Upload of COGs to AWS S3 Bucket or delivered on an external hard drive - If one-time upload, use of AWS web interface may be sufficient - If there is a need for longer term admin of this, a Python CLI could be developed using Boto3 library - Loading of all vintages of VT LiDAR and Imagery holdings at VCGI into HPC environment - First request inventory of holdings - file types, size, security needs - Not in final SOW? - Development of protocols to enable VCGI staff to deploy the routines on the VACC once access is granted - Connect and Train VCGI staff on LiDAR Dataprep (and other tools as developed in this project) on the VACC? (Not in SOW) Other open questions: - "Data and specifications to be determined based on stakeholder feedback"
epic