Task: HPC processing and data storage for VT LiDAR
High Performance Computing (HPC) processing and data storage for cloud optimization.
- Generate Cloud Optimized Geotiffs (COGs) of 2023 LiDAR products
- Workflow should take standard rasters deliverables as input
- Statewide and countywide
- COGs of all VCGI’s LiDAR and Imagery holdings
- COGs at the county level by year
- COGs at the county level by vintage
- COGS at the statewide level by vintage
- Upload of COGs to AWS S3 Bucket or delivered on an external hard drive
- If one-time upload, use of AWS web interface may be sufficient
- If there is a need for longer term admin of this, a Python CLI could be developed using Boto3 library
- Loading of all vintages of VT LiDAR and Imagery holdings at VCGI into HPC environment
- First request inventory of holdings - file types, size, security needs
- Not in final SOW?
- Development of protocols to enable VCGI staff to deploy the routines on the VACC once access is granted
- Connect and Train VCGI staff on LiDAR Dataprep (and other tools as developed in this project) on the VACC? (Not in SOW)
Other open questions:
- "Data and specifications to be determined based on stakeholder feedback"
epic