LiDAR Dataprep v2.0
Resolves Issue #1 (closed), the sole issue for Epic &7, and so this work intends to resolve the Epic as well.
This is the occasion and place where we'll review and test LD2.0-beta and make changes to it until we're all satisfied -- which you'll indicate by clicking Approve when you have tested to your satisfaction and had all your concerns addressed -- after which I'll release it as LD2.0.0.
See Release_notes.md for an overview of the major changes in this release.
Since the work being reviewed and tested here is for a jump from the 1.x generation to the 2.x generation of this application, with so many changes tracked in the diff (aka the "Changes" tab of this page), it may be best to review the branch as a whole, at https://gitlab.uvm.edu/SAL/lidar-dataprep/-/tree/code-review-2.0.0-beta . Still good to enter your feedback in the diff if possible, but all code in all files are fair game for feedback/changes, so if you can't find where to enter your feedback for a certain issue in the diff, feel free to make it a general comment (including file name and line #) here on the Overview tab.
The documentation has been greatly expanded, and so README.md (the default document shown for the branch on GitLab) is still the starting page for the documentation, but there are multiple other .md documentation files, which are linked from the README, and to some extent, between each other (links work on GitLab and PyCharm).
This version of LiDAR Dataprep has been successfully tested on Windows 10 (.69), Windows Server 2016 (salbatch1), Red Hat Enterprise Linux 7.9 (VACC Bluemoon and the VM on .69) with Python3.7, ArcPro2/ArcServer10, FME2021, LAStools2023, Conda4. Ideally everyone would deploy and test on at least one Windows and one Linux platform. I will orient you all (will schedule a meeting soon) to which steps of the setup procedures (see the setup folder) are already done for you and which steps you'll need to perform (with my help).
There is still work to be done to license folx other than me to run Arc on VACC, but since the main code here now runs on its own conda env (named dataprep), you all should be able to deploy and run this code on Bluemoon, with workflows that don't include Arc steps (or can they still be in the workflow and will just be skipped? We need to see how this will work. Your running of the test suite there will give good feedback on this).
Once testing is done and this code is released, and until I sort out your Arc-on-VACC licensing, I will be available to kickoff your Arc-included production workflows on Bluemoon as needed.
Feedback on deployment and setup -- and on the repo's documentation in general -- is as important and welcome as is feedback on the actual code.
This application now has complete submodule coverage with automated INI testing, and so the first thing to run after you setup is the test suite - see test/Automated_testing.md. If these all pass, then good to proceed to your own testing as you see fit.
All aspects of review and testing are welcome from you all, but here are my recommendations for your focus:
- Ernie: Deployment & Setup (including automated test suite, since the ease of its use will be important to have deployment and setup be reasonable tasks for this application)
- Maeve, Max: Suitability for Production
- Cale: Code, Design Patterns, Repo Structure