-
Notifications
You must be signed in to change notification settings - Fork 9
Open
Description
File: geest/core/tasks/study_area_processing_task.py
Method: create_and_save_grid()
Steps:
- Import ThreadPoolExecutor, as_completed from concurrent.futures
- Import threading for lock
- Get worker count from settings: grid_creation_workers (default 4)
- Create thread-safe lock: threading.Lock()
- Collect all chunks to process (filter out type="undefined")
- Create ThreadPoolExecutor with max_workers
- Submit all chunk tasks as futures
- Store futures in dict mapping to task and chunk index
- Iterate completed futures with as_completed()
- For each completion:
- Get result (raises exception if failed)
- Acquire lock and write to GeoPackage
- Update progress counter (thread-safe)
- Handle exceptions per chunk (log, don't fail entire process)
Error Handling:
- Catch individual chunk failures
- Log warning with chunk index
- Increment error count
- Continue processing remaining chunks
Metadata
Metadata
Assignees
Labels
No labels