Skip to content

5.0.3

Choose a tag to compare

View all tags
@jqnatividad jqnatividad released this 22 May 04:33
· 1971 commits to master since this release
5.0.3
bcc3514

[5.0.3] - 2025-05-22 "The Geo Release" 🌍

qsv 5.0.3 represents a major milestone with significant enhancements to its geospatial data processing capabilities.
They're targeted to support the Datapusher+ Data Resource Upload First (DRUF) workflow for "automagical metadata inferencing" - focusing on DCAT-US v3 recommended spatial and temporal properties that would otherwise be too tedious to manually compile:

New Geocoding Capabilities

  • Added IP geolocation with new --iplookup and --iplookupnow subcommands in the geocode command
  • Integrated Maxmind GeoLite2 database support for accurate IP-to-location mapping
  • Enhanced geocoding performance (up to 5x faster) with rkyv serialization (contributed by @estin)

Enhanced geoconvert Command

  • Added CSV input support alongside existing geospatial formats
  • Introduced GeoJSONL output format for streaming workflows
  • Added stdin support for all formats except SHP input
  • New coordinate handling options: --latitude and --longitude parameters
  • Added --max-length option for output control
  • Comprehensive test coverage additions
  • all contributed by @rzmk!

🚀 Performance & Infrastructure Improvements

Polars Integration

  • Upgraded Polars from 0.46.0 to 0.48.1 with intermediate releases
  • Enhanced Polars schema support across multiple commands (schema, joinp, pivotp, sqlp)
  • Added --polars mode to the schema command to explicitly create a polars schema file on demand, rather than as a side-effect of the sqlp command using its --cache-schema option.

Core Performance

  • Microoptimizations in the sort command
  • Improved file handling with tempfile usage in edit --in-place
  • Enhanced auto-decompression support now available suite-wide for gz, zlib, and zst files

🛠️ New Features & Usability

Enhanced Commands

  • edit: New --in-place option for direct file modification with automatic backup (.bak) creation
  • foreach: Added "/" to splitter pattern for improved path handling
  • stats: New QSV_STATS_STRING_MAX_LENGTH environment variable for string analysis control
  • to: Added --all-strings option for simplified data type handling

Distribution & Installation

  • Added conda package support with installation instructions
  • New download badges and streamlined installation documentation
  • Retired older glibc-2.31 and musl-1.2.3 "prebuilt-older" binaries as Ubuntu 20.04 has been retired and no longer supported with GitHub Actions.
  • Discontinued MSI installer in favor of the easier qsv Windows Easy Installer (thanks @rzmk!)

Quality & Stability

  • Applied multiple clippy lint suggestions for code quality
  • Enhanced test coverage, particularly for geospatial functions
  • Improved documentation with better examples and clearer explanations
  • Fixed stdin handling issues in the split command

🎯 Default Feature Changes

The qsvdp variant now includes geocode and geoconvert commands by default, making geospatial functionality more accessible to Datapusher+ users with Jinja2-powered metadata formulas.

NOTE:

  • for qsv v5.0.3, cargo install will NOT worked as the calamine crate (which powers the excel command) is pinned to zip 2.5.0 which was yanked.
  • unfortunately, the broken zip dependency also prevents us from publishing qsv 5.0.3 to crates.io
  • for both cases, either install the prebuilts or compile from source with cargo build.

Added

  • edit: add --in-place (and test) which uses tempfile by @rzmk in #2744
  • foreach: add "/" to splitter pattern #2754
  • geoconvert: add CSV input and GeoJSONL output and use buf by @rzmk in #2690
  • geoconvert: add stdin support (except for SHP input) by @rzmk in #2699
  • geoconvert: add --latitude and --longitude options by @rzmk in #2707
  • geoconvert: add --max-length option #2711
  • geocode: add iplookup and iplookupnow subcommands #2741
  • tests: geoconvert - add basic tests and move tests to test_geoconvert.rs by @rzmk in #2717
  • qsvdp now include geocode & geoconvert commands by default #2697
  • stats: QSV_STATS_STRING_MAX_LENGTH env var #2709
  • to: add --all-strings option #2746
  • docs: add conda install command by @rzmk in #2718
  • docs: add qsv download badges and update install instructions by @rzmk in #2721

Changed

Fixed:

New Contributors

Full Changelog: 4.0.0...5.0.3

Contributors

estin, HarrisonMc555, and 2 other contributors
Assets 12
Loading