Support for Rapid Buckets
- Native Rapid Bucket Creation: You can now create Rapid buckets directly via the API, enabling high-throughput and low-latency workloads for co-located compute PR #769.
- Zonal Write Tracking: Improved accuracy in byte-count tracking for writes to Rapid storage to prevent data corruption during large transfers PR #777.
- ExtendedGcsFileSystem is now the default implementation. This allows GCSFS to automatically handle modern GCS features like Hierarchical Namespace (HNS) and Rapid buckets without manual configuration.
Note: If you need to disable these new defaults and return to the legacy GCSFileSystem behavior, set the environment variable:
GCSFS_EXPERIMENTAL_ZB_HNS_SUPPORT=disable
Support for Hierarchical Namespace (HNS)
- Atomic Folder Moves: Leverages HNS to make folder renames/moves a single metadata operation (O(1)) rather than a slow "copy-all-files-then-delete" process PR #758.
- Rename Race Conditions: Fixed a bug where rapid folder renames in HNS buckets could cause temporary inconsistencies PR #771.
API & Core Improvements
- Cleaner Listing: The ls and list methods now filter out invalid keyword arguments, ensuring more predictable behavior and better error messages PR #759.
- Graceful Shutdown: Enhanced the close() logic to ensure that all background writes and sessions are finished properly before the process exits PR #763.
- Testing, Benchmarks, & Docs
- New Documentation: Added extensive guides for using Zonal and Rapid storage PR #792, PR #788.
- Benchmarking Framework: Integrated microbenchmarks to track performance, specifically targeting the 8x QPS improvements promised by Rapid buckets PR #775, PR #770.
- CI Stability: Significant fixes to the CI/CD pipeline and test coverage to handle the new default bucket types PR #785, PR #772.