fix: Percent Encoding of paths for hive style partitioning

[greedAuguria]

Which issue does this PR close?

• Closes Partition values are not URL-decoded when extracted from Hive-style paths #19650.

Rationale for this change

Currently, when DataFusion parses Hive-style partitioned paths (e.g., s3://bucket/table/city=San%20Francisco/), it extracts the partition value literally as San%20Francisco.

Standard practice in tools like Apache Spark and Apache Hive is to URL-encode special characters in partition values when writing to object stores. This PR ensures DataFusion correctly decodes these values (to San Francisco) during the listing process, preventing data mismatches and ensuring consistent behavior with other engines.

What changes are included in this PR?

1. Logic Update: Updated parse_partitions_for_path in datafusion/catalog-listing/src/helpers.rs to percent-decode partition values.
2. Signature Change: Changed the return type of parse_partitions_for_path from Option<Vec<&str>> to Option<Vec<String>> because decoded strings require new allocations.
3. Call Site Updates: Updated internal callers in datafusion-catalog-listing to accommodate the owned String return type.
4. Dependencies: Added percent-encoding crate to datafusion-catalog-listing.

Are these changes tested?

Yes, new unit tests were added to datafusion/catalog-listing/src/helpers.rs covering:

• Standard URL-encoded characters (e.g., %2F for /).
• Spaces encoded as %20.
• Multi-byte UTF-8 characters.
• Forgiving behavior for malformed encoding (matching percent-encoding crate defaults).

Are there any user-facing changes?

Yes:

• Data Behavior: Partition values extracted from file paths will now be correctly decoded instead of remaining URL-encoded.
• API Change: The public helper function parse_partitions_for_path now returns Vec<String> instead of Vec<&str>.

[martin-g]

nit: Not sure whether it is worth it to return a Vec<Cow<'a, str>> here ?! E.g. if the partition contains % then decode it and return Cow::Owned(decoded.into_owned()), otherwise `Cow::Borrowed(val).

[martin-g]

Actually this is what https://docs.rs/percent-encoding/latest/percent_encoding/struct.PercentDecode.html#method.decode_utf8 returns. It might be a good idea.

[martin-g]

Should invalid values (like %FF in one of the tests below) return None or val ?!
I think it should behave like %XX - i.e. return val

[greedAuguria]

@martin-g I am sorry for the late reply, I will get on this now! Thank you for taking a look!

[greedAuguria]

@martin-g hey, sorry again for the late reply! I've rebased on main and addressed both of your comments:

Cow<'a, str> instead of String: you were right, percent_decode_str().decode_utf8() already hands back a Cow<str>, so it's a natural fit. Now we only allocate when the value actually had encoded characters in it. Most partition values won't, so this should be a nice win for the common case. Updated the caller to use into_owned() instead of to_string() too so we don't double-allocate on the owned path.

Fallback for invalid encodings: good catch on the %FF case. Previously .ok()? would bail out and return None for the whole file, which silently drops it from query results, definitely not what we want. Now if percent-decoding produces invalid UTF-8, we just pass through the raw value as-is (via unwrap_or(Cow::Borrowed(val))). So %FF comes back as the literal string %FF instead of the file disappearing. Same idea as your suggestion to treat it like %XX.

The taplo CI failure was just an alphabetical ordering thing in the root Cargo.toml: fixed during the rebase.

Let me know if there's anything else!