Skip to content

Commit 5a86142

Browse files
alambcomphead
andauthored
Add change to VARCHAR in the upgrade guide (#16216)
* Add change to VARCHAR in the upgrade guide * Update docs/source/library-user-guide/upgrading.md Co-authored-by: Oleks V <comphead@users.noreply.github.com> --------- Co-authored-by: Oleks V <comphead@users.noreply.github.com>
1 parent 6844e56 commit 5a86142

1 file changed

Lines changed: 49 additions & 0 deletions

File tree

docs/source/library-user-guide/upgrading.md

Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,55 @@
2121

2222
## DataFusion `48.0.0`
2323

24+
### The `VARCHAR` SQL type is now represented as `Utf8View` in Arrow.
25+
26+
The mapping of the SQL `VARCHAR` type has been changed from `Utf8` to `Utf8View`
27+
which improves performance for many string operations. You can read more about
28+
`Utf8View` in the [DataFusion blog post on German-style strings]
29+
30+
[datafusion blog post on german-style strings]: https://datafusion.apache.org/blog/2024/09/13/string-view-german-style-strings-part-1/
31+
32+
This means that when you create a table with a `VARCHAR` column, it will now use
33+
`Utf8View` as the underlying data type. For example:
34+
35+
```sql
36+
> CREATE TABLE my_table (my_column VARCHAR);
37+
0 row(s) fetched.
38+
Elapsed 0.001 seconds.
39+
40+
> DESCRIBE my_table;
41+
+-------------+-----------+-------------+
42+
| column_name | data_type | is_nullable |
43+
+-------------+-----------+-------------+
44+
| my_column | Utf8View | YES |
45+
+-------------+-----------+-------------+
46+
1 row(s) fetched.
47+
Elapsed 0.000 seconds.
48+
```
49+
50+
You can restore the old behavior of using `Utf8` by changing the
51+
`datafusion.sql_parser.map_varchar_to_utf8view` configuration setting. For
52+
example
53+
54+
```sql
55+
> set datafusion.sql_parser.map_varchar_to_utf8view = false;
56+
0 row(s) fetched.
57+
Elapsed 0.001 seconds.
58+
59+
> CREATE TABLE my_table (my_column VARCHAR);
60+
0 row(s) fetched.
61+
Elapsed 0.014 seconds.
62+
63+
> DESCRIBE my_table;
64+
+-------------+-----------+-------------+
65+
| column_name | data_type | is_nullable |
66+
+-------------+-----------+-------------+
67+
| my_column | Utf8 | YES |
68+
+-------------+-----------+-------------+
69+
1 row(s) fetched.
70+
Elapsed 0.004 seconds.
71+
```
72+
2473
### `ListingOptions` default for `collect_stat` changed from `true` to `false`
2574

2675
This makes it agree with the default for `SessionConfig`.

0 commit comments

Comments
 (0)