|
21 | 21 |
|
22 | 22 | ## DataFusion `48.0.0` |
23 | 23 |
|
| 24 | +### The `VARCHAR` SQL type is now represented as `Utf8View` in Arrow. |
| 25 | + |
| 26 | +The mapping of the SQL `VARCHAR` type has been changed from `Utf8` to `Utf8View` |
| 27 | +which improves performance for many string operations. You can read more about |
| 28 | +`Utf8View` in the [DataFusion blog post on German-style strings] |
| 29 | + |
| 30 | +[datafusion blog post on german-style strings]: https://datafusion.apache.org/blog/2024/09/13/string-view-german-style-strings-part-1/ |
| 31 | + |
| 32 | +This means that when you create a table with a `VARCHAR` column, it will now use |
| 33 | +`Utf8View` as the underlying data type. For example: |
| 34 | + |
| 35 | +```sql |
| 36 | +> CREATE TABLE my_table (my_column VARCHAR); |
| 37 | +0 row(s) fetched. |
| 38 | +Elapsed 0.001 seconds. |
| 39 | + |
| 40 | +> DESCRIBE my_table; |
| 41 | ++-------------+-----------+-------------+ |
| 42 | +| column_name | data_type | is_nullable | |
| 43 | ++-------------+-----------+-------------+ |
| 44 | +| my_column | Utf8View | YES | |
| 45 | ++-------------+-----------+-------------+ |
| 46 | +1 row(s) fetched. |
| 47 | +Elapsed 0.000 seconds. |
| 48 | +``` |
| 49 | + |
| 50 | +You can restore the old behavior of using `Utf8` by changing the |
| 51 | +`datafusion.sql_parser.map_varchar_to_utf8view` configuration setting. For |
| 52 | +example |
| 53 | + |
| 54 | +```sql |
| 55 | +> set datafusion.sql_parser.map_varchar_to_utf8view = false; |
| 56 | +0 row(s) fetched. |
| 57 | +Elapsed 0.001 seconds. |
| 58 | + |
| 59 | +> CREATE TABLE my_table (my_column VARCHAR); |
| 60 | +0 row(s) fetched. |
| 61 | +Elapsed 0.014 seconds. |
| 62 | + |
| 63 | +> DESCRIBE my_table; |
| 64 | ++-------------+-----------+-------------+ |
| 65 | +| column_name | data_type | is_nullable | |
| 66 | ++-------------+-----------+-------------+ |
| 67 | +| my_column | Utf8 | YES | |
| 68 | ++-------------+-----------+-------------+ |
| 69 | +1 row(s) fetched. |
| 70 | +Elapsed 0.004 seconds. |
| 71 | +``` |
| 72 | + |
24 | 73 | ### `ListingOptions` default for `collect_stat` changed from `true` to `false` |
25 | 74 |
|
26 | 75 | This makes it agree with the default for `SessionConfig`. |
|
0 commit comments