fix: Fix Spark slice function Null type to GenericListArray casting issue

[erenavsarogullari]

Which issue does this PR close?

• Closes Fix Spark slice function Null type to GenericListArray casting issue #20466.

Rationale for this change

Currently, Spark slice function accepts Null Arrays and return Null for this particular queries. DataFusion-Spark slice function also needs to return NULL when Null Array is set.
Spark Behavior (tested with latest Spark master):

> SELECT slice(NULL, 1, 2);
+-----------------+
|slice(NULL, 1, 2)|
+-----------------+
|             null|
+-----------------+

DF Behaviour:
Current:

query error
SELECT slice(NULL, 1, 2);
----
DataFusion error: Internal error: could not cast array of type Null to arrow_array::array::list_array::GenericListArray<i32>.
This issue was likely caused by a bug in DataFusion's code. Please help us to resolve this by filing a bug report in our issue tracker: https://github.com/apache/datafusion/issues

New:

query ?
SELECT slice(NULL, 1, 2);
----
NULL

What changes are included in this PR?

Explained under first section.

Are these changes tested?

Added new UT cases for both slice.rs and slice.slt.

Are there any user-facing changes?

Yes, currently, slice function returns error message for Null Array inputs, however, expected behavior is to be returned NULL so end-user will get expected result instead of error message.

[Jefffrey]

Does Spark return null (void) or an array of null (void)?

I tested on PySpark 4.1.1

>>> spark.sql("select slice(NULL, 1, 2) as a").printSchema()
root
 |-- a: array (nullable = true)
 |    |-- element: void (containsNull = true)

• Seems to suggest the latter?

[erenavsarogullari]

Spark returns Array as (nullable = true / false) and there are 2 representations as follows in terms of input array so it returns input array for both cases as either NULL or [NULL]:
Case1: input array is null (array (nullable = true)):

+-----------------+
|slice(NULL, 1, 2)|
+-----------------+
|             NULL|
+-----------------+

root
 |-- slice(NULL, 1, 2): array (nullable = true)
 |    |-- element: void (containsNull = true)

Case2: input array has null element (array (nullable = false)):

+------------------------+
|slice(array(NULL), 1, 2)|
+------------------------+
|                  [NULL]|
+------------------------+

root
 |-- slice(array(NULL), 1, 2): array (nullable = false)
 |    |-- element: void (containsNull = true)

[Jefffrey]

This seems to suggest return type should be list of nulls instead of just null

[erenavsarogullari]

This seems to suggest return type should be list of nulls instead of just null

Yes, latest fix aims to address this by returning NullArray instead of Scalar Null value.

[Jefffrey]

This seems to suggest return type should be list of nulls instead of just null

Yes, latest fix aims to address this by returning NullArray instead of Scalar Null value.

Sorry I don't see how this relates? I mean it seems to suggest that a DataType::Null input has a DataType::List(Null) output, according to my PySpark test. But maybe the test was incorrect, since perhaps Spark introduces some casts? Could we confirm the behaviour if we want to align with Spark here?

[erenavsarogullari]

@Jefffrey Sorry for the delay and thanks again for the review. I have just submitted the latest commit which returns DataType::List(Null) for DataType::Null input and added test cases verifying the output type.

[erenavsarogullari]

Last review comment has been addressed and last CI run has been successful. Thanks for the reviews.

[Jefffrey]

Thanks @erenavsarogullari & @martin-g