Supporting casts to timestamptz: Eq Utf8 of binary physical should be same"

[alamb]

Describe the bug
After #5140 and #5117 we can cast strings to Timestamp without a timezone. However, casting with timezone is not working yet, as noticed by @waitingkuo

To Reproduce
In datafusion-cli:

❯ select '2000-01-01T00:00:00'::timestamp::timestamptz = '2000-01-01T00:00:00';
Internal("The type of Timestamp(Nanosecond, Some(\"+00:00\")) Eq Utf8 of binary physical should be same")
❯ select '2000-01-01T00:00:00'::timestamp = '2000-01-01T00:00:00';
+-----------------------------------------------------------+
| Utf8("2000-01-01T00:00:00") = Utf8("2000-01-01T00:00:00") |
+-----------------------------------------------------------+
| true                                                      |
+-----------------------------------------------------------+
1 row in set. Query took 0.004 seconds.
❯ select '2000-01-01T00:00:00'::timestamptz = '2000-01-01T00:00:00';
NotImplemented("Unsupported CAST from Utf8 to Timestamp(Nanosecond, Some(\"+00:00\"))")

Expected behavior
I expect all three queries to run and return true

Additional context
Add any other context about the problem here.

[comphead]

@alamb I'll take it if noone else will. Need to finish this off

[comphead]

@alamb @waitingkuo likely we have some design issues on TimestampTz

please consider such simple case

        let valid = StringArray::from(vec![
            "2023-01-01 04:05:06.789-8",
            "2023-01-01 04:05:06.789-7",
        ]);

        let array = Arc::new(valid) as ArrayRef;
        let b = cast(&array, &DataType::Timestamp(TimeUnit::Nanosecond,?????????)).unwrap();

Ive put ???????? as its not really clear what has to be there, Some("+07:00") or Some("+08:00")? cast works for array which can contain mixed timezones, however the output datatype is a single value with only 1 supported tz

[alamb]

In terms of ???????? I would expect that one could put any None, Some("+07:00") or Some("+08:00") and the resulting timestamps would be adjusted on input to reflect what timezone they appeared in

cc @tustvold in case he has additional thoughts

[tustvold]

cc @tustvold in case he has additional thoughts

I agree, we should adjust the timestamp from the timezone in the string, if any, to time since UTC epoch, and store this in the value. The timezone of the output can then be the DataFusion default, unless otherwise specified, regardless this doesn't impact the encoded value which is always time since UTC epoch

[comphead]

Waiting arrow 34.0.0

[comphead]

DataFusion CLI v21.0.0
❯ select '2000-01-01T00:00:00'::timestamp::timestamptz = '2000-01-01T00:00:00';

+-----------------------------------------------------------+
| Utf8("2000-01-01T00:00:00") = Utf8("2000-01-01T00:00:00") |
+-----------------------------------------------------------+
| true                                                      |
+-----------------------------------------------------------+
1 row in set. Query took 0.006 seconds.
❯ 

That finally works after Arrow 36.0.0, I'll create a PR with tests
@alamb