`from_dataset` breaks mental model / parallelism of xarray

[ahuang11]

I was expecting to do this:

ctx.from_dataset("rasm", ds, chunks={"time": 10}).sql("""
SELECT
    MAX(rasm."Tair")
FROM
    rasm
""")

However, from_dataset returns None, so I had to do it like this:

ctx.from_dataset("rasm", ds, chunks={"time": 10})
ctx.sql("""
SELECT
    MAX(rasm."Tair")
FROM
    rasm
""")

Was wondering, whether this is part of the design?

Early on, XArray steered away from inplace: pydata/xarray#1756

[alxmrs]

TODO:

• Find out what the datafusion convention is
• Make executive decision if we want to support this if it breaks from the datafusion convention.

[ahuang11]

Looking at https://datafusion.apache.org/python/user-guide/data-sources.html#create-in-memory,

In [5]: import pyarrow as pa

In [6]: ctx.from_pylist([
   ...:     { "a": 1, "b": 10.0, "c": "alpha" },
   ...:     { "a": 2, "b": 20.0, "c": "beta" },
   ...:     { "a": 3, "b": 30.0, "c": "gamma" },
   ...: ]).show()
   ...: 
DataFrame()
+---+------+-------+
| a | b    | c     |
+---+------+-------+
| 1 | 10.0 | alpha |
| 2 | 20.0 | beta  |
| 3 | 30.0 | gamma |
+---+------+-------+

It does seem like it does return the object, which allows chaining.

Same for read_* https://datafusion.apache.org/python/user-guide/data-sources.html#local-file