Skip to content

Add cftime support#149

Merged
alxmrs merged 13 commits intoalxmrs:mainfrom
ahuang11:add_cftime_support
Mar 30, 2026
Merged

Add cftime support#149
alxmrs merged 13 commits intoalxmrs:mainfrom
ahuang11:add_cftime_support

Conversation

@ahuang11
Copy link
Copy Markdown
Collaborator

@ahuang11 ahuang11 commented Mar 19, 2026

Closes #144

After attempting this, I realized it did not feel like a good first issue~ Note I used Claude Opus to plan and write this.

Two tiers based on calendar type:

  1. Gregorian-like (standard, noleap, all_leap) → pa.timestamp('us'). String SQL filters work naturally (WHERE time > '1980-01-01'). Microsecond resolution covers ±292k years, avoiding the nanosecond 1678–2262 overflow.
  2. Non-Gregorian (360_day, julian) → pa.int64() with xarray:units/xarray:calendar metadata on the Arrow field. Lossless — a 360-day "Feb 30th" isn't forced into Gregorian. Auto-registered cftime() UDF provides ergonomic filtering: WHERE time > cftime('2000-07-01').

Now this runs:

import xarray as xr
from metpy.cbook import get_test_data
from xarray_sql import XarrayContext

ds = xr.tutorial.open_dataset("rasm")
context = XarrayContext().from_dataset("rasm", ds, chunks={"time": 2})
context.sql("SELECT * FROM rasm LIMIT 1000")
image

However as you can see, there are a lot of NaNs; to be fixed in a separate PR in #151

@alxmrs
Copy link
Copy Markdown
Owner

alxmrs commented Mar 20, 2026

After attempting this, I realized it did not feel like a good first issue

Ok, full confession: I may be bad at scoping what a good first issue is or not. It's also hard to know what's involved before you've implemented the thing. I appreciate this feedback.

Copy link
Copy Markdown
Owner

@alxmrs alxmrs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this PR! Here is my initial round of feedback. Mostly questions, some suggestions.

@ahuang11
Copy link
Copy Markdown
Collaborator Author

Thanks for the review; a bit busy this early week, but will definitely get to this later. I'm also planning to first make a docs infra PR first so this can build on that docs.

@alxmrs alxmrs force-pushed the add_cftime_support branch from 932c8b3 to 8789b1e Compare March 29, 2026 23:59
@alxmrs alxmrs merged commit 65d8464 into alxmrs:main Mar 30, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

cftime support

2 participants