Alignment of n-dimensional indexes with partially excluded dims

[benbovy]

• Closes Support alignment with partial overlap between index vs. excluded dimensions #10243
• Tests added
• User visible changes (including notable bug fixes) are documented in whats-new.rst

Support checking exact alignment of indexes that use multiple dimensions in the cases where some of those dimensions are included in alignment while others are excluded.

Add the exclude_dims keyword argument to Index.equals() (still support old signature with FutureWarning).

Also fixed a bug: indexes associated with scalar coordinates were ignored during alignment.

TODO:

• add exclude_dims kwarg to CoordinateTransform.equals() as well
• copy or assign the index of each original object in the new aligned objects instead of assigning the aligned index (when copying this may have an impact on performance).

[benbovy]

I added those overloads so Mypy doesn't fail for 3rd-party Xarray indexes. But maybe we want Mypy to fail and thereby encourage maintainers updating their code?

[dcherian]

I'm having trouble deciding if any of the other cases make any sense. Seems OK to remove anything that requires actual reindexing, which leaves "override" and I'm not sure about that. Let's have the error message recommend the user to open an issue if they feel it should work.

[benbovy]

I think that you are right. I'll try with other join options except "override".

[dcherian]

This seems OK to me, but is definitely hard to reason about.

[benbovy]

Yeah this certainly doesn't help better understand the alignment rules, which are already complex. It looks to me like a pretty robust approach for dealing with "multi-dimensional" indexes and dimensions excluded from alignment, though.

[benbovy]

I've made this to avoid any general performance regression for the pretty narrow-scope feature added here. However, self.keep_original_indexes may not be needed and we could write instead:

               if self.reindex[key] is False:
                    new_idx = obj_idx.copy(deep=self.copy)
                    new_idx_vars = new_idx.create_variables(obj_idx_vars)
                else:
                    new_idx = aligned_idx
                    new_idx_vars = {
                        k: v.copy(deep=self.copy) for k, v in aligned_idx_vars.items()
                    }

(i.e., always copy the index of each original object when no-reindexing is required)

I'm not sure about the performance impact of obj_idx.copy(deep=self.copy). Actually, I'm not sure either about v.copy(deep=self.copy) below (what we do in the main branch) and if we even want that.

This needs further investigation / performance assessment. Maybe in a follow-up PR?

[dcherian]

IMO it's fine to address in a followup PR

[dcherian]

LGTM. Thanks @benbovy . It's hard to reason about so I expect we'll need to use this feature a bit to find the edge cases we may have missed.

[benbovy]

@shoyer any thoughts on this?

[benbovy]

Thanks for the review @dcherian and @shoyer!