Skip to content

Conversation

@Fokko
Copy link
Contributor

@Fokko Fokko commented Mar 5, 2025

Don't use internal fields :)

I want to remove this one in apache/iceberg-python#1768. It should be okay since the Record also has a __len__.

@codecov
Copy link

codecov bot commented Mar 5, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 78.45%. Comparing base (7ef32fc) to head (ca7e6e1).
Report is 9 commits behind head on main.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #3917      +/-   ##
==========================================
+ Coverage   75.35%   78.45%   +3.09%     
==========================================
  Files         767      768       +1     
  Lines      103619    97902    -5717     
==========================================
- Hits        78080    76807    -1273     
+ Misses      25539    21095    -4444     
Files with missing lines Coverage Δ
daft/iceberg/iceberg_scan.py 90.62% <100.00%> (ø)

... and 34 files with indirect coverage changes

🚀 New features to boost your workflow:
  • Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@kevinzwang
Copy link
Member

Hey @Fokko, thanks for bringing this up!

From what I can tell, the Record class implements __len__ starting in PyIceberg v0.7. We currently support pyiceberg >= v0.4.0 so if we want to make this change, we should probably also bump our minimum required version as well.

I'm okay with doing that, but we should get it in with these changes. Would you mind changing the version requirement in pyproject.toml?

@kevinzwang kevinzwang self-requested a review March 5, 2025 22:26
@Fokko
Copy link
Contributor Author

Fokko commented Mar 6, 2025

@kevinzwang Thanks, that's a great point! I wouldn't recommend anyone using 0.4.0 anyway, I've bumped it to >=0.7.0 as you suggested 👍

@Fokko Fokko force-pushed the fd-remove_position_to_field_name branch from 11701c8 to ca7e6e1 Compare March 6, 2025 06:50
Copy link
Member

@kevinzwang kevinzwang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great! thanks again for the help

@kevinzwang kevinzwang merged commit a1e5ff3 into Eventual-Inc:main Mar 7, 2025
47 checks passed
kevinzwang pushed a commit that referenced this pull request Mar 8, 2025
Don't use internal fields :)

I want to remove this one in
apache/iceberg-python#1768. It should be okay
since the `Record` also has a `__len__`.
Fokko added a commit to apache/iceberg-python that referenced this pull request Apr 22, 2025
This aligns the implementation with Java.

We had the keywords there mostly for the tests, but they should not be
used, and it seems like that's already the case :'( I was undecided if
the costs of this PR (all the changes), are worth it, but I see more PRs
using the Record in a bad way (example
#1743) that might lead to
very subtle bugs where the position might sometime change based on the
ordering of the dict.

Blocked by Eventual-Inc/Daft#3917
gabeiglio pushed a commit to Netflix/iceberg-python that referenced this pull request Aug 13, 2025
This aligns the implementation with Java.

We had the keywords there mostly for the tests, but they should not be
used, and it seems like that's already the case :'( I was undecided if
the costs of this PR (all the changes), are worth it, but I see more PRs
using the Record in a bad way (example
apache#1743) that might lead to
very subtle bugs where the position might sometime change based on the
ordering of the dict.

Blocked by Eventual-Inc/Daft#3917
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants