ARROW-11239: [Rust] Show some unit test failures#9193
Closed
mqy wants to merge 8 commits intoapache:masterfrom
Closed
ARROW-11239: [Rust] Show some unit test failures#9193mqy wants to merge 8 commits intoapache:masterfrom
mqy wants to merge 8 commits intoapache:masterfrom
Conversation
Member
|
I am aware of this, but I was unable to work on it. I can only work on this over this weekend :( |
mqy
changed the title
mqy
changed the title
Contributor
Author
|
Both ARROW-11239 and code of this PR were updated. It looks like the failures are relevant to |
mqy
commented
Jan 16, 2021
| array.data_type().clone(), | ||
| child_datum.null_buffer().is_some(), | ||
| )); | ||
| let null_buffer = child_datum.null_buffer(); |
Contributor
Author
There was a problem hiding this comment.
mqy
commented
Jan 16, 2021
|
|
||
| new_data.len = length; | ||
| new_data.offset = offset + self.offset; | ||
| self.copy_range(offset, length) |
Contributor
Author
There was a problem hiding this comment.
This is the attempt to implement slice with MutableArrayData, but triggers ARROW-11263
Contributor
Author
|
e20ee33 added new tests that passed locally. |
mqy
commented
Jan 16, 2021
| new_data | ||
| fn copy_range(&self, offset: usize, length: usize) -> ArrayData { | ||
| assert!((offset + length) <= self.len()); | ||
| let arrays = vec![self]; |
Contributor
Author
There was a problem hiding this comment.
Perhaps this hints the right direction:
- Possible performance speedup because no longer clone all array data and use only part of it (think about the total length is large and the slice length very small)
- Make the resulting ArrayData self-contained, thus less chances to mystery bugs. In this case, perhaps it's better to rename
sliceascopy_rangeto avoid mis-understanding.
The problem is: there are still 5 failures due to "not yet implemented" or "not supported"
Contributor
Author
There was a problem hiding this comment.
Possible performance speedup
Totally wrong, the benchmark of array_slice shows 4x - 11x regression.
mqy
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR is used to demonstrate some test failures, but not intended to fix the problems.