ARROW-15665: [C++] Add error handling option to StrptimeOptions by rok · Pull Request #12464 · apache/arrow

rok · 2022-02-18T15:41:53Z

This is to resolve ARROW-15665.

github-actions · 2022-02-18T15:42:14Z

https://issues.apache.org/jira/browse/ARROW-15665

rok · 2022-02-23T12:30:37Z

@pitrou I'm not sure what's causing this (Check failed: out.array()->buffers[0] Null bitmap deleted by kernel but can_write_into_slices = true). Any insight?
Would kernel preallocation settings help this or am I missing something else?

pitrou · 2022-02-28T17:29:41Z

can_write_into_slices means the kernel exec machinery won't let you redefine buffer addresses: you can write into the buffers given to you by the machinery but you are not allowed to point them elsewhere.

rok · 2022-03-17T04:06:12Z

@pitrou I think this is ready for review. I'm not sure why can_write_into_slices=true is required here, could you please check that I'm not doing something ill advised.

rok · 2022-03-17T04:22:36Z

@lidavidm I moved Strptime to scalar_temporal_unary.cc. Do you think that makes sense or better keep it in scalar_string_ascii.cc?

lidavidm

I think it's fine to remove it from scalar_string_ascii.cc.

cpp/src/arrow/compute/kernels/scalar_temporal_unary.cc

lidavidm · 2022-03-17T12:09:45Z

cpp/src/arrow/compute/api_scalar.h

What is the behavior if we don't raise parse errors?

Indeed, we may want to make this more descriptive, e.g. bool error_is_null or something. Opinions?

Changed to error_is_null. How about error_returns_null?

cpp/src/arrow/compute/kernels/scalar_string_test.cc

lidavidm · 2022-03-17T12:12:14Z

cpp/src/arrow/compute/kernels/scalar_temporal_unary.cc

Do we want to respect %% for escaping?

Switched to regex replace to do escaping. Feels a bit over engineered.

Regex is a bit much, how about just checking if the next char is '%' and advancing the string if so? (I think we do such things elsewhere)

Switched to a sequence counting approach.

cpp/src/arrow/compute/kernels/scalar_temporal_unary.cc

python/pyarrow/_compute.pyx

cpp/src/arrow/compute/kernels/scalar_temporal_unary.cc

cpp/src/arrow/compute/kernels/scalar_string_test.cc

cpp/src/arrow/compute/kernels/scalar_temporal_unary.cc

jorisvandenbossche · 2022-03-21T18:43:09Z

python/pyarrow/_compute.pyx

This should be False for the default?

Can you also add a test for this default? (a test that checks an error is raised for the default options without explicitly passing error_is_null)

Ah sorry, it was a remainder from the previous raise_errors parameter. Changed and added a test.

lidavidm · 2022-03-22T19:29:40Z

cpp/src/arrow/compute/kernels/scalar_temporal_unary.cc

Hmm, we might need to test this more. I thought we merged an optimization where INTERSECTION wouldn't allocate a null bitmap if all inputs were non-null.

Hm, what would you recommend we test? Bigger arrays?

Arrays with no input nulls

Got it. Added bitmap allocation and tests. Could you please check if we're still missing something?

rok · 2022-03-23T15:20:02Z

@lidavidm thanks for the review. Could you take another look?

lidavidm

Thanks, LGTM overall, just a couple comments

lidavidm · 2022-03-24T12:09:55Z

cpp/src/arrow/compute/kernels/scalar_temporal_unary.cc

+    int64_t* out_data = out->GetMutableValues<int64_t>(1);
+
+    if (self.error_is_null) {
+      if (!in.MayHaveNulls()) {


nit: it might be better to directly check out->buffers[0] instead of relying on the optimization being performed

Switched to if (out->buffers[0] == nullptr).

ursabot · 2022-03-24T15:32:01Z

Benchmark runs are scheduled for baseline = 623a15e and contender = ddb663b. ddb663b is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Failed ⬇️0.0% ⬆️0.0%] ec2-t3-xlarge-us-east-2
[Finished ⬇️0.34% ⬆️0.08%] test-mac-arm
[Failed ⬇️0.36% ⬆️0.0%] ursa-i9-9960x
[Finished ⬇️0.51% ⬆️0.04%] ursa-thinkcentre-m75q
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True
test-mac-arm: Supported benchmark langs: C++, Python, R
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java

github-actions bot added Component: C++ Component: Python Component: R labels Feb 18, 2022

rok force-pushed the ARROW-15665 branch 2 times, most recently from da7c0b5 to 6352488 Compare February 23, 2022 02:58

rok force-pushed the ARROW-15665 branch 6 times, most recently from 4df261b to a71ed14 Compare March 17, 2022 03:06

rok marked this pull request as ready for review March 17, 2022 03:52

rok force-pushed the ARROW-15665 branch from a71ed14 to 45c86ef Compare March 17, 2022 04:11

lidavidm reviewed Mar 17, 2022

View reviewed changes

rok force-pushed the ARROW-15665 branch 2 times, most recently from 33ce31a to 70d6b28 Compare March 17, 2022 20:37

lidavidm reviewed Mar 18, 2022

View reviewed changes

cpp/src/arrow/compute/kernels/scalar_string_test.cc Outdated Show resolved Hide resolved

cpp/src/arrow/compute/kernels/scalar_temporal_unary.cc Outdated Show resolved Hide resolved

rok force-pushed the ARROW-15665 branch from 70d6b28 to faa74cc Compare March 18, 2022 15:10

rok commented Mar 18, 2022

View reviewed changes

cpp/src/arrow/compute/kernels/scalar_temporal_unary.cc Outdated Show resolved Hide resolved

jorisvandenbossche reviewed Mar 21, 2022

View reviewed changes

rok force-pushed the ARROW-15665 branch from 8d07f87 to 6546d81 Compare March 22, 2022 13:11

rok added 4 commits March 22, 2022 19:09

Adding raise_errors to StrptimeOptions

b53a2e8

Some changes and kernel.can_write_into_slices=false

cb73a00

Make raise_errors=false default in R

0b9be59

Review feedback

66a2883

Using FirstTimeBitmapWriter

1b3f7bb

rok force-pushed the ARROW-15665 branch 3 times, most recently from e721a39 to 04202c9 Compare March 22, 2022 19:14

lidavidm reviewed Mar 22, 2022

View reviewed changes

Review feedback

b958a09

rok force-pushed the ARROW-15665 branch from 04202c9 to b958a09 Compare March 23, 2022 14:57

lidavidm approved these changes Mar 23, 2022

View reviewed changes

rok force-pushed the ARROW-15665 branch from baaab73 to 85b83e9 Compare March 24, 2022 02:53

Review feedback

06ab469

rok force-pushed the ARROW-15665 branch from 85b83e9 to 06ab469 Compare March 24, 2022 11:32

lidavidm approved these changes Mar 24, 2022

View reviewed changes

Review feedback

b4a6e75

rok requested a review from lidavidm March 24, 2022 15:18

lidavidm approved these changes Mar 24, 2022

View reviewed changes

lidavidm closed this in ddb663b Mar 24, 2022

asfimport mentioned this pull request Mar 30, 2022

[C++] Add error handling option to StrptimeOptions #20115

Closed

Comments

Conversation

rok commented Feb 18, 2022

Uh oh!

github-actions bot commented Feb 18, 2022

Uh oh!

rok commented Feb 23, 2022

Uh oh!

pitrou commented Feb 28, 2022

Uh oh!

rok commented Mar 17, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rok commented Mar 17, 2022

Uh oh!

lidavidm left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rok commented Mar 23, 2022

Uh oh!

lidavidm left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ursabot commented Mar 24, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

rok commented Mar 17, 2022 •

edited

Loading

ursabot commented Mar 24, 2022 •

edited

Loading