[KafkaIO] Remove duplicate offset in range check#34201
[KafkaIO] Remove duplicate offset in range check#34201scwhittle merged 6 commits intoapache:masterfrom
Conversation
|
Assigning reviewers. If you would like to opt out of this review, comment R: @m-trieu for label java. Available commands:
The PR bot will only process comments in the main thread (not review comments). |
|
Reminder, please take a look at this pr: @m-trieu @damccorm @johnjcasey @fozzie15 |
|
@johnjcasey @fozzie15 could you please take a look? |
275d39a to
3f172ec
Compare
|
@johnjcasey @fozzie15 I've rebased this PR on master instead of #34165 since there's no dependency between the two PRs. |
3f90e92 to
9709603
Compare
1a6c16f to
07d7567
Compare
07d7567 to
a0878ba
Compare
|
assign to next reviewer |
|
Reminder, please take a look at this pr: @m-trieu @damccorm @johnjcasey @fozzie15 |
|
Assigning new set of reviewers because Pr has gone too long without review. If you would like to opt out of this review, comment R: @robertwb for label java. Available commands:
|
|
Reminder, please take a look at this pr: @robertwb @Abacn @johnjcasey @jrmccluskey |
|
Assigning new set of reviewers because Pr has gone too long without review. If you would like to opt out of this review, comment R: @chamikaramj for label java. Available commands:
|
|
Reminder, please take a look at this pr: @chamikaramj @damccorm @johnjcasey @lostluck |
|
Assigning new set of reviewers because Pr has gone too long without review. If you would like to opt out of this review, comment R: @ahmedabu98 for label java. Available commands:
|
|
Reminder, please take a look at this pr: @ahmedabu98 @Abacn @johnjcasey @robertwb |
sdks/java/io/kafka/src/main/java/org/apache/beam/sdk/io/kafka/KafkaUnboundedReader.java
Show resolved
Hide resolved
|
Assigning new set of reviewers because Pr has gone too long without review. If you would like to opt out of this review, comment R: @chamikaramj for label java. Available commands:
|
|
stop reviewer notifications |
|
Stopping reviewer notifications for this pull request: requested by reviewer. If you'd like to restart, comment |
sdks/java/io/kafka/src/main/java/org/apache/beam/sdk/io/kafka/KafkaUnboundedReader.java
Show resolved
Hide resolved
sdks/java/io/kafka/src/main/java/org/apache/beam/sdk/io/kafka/ReadFromKafkaDoFn.java
Outdated
Show resolved
Hide resolved
…he poll loop for poll timing metric
|
Failure looks unrelated, merging. |
The fetcher (or fetch collector in later versions) in Kafka's client library already checks if a fetched record batch starts at the consumer's current position before returning any records in response to polling and otherwise ignores the batch.
See https://github.com/apache/kafka/blob/2.4.1/clients/src/main/java/org/apache/kafka/clients/consumer/internals/Fetcher.java#L675 and https://github.com/apache/kafka/blob/2.4.1/clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java#L1605.
Stacked on top of #34165.Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, commentfixes #<ISSUE NUMBER>instead.CHANGES.mdwith noteworthy changes.See the Contributor Guide for more tips on how to make review process smoother.
To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md
GitHub Actions Tests Status (on master branch)
See CI.md for more information about GitHub Actions CI or the workflows README to see a list of phrases to trigger workflows.