[Bug]: [Flink] JdbcIO metrics are always `NaN` when using the FlinkRunner

[oswidan97]

What happened?

Specifically these 2 metrics in JdbcIO that should be updated when inserting into the sink database, always have the value NaN, even though data is being ingested in the sink database that I am using.

I am using the apache beam version 2.56.0 and beam-runners-flink-1.17

Screenshot from the FlinkUI

Issue Priority

Priority: 2 (default / most bugs should be filed as P2)

Issue Components

• Component: Python SDK
• Component: Java SDK
• Component: Go SDK
• Component: Typescript SDK
• Component: IO connector
• Component: Beam YAML
• Component: Beam examples
• Component: Beam playground
• Component: Beam katas
• Component: Website
• Component: Spark Runner
• Component: Flink Runner
• Component: Samza Runner
• Component: Twister2 Runner
• Component: Hazelcast Jet Runner
• Component: Google Cloud Dataflow Runner

[je-ik]

Thanks for the report. A few questions, if you don't mind:

• is your Pipeline bounded or unbounded?
• can you please share any other command-line arguments/configuration passed to the job?
• if bounded, can you verify if the metrics are available after the job terminates via PipelineResult?

[oswidan97]

Thanks for the report. A few questions, if you don't mind:

• is your Pipeline bounded or unbounded?
• can you please share any other command-line arguments/configuration passed to the job?
• if bounded, can you verify if the metrics are available after the job terminates via PipelineResult?

• It's unbounded, I am using KafkaIO as the source and writing using JdbcIO
• These are my only passed arguments --runner=FlinkRunner", "--attachedMode=false", "--checkpointingInterval=1000", "--applicationConfigFile={path_to_file}"
• N/A

[je-ik]

Thanks. Could build patched version of Beam and try to log the contents of the container in here?

[kennknowles]

I'm going to be taking a peek at Flink metrics soon. I'm not sure if it will fix these, but I think there's some general issues.

[github-actions]

This issue has been marked as stale due to 150 days of inactivity. It will be closed in 30 days if no further activity occurs. If you think that’s incorrect or this issue still needs to be addressed, please simply write any comment. If closed, you can reopen the issue at any time. Thank you for your contributions.

[github-actions]

This issue has been closed due to lack of activity. If you think that is incorrect, you can reopen the issue at any time.