[Bug]: BigQueryIO performance overhead on DynamicDestinations on parsing json

[stankiewicz]

What happened?

Where: https://github.com/apache/beam/blob/master/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/DynamicDestinationsHelpers.java#L305-L314

Why: validation of static valueprovider is per element
Impact: 20% overhead on this stage.

Request: store outcome of validation within the object itself.

Issue Priority

Priority: 2 (default / most bugs should be filed as P2)

Issue Components

• Component: Python SDK
• Component: Java SDK
• Component: Go SDK
• Component: Typescript SDK
• Component: IO connector
• Component: Beam YAML
• Component: Beam examples
• Component: Beam playground
• Component: Beam katas
• Component: Website
• Component: Infrastructure
• Component: Spark Runner
• Component: Flink Runner
• Component: Samza Runner
• Component: Twister2 Runner
• Component: Hazelcast Jet Runner
• Component: Google Cloud Dataflow Runner

[stankiewicz]

.add-labels "good first issue"

[github-actions]

Label "good cannot be managed because it does not exist in the repo. Please check your spelling.

[stankiewicz]

.add-labels "good%20first%20issue"

[github-actions]

Label "good%20first%20issue" cannot be managed because it does not exist in the repo. Please check your spelling.

[stankiewicz]

.add-labels good first issue

[github-actions]

Label good cannot be managed because it does not exist in the repo. Please check your spelling.

[stankiewicz]

.take_issue

[Vicky-8223]

@stankiewicz I am trying to understand the current behaviour before proposing the soln.
Can you please clarify how the static valueProvider validation currently works in this code path like where the validation result is computed, why it is evaluated per element rather than once , whether there are constraints that influenced this design

[stankiewicz]

@Vicky-8223 here is the solution #37014