LOG-9171: Add support for GCP workload identity federation v0.54 by vparfonov · Pull Request #263 · ViaQ/vector

vparfonov · 2026-04-22T16:45:09Z

Following PR: #249

Core changes (`src/gcp.rs`)

Replace goauth + smpl_jwt libraries with official google-cloud-auth v1.6
Implement scopes for External Account credentials (Workload Identity Federation)
Add token refresh logic with 1-hour expiry
Maintain backward compatibility with all existing authentication methods

New changes:

Compilation fix:
- PUBSUB_ADDRESS static was accidentally removed from src/gcp.rs but is still referenced in 4 integration test files (src/sinks/gcp/pubsub.rs, src/sources/gcp_pubsub.rs). This was causing the unit: FAILED CI status.
- Restored the LazyLock static and its import.
Security (least-privilege):
- Chronicle sink was changed from Scope::MalachiteIngestion to scopes::CLOUD_PLATFORM, which grants access to all GCP APIs. Restored the original narrow scope by adding MALACHITE_INGESTION (https://www.googleapis.com/auth/malachite-ingestion) as a scope constant.
Test coverage:
- The fails_missing_creds test was removed from gcp::tests without replacement.
- Added a falls_back_to_adc test that verifies the ADC fallback path doesn't panic.
Code cleanup:
- Applied cargo fmt
- Fix the cargo-fmt-check

/assign @jcantrill
/cc @cahartma @Clee2691

https://redhat.atlassian.net/browse/LOG-9171

- Restore PUBSUB_ADDRESS static removed by accident (fixes compilation) - Add MALACHITE_INGESTION scope for Chronicle sink (least-privilege) - Add falls_back_to_adc test to gcp::tests - Document ScopedEnv rationale (no with_credentials_file() API) - Remove dev comment from spawn_regenerate_token - Trim verbose test comments in pubsub and stackdriver tests - Apply cargo fmt Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

vparfonov · 2026-04-22T16:45:18Z

/hold

openshift-ci · 2026-04-22T16:45:57Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: vparfonov
Once this PR has been reviewed and has the lgtm label, please assign jcantrill for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

jcantrill · 2026-04-22T17:20:18Z

+        // The google-cloud-auth Builder has no with_credentials_file() method.
+        // The GOOGLE_APPLICATION_CREDENTIALS env var is the only way to pass
+        // a credentials path. ScopedEnv restores the original value on drop.
+        let _guard = ScopedEnv::set("GOOGLE_APPLICATION_CREDENTIALS", path);


Is this correct in there is no way for us to support isolating credentials for multiple GCP outputs?

Yes, this is a limitation of the google-cloud-auth library — its Builder has no with_credentials_file() method, so the GOOGLE_APPLICATION_CREDENTIALS env var is the only way to pass a credentials path.
The env var is only set temporarily during build_access_token_credentials() call. After that the ScopedEnv guard drops, restores the original value, and each GCP output holds its own independent AccessTokenCredentials instance with its own credentials. So at runtime, multiple GCP outputs with different credentials work fine.
The only theoretical risk is during initialization: if two GCP outputs with different credentials_path values call build() concurrently, they could race on the env var. In practice Vector builds sink configs sequentially during startup, so this doesn't happen.

What does this mean in practice? Do we need to set this var? Do these credentials need to be a parent or some set of credentials that delegates to the individual outputs? Maybe these questions are not relevant for the collector but more so for the operator

openshift-ci-robot · 2026-04-22T17:26:53Z

@vparfonov: This pull request references LOG-9171 which is a valid jira issue.

Details

In response to this:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

vparfonov · 2026-04-23T04:12:20Z

/test unit

vparfonov · 2026-04-23T12:49:09Z

/test unit

vparfonov · 2026-04-27T11:58:49Z

/test unit

jcantrill · 2026-04-27T19:19:57Z

This should merge after #249? Replace #249?

- make_token(): replace .ok() with proper error logging and rate limiting so token fetch failures are visible instead of causing opaque 403 responses - from_file(): add mutex around ScopedEnv to prevent concurrent sink initializations from racing on GOOGLE_APPLICATION_CREDENTIALS Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

vparfonov · 2026-04-28T11:11:25Z

/test unit

google-cloud-auth 1.8.0 requires serde_json >= 1.0.148, which changed float exponent formatting from e308 to e+308. Update fixtures to match. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

vparfonov · 2026-04-28T14:36:33Z

This should merge after #249? Replace #249?

Replace #249

openshift-ci · 2026-04-28T15:22:23Z

@vparfonov: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/unit	`3cd574e`	link	true	`/test unit`
ci/prow/cluster-logging-operator-e2e	`090cf73`	link	true	`/test cluster-logging-operator-e2e`
ci/prow/clo-functional	`090cf73`	link	true	`/test clo-functional`

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

…json 1.0.149+ serde_json 1.0.149 replaced ryu with zmij for float serialization, which uses scientific notation for very large floats. This caused a mismatch between ryu::Buffer::format_finite() and actual JSON output, breaking the estimate_f32 and serialize_f64 quickcheck tests. Use serde_json::to_string() directly so the estimation always matches the actual serialized output regardless of future serde_json changes. Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>

cahartma and others added 2 commits April 8, 2026 17:48

LOG-9171: Add GCP WIF support

0f7097d

openshift-ci Bot added the do-not-merge/hold label Apr 22, 2026

vparfonov changed the title ~~Fix gcp wif~~ WIP:Fix gcp wif Apr 22, 2026

openshift-ci Bot added the do-not-merge/work-in-progress label Apr 22, 2026

openshift-ci Bot requested review from cahartma and jcantrill April 22, 2026 16:45

jcantrill reviewed Apr 22, 2026

View reviewed changes

jcantrill added the v0.54 label Apr 22, 2026

vparfonov changed the title ~~WIP:Fix gcp wif~~ LOG-9171: Add support for GCP workload identity federation v0.54 Apr 22, 2026

openshift-ci Bot removed the do-not-merge/work-in-progress label Apr 22, 2026

openshift-ci-robot added the jira/valid-reference label Apr 22, 2026

vparfonov force-pushed the fix-gcp-wif branch from a41af36 to 3cd574e Compare April 28, 2026 09:35

fix(codecs): update native JSON fixtures for serde_json 1.0.149

090cf73

google-cloud-auth 1.8.0 requires serde_json >= 1.0.148, which changed float exponent formatting from e308 to e+308. Update fixtures to match. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

vparfonov mentioned this pull request Apr 28, 2026

LOG-9171: Add support for gcp workload identity federation v0.54 #249

Closed

Conversation

vparfonov commented Apr 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Core changes (src/gcp.rs)

New changes:

Uh oh!

vparfonov commented Apr 22, 2026

Uh oh!

openshift-ci Bot commented Apr 22, 2026

Uh oh!

jcantrill Apr 22, 2026

Choose a reason for hiding this comment

Uh oh!

vparfonov Apr 22, 2026

Choose a reason for hiding this comment

Uh oh!

jcantrill Apr 27, 2026

Choose a reason for hiding this comment

Uh oh!

openshift-ci-robot commented Apr 22, 2026 • edited by openshift-ci Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vparfonov commented Apr 23, 2026

Uh oh!

vparfonov commented Apr 23, 2026

Uh oh!

vparfonov commented Apr 27, 2026

Uh oh!

jcantrill commented Apr 27, 2026

Uh oh!

vparfonov commented Apr 28, 2026

Uh oh!

vparfonov commented Apr 28, 2026

Uh oh!

openshift-ci Bot commented Apr 28, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

vparfonov commented Apr 22, 2026 •

edited

Loading

Core changes (`src/gcp.rs`)

openshift-ci-robot commented Apr 22, 2026 •

edited by openshift-ci Bot

Loading