Skip to content

OAPE-693:Add changes for checking E2E coverage with lgtm based approach for External-Secrets-Operator#81660

Open
siddhibhor-56 wants to merge 2 commits into
openshift:mainfrom
siddhibhor-56:more-codecov
Open

OAPE-693:Add changes for checking E2E coverage with lgtm based approach for External-Secrets-Operator#81660
siddhibhor-56 wants to merge 2 commits into
openshift:mainfrom
siddhibhor-56:more-codecov

Conversation

@siddhibhor-56

@siddhibhor-56 siddhibhor-56 commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Adds Separates E2E coverage instrumentation from the main e2e-operator presubmit into a standalone e2e-operator-coverage test with always_run: false, gated on LGTM via pipeline-controller

Summary by CodeRabbit

This PR updates the OpenShift CI configuration for openshift/external-secrets-operator to decouple E2E coverage instrumentation from the main E2E presubmit.

  • In ci-operator/config/openshift/external-secrets-operator/openshift-external-secrets-operator-main.yaml, the existing e2e-operator presubmit now only installs the operator bundle (restricted install mode) and runs make test-e2e with cpu: 100m, removing the coverage setup/collection flow from that job.
  • A new e2e-operator-coverage presubmit job is added with always_run: false. It performs coverage collection via hack/e2e-coverage.sh (setup during the test flow and collect in post), while running the same make test-e2e verification and using the same cpu: 100m constraint.
  • In core-services/pipeline-controller/lgtm-config.yaml, the LGTM configuration for the openshift org is extended to include external-secrets-operator on the main branch, gating when the coverage job runs.

Net effect: standard presubmits stay lighter, while coverage is still exercised through an LGTM-triggered, dedicated coverage presubmit for main.

@openshift-ci
openshift-ci Bot requested review from Prucek and bharath-b-rh July 9, 2026 08:54
@siddhibhor-56 siddhibhor-56 changed the title Add changes for checking E2E coverage with lgtm based approach Add changes for checking E2E coverage with lgtm based approach for External-Secrets-Operator Jul 9, 2026
@coderabbitai

coderabbitai Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

Walkthrough

The CI configuration separates External Secrets Operator end-to-end testing from coverage execution and adds the repository to OpenShift LGTM configuration for the main branch.

Changes

External Secrets Operator CI

Layer / File(s) Summary
Separate e2e and coverage workflows
ci-operator/config/openshift/external-secrets-operator/...
The main e2e workflow runs bundle installation and make test-e2e; coverage is handled by a new optional job using the existing cluster claim settings.
Register the main branch for LGTM
core-services/pipeline-controller/lgtm-config.yaml
Adds external-secrets-operator under the OpenShift organization for the main branch.

Estimated code review effort: 2 (Simple) | ~10 minutes

Suggested labels: lgtm, rehearsals-ack

Suggested reviewers: prucek, bharath-b-rh

🚥 Pre-merge checks | ✅ 15
✅ Passed checks (15 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names ✅ Passed The PR only changes CI YAML; no Ginkgo It/Describe/Context/When titles were added or modified, and the new job names are static.
Test Structure And Quality ✅ Passed PR only changes ci-operator/LGTM config; no Ginkgo test code was modified, so the test-structure check is not applicable.
Microshift Test Compatibility ✅ Passed The PR only changes CI/job config and LGTM gating; no new Ginkgo test code was added, so MicroShift compatibility checks aren’t triggered.
Single Node Openshift (Sno) Test Compatibility ✅ Passed The PR only changes CI/pipeline config; no new Ginkgo e2e tests or test logic were added, so the SNO compatibility check doesn't apply.
Topology-Aware Scheduling Compatibility ✅ Passed Only ci-operator and LGTM config changed; no deployment manifests/controllers or scheduling constraints were added.
Ote Binary Stdout Contract ✅ Passed The PR only changes CI YAML (ci-operator and LGTM config); it doesn't modify any OTE binary process setup or stdout logging paths.
Ipv6 And Disconnected Network Test Compatibility ✅ Passed Only CI config changed; no new Ginkgo tests or network-sensitive code were added, and the modified job commands show no IPv4 or external-host assumptions.
No-Weak-Crypto ✅ Passed Diff only changes a presubmit cluster name and LGTM YAML syntax; no MD5/SHA1/DES/RC4/3DES/Blowfish/ECB, custom crypto, or secret comparisons appear.
Container-Privileges ✅ Passed No touched manifest sets privileged/host namespaces/capabilities or allowPrivilegeEscalation; the new install step uses --security-context-config=restricted.
No-Sensitive-Data-In-Logs ✅ Passed No new secret-bearing logs were added; the changes only split CI jobs and add LGTM gating, with no passwords/tokens/PII echoed.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title is related to the main change: splitting External-Secrets-Operator E2E coverage into a separate LGTM-gated flow.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
core-services/pipeline-controller/lgtm-config.yaml (1)

17-17: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Stray blank line in repos list.

An empty line was inserted into the openshift org's repos: list right where external-secrets-operator would alphabetically sort. This looks like an editing artifact rather than an intentional change — the actual entry was correctly added separately at Lines 35-37.

🧹 Proposed cleanup
       - egress-router-cni
-
       - gcp-workload-identity-federation-webhook
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@core-services/pipeline-controller/lgtm-config.yaml` at line 17, Remove the
stray blank line from the openshift org repos list in lgtm-config.yaml so the
list stays alphabetically contiguous; keep the external-secrets-operator entry
in its correct sorted position and do not add any empty separator line in that
section.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@core-services/pipeline-controller/lgtm-config.yaml`:
- Line 17: Remove the stray blank line from the openshift org repos list in
lgtm-config.yaml so the list stays alphabetically contiguous; keep the
external-secrets-operator entry in its correct sorted position and do not add
any empty separator line in that section.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: 65acbb1c-b1dc-459c-8c57-59f44bd48d7b

📥 Commits

Reviewing files that changed from the base of the PR and between f159d36 and a9b4f0b.

⛔ Files ignored due to path filters (1)
  • ci-operator/jobs/openshift/external-secrets-operator/openshift-external-secrets-operator-main-presubmits.yaml is excluded by !ci-operator/jobs/**
📒 Files selected for processing (2)
  • ci-operator/config/openshift/external-secrets-operator/openshift-external-secrets-operator-main.yaml
  • core-services/pipeline-controller/lgtm-config.yaml

@siddhibhor-56
siddhibhor-56 force-pushed the more-codecov branch 3 times, most recently from 01762cb to 3e3d004 Compare July 9, 2026 12:25
@siddhibhor-56

Copy link
Copy Markdown
Contributor Author

/pj-rehearse max

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

@siddhibhor-56: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

@siddhibhor-56

Copy link
Copy Markdown
Contributor Author

/pj-rehearse pull-ci-openshift-external-secrets-operator-main-e2e-operator-coverage

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

@siddhibhor-56: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

@siddhibhor-56

Copy link
Copy Markdown
Contributor Author

/pj-rehearse pull-ci-openshift-external-secrets-operator-main-e2e-operator

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

@siddhibhor-56: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

@siddhibhor-56

Copy link
Copy Markdown
Contributor Author

/pj-rehearse pull-ci-openshift-external-secrets-operator-main-e2e-operator

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

@siddhibhor-56: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

@bharath-b-rh

Copy link
Copy Markdown
Contributor

/pj-rehearse pull-ci-openshift-external-secrets-operator-main-e2e-operator

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

@bharath-b-rh: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

@siddhibhor-56

Copy link
Copy Markdown
Contributor Author

/pj-rehearse pull-ci-openshift-external-secrets-operator-main-e2e-operator

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

@siddhibhor-56: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

@openshift-ci openshift-ci Bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jul 15, 2026
@openshift-merge-bot

Copy link
Copy Markdown
Contributor

@siddhibhor-56, pj-rehearse: unable prepare a candidate for rehearsal; rehearsals will not be run. This could be due to a branch that needs to be rebased. ERROR:

couldn't rebase candidate onto 948a8491ec474f99eb537afaad0b44749d3a155e due to conflicts

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

@siddhibhor-56, pj-rehearse: unable to determine affected jobs. This could be due to a branch that needs to be rebased. ERROR:

couldn't prepare candidate: couldn't rebase candidate onto 948a8491ec474f99eb537afaad0b44749d3a155e due to conflicts
Interacting with pj-rehearse

Comment: /pj-rehearse to run up to 5 rehearsals
Comment: /pj-rehearse skip to opt-out of rehearsals
Comment: /pj-rehearse {test-name}, with each test separated by a space, to run one or more specific rehearsals
Comment: /pj-rehearse more to run up to 10 rehearsals
Comment: /pj-rehearse max to run up to 25 rehearsals
Comment: /pj-rehearse auto-ack to run up to 5 rehearsals, and add the rehearsals-ack label on success
Comment: /pj-rehearse list to get an up-to-date list of affected jobs
Comment: /pj-rehearse abort to abort all active rehearsals
Comment: /pj-rehearse network-access-allowed to allow rehearsals of tests that have the restrict_network_access field set to false. This must be executed by an openshift org member who is not the PR author

Once you are satisfied with the results of the rehearsals, comment: /pj-rehearse ack to unblock merge. When the rehearsals-ack label is present on your PR, merge will no longer be blocked by rehearsals.
If you would like the rehearsals-ack label removed, comment: /pj-rehearse reject to re-block merging.

@openshift-ci openshift-ci Bot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jul 15, 2026
@siddhibhor-56

Copy link
Copy Markdown
Contributor Author

/pj-rehearse max

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

@siddhibhor-56: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

@siddhibhor-56, pj-rehearse: unable prepare a candidate for rehearsal; rehearsals will not be run. This could be due to a branch that needs to be rebased. ERROR:

couldn't rebase candidate onto 948a8491ec474f99eb537afaad0b44749d3a155e due to conflicts

Co-authored-by: Cursor <cursoragent@cursor.com>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In
`@ci-operator/config/openshift/external-secrets-operator/openshift-external-secrets-operator-main.yaml`:
- Around line 81-84: Update all three oc wait invocations in the
external-secrets operator installation steps to include --timeout=10m, while
preserving their existing conditions, namespaces, and resources.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: 56ab862c-c1cc-492c-b83b-4abc77324198

📥 Commits

Reviewing files that changed from the base of the PR and between 3e3d004 and d12b1b4.

📒 Files selected for processing (1)
  • ci-operator/config/openshift/external-secrets-operator/openshift-external-secrets-operator-main.yaml

Comment on lines +81 to +84
commands: |
oc create namespace external-secrets-operator
operator-sdk run bundle --timeout=10m --security-context-config=restricted --install-mode=AllNamespaces -n external-secrets-operator "$OO_BUNDLE" --verbose
oc wait --for condition=Available -n external-secrets-operator deployment external-secrets-operator-controller-manager

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

file='ci-operator/config/openshift/external-secrets-operator/openshift-external-secrets-operator-main.yaml'

echo '--- relevant lines ---'
nl -ba "$file" | sed -n '70,140p'

echo
echo '--- search for oc wait / timeout ---'
rg -n 'oc wait|--timeout|operator-sdk run bundle' "$file"

Repository: openshift/release

Length of output: 218


🌐 Web query:

kubectl wait default timeout oc wait default timeout docs

💡 Result:

For both kubectl wait and oc wait, the default timeout duration is 30 seconds [1][2]. The --timeout parameter specifies the length of time to wait before giving up on the condition being met [1][3]. The following behaviors apply to the timeout value: - Default: 30s [1][2] - Zero: The command checks the condition once and exits immediately without waiting [1][3]. - Negative: The command will wait for one week (effectively an indefinite wait) [3][4]. Since oc is a wrapper around kubectl (specifically for OpenShift clusters), it inherits this behavior and default configuration from the underlying kubectl implementation [2].

Citations:


🏁 Script executed:

#!/bin/bash
set -euo pipefail

file='ci-operator/config/openshift/external-secrets-operator/openshift-external-secrets-operator-main.yaml'

echo '--- relevant lines with numbers ---'
sed -n '70,140p' "$file" | cat -n

echo
echo '--- matches ---'
rg -n 'oc wait|operator-sdk run bundle|--timeout' "$file"

Repository: openshift/release

Length of output: 3808


Add --timeout=10m to all three oc wait calls. oc wait defaults to 30s, so the deployment can still be coming up when these install steps give up and make the jobs flaky.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@ci-operator/config/openshift/external-secrets-operator/openshift-external-secrets-operator-main.yaml`
around lines 81 - 84, Update all three oc wait invocations in the
external-secrets operator installation steps to include --timeout=10m, while
preserving their existing conditions, namespaces, and resources.

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

[REHEARSALNOTIFIER]
@siddhibhor-56: the pj-rehearse plugin accommodates running rehearsal tests for the changes in this PR. Expand 'Interacting with pj-rehearse' for usage details. The following rehearsable tests have been affected by this change:

Test name Repo Type Reason
pull-ci-openshift-external-secrets-operator-main-e2e-operator-coverage openshift/external-secrets-operator presubmit Presubmit changed
pull-ci-openshift-external-secrets-operator-main-e2e-operator openshift/external-secrets-operator presubmit Ci-operator config changed
Interacting with pj-rehearse

Comment: /pj-rehearse to run up to 5 rehearsals
Comment: /pj-rehearse skip to opt-out of rehearsals
Comment: /pj-rehearse {test-name}, with each test separated by a space, to run one or more specific rehearsals
Comment: /pj-rehearse more to run up to 10 rehearsals
Comment: /pj-rehearse max to run up to 25 rehearsals
Comment: /pj-rehearse auto-ack to run up to 5 rehearsals, and add the rehearsals-ack label on success
Comment: /pj-rehearse list to get an up-to-date list of affected jobs
Comment: /pj-rehearse abort to abort all active rehearsals
Comment: /pj-rehearse network-access-allowed to allow rehearsals of tests that have the restrict_network_access field set to false. This must be executed by an openshift org member who is not the PR author

Once you are satisfied with the results of the rehearsals, comment: /pj-rehearse ack to unblock merge. When the rehearsals-ack label is present on your PR, merge will no longer be blocked by rehearsals.
If you would like the rehearsals-ack label removed, comment: /pj-rehearse reject to re-block merging.

@siddhibhor-56

Copy link
Copy Markdown
Contributor Author

/pj-rehearse max

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

@siddhibhor-56: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

@openshift-ci

openshift-ci Bot commented Jul 15, 2026

Copy link
Copy Markdown
Contributor

@siddhibhor-56: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@bharath-b-rh bharath-b-rh left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm
/approve

@openshift-ci openshift-ci Bot added the lgtm Indicates that a PR is ready to be merged. label Jul 16, 2026
@openshift-ci

openshift-ci Bot commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: bharath-b-rh, siddhibhor-56
Once this PR has been reviewed and has the lgtm label, please assign hector-vido for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@siddhibhor-56 siddhibhor-56 changed the title Add changes for checking E2E coverage with lgtm based approach for External-Secrets-Operator OAPE-693:Add changes for checking E2E coverage with lgtm based approach for External-Secrets-Operator Jul 16, 2026
@openshift-ci-robot openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Jul 16, 2026
@openshift-ci-robot

openshift-ci-robot commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

@siddhibhor-56: This pull request references OAPE-693 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "5.0.0" version, but no target version was set.

Details

In response to this:

Adds Separates E2E coverage instrumentation from the main e2e-operator presubmit into a standalone e2e-operator-coverage test with always_run: false, gated on LGTM via pipeline-controller

Summary by CodeRabbit

This PR updates the OpenShift CI configuration for openshift/external-secrets-operator to decouple E2E coverage instrumentation from the main E2E presubmit.

  • In ci-operator/config/openshift/external-secrets-operator/openshift-external-secrets-operator-main.yaml, the existing e2e-operator presubmit now only installs the operator bundle (restricted install mode) and runs make test-e2e with cpu: 100m, removing the coverage setup/collection flow from that job.
  • A new e2e-operator-coverage presubmit job is added with always_run: false. It performs coverage collection via hack/e2e-coverage.sh (setup during the test flow and collect in post), while running the same make test-e2e verification and using the same cpu: 100m constraint.
  • In core-services/pipeline-controller/lgtm-config.yaml, the LGTM configuration for the openshift org is extended to include external-secrets-operator on the main branch, gating when the coverage job runs.

Net effect: standard presubmits stay lighter, while coverage is still exercised through an LGTM-triggered, dedicated coverage presubmit for main.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. lgtm Indicates that a PR is ready to be merged.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants