fix: enforce policy-based access control on artifact downloads

[ycombinator]

What is the problem this PR solves?

The artifact download endpoint (/api/fleet/artifacts/{id}/{sha256}) only validates the agent's API key but never checks whether the requested artifact belongs to the agent's assigned policy. This means an agent enrolled under one policy can download artifacts belonging to a different policy if it knows the artifact ID and SHA256 hash. For example, an agent enrolled under a policy with no integrations can retrieve Elastic Defend trust lists, exception lists, and other security artifacts from another policy.

How does this PR solve the problem?

Implements the authorizeArtifact() function (previously a no-op that returned nil) to enforce policy-based access control:

1. Adds a GetPolicy(ctx, policyID) method to the policy.Monitor interface that returns the cached policy for a given ID (reloads from ES on cache miss).
2. In authorizeArtifact, fetches the agent's policy via the monitor using agent.AgentPolicyID and verifies that the requested artifact (identifier + decoded_sha256) appears in the policy's inputs[].artifact_manifest.artifacts.
3. Returns 403 Forbidden (ErrUnauthorizedArtifact) if the artifact is not listed in the agent's assigned policy.

How to test this PR locally

1. Set up Fleet Server with Elasticsearch and Kibana
2. Create two agent policies: Victim-Policy with Elastic Defend integration (add a trusted application), and Attacker-Policy with no integrations
3. Create an enrollment token for Attacker-Policy and enroll an agent
4. Attempt to download an artifact belonging to Victim-Policy using the attacker agent's API key — should now receive 403 Forbidden instead of the artifact contents
5. Verify that an agent enrolled under Victim-Policy can still download its own artifacts normally (200 OK)

Design Checklist

• I have ensured my design is stateless and will work when multiple fleet-server instances are behind a load balancer.
• I have or intend to scale test my changes, ensuring it will work reliably with 100K+ agents connected.
• I have included fail safe mechanisms to limit the load on fleet-server: rate limiting, circuit breakers, caching, load shedding, etc.

Checklist

• I have added tests that prove my fix is effective or that my feature works
• I have added an entry in ./changelog/fragments using the changelog tool

[mergify]

This pull request does not have a backport label. Could you fix it @ycombinator? 🙏
To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

• backport-./d./d is the label to automatically backport to the 8./d branch. /d is the digit
• backport-active-all is the label that automatically backports to all active branches.
• backport-active-8 is the label that automatically backports to all active minor branches for the 8 major.
• backport-active-9 is the label that automatically backports to all active minor branches for the 9 major.

[michel-laterman]

lgtm

[michel-laterman]

lgtm, just have nitpicks

[github-actions]

TL;DR

The Buildkite failure is a hung E2E test in TestOpAMPWithUpstreamCollector, not a compile error. The test blocks in a deferred otelCmd.Wait() and eventually the package-level test timeout aborts the run.

Remediation

• Add a hard stop for the collector process in testing/e2e/stand_alone_test.go after graceful shutdown is attempted (for example: set otelCmd.WaitDelay and/or fall back to Process.Kill if SIGTERM does not exit promptly).
• Re-run the E2E step after updating process shutdown handling, and specifically verify TestStandAloneRunningSuite/TestOpAMPWithUpstreamCollector exits cleanly.

Investigation details

Root Cause

The stack trace in the failed job shows the test goroutine waiting in os/exec.(*Cmd).Wait from TestOpAMPWithUpstreamCollector:

• testing/e2e/stand_alone_test.go:712 (defer otelCmd.Wait())
• function returns at testing/e2e/stand_alone_test.go:729, where the deferred wait executes.

The collector command is started with context cancellation and a custom cancel handler:

• testing/e2e/stand_alone_test.go:703-706 sets otelCmd.Cancel to only send SIGTERM.

If the child process does not terminate promptly on SIGTERM, deferred Wait() can block and the suite eventually times out, which matches the Buildkite log (long-running goroutine blocked in Cmd.Wait, package fails after ~1980s).

Evidence

• Build: https://buildkite.com/elastic/fleet-server/builds/14999
• Job/step: E2E Test (.buildkite/scripts/e2e_test.sh)
• Key log excerpt:
  • github.com/elastic/fleet-server/testing/e2e.(*StandAloneSuite).TestOpAMPWithUpstreamCollector
  • /opt/buildkite-agent/.../stand_alone_test.go:729 +0x18de
  • os/exec.(*Cmd).Wait(...)/src/os/exec/exec.go:930
  • FAIL github.com/elastic/fleet-server/testing/e2e 1980.062s

Related flaky-test tracking: #6590 (same test area under stand_alone_test.go, previously reported instability).

Verification

• Not run locally (requires full E2E environment and external dependency build path in CI).

Follow-up

If this test still flakes after adding a forced-stop path, consider explicitly asserting collector shutdown completion (with a bounded timeout) so hangs fail fast with a direct assertion instead of package timeout.

Note

🔒 Integrity filter blocked 2 items

The following items were blocked because they don't meet the GitHub integrity level.

• fix: enforce policy-based access control on artifact downloads #7009 pull_request_read: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
• #7009 pull_request_read: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".

To allow these resources, lower min-integrity in your GitHub frontmatter:

tools:
  github:
    min-integrity: approved  # merged | approved | unapproved | none

---

What is this? | From workflow: PR Buildkite Detective

Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.

[github-actions]

@Mergifyio backport 9.4 9.3 8.19

[mergify]

backport 9.4 9.3 8.19

✅ Backports have been created

Details

• #7162 [9.4](backport #7009) fix: enforce policy-based access control on artifact downloads has been created for branch 9.4
• #7163 [9.3](backport #7009) fix: enforce policy-based access control on artifact downloads has been created for branch 9.3
• #7164 [8.19](backport #7009) fix: enforce policy-based access control on artifact downloads has been created for branch 8.19 but encountered conflicts

Cherry-pick of caa8b2d has failed:

On branch mergify/bp/8.19/pr-7009
Your branch is up to date with 'origin/8.19'.

You are currently cherry-picking commit caa8b2d.
  (fix conflicts and run "git cherry-pick --continue")
  (use "git cherry-pick --skip" to skip this patch)
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

Changes to be committed:
	new file:   changelog/fragments/1778540235-fix-artifact-access-control.yaml
	modified:   internal/pkg/api/error.go
	modified:   internal/pkg/api/handleArtifacts.go
	new file:   internal/pkg/api/handleArtifacts_test.go
	modified:   internal/pkg/dl/constants.go
	modified:   internal/pkg/server/fleet.go
	modified:   testing/e2e/api_version/client_api_2023_06_01.go
	modified:   testing/e2e/api_version/client_api_current.go
	modified:   testing/e2e/scaffold/scaffold.go

Unmerged paths:
  (use "git add <file>..." to mark resolution)
	both modified:   internal/pkg/api/handleCheckin_test.go
	both modified:   internal/pkg/policy/monitor.go

To fix up this pull request, you can check it out locally. See documentation: https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/checking-out-pull-requests-locally