feat(experimentation): add per-organisation ingestion infrastructure service#8035
feat(experimentation): add per-organisation ingestion infrastructure service#8035gagantrivedi wants to merge 1 commit into
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
2 Skipped Deployments
|
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #8035 +/- ##
========================================
Coverage 98.64% 98.65%
========================================
Files 1506 1508 +2
Lines 59572 59707 +135
========================================
+ Hits 58766 58901 +135
Misses 806 806 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Thanks for submitting a PR! Please check the boxes below:
docs/if required so people know about the feature.Changes
Adds a service layer for provisioning per-organisation experiment events ingestion infrastructure, to be consumed by the events ingestion server and hooked up to an API later.
experimentation/ingestion_infra_service.py:provision_ingestion_infrastructure(organisation_id)idempotently creates an S3 events bucket and a DirectPut Firehose delivery stream per organisation.environment_key, so objects land under per-environment S3 namespaces (events/env_key=.../year=.../...) and are batched per environment.INGESTION_EVENTS_BUCKET_PREFIX,INGESTION_FIREHOSE_DELIVERY_ROLE_ARN(a single shared Firehose delivery role, provisioned out-of-band, scoped by bucket naming convention).Note: with a shared delivery role, per-stream/bucket isolation is enforced by the control plane (only this service sets stream destinations), not IAM; per-organisation roles are a possible v2 hardening.
How did you test this code?
Unit tests with moto covering fresh provisioning (full destination config), idempotent re-runs, missing-settings errors, and AWS client error propagation.
make lint,make typecheck, and the full experimentation unit suite pass locally.