Gateway SUP integration

[julienduquesnay-se]

Description

Specification update related to the Gateway SUP. Add support of gateway service.

Issues Addressed

#137

Change Type

Please select the relevant options:

• [] Fix (change that resolves an issue)
• New enhancement (change that adds specification content)
• Content edits (change that edits existing content)

Checklist

• I have read the CONTRIBUTING document.
• My changes adhere to the established patterns, and best practices.

[phil-abb]

@julienduquesnay-se - I won't be able to review this until next week.

[phil-abb]

@julienduquesnay-se - would you mind updating your branch? It's showing changes related to the helm PR as changes in your branch

[julienduquesnay-se]

@julienduquesnay-se - would you mind updating your branch? It's showing changes related to the helm PR as changes in your branch

@phil-abb, I was trying to merge the latest changes from the "pre-draft" branch, and it looks like it grabbed your latest commit. I might need help to clean that up.

[phil-abb]

@julienduquesnay-se - would you mind updating your branch? It's showing changes related to the helm PR as changes in your branch

@phil-abb, I was trying to merge the latest changes from the "pre-draft" branch, and it looks like it grabbed your latest commit. I might need help to clean that up.

I rebased via the GitHub PR UI, and it looks ok now.

[matlec]

This PR implements an approved SUP, so I'm not asking to block or restructure it here. However, I'd like to flag a broader question this PR surfaces that's worth picking up separately.

@ajcraig @phil-abb

This is a bigger thread, and it ties into a comment I left on the "Device roles to capabilities" SUP. Working through this gateway PR, I keep bumping into the same question:

When the WFM sends a deployment, what is it targeting?

Today the implicit answer is "a WFM client". Routing happens via clientId in the URL, and the ApplicationDeployment YAML stays target-agnostic. This PR shifts that implicit answer by putting deviceId inside the ApplicationDeployment YAML's metadata. That works neatly for the see-thru gateway case, but it doesn't generalize. Multi-node Kubernetes clusters are the clearest example: kube-scheduler picks where pods run, so the WFM really has no business addressing nodes directly.

That points at something deeper. Standalone Device, Standalone Cluster, and Gateway look like three different things in the spec - but to me they read as the same shape, and the only real difference between them is: how many deployment targets the WFM client speaks for. A Standalone Device speaks for one. A Standalone Cluster speaks for one (the cluster as a whole, single-node or multi-node). A Gateway speaks for several. The PR introduces Gateway as a new category to handle the "several" case; I'd suggest framing it instead as extending the existing pattern to support more than one deployment target per client.

The most concrete future case this opens up is the DFM/WFM split for multi-node clusters. For a multi-node cluster, devices (the nodes) aren't really a WFM concern - the WFM just needs to know there's a deployment target (the cluster) and what it can run. Node-level identity, vendor, lifecycle - that's all DFM territory (post-GA). The PR currently surfaces device identity through WFM artifacts: deviceId in deployment metadata, the capabilities endpoint keyed on deviceId, the Gateway role naming child devices. When the DFM lands and we want to add device-level visibility, we'd ideally do that without having to pull device identity through every WFM interaction. One framing that would help: separate "what the WFM addresses" from "what counts as a device." The WFM-side concept becomes a "deployment target," orthogonal to whatever the DFM tracks. The WFM addresses targets; the DFM (when it arrives) tracks devices; the two surfaces stay independent.

Sketch of the alternative framing

• WFM Client = protocol participant. Has clientId, X.509 identity. Speaks for one or more deployment targets.
• Deployment target = an execution surface the WFM can address. Has a targetId unique per client. Has its own capabilities document.
• Target capabilities are scoped to deployment concerns: supported runtimes, deployment types, resources. Device-shaped fields (vendor, modelNumber, serialNumber) move to the DFM; they describe what the device is, not what it can host.
• What "resources" means depends on what schedules at this target. A standalone device or a gateway-managed sub-device reports concrete hardware (camera, GPU, ...) because Margo schedules at that level. For a cluster target, it probably makes less sense to report hardware at the cluster-level. In the cluster case, kube-scheduler can handle node-specific / hardware-aware placement via the chart's own affinity rules. Margo doesn't need to know which cluster node has the camera - that's Kubenetes' job.
• Target identity inherits from the client. No separate certs; the client is the trust boundary.
• Deployment routing = (clientId, targetId) pair. clientId stays where it is today. targetId lives in the bundle manifest's in a deployment entry - not in the ApplicationDeployment YAML, which stays target-agnostic. One bundle manifest carries routing for all of a client's deployments; a multi-target client fetches once rather than per target.
• Delegation ("autonomous placement") = a client-level capability flag, orthogonal to target count. To request delegation for a deployment, the WFM omits targetId from the bundle manifest entry; the client picks among its eligible targets. Single-target clients trivially satisfy this. A multi-target client that hasn't reported delegation capability must receive an explicit targetId.
• No hierarchy in targetId. Flat IDs unique per client. "Parent/child" is the client's internal concern, not protocol-visible.

How this covers the cases

Case	clientId	Targets (WFM)	Devices (DFM, post-GA)
Standalone Device	1	1 (the device)	1 (the device)
Standalone Cluster	1	1 (the cluster)	1 (the node)
Multi-node cluster (when it arrives)	1	1 (the cluster)	N (one per node)
See-thru gateway	1	N+1 or N (gw + children)	1 (just the gateway)
Opaque gateway	1	1 (the gateway)	1 (just the gateway)

Clusters don't need special-casing - a cluster is just "1 target." Gateways become "clients with >1 targets." The opaque/see-thru distinction collapses into "how many targets do you expose."

If there's interest, I'm happy to draft this as a SUP (post-PlugFest 2 ;)).

[matlec]

The PR carries forward the SUP's distinction between "opaque gateway" and "see-thru gateway" into the MUST rules. Reading through, I think only see-thru really needs to live there - and even then, mostly as a name for an underlying mechanism.

From the WFM's side, an opaque gateway looks identical to a regular device: no Gateway role, no hierarchical deviceId, no special error codes, ... The "Opaque gateways MUST report the combined capabilities..." boils down to "report what you can actually offer", which any device does anyway. And since the WFM can't tell an opaque gateway from a regular device, there's no way the spec could enforce that rule even if it wanted to. It's good guidance for someone building an aggregation device, but it doesn't really have a job in the MUST rules. (Side note: the example uses Standalone Device, but the rules don't seem to stop an opaque gateway from reporting Gateway instead - so even the boundary of the category isn't clear to me from the text.)

See-thru is different. It does point at something real. But that "something real" is the combination of mechanisms already in this PR: the Gateway role plus hierarchical deviceId. The name is a convenient handle for that combination, but it's the mechanisms themselves doing the work.

What I'd suggest:

• Drop opaque from the MUST rules, maybe just provide an informative note such as: "A device may aggregate several sub-devices behind it and report itself as a single Margo device."
• Rewrite the see-thru MUSTs to point at the mechanism directly: "A WFM client reporting the Gateway role MUST..." You could keep "see-thru gateway" as an informal name in the spec (as a useful shorthand) but don't pin protocol rules to that term.

[chrisgclayton]

I think the key is addressability. In the case of single node, cluster, and opaque I am addressing the target directly. In the case of transparent I am targeting the leaf device via the gateway. That means I have to consider the leaf device the same way I would any other target, but then have to add in the targeting parameters associated with the gateway it must communicate through.

i.e., I can target a leaf device of a gateway just like any other device but what happens if the gateway does not support the particular constructs that must flow through to it for "starting the camera". If we assume it is "just flow through" it would devalue the gateway and become not much more than a proxy server that security teams may not be happy with.

[julienduquesnay-se]

@matlec
Regarding your first suggestion: I have dropped the requirement (it was not a requirement in the SUP) and rephrase a bit based on your proposal.
Regarding your second suggestion: it makes sense and I tried to rewrite the requirements accordingly.

@chrisgclayton the original intent for the gateway concept was to allow connecting non-margo devices to the Margo ecosystem. How the gateway communicates with its child-devices is on purpose outside of the scope of the specification.

[ajcraig]

@julienduquesnay-se @matlec

Review Comments

Device ID

• I think we should move away from deviceId and utilize targetName
  • This appears to be a user-assigned name that the device owner defines — deviceId implies a system-generated unique identifier, which could cause confusion later.
  • deviceId should be reserved for universally unique identifiers, such as one assigned by the Device Fleet Manager.
• Within the new documentation, the properties attribute id has a description referencing deviceId. I propose both the route parameter {deviceId} and the body field id be renamed to targetName.

Device Capabilities

• It would help to add concrete HTTP call examples within the gateway section, something like:
  • Gateway only: POST .../capabilities/gateway1 → body "targetName": "gateway1"
  • Gateway + child: POST .../capabilities/gateway1/deviceA → body "targetName": "gateway1/deviceA"
  • Gateway + intermediate + child: POST .../capabilities/gateway1/zone1/sensorB → body "targetName": "gateway1/zone1/sensorB"
• An additional see-thru gateway payload example would also help adopters understand the correct format when reporting gateway metadata.

Deployment Status

• deviceId is listed as required in the body attributes but is absent from the example payload — these should be consistent.
• I think targetName should be optional here, only included when a see-thru gateway is reporting status on behalf of a child device. This makes the intent explicit: targetName identifies which child device the status applies to, since the reporting client and the executing device differ in that scenario.

Desired State

• Examples showing how desired state YAML files are structured for see-thru gateway deployments would be helpful. It would also be worth showing the wildcard targeting case (e.g. targetName: gateway1/*) since that behavior is unique to gateway scenarios and has no example today.

[phil-abb]

@julienduquesnay-se @matlec

Review Comments

Device ID

...

In light of changes that have been discussed as part of the identity and authorization framework, I think it makes sense to move away from using "deviceId" in this way. Previously, we really didn't have any real meaning behind "deviceId" other than some random ID in the device capabilities magically assigned by the device supplier. No "device ID" is starting to mean something more specific than this, so I think by using device ID here, it's just going to require us to change it later. May as well just change it now as part of the original PR since we know it will need to change.,

[julienduquesnay-se]

@julienduquesnay-se @matlec

Review Comments

Device ID

...

In light of changes that have been discussed as part of the identity and authorization framework, I think it makes sense to move away from using "deviceId" in this way. Previously, we really didn't have any real meaning behind "deviceId" other than some random ID in the device capabilities magically assigned by the device supplier. No "device ID" is starting to mean something more specific than this, so I think by using device ID here, it's just going to require us to change it later. May as well just change it now as part of the original PR since we know it will need to change.,

@phil-abb @ajcraig @matlec
While I don't necessary disagree with the proposed change, it raises a concern to me. The content on the SUP was reviewed and approved as part of a formal process. Here we are talking about making changes to what was approved outside of this process and potentially without the knowledge/approval of the original approvers for the SUP.
While it might add some work and time, it is cleaner to complete the changes to the specification as specified in the SUP and then submit an official change request as per the process to make these changes (either has a direct PR or a SUP depending on the scope).
Changing an attribute name, without change of semantic meaning, might be fine. But if we accept this type of change they should be clearly reported, maybe in the SUP itself in a new section. And the original voters need to be made aware so they can react if they disagree with the change. I don't think we can assume they are monitoring the PR.

[phil-abb]

@phil-abb @ajcraig @matlec While I don't necessary disagree with the proposed change, it raises a concern to me...

Fair comment.

I guess the question is, at what point do changes made while updating the specification with the information from an approved SUP start to matter?

For this case, I view it as just changing a word. The SUP used the word "foo," but after thinking about it, "bar" seemed better. It doesn't fundamentally change the behavior that was approved in the SUP.

For example, if this change isn't made with this PR, I wouldn't expect a new SUP to be created just to change "deviceId" to "targetName" because it's not changing any behaviors.

Maybe a topic we can discuss in the next TWG call to see what people think about how much change is acceptable when updating the spec for a SUP?

[ajcraig]

@phil-abb @ajcraig @matlec While I don't necessary disagree with the proposed change, it raises a concern to me...

Fair comment.

I guess the question is, at what point do changes made while updating the specification with the information from an approved SUP start to matter?

Fair response indeed @julienduquesnay-se. I'm leaning towards approving this SUP without the change, and then I can drive the word change in a PR directly to spec without a full SUP. Since, as Phil mentioned, it is just a word change.

However, it would be "burried" inside the "Gateway SUP", which didn't propose a change to the original id.

But we are learning these proccesses on the fly, so @phil-abb thoughts on my strategy above to address this particular FB to Julien?

[phil-abb]

But we are learning these proccesses on the fly, so @phil-abb thoughts on my strategy above to address this particular FB to Julien?

@julienduquesnay-se / @ajcraig / @matlec

I don't have too strong an opinion on whether we do it now or later; it just seems like doing it now as part of the original change means people don't have to make changes later after things have been implemented.

If we want to keep it how it is for now and revisit it later after we know the results of the identity and authentication framework vote, then that is fine as well. Even if that SUP is rejected, I think we'll want to reconsider how we are using ID/Device ID because our use right now seems too generic.