Allowing A/B testing of origin policies

[domenic]

Some folks testing Origin-Policy behind a flag noticed that the current design isn't very conducive to A/B testing. That is, it is hard to send different origin policies to different users.

This is something sites often do today with headers. For example gradual rollout of a new CSP or FP, or testing the effect of client hints infrastructure. It seems like something that should be possible for origin policy as well.

The problem is that it comes into conflict with our design goal of "one origin policy per origin", which led to things like moving away from the /.well-known/origin-policy/origin-policy-id design into the current single /.well-known/origin-policy manifest in #47. But upon further consideration, I think they're compatible: our design goal is more like "one origin policy per origin per user", where here "user" is defined as something like "user agent" or "user agent session" or "browser user who hasn't cleared their cache". We don't mind if the server has multiple origin policies, as long as one user doesn't see multiple.

I see a few options to enable A/B testing in this way:

1. Add cookies/credentials to the /.well-known/origin-policy request. (See Credentials mode "omit" incompatible with HTML's navigate #21.) This would allow servers to vary the contents of /.well-known/origin-policy on a user cookie.

2. Add forwarding of the allowed/preferred IDs from the main-request Origin-Policy header to the /.well-known/origin-policy request. E.g. add a request header Origin-Policy-Requested (Sec-Origin-Policy-Requested?) to contain this information, which the browser sends when requesting /.well-known/origin-policy. This would allow servers to vary the contents of the Origin-Policy header per user, and then use that information to vary the contents of /.well-known/origin-policy.

3. Move back to the /.well-known/origin-policy/origin-policy-id model. This would probably also allow simplifying the infrastructure by removing the "ids" field and the matching that takes place there, since New version negotiation mechanism makes updates impossible for cached resources #66 would no longer be an issue. This would allow servers to vary the contents of the Origin-Policy header per user, and the browser would automatically fetch the desired /.well-known/origin-policy/X policy for that user.

4. Do nothing. Servers cannot AB test per user, but they can do things like randomly varying the contents of /.well-known/origin-policy (or varying the contents based on which backend machine that gets hit) and using self.originPolicyIds on the client side to see which one actually got used.

Currently (1) seems most attractive to me. It is a minimal delta from the existing spec, and has the bonus of making H/2 PUSH more feasible (as discussed in #21). In offline discussions @sleevi was the one most concerned with avoiding credentials; I wonder if the above changes his mind at all.

Between (3) and (2):

• (2) feels like a roundabout version of (3), requiring more server intervention. One could also see it as a roundabout version of (1), adding a very tailored cookie for this one request. Either way, (2) feels fairly inelegant to me.
• However, maybe it is a feature that (2) requires more server intervention: it makes it slightly harder for servers to unintentionally serve multiple origin policies for the same user. The default path with (2) still ends up with one origin policy per origin, whereas the default path for (3) makes it really easy to end up with multiple origin policies per origin.
• On a personal level, (3) would be a lot of churn in the spec, web platform tests, and Chrome implementation, which would make me sad about all the work I'd be rolling back. That shouldn't really matter though...