HTTP content negotiation does not respect media type parameters [SPR-13365]

[spring-projects-issues]

Benedict Adamson opened SPR-13365 and commented

The
org.springframework.web.servlet.mvc.method.annotation.AbstractMessageConverterMethodProcessor
class does not respect media-type parameters when doing content negotiation. In the writeWithMessageConverters method, if a requestedType has a non empty MimeType.getParameters() map, those parameters have no effect on which type is the selectedMediaType. I believe that HTTP standard conformance requires that those parameters do have an effect on content negotiation.

The cause of this seems to be that writeWithMessageConverters uses MimeType.isCompatibleWith(MimeType), which ignores the parameters, and then (indirectly) sorts the produceable media types using MediaType.SPECIFICITY_COMPARATOR, which also ignores the parameters.

---

Worse still, if the selectedMediaType (the supported media type of the chosen HTTP message converter) differs from the requested type only in the media-type parameters, the Content-Type header of the response will contain the requested media type even though the response body is actually of the selectedMediaType.

The cause of that seems to be that the AbstractMessageConverterMethodProcessor.getMostSpecificMediaType(MediaType, MediaType) chooses the acceptType rather than the produceTypeToUse when the two types have equal specificity, according to the MediaType.SPECIFICITY_COMPARATOR, and that comparator ignores the media-type parameters.

---

Affects: 4.1 GA

Issue Links:

• Content negotiation ignores media type parameters [SPR-10903] #15531 Content negotiation ignores media type parameters ("duplicates")

[spring-projects-issues]

Rossen Stoyanchev commented

Can you provide a specific example or two with the requested media type(s) and expected vs actual behavior?

[spring-projects-issues]

Benedict Adamson commented

Here is some more information about the situation that lead me to discover this problem, with the details anonymized.

My web client and server communicate using a custom binary format. I gave the first version of the format a media-type like application/vnd.example.app.thing; version=1, with a custom HTTP message converter, BinaryThingConverter1, which has that media-type as its sole supported media type. Both client and server had that HTTP message converter, and everything worked fine.

I need to alter the custom binary format to add some new functionality. I need backwards compatibility, some my client and server should be able to communicate with the new version of the format and the old version of the format. I therefore chose to give the new version of the format a new media-type, like application/vnd.example.app.thing; version=2: I used the version parameter of the media-type to communicate the version of the format.

I'm using Test Driven Development, so I added a test-case for my current version of the web server (using AbstractJUnit4SpringContextTests) in which I simulate the client making a GET with an Accept header requesting the new format, application/vnd.example.app.thing; version=2. At this stage the web server does not have a new message converter that has that new media type as its supported media type; it still had only the old BinaryThingConverter1, which supports only application/vnd.example.app.thing; version=1. I expected my test-case to fail with an HTTP status code of 406 (Not Acceptable). I was surprised to discover that my test-case passed: the server responded with 200 (OK). I did some debugging to track down what was happening.

[spring-projects-issues]

Benedict Adamson commented

I see now that this is a duplicate of #15531.

[spring-projects-issues]

Brian Clozel commented

Hi Raedwald

In my opinion, "application/vnd.example.app.thing; version=1" and "application/vnd.example.app.thing; version=2" are compatible media types and should be processed by the same HttpMessageConverter. It means that both media types should have "similar content", but that the converter can process them differently.

According to RFC 7231 section 3.1.1.1:

The presence or absence of a parameter might be significant to the processing of a media-type, depending on its definition within the media type registry.

Given that your custom media type is not part of the registry, we can't really decide with this information.

RFC2046 section 1 says:

Parameters are modifiers of the media subtype, and as such do not fundamentally affect the nature of the content.

If you're changing the nature of the representation between those two, then you should probably use something like application/vnd.example.app.v1.thing and application/vnd.example.app.v2.thing. Check out GitHub's API documentation on Media Type.

Anyway, I think this question may duplicate #15531 - could you take a look at the comments?

[spring-projects-issues]

Brian Clozel commented

About the second part of this issue, the selectedMediaType for the response.

Are you sure MediaType.SPECIFICITY_COMPARATOR ignores parameters? To me, it is:

1. comparing the type and subtype specificity "text/*" < "text/html"
2. comparing types with their quality parameters
3. then comparing types with their number of parameters (see MimeType.SpecificityComparator.compareParameters)

Now AbstractMessageConverterMethodProcessor sorts from the least specific media-type to the most specific and loops on those; the first concrete media-type in that list is selected as a Content-Type. In other words, we select the least specific media-type we can.

I may be missing something here. If you've got pointers for this (what's the expected behavior here for a server) or if you think this should be the opposite (and why?) - please let me know!

[spring-projects-issues]

Rossen Stoyanchev commented

Given that your custom media type is not part of the registry, we can't really decide with this information.

Precisely, how would we compare the version parameter? A numeric comparison would be the obvious place to start but what if it was 2a, 2.0.1.A, or other. Also suppose there were additional parameters, perhaps major and minor versions, or perhaps completely unrelated to versioning. From a framework perspective we have no knowledge about the meaning of those parameters and their possible values.

As Brian outlined above we are quite intentionally sorting based on as much as we can assume, the type and sub-type, quality parameters, and number of parameters.

[spring-projects-issues]

Benedict Adamson commented

Precisely, how would we compare the version parameter

Compare for equality.

Here is a possible ranking scheme:

1. Prefer converters that have the same set of parameters as the request and equivalent values for all the parameters (that is, prefer exact matches)
2. Prefer converters that have a super-set of parameters, with equivalent values for the parameters in the request.
3. Prefer converters that have the fewest parameters that are present but have different values for the parameters (this will tend to prefer converters that have no parameters over converts with many parameters that do not match).
4. Prefer converters that have the most parameters that match the request.

[spring-projects-issues]

Benedict Adamson commented

Are you sure MediaType.SPECIFICITY_COMPARATOR ignores parameters

As you point out, it examines the number of parameters. But it ignores the values of the parameters, which is what I meant.

[spring-projects-issues]

Brian Clozel commented

Raedwald,

I'm closing this issue as a duplicate, but feel free to open a new issue (improvement proposal) regarding mediatype comparison + media type parameters.

This is an interesting issue, but it will require developer interest (votes!) and use cases descriptions to really understand what we can do about that.

Thanks!