Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MSC3266: Room summary API #3266

Open
wants to merge 27 commits into
base: old_master
Choose a base branch
from

Conversation

deepbluev7
Copy link
Contributor

@deepbluev7 deepbluev7 commented Jul 4, 2021

Rendered.

Somewhat related to MSC2946 this provides an API to get a summary for a specific room either from the local server or over federation.

Useful for a few cases, where you need to show a summary for a room like matrix.to, traveler bots, showing spaces, lightweight clients, etc.

Open design questions looking for feedback: see this comment chain: #3266 (comment) resolved as of 2021/10/06

Implementations:

Signed-off-by: Nicolas Werner nicolas.werner@hotmail.de

Signed-off-by: Nicolas Werner <nicolas.werner@hotmail.de>
@deepbluev7 deepbluev7 changed the title Room summary proposal MSC3266: Room summary proposal Jul 4, 2021
@deepbluev7 deepbluev7 changed the title MSC3266: Room summary proposal MSC3266: Room summary Jul 4, 2021
@turt2live turt2live added client-server Client-Server API kind:feature MSC for not-core and not-maintenance stuff needs-implementation This MSC does not have a qualifying implementation for the SCT to review. The MSC cannot enter FCP. proposal A matrix spec change proposal proposal-in-review labels Jul 5, 2021
@deepbluev7 deepbluev7 changed the title MSC3266: Room summary MSC3266: Room summary API Jul 11, 2021
Signed-off-by: Nicolas Werner <nicolas.werner@hotmail.de>
Comment on lines 154 to 156
- The `/state` API could be used, but the response is much bigger than needed,
can't be cached as easily and may need more requests. This also doesn't work
over federation (yet).
Copy link
Member

@t3chguy t3chguy Jul 14, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also the /state API also doesn't return stripped state events so would not have access to said data based on the Stripped State MSC https://matrix.org/docs/spec/client_server/r0.6.1#get-matrix-client-r0-rooms-roomid-state

The specific state API does return stripped state (because consistency, what's that) https://matrix.org/docs/spec/client_server/r0.6.1#get-matrix-client-r0-rooms-roomid-state-eventtype-statekey

But would require MANY API hits and not be able to reach all the data (e.g num_joined_members would not be possible)

Signed-off-by: Nicolas Werner <nicolas.werner@hotmail.de>
…est of the path separate

Signed-off-by: Nicolas Werner <nicolas.werner@hotmail.de>
That way the requesting server knows, if any user would have access to
that room and it can forward the room to the user.

Signed-off-by: Nicolas Werner <nicolas.werner@hotmail.de>
Copy link
Member

@turt2live turt2live left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

overall I'm more convinced on this MSC now, particularly for the need of seeing if a room is joinable or knockable in a client, though still have a few concerns before this can go into FCP (imo).

I don't think they're unsurmountable, but some are not as easy as others (potentially).

Comment on lines +33 to +37
The API returns a summary of the given room, provided the user is either already
a member, or has the necessary permissions to join. (For example, the user may
be a member of a room mentioned in an `allow` condition in the join rules of a
restricted room.) For unauthenticated requests a response should only be
returned if the room is publicly accessible.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This MSC is still unclear in what it wants, imo. If it just wants to use the same conditions as /hierarchy then it can just say that (independent of us fixing matrix-org/matrix-spec#1184). Currently, it forbids knockable rooms which appears to be an anti-goal of this MSC.

Suggested phrasing:

The API returns a summary of the given room, provided the user has reasonable justification to see those details. At present, this would mean that the room would normally be returned by /hierarchy, which includes knockable, restricted, and public rooms. Invite-only rooms would not be accessible by this API unless the user was already joined to that room.

via=neko.dev
```

(This is not under `/rooms`, because it can be used with an alias.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's call it /room_summary - there's a good chance we'll want other entities to be summarized in the future and it'd be annoying to fix/replace the API endpoint for them.

Comment on lines +50 to +52
- `via` are servers that should be tried to request a summary from, if it can't
be generated locally. These can be from a matrix URI, matrix.to link or a
`m.space.child` event for example.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The vias feel insufficiently specified, despite I think everyone "knowing" what you mean:

  • Are they required? (no)
  • Are they the same as https://spec.matrix.org/v1.3/appendices/#routing ? (yes - please add the link)
  • Where is "locally"? (the server, but could be argued to be on the client side with the current language)
  • How does the client pick servers for an arbitrary room? (it is most often told what they are, but in the case of the room directory or bare room ID it might have to guess/hope for the best, just like with regular joins)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The spec text for this same parameter (but with a different name) on the /join/ endpoint is this:

The servers to attempt to join the room through. One of the servers must be participating in the room.

Why does an MSC need to be more specific then the spec? It is needed when otherwise the room can't be routed to. If that is the case, depends on what room you are previewing. If you know you are joined already, you don't need it. If you pass an alias, you don't need it. If you got a random room and you know some vias, there is no harm in just passing them. Some examples where those vias can be from is already listed in the line you are commenting on, but if a client provides more ways to preview a room it might have client side fields for that, like a text field a user can enter servers in, and I don't think that needs to be specified?

Copy link
Member

@KitsuneRal KitsuneRal Dec 16, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AIUI, linking to appendices/#routing effectively answers the questions regarding "locally" and the choice of servers?

Comment on lines +73 to +76
These are the same fields as those returned by `/publicRooms` or
[`/hierarchy`](https://spec.matrix.org/v1.3/client-server-api/#get_matrixclientv1roomsroomidhierarchy)
, with a few additions: `room_type`, `membership`, `room_version` and
`encryption`.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of taking this approach, why not just do what /hierarchy did and use a PublicRoomsChunk explicitly, extending the schema of that? (the benefit being that all 3 related endpoints get a room_type, membership, room_version, and encryption instead of having to subschema the nightmare a second time)

Maintaining several similar objects is not ideal from a spec perspective.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, this implicitly specifies to reuse the "PublicRoomsChunk", but most people probably would not know what that means. I think it is clearer to name the endpoint, that already returns this value than using the name of the value, that most people probably don't remember what it is. At least I mostly ignore those names unless I work with the OpenAPI JSON directly.

I also copied this wording originally from the space summary MSCs. It only uses the PublicRoomsChunk name in one example, it generally talks about the return value of publicRooms otherwise.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given that we tend to change sub-structure names in OpenAPI occasionally (that doesn't alter the shape of the OpenAPI, generated code notwithstanding), I agree with @deepbluev7 that the current text is good enough.

`room_type`, `room_version` and `encryption` are already accessible as part of
the stripped state according to
https://spec.matrix.org/v1.3/client-server-api/#stripped-state . The
`membership` is not, but a client could access that in various different ways
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How is the client meant to use the membership field? Even with the table below, the client should already have several other ways of knowing its current membership - is this really the best endpoint to reveal such information?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some clients don't call /sync. Having to use an extra endpoint to fetch this endpoint is opposite to the goals of this endpoint. In general the reason to call a summary endpoint is to do less calls than generating all that info yourself.

For example, if you want to join a room, you might want to generate a preview first. Then you want to name the button differently depending on if you are joined to the room or not. If you have all joined rooms cached locally, you can fetch that info from there, but some clients do not have that info without doing another request. And the server needs to check the join state anyway to generate some of the results, so it should not have an additional cost to just put that in the summary.

Comment on lines +117 to +118
For symmetry the `room_version` and `encryption` fields are also added to the
`/hierarchy` API.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please just use PublicRoomsChunk 😭

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You mean ChildRoomsChunk. And again, I think those names are confusing.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(FWIW - the name ChildRoomsChunk was introduced (by me) in a clarification PR (not MSC) solely for the purpose of describing, well, child rooms in /hierarchy. Refactoring those sub-schema names is certainly not something for this MSC (we rarely, if ever, did it in the previous MSC practice). PublicRoomsChunk is used in a couple of places across the API; I agree the name could be better but it actually reflects the current usage of this group of fields.)

I think it's a good question for SCT on the status of those names, how much we have to preserve them etc.; in the meantime, I would suggest keeping the text as it is to avoid accidental confusion.

Comment on lines +164 to +170
Additionally the `/hierarchy` API doesn't work using aliases. This currently
doesn't allow users to preview rooms not known to the local server over
federation. While the user can resolve the alias and then call the `/hierarchy`
API using the resolved roomid, a roomid is not a routable entity, so the server
never receives the information which servers to ask about the requested rooms.
This MSC resolves that by providing a way to pass server names to ask for the
room as well as the alias directly.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is all made invalid by the summary endpoint having to reduce itself down to a room ID before going out over federation? How is it any more routable if the server does the resolution instead of the client?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the client resolves the alias, only the client knows the server that resolved the alias.

If the server resolves the alias, it receives a list of servers that it can join via, which it can then use to query for the summary. This is the same behavior as with every other endpoint that takes an alias (apart from the room directory endpoints themselves, I guess). Usually asking the server about the room the alias is hosted on is entirely sufficient though. Apart from the directory, there is no endpoint which passes an alias over federation, so I don't see how that would be an issue in this case now?

Comment on lines +161 to +162
space, but also is missing the room version, membership and the encryption
field.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this hardly seems like an unsurmountable problem... The argument for it dealing mostly with spaces is quite strong, but saying it's missing fields is just asking for an MSC to add the fields :p

The key with the alternatives section is it needs to describe why a solution isn't valid, so, why is writing an MSC for adding the fields a bad idea? (answer: it's because of the focus on spaces - it would be confusing to client developers to call a Spaces endpoint to get a non-Space summary, though there is an unargued point about shifting the complexity onto the server instead, as this MSC does).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This MSC actually adds most of those fields to the /hierarchy endpoint as well. But the EXISTING endpoint is not an alternative to this MSC, since it lacks those fields. Of course those could be added, but it is not the only reason why it currently can't be used for this. This sentence is just an additional complication that people would notice, when they try to use the /hierarchy endpoint as is without extending it.

Comment on lines +234 to +235
This API may leak data, if implemented incorrectly or malicious servers could
return wrong results for a summary.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is true with any endpoint - the leak for this endpoint would more be that people could be surprised by the amount of metadata returned, such as in the case of matrix-org/matrix-spec#1186

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So should I just delete that line? I do think it is a genuine concern.

over federation and are much heavier than necessary or need a lot of http calls
for each room.

## Proposal
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We appear to be missing error conditions from the MSC: what if the room doesn't exist? is invalid? unroutable?

@turt2live turt2live removed the needs-implementation This MSC does not have a qualifying implementation for the SCT to review. The MSC cannot enter FCP. label Aug 9, 2022
@t3chguy
Copy link
Member

t3chguy commented Oct 24, 2022

An interesting future consumer of this work could be shields.io - https://github.com/badges/shields/blob/master/services/matrix/matrix.service.js currently registers a bunch of guests and uses the /state/ API so a dedicated API which doesn't require auth would be ideal.

@richvdh
Copy link
Member

richvdh commented Dec 13, 2022

Still lots of outstanding questions from previous reviews here. @deepbluev7 are you still interested in pursuing this?

@deepbluev7
Copy link
Contributor Author

Yeah, I am still pursuing this, but I need to update it to work with the new room join rule, since there knocking and restricted joins are not distinguishable, so the allowed roomids need to be exposed, possibly prefiltered. But since my clients now need the unstable endpoint anyway in a few released versions, I am not as much in a hurry anymore.

@turt2live
Copy link
Member

I'm going to take this off the SCT's priority list in that case. There is a bit of a problem if clients are relying on the unstable endpoint existing (as we're seeing happen outside of just your clients) - ideally, the MSC moves forward at a more hurried pace to avoid problems related to defacto spec.

For reference, how we'd likely avoid defacto spec being an issue here is recommend the feature for removal from Synapse, forcing clients to treat the functionality as unstable.

@deepbluev7
Copy link
Contributor Author

Well, the spec team told me to prioritize #3664, so that's what I did. Now that is stuck again. Before that, this MSC was stuck because there weren't enough implementations and half of the comments are nitpicking about wording or names, instead of actually being helpful to the content of the MSC, so it simply takes a bit of time to work on things. I changed some sentences 3 or more times. It simply takes a bit of time to please everyone with the wording, since if you react quickly, you end up just changing it back again on the next comment.

I think the 2 open points, that are actually important for the functionality of the MSC are:

  • error codes
  • No information about if you can join a restricted_knock room

The rest, in my opinion, is arguing about how specific things are worded or if a specific sentence "is important".

Co-authored-by: Alexey Rusakov <Kitsune-Ral@users.sf.net>
@melroy89
Copy link

Also still not merged? We have a lot of users now complaining they can't see the rooms in the Matrix space. Most likely people are using Dendrite. Please consider making it out of beta this year...

@deepbluev7
Copy link
Contributor Author

@melroy89 Previewing rooms in a space is already part of the spec. This is for rooms outside the space hierarchy.

@melroy89
Copy link

@melroy89 Previewing rooms in a space is already part of the spec. This is for rooms outside the space hierarchy.

Ah I see, since that still needs to be enabled manually in Dendrite?
https://github.com/matrix-org/dendrite/blob/main/docs/FAQ.md#does-dendrite-support-space-summaries

@deepbluev7
Copy link
Contributor Author

That's because the devs didn't get around to stabilizing it and has little to do with the spec.

@aine-etke
Copy link

yingziwu added a commit to yingziwu/synapse that referenced this pull request May 15, 2024
Synapse 1.107.0 (2024-05-14)
============================

No significant changes since 1.107.0rc1.

- Add preliminary support for [MSC3823: Account Suspension](matrix-org/matrix-spec-proposals#3823). ([\#17051](element-hq/synapse#17051))
- Declare support for [Matrix v1.10](https://matrix.org/blog/2024/03/22/matrix-v1.10-release/). Contributed by @clokep. ([\#17082](element-hq/synapse#17082))
- Add support for [MSC4115: membership metadata on events](matrix-org/matrix-spec-proposals#4115). ([\#17104](element-hq/synapse#17104), [\#17137](element-hq/synapse#17137))

- Fixed search feature of Element Android on homesevers using SQLite by returning search terms as search highlights. ([\#17000](element-hq/synapse#17000))
- Fixes a bug introduced in v1.52.0 where the `destination` query parameter for the  [Destination Rooms Admin API](https://element-hq.github.io/synapse/v1.105/usage/administration/admin_api/federation.html#destination-rooms) failed to actually filter returned rooms. ([\#17077](element-hq/synapse#17077))
- For MSC3266 room summaries, support queries at the recommended endpoint of `/_matrix/client/unstable/im.nheko.summary/summary/{roomIdOrAlias}`. The existing endpoint of `/_matrix/client/unstable/im.nheko.summary/rooms/{roomIdOrAlias}/summary` is deprecated. ([\#17078](element-hq/synapse#17078))
- Apply user email & picture during OIDC registration if present & selected. ([\#17120](element-hq/synapse#17120))
- Improve error message for cross signing reset with [MSC3861](matrix-org/matrix-spec-proposals#3861) enabled. ([\#17121](element-hq/synapse#17121))
- Fix a bug which meant that to-device messages received over federation could be dropped when the server was under load or networking problems caused problems between Synapse processes or the database. ([\#17127](element-hq/synapse#17127))
- Fix bug where `StreamChangeCache` would not respect configured cache factors. ([\#17152](element-hq/synapse#17152))

- Correct licensing metadata on Docker image. ([\#17141](element-hq/synapse#17141))

- Update the `event_cache_size` and `global_factor` configuration options' documentation. ([\#17071](element-hq/synapse#17071))
- Remove broken sphinx docs. ([\#17073](element-hq/synapse#17073), [\#17148](element-hq/synapse#17148))
- Add RuntimeDirectory to example matrix-synapse.service systemd unit. ([\#17084](element-hq/synapse#17084))
- Fix various small typos throughout the docs. ([\#17114](element-hq/synapse#17114))
- Update enable_notifs configuration documentation. ([\#17116](element-hq/synapse#17116))
- Update the Upgrade Notes with the latest minimum supported Rust version of 1.66.0. Contributed by @jahway603. ([\#17140](element-hq/synapse#17140))

- Enable [MSC3266](matrix-org/matrix-spec-proposals#3266) by default in the Synapse Complement image. ([\#17105](element-hq/synapse#17105))
- Add optimisation to `StreamChangeCache.get_entities_changed(..)`. ([\#17130](element-hq/synapse#17130))

* Bump furo from 2024.1.29 to 2024.4.27. ([\#17133](element-hq/synapse#17133))
* Bump idna from 3.6 to 3.7. ([\#17136](element-hq/synapse#17136))
* Bump jsonschema from 4.21.1 to 4.22.0. ([\#17157](element-hq/synapse#17157))
* Bump lxml from 5.1.0 to 5.2.1. ([\#17158](element-hq/synapse#17158))
* Bump phonenumbers from 8.13.29 to 8.13.35. ([\#17106](element-hq/synapse#17106))
- Bump pillow from 10.2.0 to 10.3.0. ([\#17146](element-hq/synapse#17146))
* Bump pydantic from 2.6.4 to 2.7.0. ([\#17107](element-hq/synapse#17107))
* Bump pydantic from 2.7.0 to 2.7.1. ([\#17160](element-hq/synapse#17160))
* Bump pyicu from 2.12 to 2.13. ([\#17109](element-hq/synapse#17109))
* Bump serde from 1.0.197 to 1.0.198. ([\#17111](element-hq/synapse#17111))
* Bump serde from 1.0.198 to 1.0.199. ([\#17132](element-hq/synapse#17132))
* Bump serde from 1.0.199 to 1.0.200. ([\#17161](element-hq/synapse#17161))
* Bump serde_json from 1.0.115 to 1.0.116. ([\#17112](element-hq/synapse#17112))
- Update `tornado` Python dependency from 6.2 to 6.4. ([\#17131](element-hq/synapse#17131))
* Bump twisted from 23.10.0 to 24.3.0. ([\#17135](element-hq/synapse#17135))
* Bump types-bleach from 6.1.0.1 to 6.1.0.20240331. ([\#17110](element-hq/synapse#17110))
* Bump types-pillow from 10.2.0.20240415 to 10.2.0.20240423. ([\#17159](element-hq/synapse#17159))
* Bump types-setuptools from 69.0.0.20240125 to 69.5.0.20240423. ([\#17134](element-hq/synapse#17134))
| num_joined_members | Required. Member count of the room | Copied from `publicRooms`. |
| topic | Optional. Topic of the room | Copied from `publicRooms`. |
| world_readable | Required. If the room history can be read without joining. | Copied from `publicRooms`. |
| join_rule | Optional. Join rules of the room | Copied from `publicRooms`. |
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With the introduction of knock_restricted, knowing the join rule isn't sufficient to know which method the room can be joined via. As such the response should also include the allowedRoomIds, that the user is joined to or so.

Comment on lines +33 to +35
The API returns a summary of the given room, provided the user is either already
a member, or has the necessary permissions to join. (For example, the user may
be a member of a room mentioned in an `allow` condition in the join rules of a
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not clear if this includes invites. It probably should, but that should be explicitly mentioned in the MSC.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
client-server Client-Server API kind:feature MSC for not-core and not-maintenance stuff proposal A matrix spec change proposal
Projects
None yet
Development

Successfully merging this pull request may close these issues.