From cf501377d73bb6a92ffcc5310a69533fe0de857c Mon Sep 17 00:00:00 2001 From: David Baker Date: Fri, 21 Aug 2020 13:03:19 +0100 Subject: [PATCH 01/80] Placeholder for reliable VoIP MSC --- proposals/2746-reliable-voip.md | 3 +++ 1 file changed, 3 insertions(+) create mode 100644 proposals/2746-reliable-voip.md diff --git a/proposals/2746-reliable-voip.md b/proposals/2746-reliable-voip.md new file mode 100644 index 0000000000..3d0398c1a1 --- /dev/null +++ b/proposals/2746-reliable-voip.md @@ -0,0 +1,3 @@ +# MSC2746: Improved Signalling for 1:1 VoIP + +Placeholder MSC for improved 1:1 VoIP events From a9b17fcc08ecd38241a3bd45cefde713c93995e1 Mon Sep 17 00:00:00 2001 From: David Baker Date: Fri, 21 Aug 2020 19:14:14 +0100 Subject: [PATCH 02/80] First version written up --- proposals/2746-reliable-voip.md | 193 +++++++++++++++++++++++++++++++- 1 file changed, 192 insertions(+), 1 deletion(-) diff --git a/proposals/2746-reliable-voip.md b/proposals/2746-reliable-voip.md index 3d0398c1a1..aa336de30f 100644 --- a/proposals/2746-reliable-voip.md +++ b/proposals/2746-reliable-voip.md @@ -1,3 +1,194 @@ # MSC2746: Improved Signalling for 1:1 VoIP -Placeholder MSC for improved 1:1 VoIP events +Matrix has basic support for signalling 1:1 WebRTC calls, but has a number of shortcomings: + + * If several devices try to answer the same call, there is no way for them to determine clearly + that the caller has set up the call with a different device, and no way for the caller to + determine candidate events map to which answer. + * Hangup reasons are often incorrect. + * There is confusion and no clear guidance on how clients should determine whether an incoming + invite is stale or not. + * There is no support for renegotiation of SDP, for changing ICE candidates / hold/resume + functionality, etc. + * There is no distinction between rejecting a call and ending it, which means that in trying + to reject a call, a client can inadvertantly cause a call that has been sucessfully set up + on a different device to be hung up. + +## Proposal +### Change the `version` field in all VoIP events to `1` +This will be used to determine whether determine whether devices support this new version of the protocol. +If clients see events with `version` other than `0` or `1`, they should treat these the same as if they had +`version` == `1`. + +### Add `answer_id` to `m.call.answer` +This is a random string generated by the sending device in the same way as `call_id` (ie. in this case, +it should be sufficiently unique in the context of this VoIP call). This identifies each answer sent to +a given `m.call.invite`. + +*Also considered: use the `event_id` of the answer event: this is rejected for similarity with `call_id` +where it is desirable to know the ID of a call before receving the remote echo of the invite event (this +will be useful in future for call transfers where the transferor can assign a `call_id` for the transferee +to use).* + +### Add `answer_id` to `m.call.candidates` +This allows the caller to determine which candidate events correspond to which answer (for the callee, +all candidates with matching `call_id` not from its own user are from the caller party, of which there +is only one.) + +### Introduce `m.call.ack` +This event is sent by the caller once it has chosen an answer. Its `answer_id` +field indicates the answer it's chosen (and has `call_id` too). If the callee +sees an ack for an answer ID other than the one it sent, it ends the call and +informs the user the call was answered elsewhere. It does not send any events. +Media can start flowing before this event is seen or even sent. Clients that +implement previous versions of this specification will ignore this event and +behave as they did before. + +Example: +``` +{ + "type": "m.call.ack", + "content": { + "call_id": "12345", + "answer_id": "67890", + }, +} +``` + +### Introduce `m.call.reject` + + * If the `m.call.invite` event has `version` `1`, a client wishing to reject a call instead + sends an `m.call.reject` event. This rejects the call on all devices, but if another device + has already sent an accept, it disregards the reject and carries on. The reject has an + `answer_id` just like an answer, and the caller acks it just like an answer. If the other + client that had already sent an answer sees the caller ack the reject instead of its answer, + it ends the call. + * If the `m.call.invite` event has `version` `0`, the callee send an `m.call.hangup` event before. + +Example: +``` +{ + "type": "m.call.reject", + "content" : { + "version": 1, + "call_id": "12345", + "answer_id": "67890", + } +} +``` + +If the calling user chooses to end the call before setup is complete, the client sends `m.call.hangup` +as previously. + +### Clarify what actions a client may take in response to an invite +The client may: + * Attempt to accept the call by sending an answer + * Actively reject the call everywhere: reject the call as per above, which will stop the call from + ringing on all the user's devices and the caller's client will inform them that the user has + rejected their call. + * Ignore the call: send no events, but stop alerting the user about the call. The user's other + devices will continue to ring, and the caller's device will continue to indicate that the call + is ringing, and will time the call out in the normal way if no other device responds. + +### Introduce more reason codes to `m.call.hangup` + * `ice_timeout`: The connection failed after some media was exchanged (as opposed to current + `ice_failed` which means no media connection could be established). Note that, in the case of + an ICE renegotiation, a client should be sure to send `ice_timeout` rather than `ice_failed` if + media had previously been received successfully, even if the ICE renegotiation itself failed. + * `user_hangup`: Clients must now send this code when the user chooses to end the call, although + for backwards compatability, a clients should treat an absence of the `reason` field as + `user_hangup`. + * `user_media_failed`: The client was unable to start capturing media in such a way as it is unable + to continue the call. + * `unknown_error`: Some other failure occurred that meant the client was unable to continue the call + rather than the user choosing to end it. + +### Introduce `m.call.negotiate` +This introduces SDP negotiation semantics for media pause, hold/resume, ICE restarts and voice/video +call up/downgrading. Clients should implement & honour hold functionality as per WebRTC's +recommendation: https://www.w3.org/TR/webrtc/#hold-functionality + +If both the invite event and the accepted answer event have `v` equal to `1`, either party may +send `m.call.negotiate` with an `sdp` field to offer new SDP to the other party. This event has +`call_id` with the ID of the call and those sent by the callee have `answer_id` equal to the ID +of the client's answer. The caller ignores any negotiate events with `answer_id` not equal to the +answer it accepted. Clients may either use the same mechanism used for remote echo of messages +to recognise and ignore their own negotiate messages (ie. txn id) or they may ignore messages +from their own user, or they may use the presence or absence of an `answer_id` field. + +This has a `lifetime` field as in `m.call.invite`, after which the sender of the negotiate event +should consider the negotiation failed (timed out) and the recipient should ignore it. + +Example: +``` +{ + "type": "m.call.negotiate", + "content": { + "call_id": "12345", + "answer_id": "67890", + "sdp": "[some sdp]", + "lifetime": 10000, + } +} +``` + +### Designate one party as 'polite' +In line with WebRTC perfect negotiation (https://w3c.github.io/webrtc-pc/#perfect-negotiation-example) +we introduce rules to establish which party is polite. By default, the callee is the polite party. +In a glare situation, if the client receives an invite whilst preparing to send, it becomes the callee +and therefore becomes the polite party. If an invite is received after the client has sent one, the +party whose invite had the lexicographically greater call ID becomes the polite party. + +### Add explicit recommendations for call event liveness. +`m.call.invite` contains a `lifetime` field that indicates how long the offer is valid for. When +a client receives an invite, it should use the `age` field of the event plus the time since it +received the event from the homeserver to determine whether the invite is still valid. If the +invite is still valid *and will remain valid for long enough for the user to accept the call*, +it should signal an incoming call. The amount of time allowed for the user to accept the call may +vary between clients, for example, it may be longer on a locked mobile device than on an unlocked +desktop device. + +The client should only signal an incoming call once it has completed processing the entire sync +response. This ensures that if the sync response contains subsequent events that indicate the call +has been hung up, rejected, or answered elsewhere, the client does not signal it. + +If on startup, after processing locally stored events, the client determines that there is an invite +that is still valid, it should still signal it but only after it has completed a sync from the homeserver. + +### Introduce recommendations for batching of ICE candidates +Clients should aim to send a small number of candidate events, with guidelines: + * ICE candidates which can be discovered immediately or almost immediately in the invite/answer + event itself (eg. host candidates). If server reflexive or relay candiates can be gathered in + a sufficiently short period of time, these should be sent here too. A delay of around 200ms is + suggested as a starting point. + * The client should then allow some time for further candidates to be gathered in order to batch them, + rather than sending each candidate as it arrives. A starting point of 2 seconds after sending the + invite or 500ms after sending the answer is suggested as starting point (since a delay is natural + anyway after the invite whilst the client waits for the user to accept it). + +### Add DTMF +Add that Matrix clients can send DTMF as specified by WebRTC. The WebRTC standard as of August +2020 does not support receiving DTMF but a Matrix client can receive and interpret the DTMF sent +in the RTP payload. + +### Deprecate `type` in `m.call.invite` and `m.call.answer` +These are redundant: clients should continue to send them but must not require +them to be present on events they receive. + +## Potential issues + * It remains explicity impossible to place a call to yourself. Matrix uses a shared medium for + signalling so a client will always see invites from other devices. We would need to introduce + a way for a client to signal to other devices that they should treat the invite as an incoming + call and mechanisms to clarify what events were from which party in a call. This would mean a + significant amount of protocol dedcated to just this feature, so this MSC omits it. + +## Alternatives + * We could use event IDs for `call_id` and `answer_id` as discussed above. + * The event type of `m.call.ack` mirrors that of SIP, although gives few other clues on its purpose. + `m.call.choose_answer` was considered but is quite verbose. +## Security considerations + * IP addresses remain in the room in candidates, as they did in the previous version of the spec. + This is not ideal, but alternatives were either sending candidates over to-device messages + (would slow down call setup because a target device would have to be established before sending + candidates) or redacting them afterwards (the volume of events sent during calls can already + cause rate limiting issues and this would exacerbate this). From 37c1f9833b5bc1b6f8e786d41ed2b42181086b93 Mon Sep 17 00:00:00 2001 From: David Baker Date: Mon, 24 Aug 2020 09:50:06 +0100 Subject: [PATCH 03/80] Typo Co-authored-by: Matthew Hodgson --- proposals/2746-reliable-voip.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/2746-reliable-voip.md b/proposals/2746-reliable-voip.md index aa336de30f..205ace71ef 100644 --- a/proposals/2746-reliable-voip.md +++ b/proposals/2746-reliable-voip.md @@ -26,7 +26,7 @@ it should be sufficiently unique in the context of this VoIP call). This identif a given `m.call.invite`. *Also considered: use the `event_id` of the answer event: this is rejected for similarity with `call_id` -where it is desirable to know the ID of a call before receving the remote echo of the invite event (this +where it is desirable to know the ID of a call before receiving the remote echo of the invite event (this will be useful in future for call transfers where the transferor can assign a `call_id` for the transferee to use).* From 5156caf11ba9d48ed0451fc09fb53ecd2b8f4188 Mon Sep 17 00:00:00 2001 From: David Baker Date: Mon, 24 Aug 2020 12:10:49 +0100 Subject: [PATCH 04/80] Typo Co-authored-by: Matthew Hodgson --- proposals/2746-reliable-voip.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/2746-reliable-voip.md b/proposals/2746-reliable-voip.md index 205ace71ef..4b5796f3ac 100644 --- a/proposals/2746-reliable-voip.md +++ b/proposals/2746-reliable-voip.md @@ -180,7 +180,7 @@ them to be present on events they receive. signalling so a client will always see invites from other devices. We would need to introduce a way for a client to signal to other devices that they should treat the invite as an incoming call and mechanisms to clarify what events were from which party in a call. This would mean a - significant amount of protocol dedcated to just this feature, so this MSC omits it. + significant amount of protocol dedicated to just this feature, so this MSC omits it. ## Alternatives * We could use event IDs for `call_id` and `answer_id` as discussed above. From fe8b1ebbd535103541c7d62d55d1d96da26edceb Mon Sep 17 00:00:00 2001 From: David Baker Date: Mon, 24 Aug 2020 14:54:50 +0100 Subject: [PATCH 05/80] Switch to `m.call.select_answer` --- proposals/2746-reliable-voip.md | 24 ++++++++++++------------ 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/proposals/2746-reliable-voip.md b/proposals/2746-reliable-voip.md index 4b5796f3ac..9d9d9bb050 100644 --- a/proposals/2746-reliable-voip.md +++ b/proposals/2746-reliable-voip.md @@ -35,19 +35,19 @@ This allows the caller to determine which candidate events correspond to which a all candidates with matching `call_id` not from its own user are from the caller party, of which there is only one.) -### Introduce `m.call.ack` +### Introduce `m.call.select_answer` This event is sent by the caller once it has chosen an answer. Its `answer_id` field indicates the answer it's chosen (and has `call_id` too). If the callee -sees an ack for an answer ID other than the one it sent, it ends the call and -informs the user the call was answered elsewhere. It does not send any events. -Media can start flowing before this event is seen or even sent. Clients that -implement previous versions of this specification will ignore this event and -behave as they did before. +sees a `select_answer` for an answer ID other than the one it sent, it ends the +call and informs the user the call was answered elsewhere. It does not send any +events. Media can start flowing before this event is seen or even sent. +Clients that implement previous versions of this specification will ignore this +event and behave as they did before. Example: ``` { - "type": "m.call.ack", + "type": "m.call.select_answer", "content": { "call_id": "12345", "answer_id": "67890", @@ -60,9 +60,9 @@ Example: * If the `m.call.invite` event has `version` `1`, a client wishing to reject a call instead sends an `m.call.reject` event. This rejects the call on all devices, but if another device has already sent an accept, it disregards the reject and carries on. The reject has an - `answer_id` just like an answer, and the caller acks it just like an answer. If the other - client that had already sent an answer sees the caller ack the reject instead of its answer, - it ends the call. + `answer_id` just like an answer, and the caller sends a `select_answer` for it just like an + answer. If the other client that had already sent an answer sees the caller select the + reject response instead of its answer, it ends the call. * If the `m.call.invite` event has `version` `0`, the callee send an `m.call.hangup` event before. Example: @@ -184,8 +184,8 @@ them to be present on events they receive. ## Alternatives * We could use event IDs for `call_id` and `answer_id` as discussed above. - * The event type of `m.call.ack` mirrors that of SIP, although gives few other clues on its purpose. - `m.call.choose_answer` was considered but is quite verbose. + * `m.call.select_answer` was chosen such that its name reflect the intention of the event. `m.call.ack` + is more succinct and mirrors SIP, but this MSC opts for the more descriptive name. ## Security considerations * IP addresses remain in the room in candidates, as they did in the previous version of the spec. This is not ideal, but alternatives were either sending candidates over to-device messages From 6c4a077a2184590bf27c1e3a21e387e96025217c Mon Sep 17 00:00:00 2001 From: David Baker Date: Tue, 25 Aug 2020 12:32:51 +0100 Subject: [PATCH 06/80] Make self-calling possible Replaces `answer_id` with `party_id` and introduces the `invitee` field on `m.call.invite`. --- proposals/2746-reliable-voip.md | 100 ++++++++++++++++++++------------ 1 file changed, 63 insertions(+), 37 deletions(-) diff --git a/proposals/2746-reliable-voip.md b/proposals/2746-reliable-voip.md index 9d9d9bb050..1702d7e1ae 100644 --- a/proposals/2746-reliable-voip.md +++ b/proposals/2746-reliable-voip.md @@ -20,29 +20,37 @@ This will be used to determine whether determine whether devices support this ne If clients see events with `version` other than `0` or `1`, they should treat these the same as if they had `version` == `1`. -### Add `answer_id` to `m.call.answer` -This is a random string generated by the sending device in the same way as `call_id` (ie. in this case, -it should be sufficiently unique in the context of this VoIP call). This identifies each answer sent to -a given `m.call.invite`. - -*Also considered: use the `event_id` of the answer event: this is rejected for similarity with `call_id` -where it is desirable to know the ID of a call before receiving the remote echo of the invite event (this -will be useful in future for call transfers where the transferor can assign a `call_id` for the transferee -to use).* - -### Add `answer_id` to `m.call.candidates` -This allows the caller to determine which candidate events correspond to which answer (for the callee, -all candidates with matching `call_id` not from its own user are from the caller party, of which there -is only one.) +### Add `invitee` field to `m.call.invite` +This allows for the following use cases: + * Placing a call to a specifc user in a room where other users are also present. + * Placing a call to oneself. +The field should be added for all invites where the target is a specific user. Invites without an `invitee` +field are defined to be intended for any member of the room other than the sender of the event. Clients +should consider an incoming call if they see a non-expired invite event where the `invitee` field is either +absent or equal to their user's Matrix ID, however they should evaluate whether or not to ring based on their +user's trust relationship with the caller, eg. ignoring call invites from users in public rooms that they have +no other connection with. As a starting point, it is suggested that clients ring for any call invite from a user +that they have a direct message room with. It is strongly recommended that when clients do not ring for an +incoming call invite, they still display the invite in the room and annotate that it was ignored. + +### Add `party_id` to all VoIP events +Whenever a client first participates in a new call, it generates a `party_id` for itself to use for the +duration of the call. It adds a `party_id` field containing this ID to all VoIP events it sends on the +call. Clients use this to identify remote echo of their own events, since a user may now call themselves, +they can no longer ignore events from their own user. This field also identifies different answers sent +by different clients to an invite, and matches `m.call.candidate` events to their respective answer/invite. + +A client implementation may choose to use the device ID used in end-to-end cryptography for this purpose. ### Introduce `m.call.select_answer` -This event is sent by the caller once it has chosen an answer. Its `answer_id` -field indicates the answer it's chosen (and has `call_id` too). If the callee -sees a `select_answer` for an answer ID other than the one it sent, it ends the -call and informs the user the call was answered elsewhere. It does not send any -events. Media can start flowing before this event is seen or even sent. -Clients that implement previous versions of this specification will ignore this -event and behave as they did before. +This event is sent by the caller once it has chosen an answer. Its +`selected_party_id` field indicates the answer it's chosen (and has `call_id` +and its own `party_id` too). If the callee sees a `select_answer` for an answer +ID other than the one it sent, it ends the call and informs the user the call +was answered elsewhere. It does not send any events. Media can start flowing +before this event is seen or even sent. Clients that implement previous +versions of this specification will ignore this event and behave as they did +before. Example: ``` @@ -50,7 +58,8 @@ Example: "type": "m.call.select_answer", "content": { "call_id": "12345", - "answer_id": "67890", + "party_id": "67890", + "selected_party_id": "111213", }, } ``` @@ -59,8 +68,8 @@ Example: * If the `m.call.invite` event has `version` `1`, a client wishing to reject a call instead sends an `m.call.reject` event. This rejects the call on all devices, but if another device - has already sent an accept, it disregards the reject and carries on. The reject has an - `answer_id` just like an answer, and the caller sends a `select_answer` for it just like an + has already sent an accept, it disregards the reject and carries on. The reject has a + `party_id` just like an answer, and the caller sends a `select_answer` for it just like an answer. If the other client that had already sent an answer sees the caller select the reject response instead of its answer, it ends the call. * If the `m.call.invite` event has `version` `0`, the callee send an `m.call.hangup` event before. @@ -72,7 +81,7 @@ Example: "content" : { "version": 1, "call_id": "12345", - "answer_id": "67890", + "party_id": "67890", } } ``` @@ -110,11 +119,10 @@ recommendation: https://www.w3.org/TR/webrtc/#hold-functionality If both the invite event and the accepted answer event have `v` equal to `1`, either party may send `m.call.negotiate` with an `sdp` field to offer new SDP to the other party. This event has -`call_id` with the ID of the call and those sent by the callee have `answer_id` equal to the ID -of the client's answer. The caller ignores any negotiate events with `answer_id` not equal to the -answer it accepted. Clients may either use the same mechanism used for remote echo of messages -to recognise and ignore their own negotiate messages (ie. txn id) or they may ignore messages -from their own user, or they may use the presence or absence of an `answer_id` field. +`call_id` with the ID of the call and `party_id` equal to the client's party ID for that call. +The caller ignores any negotiate events with `party_id` not equal to the `party_id` of the +answer it accepted. Clients should use the `party_id` field to ignore the remote echo of their +own negotiate events. This has a `lifetime` field as in `m.call.invite`, after which the sender of the negotiate event should consider the negotiation failed (timed out) and the recipient should ignore it. @@ -125,7 +133,7 @@ Example: "type": "m.call.negotiate", "content": { "call_id": "12345", - "answer_id": "67890", + "party_id": "67890", "sdp": "[some sdp]", "lifetime": 10000, } @@ -176,19 +184,37 @@ These are redundant: clients should continue to send them but must not require them to be present on events they receive. ## Potential issues - * It remains explicity impossible to place a call to yourself. Matrix uses a shared medium for - signalling so a client will always see invites from other devices. We would need to introduce - a way for a client to signal to other devices that they should treat the invite as an incoming - call and mechanisms to clarify what events were from which party in a call. This would mean a - significant amount of protocol dedicated to just this feature, so this MSC omits it. + * The ability to call yourself makes the protocol a little more complex for clients to implement, + and is somewhat of a special case. However, some of the hnecessary additions are necessary for + other features so this MSC elects to make it possible. + * Clients must make a decision on whether to ring for any given call: defining this in the spec + would be cumbersome and would limit clients' ability to use reputation-based systems for this + decision in the future. However, having a call ring on one client and not the other because one + had categorised it as a junk call and not the other would be confusing for the user. ## Alternatives - * We could use event IDs for `call_id` and `answer_id` as discussed above. + * We could define that the ID of a call is implcitly the event IDs of the invite event rather than + having a specific `call_id` field. This would mean that a client would be unable to know the ID of + a call before the remote echo of the invite came back, which could complicate implementations. + There is probably no compelling reason to change this. * `m.call.select_answer` was chosen such that its name reflect the intention of the event. `m.call.ack` is more succinct and mirrors SIP, but this MSC opts for the more descriptive name. + * This MSC elects to allow invites without an `invitee` field to mean a call for anyone in the room. + This could be useful for hunt group style semantics where an incoming call causes many different + users' phones to ring and any one of them may pick up the call. This does mean clients will need + to not blindly ring for any call invites in any room, since this would make unsolicited calls in + easy in public rooms. We could opt to leave this out, or make it more explicit with a specific value + for the `invitee` field. + * `party_id` is one of many potential solutions: callees could add `answer_id`s to their events and + callers could be identified by the lack of an `answer_id`. An explicit field on every event may be + easier to comprehend, less error-prone and clearer in the backwards-compatibility scenario. + * We could make `party_id`s more prescriptive, eg. the caller could always have a `party_id` of the + empty string, the word `caller` or equal to the `call_id`, which may make debugging simpler. + ## Security considerations * IP addresses remain in the room in candidates, as they did in the previous version of the spec. This is not ideal, but alternatives were either sending candidates over to-device messages (would slow down call setup because a target device would have to be established before sending candidates) or redacting them afterwards (the volume of events sent during calls can already cause rate limiting issues and this would exacerbate this). + * Clients must take care to not ring for any call, as per the 'alternatives' section. From 25ed29aa7673d1bc59423748e3e78a36d7b69075 Mon Sep 17 00:00:00 2001 From: David Baker Date: Thu, 3 Sep 2020 16:00:52 +0100 Subject: [PATCH 07/80] Nobody spotted the deliberate typo --- proposals/2746-reliable-voip.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/2746-reliable-voip.md b/proposals/2746-reliable-voip.md index 1702d7e1ae..ee9eabe08d 100644 --- a/proposals/2746-reliable-voip.md +++ b/proposals/2746-reliable-voip.md @@ -185,7 +185,7 @@ them to be present on events they receive. ## Potential issues * The ability to call yourself makes the protocol a little more complex for clients to implement, - and is somewhat of a special case. However, some of the hnecessary additions are necessary for + and is somewhat of a special case. However, some of the necessary additions are necessary for other features so this MSC elects to make it possible. * Clients must make a decision on whether to ring for any given call: defining this in the spec would be cumbersome and would limit clients' ability to use reputation-based systems for this From bec62abfb94f4b2e9913835cf0b271a7f0f437c3 Mon Sep 17 00:00:00 2001 From: David Baker Date: Fri, 4 Sep 2020 10:01:05 +0100 Subject: [PATCH 08/80] Fixes & clarifications from Brendan Co-authored-by: Brendan Abolivier --- proposals/2746-reliable-voip.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/proposals/2746-reliable-voip.md b/proposals/2746-reliable-voip.md index ee9eabe08d..48e051329b 100644 --- a/proposals/2746-reliable-voip.md +++ b/proposals/2746-reliable-voip.md @@ -43,9 +43,9 @@ by different clients to an invite, and matches `m.call.candidate` events to thei A client implementation may choose to use the device ID used in end-to-end cryptography for this purpose. ### Introduce `m.call.select_answer` -This event is sent by the caller once it has chosen an answer. Its +This event is sent by the caller's client once it has chosen an answer. Its `selected_party_id` field indicates the answer it's chosen (and has `call_id` -and its own `party_id` too). If the callee sees a `select_answer` for an answer +and its own `party_id` too). If the callee's client sees a `select_answer` for an answer ID other than the one it sent, it ends the call and informs the user the call was answered elsewhere. It does not send any events. Media can start flowing before this event is seen or even sent. Clients that implement previous @@ -117,7 +117,7 @@ This introduces SDP negotiation semantics for media pause, hold/resume, ICE rest call up/downgrading. Clients should implement & honour hold functionality as per WebRTC's recommendation: https://www.w3.org/TR/webrtc/#hold-functionality -If both the invite event and the accepted answer event have `v` equal to `1`, either party may +If both the invite event and the accepted answer event have `version` equal to `1`, either party may send `m.call.negotiate` with an `sdp` field to offer new SDP to the other party. This event has `call_id` with the ID of the call and `party_id` equal to the client's party ID for that call. The caller ignores any negotiate events with `party_id` not equal to the `party_id` of the From 019bcddd48b16d7f788f29d4cc99d914f7bfaf55 Mon Sep 17 00:00:00 2001 From: David Baker Date: Fri, 4 Sep 2020 10:03:09 +0100 Subject: [PATCH 09/80] answers ID -> party ID --- proposals/2746-reliable-voip.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/2746-reliable-voip.md b/proposals/2746-reliable-voip.md index 48e051329b..1dcb4d796e 100644 --- a/proposals/2746-reliable-voip.md +++ b/proposals/2746-reliable-voip.md @@ -46,7 +46,7 @@ A client implementation may choose to use the device ID used in end-to-end crypt This event is sent by the caller's client once it has chosen an answer. Its `selected_party_id` field indicates the answer it's chosen (and has `call_id` and its own `party_id` too). If the callee's client sees a `select_answer` for an answer -ID other than the one it sent, it ends the call and informs the user the call +with party ID other than the one it sent, it ends the call and informs the user the call was answered elsewhere. It does not send any events. Media can start flowing before this event is seen or even sent. Clients that implement previous versions of this specification will ignore this event and behave as they did From 66179f11c1d4024a6a4df961f0761902c49b47ac Mon Sep 17 00:00:00 2001 From: David Baker Date: Fri, 4 Sep 2020 10:19:04 +0100 Subject: [PATCH 10/80] clarify party_id --- proposals/2746-reliable-voip.md | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/proposals/2746-reliable-voip.md b/proposals/2746-reliable-voip.md index 1dcb4d796e..0edc30a9cb 100644 --- a/proposals/2746-reliable-voip.md +++ b/proposals/2746-reliable-voip.md @@ -35,7 +35,12 @@ incoming call invite, they still display the invite in the room and annotate tha ### Add `party_id` to all VoIP events Whenever a client first participates in a new call, it generates a `party_id` for itself to use for the -duration of the call. It adds a `party_id` field containing this ID to all VoIP events it sends on the +duration of the call. This needs to be long enough that the chance of a collision between multiple devices +both generating an answer at the same time generating the same party ID is vanishingly small: 8 uppercase ++ lowercase alphanumeric characters is recommended. Parties in the call are identified by the tuple of +`(user_id, party_id)`. + +The client adds a `party_id` field containing this ID alongside the `user_id` field to all VoIP events it sends on the call. Clients use this to identify remote echo of their own events, since a user may now call themselves, they can no longer ignore events from their own user. This field also identifies different answers sent by different clients to an invite, and matches `m.call.candidate` events to their respective answer/invite. From 9e8c8292dd6127d0961f245968d5cf829678e96e Mon Sep 17 00:00:00 2001 From: David Baker Date: Fri, 4 Sep 2020 10:25:10 +0100 Subject: [PATCH 11/80] require that the client tries to decrypt all events before ringing --- proposals/2746-reliable-voip.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/proposals/2746-reliable-voip.md b/proposals/2746-reliable-voip.md index 0edc30a9cb..11a09bc27d 100644 --- a/proposals/2746-reliable-voip.md +++ b/proposals/2746-reliable-voip.md @@ -161,9 +161,10 @@ it should signal an incoming call. The amount of time allowed for the user to ac vary between clients, for example, it may be longer on a locked mobile device than on an unlocked desktop device. -The client should only signal an incoming call once it has completed processing the entire sync -response. This ensures that if the sync response contains subsequent events that indicate the call -has been hung up, rejected, or answered elsewhere, the client does not signal it. +The client should only signal an incoming call in a given room once it has completed processing the +entire sync response and, for encrypted rooms, attempted to decrypt all encrypted events in the +sync response for that room. This ensures that if the sync response contains subsequent events that +indicate the call has been hung up, rejected, or answered elsewhere, the client does not signal it. If on startup, after processing locally stored events, the client determines that there is an invite that is still valid, it should still signal it but only after it has completed a sync from the homeserver. From e224af33c449f378c95d49428c27cc0444b231db Mon Sep 17 00:00:00 2001 From: David Baker Date: Fri, 4 Sep 2020 10:26:27 +0100 Subject: [PATCH 12/80] not all of these necessary's were necessary --- proposals/2746-reliable-voip.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/2746-reliable-voip.md b/proposals/2746-reliable-voip.md index 11a09bc27d..37228e7d10 100644 --- a/proposals/2746-reliable-voip.md +++ b/proposals/2746-reliable-voip.md @@ -191,7 +191,7 @@ them to be present on events they receive. ## Potential issues * The ability to call yourself makes the protocol a little more complex for clients to implement, - and is somewhat of a special case. However, some of the necessary additions are necessary for + and is somewhat of a special case. However, some of the necessary additions are also required for other features so this MSC elects to make it possible. * Clients must make a decision on whether to ring for any given call: defining this in the spec would be cumbersome and would limit clients' ability to use reputation-based systems for this From 25618205ed9e10851d7f8b93690d618513c70e1c Mon Sep 17 00:00:00 2001 From: David Baker Date: Fri, 11 Sep 2020 13:48:43 +0100 Subject: [PATCH 13/80] Apply suggestions from code review Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> --- proposals/2746-reliable-voip.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/proposals/2746-reliable-voip.md b/proposals/2746-reliable-voip.md index 37228e7d10..93eccf5cde 100644 --- a/proposals/2746-reliable-voip.md +++ b/proposals/2746-reliable-voip.md @@ -4,7 +4,7 @@ Matrix has basic support for signalling 1:1 WebRTC calls, but has a number of sh * If several devices try to answer the same call, there is no way for them to determine clearly that the caller has set up the call with a different device, and no way for the caller to - determine candidate events map to which answer. + determine which candidate events map to which answer. * Hangup reasons are often incorrect. * There is confusion and no clear guidance on how clients should determine whether an incoming invite is stale or not. @@ -71,13 +71,13 @@ Example: ### Introduce `m.call.reject` - * If the `m.call.invite` event has `version` `1`, a client wishing to reject a call instead + * If the `m.call.invite` event has `version` `1`, a client wishing to reject the call sends an `m.call.reject` event. This rejects the call on all devices, but if another device has already sent an accept, it disregards the reject and carries on. The reject has a `party_id` just like an answer, and the caller sends a `select_answer` for it just like an answer. If the other client that had already sent an answer sees the caller select the reject response instead of its answer, it ends the call. - * If the `m.call.invite` event has `version` `0`, the callee send an `m.call.hangup` event before. + * If the `m.call.invite` event has `version` `0`, the callee sends an `m.call.hangup` event before. Example: ``` @@ -148,7 +148,7 @@ Example: ### Designate one party as 'polite' In line with WebRTC perfect negotiation (https://w3c.github.io/webrtc-pc/#perfect-negotiation-example) we introduce rules to establish which party is polite. By default, the callee is the polite party. -In a glare situation, if the client receives an invite whilst preparing to send, it becomes the callee +In a glare situation, if the client receives an invite whilst preparing to send one, it becomes the callee and therefore becomes the polite party. If an invite is received after the client has sent one, the party whose invite had the lexicographically greater call ID becomes the polite party. From 63cecd15358cffec60dcad363f6e4e184bf4a8ab Mon Sep 17 00:00:00 2001 From: David Baker Date: Fri, 11 Sep 2020 13:57:20 +0100 Subject: [PATCH 14/80] line break --- proposals/2746-reliable-voip.md | 1 + 1 file changed, 1 insertion(+) diff --git a/proposals/2746-reliable-voip.md b/proposals/2746-reliable-voip.md index 93eccf5cde..5dbc5e816d 100644 --- a/proposals/2746-reliable-voip.md +++ b/proposals/2746-reliable-voip.md @@ -24,6 +24,7 @@ If clients see events with `version` other than `0` or `1`, they should treat th This allows for the following use cases: * Placing a call to a specifc user in a room where other users are also present. * Placing a call to oneself. + The field should be added for all invites where the target is a specific user. Invites without an `invitee` field are defined to be intended for any member of the room other than the sender of the event. Clients should consider an incoming call if they see a non-expired invite event where the `invitee` field is either From c6f6ca13d66d1be876223150a5180c71df4c61c9 Mon Sep 17 00:00:00 2001 From: David Baker Date: Fri, 11 Sep 2020 14:02:02 +0100 Subject: [PATCH 15/80] workaround markdown being awful --- proposals/2746-reliable-voip.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/proposals/2746-reliable-voip.md b/proposals/2746-reliable-voip.md index 5dbc5e816d..36b627b284 100644 --- a/proposals/2746-reliable-voip.md +++ b/proposals/2746-reliable-voip.md @@ -37,8 +37,8 @@ incoming call invite, they still display the invite in the room and annotate tha ### Add `party_id` to all VoIP events Whenever a client first participates in a new call, it generates a `party_id` for itself to use for the duration of the call. This needs to be long enough that the chance of a collision between multiple devices -both generating an answer at the same time generating the same party ID is vanishingly small: 8 uppercase -+ lowercase alphanumeric characters is recommended. Parties in the call are identified by the tuple of +both generating an answer at the same time generating the same party ID is vanishingly small: 8 uppercase + +lowercase alphanumeric characters is recommended. Parties in the call are identified by the tuple of `(user_id, party_id)`. The client adds a `party_id` field containing this ID alongside the `user_id` field to all VoIP events it sends on the From e9fe3af2c976cc671962383bdcaf3183a4122efa Mon Sep 17 00:00:00 2001 From: David Baker Date: Fri, 11 Sep 2020 14:11:45 +0100 Subject: [PATCH 16/80] specify grammar for IDs --- proposals/2746-reliable-voip.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/proposals/2746-reliable-voip.md b/proposals/2746-reliable-voip.md index 36b627b284..a7118c1095 100644 --- a/proposals/2746-reliable-voip.md +++ b/proposals/2746-reliable-voip.md @@ -190,6 +190,10 @@ in the RTP payload. These are redundant: clients should continue to send them but must not require them to be present on events they receive. +### Specify exact grammar for VoIP IDs +`call_id`s and the newly introduced `party_id` are explicitly defined to be up to 32 characters +from the set of `A-Z` `a-z` `0-9` `.-_`. + ## Potential issues * The ability to call yourself makes the protocol a little more complex for clients to implement, and is somewhat of a special case. However, some of the necessary additions are also required for From 563dba52f5ee881290f72248ff2ef4693ed414fb Mon Sep 17 00:00:00 2001 From: David Baker Date: Fri, 11 Sep 2020 14:17:44 +0100 Subject: [PATCH 17/80] document why not mandate the same device IDs --- proposals/2746-reliable-voip.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/proposals/2746-reliable-voip.md b/proposals/2746-reliable-voip.md index a7118c1095..db4ca2596b 100644 --- a/proposals/2746-reliable-voip.md +++ b/proposals/2746-reliable-voip.md @@ -46,7 +46,10 @@ call. Clients use this to identify remote echo of their own events, since a user they can no longer ignore events from their own user. This field also identifies different answers sent by different clients to an invite, and matches `m.call.candidate` events to their respective answer/invite. -A client implementation may choose to use the device ID used in end-to-end cryptography for this purpose. +A client implementation may choose to use the device ID used in end-to-end cryptography for this purpose, +or it may choose, for example, to use a different one for each call to avoid lekaing information on which +devices were used in a call (in an unencrypted room) or if a single device (ie. access token were used to +send signalling for more than one call party. ### Introduce `m.call.select_answer` This event is sent by the caller's client once it has chosen an answer. Its From 070451eab3cbfe1b63e38b3412277d891299891f Mon Sep 17 00:00:00 2001 From: David Baker Date: Fri, 11 Sep 2020 14:25:00 +0100 Subject: [PATCH 18/80] rejection is about what the caller sees, not what's been sent --- proposals/2746-reliable-voip.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/proposals/2746-reliable-voip.md b/proposals/2746-reliable-voip.md index db4ca2596b..f128d5c507 100644 --- a/proposals/2746-reliable-voip.md +++ b/proposals/2746-reliable-voip.md @@ -76,8 +76,8 @@ Example: ### Introduce `m.call.reject` * If the `m.call.invite` event has `version` `1`, a client wishing to reject the call - sends an `m.call.reject` event. This rejects the call on all devices, but if another device - has already sent an accept, it disregards the reject and carries on. The reject has a + sends an `m.call.reject` event. This rejects the call on all devices, but if the calling + device sees an accept, it disregards the reject event and carries on. The reject has a `party_id` just like an answer, and the caller sends a `select_answer` for it just like an answer. If the other client that had already sent an answer sees the caller select the reject response instead of its answer, it ends the call. From 7bca76e413807f69faa40c74244b82d7d061b16a Mon Sep 17 00:00:00 2001 From: David Baker Date: Fri, 11 Sep 2020 14:44:17 +0100 Subject: [PATCH 19/80] Explain use of the age field --- proposals/2746-reliable-voip.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/proposals/2746-reliable-voip.md b/proposals/2746-reliable-voip.md index f128d5c507..01208bf896 100644 --- a/proposals/2746-reliable-voip.md +++ b/proposals/2746-reliable-voip.md @@ -158,9 +158,10 @@ party whose invite had the lexicographically greater call ID becomes the polite ### Add explicit recommendations for call event liveness. `m.call.invite` contains a `lifetime` field that indicates how long the offer is valid for. When -a client receives an invite, it should use the `age` field of the event plus the time since it -received the event from the homeserver to determine whether the invite is still valid. If the -invite is still valid *and will remain valid for long enough for the user to accept the call*, +a client receives an invite, it should use the event's `age` field in the sync response plus the +time since it received the event from the homeserver to determine whether the invite is still valid. +The use of the `age` field ensures that incorrect clocks on client devices don't break calls. +If the invite is still valid *and will remain valid for long enough for the user to accept the call*, it should signal an incoming call. The amount of time allowed for the user to accept the call may vary between clients, for example, it may be longer on a locked mobile device than on an unlocked desktop device. From e361fa9d5e13ab563314aefa6d3888271d2078b7 Mon Sep 17 00:00:00 2001 From: David Baker Date: Tue, 15 Sep 2020 10:19:09 +0100 Subject: [PATCH 20/80] Clarify party_id / user_id tuple in negotiate events --- proposals/2746-reliable-voip.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/2746-reliable-voip.md b/proposals/2746-reliable-voip.md index 01208bf896..f0507d7940 100644 --- a/proposals/2746-reliable-voip.md +++ b/proposals/2746-reliable-voip.md @@ -129,7 +129,7 @@ recommendation: https://www.w3.org/TR/webrtc/#hold-functionality If both the invite event and the accepted answer event have `version` equal to `1`, either party may send `m.call.negotiate` with an `sdp` field to offer new SDP to the other party. This event has `call_id` with the ID of the call and `party_id` equal to the client's party ID for that call. -The caller ignores any negotiate events with `party_id` not equal to the `party_id` of the +The caller ignores any negotiate events with `party_id` + `user_id` tuple not equal to that of the answer it accepted. Clients should use the `party_id` field to ignore the remote echo of their own negotiate events. From 722ee0d62a02c9366c2f760bd94869fe8563653c Mon Sep 17 00:00:00 2001 From: David Baker Date: Thu, 17 Sep 2020 16:28:48 +0100 Subject: [PATCH 21/80] Require end-of-candidates candidate --- proposals/2746-reliable-voip.md | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/proposals/2746-reliable-voip.md b/proposals/2746-reliable-voip.md index f0507d7940..263587d104 100644 --- a/proposals/2746-reliable-voip.md +++ b/proposals/2746-reliable-voip.md @@ -185,6 +185,16 @@ Clients should aim to send a small number of candidate events, with guidelines: invite or 500ms after sending the answer is suggested as starting point (since a delay is natural anyway after the invite whilst the client waits for the user to accept it). +### Mandate the end-of-candidates candidate +Mandate that an ICE candidate whose value is the empty string must be sent in an m.call.candidates +message to signal that no more ICE candidates will be sent. The WebRTC spec requires browsers to +generate such a candidate, however note that at time of writing, not all browsers do (Chrome does +not). The client should send any remaining candidates once candidate generation finishes, ignoring +timeouts above. + +This allows bridges to batch the candidates together when bridging to protocols that don't support +trickle ICE. + ### Add DTMF Add that Matrix clients can send DTMF as specified by WebRTC. The WebRTC standard as of August 2020 does not support receiving DTMF but a Matrix client can receive and interpret the DTMF sent From 8e76616d1a4a9847f514740428c678b0455bd3c4 Mon Sep 17 00:00:00 2001 From: David Baker Date: Thu, 17 Sep 2020 16:39:16 +0100 Subject: [PATCH 22/80] Add alternatives note for trickle ICE discovery mechansim and why it would be (more) awful --- proposals/2746-reliable-voip.md | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/proposals/2746-reliable-voip.md b/proposals/2746-reliable-voip.md index 263587d104..23666da9fc 100644 --- a/proposals/2746-reliable-voip.md +++ b/proposals/2746-reliable-voip.md @@ -234,7 +234,15 @@ from the set of `A-Z` `a-z` `0-9` `.-_`. callers could be identified by the lack of an `answer_id`. An explicit field on every event may be easier to comprehend, less error-prone and clearer in the backwards-compatibility scenario. * We could make `party_id`s more prescriptive, eg. the caller could always have a `party_id` of the - empty string, the word `caller` or equal to the `call_id`, which may make debugging simpler. + empty string, the word `caller` or equal to the `call_id`, which may make debugging simpler. + * To allow for bridging into protocols that don't support trickle ICE, this proposal requires that + clients send an empty candidate to signal the end of candidates. This means it will be up to bridges + to buffer the invite and edit the SDP to add the candidates once they arrive, adding complexity to + bridges. The alternative would be a discovery mechanism so clients could know whether a callee supports + trickle ICE before calling, and disable it if so. This would add complexity to every Matrix client as + well as having to assume that all current clients did not, disabling trickle ICE everywhere until clients + support the discovery mechanism. The mechanism would also have to be per-user which would make sense for + bridged users, but not where some of a users devices support trickle ICE and some do not. ## Security considerations * IP addresses remain in the room in candidates, as they did in the previous version of the spec. From 7e742d402c08dfbc53c4369e487bc16934fbaefd Mon Sep 17 00:00:00 2001 From: David Baker Date: Mon, 21 Sep 2020 12:06:09 +0100 Subject: [PATCH 23/80] add that chrome spits out `icegatheringstatechange` --- proposals/2746-reliable-voip.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/proposals/2746-reliable-voip.md b/proposals/2746-reliable-voip.md index 23666da9fc..07cccfc6db 100644 --- a/proposals/2746-reliable-voip.md +++ b/proposals/2746-reliable-voip.md @@ -189,8 +189,8 @@ Clients should aim to send a small number of candidate events, with guidelines: Mandate that an ICE candidate whose value is the empty string must be sent in an m.call.candidates message to signal that no more ICE candidates will be sent. The WebRTC spec requires browsers to generate such a candidate, however note that at time of writing, not all browsers do (Chrome does -not). The client should send any remaining candidates once candidate generation finishes, ignoring -timeouts above. +not, but does generate an `icegatheringstatechange` event). The client should send any remaining +candidates once candidate generation finishes, ignoring timeouts above. This allows bridges to batch the candidates together when bridging to protocols that don't support trickle ICE. From 8da4b7c4b971491743971c9b6c08c580958065b8 Mon Sep 17 00:00:00 2001 From: David Baker Date: Wed, 14 Oct 2020 15:59:52 +0100 Subject: [PATCH 24/80] clients must accept string version --- proposals/2746-reliable-voip.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/proposals/2746-reliable-voip.md b/proposals/2746-reliable-voip.md index 07cccfc6db..76fa7d6018 100644 --- a/proposals/2746-reliable-voip.md +++ b/proposals/2746-reliable-voip.md @@ -18,7 +18,8 @@ Matrix has basic support for signalling 1:1 WebRTC calls, but has a number of sh ### Change the `version` field in all VoIP events to `1` This will be used to determine whether determine whether devices support this new version of the protocol. If clients see events with `version` other than `0` or `1`, they should treat these the same as if they had -`version` == `1`. +`version` == `1`. In addition, clients must accept either a number or a string for the value of the `version` +field, in order to allow for namespaced versions in the future. ### Add `invitee` field to `m.call.invite` This allows for the following use cases: From 18200d0cd5209dd4474e136cd3466f6fae9966bf Mon Sep 17 00:00:00 2001 From: David Baker Date: Tue, 20 Oct 2020 15:10:20 +0100 Subject: [PATCH 25/80] Add text on unstable prefixng (and how/why we aren't) --- proposals/2746-reliable-voip.md | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/proposals/2746-reliable-voip.md b/proposals/2746-reliable-voip.md index 76fa7d6018..76e1864b2e 100644 --- a/proposals/2746-reliable-voip.md +++ b/proposals/2746-reliable-voip.md @@ -252,3 +252,11 @@ from the set of `A-Z` `a-z` `0-9` `.-_`. candidates) or redacting them afterwards (the volume of events sent during calls can already cause rate limiting issues and this would exacerbate this). * Clients must take care to not ring for any call, as per the 'alternatives' section. + +## Unstable prefix +Since VoIP events already have a 'version' field, we would ideally use a string, namespaced version during +development, but this field is defined to be a string in version 0. This MSC proposes changing the version +field to a string so that this namespacing can be used for future changes. Since there is no other easy way +to namespace events whilst in development and ensure interoperability, we have chosen not to use an unstable +prefix for this change, on the understanding that in future we will be able to use the string `version` field +for the unstable prefix. From 9e226012640d2a16c1f35985368187b8eb0fb9a9 Mon Sep 17 00:00:00 2001 From: David Baker Date: Tue, 20 Oct 2020 15:22:00 +0100 Subject: [PATCH 26/80] Specify what happens when someone leaves the room --- proposals/2746-reliable-voip.md | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/proposals/2746-reliable-voip.md b/proposals/2746-reliable-voip.md index 76e1864b2e..c9981be363 100644 --- a/proposals/2746-reliable-voip.md +++ b/proposals/2746-reliable-voip.md @@ -209,6 +209,15 @@ them to be present on events they receive. `call_id`s and the newly introduced `party_id` are explicitly defined to be up to 32 characters from the set of `A-Z` `a-z` `0-9` `.-_`. +### Specify behhaviour on room leave +If the client sees the party it is in a call with leave the room, the client should treat this +as a hangup event for any calls that are in progress. No specific requirement is given for the +situation where a client has sent an invite and the invitee leaves the room, but the client may +wish to treat it as a rejection if there are no more users in the room who could answer the call +(eg. the user is now alone or the `invitee` field was set on the invite). + +The same behaviour applies when a client is looking at historic calls. + ## Potential issues * The ability to call yourself makes the protocol a little more complex for clients to implement, and is somewhat of a special case. However, some of the necessary additions are also required for From a4466299e359a9a5425b63ab7d54f8852d5f455f Mon Sep 17 00:00:00 2001 From: David Baker Date: Thu, 22 Oct 2020 19:03:48 +0100 Subject: [PATCH 27/80] Rejig m.call.negotiate We need to know what is an offer and what is an answer. --- proposals/2746-reliable-voip.md | 20 ++++++++++++++------ 1 file changed, 14 insertions(+), 6 deletions(-) diff --git a/proposals/2746-reliable-voip.md b/proposals/2746-reliable-voip.md index c9981be363..2eebace3e5 100644 --- a/proposals/2746-reliable-voip.md +++ b/proposals/2746-reliable-voip.md @@ -128,7 +128,7 @@ call up/downgrading. Clients should implement & honour hold functionality as per recommendation: https://www.w3.org/TR/webrtc/#hold-functionality If both the invite event and the accepted answer event have `version` equal to `1`, either party may -send `m.call.negotiate` with an `sdp` field to offer new SDP to the other party. This event has +send `m.call.negotiate` with a `description` field to offer new SDP to the other party. This event has `call_id` with the ID of the call and `party_id` equal to the client's party ID for that call. The caller ignores any negotiate events with `party_id` + `user_id` tuple not equal to that of the answer it accepted. Clients should use the `party_id` field to ignore the remote echo of their @@ -137,6 +137,10 @@ own negotiate events. This has a `lifetime` field as in `m.call.invite`, after which the sender of the negotiate event should consider the negotiation failed (timed out) and the recipient should ignore it. +The `description` field is the same as the `offer` field in in `m.call.invite` and `answer` +field in `m.call.answer` and is an `RTCSessionDescriptionInit` object as per +https://www.w3.org/TR/webrtc/#dom-rtcsessiondescriptioninit. + Example: ``` { @@ -144,12 +148,20 @@ Example: "content": { "call_id": "12345", "party_id": "67890", - "sdp": "[some sdp]", "lifetime": 10000, + "description": { + "sdp": "[some sdp]", + "type": "offer", + }, } } ``` +This MSC also proposes clarifying the `m.call.invite` and `m.call.answer` events to state that +the `offer` and `answer` fields respectively are objects of type `RTCSessionDescriptionInit` +(and hence the `type` field, whilst redundant in these events, is included for ease of working +with the WebRTC API). + ### Designate one party as 'polite' In line with WebRTC perfect negotiation (https://w3c.github.io/webrtc-pc/#perfect-negotiation-example) we introduce rules to establish which party is polite. By default, the callee is the polite party. @@ -201,10 +213,6 @@ Add that Matrix clients can send DTMF as specified by WebRTC. The WebRTC standar 2020 does not support receiving DTMF but a Matrix client can receive and interpret the DTMF sent in the RTP payload. -### Deprecate `type` in `m.call.invite` and `m.call.answer` -These are redundant: clients should continue to send them but must not require -them to be present on events they receive. - ### Specify exact grammar for VoIP IDs `call_id`s and the newly introduced `party_id` are explicitly defined to be up to 32 characters from the set of `A-Z` `a-z` `0-9` `.-_`. From 1326a2261970c2eba80a03e5c037240f2de3d4b1 Mon Sep 17 00:00:00 2001 From: David Baker Date: Fri, 23 Oct 2020 11:43:06 +0100 Subject: [PATCH 28/80] Explain politness & glare in a simpler way (I hope) --- proposals/2746-reliable-voip.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/proposals/2746-reliable-voip.md b/proposals/2746-reliable-voip.md index 2eebace3e5..80c91b0a2b 100644 --- a/proposals/2746-reliable-voip.md +++ b/proposals/2746-reliable-voip.md @@ -164,10 +164,10 @@ with the WebRTC API). ### Designate one party as 'polite' In line with WebRTC perfect negotiation (https://w3c.github.io/webrtc-pc/#perfect-negotiation-example) -we introduce rules to establish which party is polite. By default, the callee is the polite party. -In a glare situation, if the client receives an invite whilst preparing to send one, it becomes the callee -and therefore becomes the polite party. If an invite is received after the client has sent one, the -party whose invite had the lexicographically greater call ID becomes the polite party. +we introduce rules to establish which party is polite. The callee is always the polite party. In a +glare situation, the politenes of a party is therefore determined by whether the inbound or outbound +call is used: if a client discards its outbound call in favour of an inbound call, it becomes the polite +party. ### Add explicit recommendations for call event liveness. `m.call.invite` contains a `lifetime` field that indicates how long the offer is valid for. When From a6698283680c7cf2a2275de9aa2993e26d7de98e Mon Sep 17 00:00:00 2001 From: David Baker Date: Mon, 26 Oct 2020 12:06:46 +0000 Subject: [PATCH 29/80] Add note on why we don't allow for ICE before an answer. --- proposals/2746-reliable-voip.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/proposals/2746-reliable-voip.md b/proposals/2746-reliable-voip.md index 80c91b0a2b..7791a3e557 100644 --- a/proposals/2746-reliable-voip.md +++ b/proposals/2746-reliable-voip.md @@ -236,6 +236,11 @@ The same behaviour applies when a client is looking at historic calls. had categorised it as a junk call and not the other would be confusing for the user. ## Alternatives + * This MSC does not allow for ICE negotiation before the user chooses to answer the call. This can + make call setup faster by allowing connectivity to be established whilst the call is ringing. This + is problematic with Matrix since any device or user could answer the call, so it is not known which + device is going to answer before the user chooses to answer. It would also leak information on which + of a user's devices were online. * We could define that the ID of a call is implcitly the event IDs of the invite event rather than having a specific `call_id` field. This would mean that a client would be unable to know the ID of a call before the remote echo of the invite came back, which could complicate implementations. From 0fa0770177c5ea1c9ddb74c8a51b2f9c5c0e3c83 Mon Sep 17 00:00:00 2001 From: David Baker Date: Mon, 26 Oct 2020 19:28:10 +0000 Subject: [PATCH 30/80] Define WebRTC track & stream configs for calls --- proposals/2746-reliable-voip.md | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/proposals/2746-reliable-voip.md b/proposals/2746-reliable-voip.md index 7791a3e557..6b85910c08 100644 --- a/proposals/2746-reliable-voip.md +++ b/proposals/2746-reliable-voip.md @@ -21,6 +21,19 @@ If clients see events with `version` other than `0` or `1`, they should treat th `version` == `1`. In addition, clients must accept either a number or a string for the value of the `version` field, in order to allow for namespaced versions in the future. +### Define the configurations of WebRTC streams and tracks in each call type +We define that: + * A voice call has at least one track of kind 'audio' in the first stream + * A video call has at least one track of kind 'video' and at least one track of kind 'audio' in the first stream +Clients implementing this specification use the first stream and will ignore any streamless tracks. Note that +in the Javascript WebRTC API, this means `addTrack()` must be passed two parameters: a track and a stream, +not just a track, and in a video call the stream must be the same for both audio and video track. + +A client may send other streams and tracks but the behaviour of the other party with respect to presenting +such streams and tracks is undefined. + +This follows the existing known implementations of v0 VoIP. + ### Add `invitee` field to `m.call.invite` This allows for the following use cases: * Placing a call to a specifc user in a room where other users are also present. From 599ad3c18d5209cfc5e6cfe55baf3b9bc87e27e7 Mon Sep 17 00:00:00 2001 From: David Baker Date: Thu, 3 Dec 2020 11:36:18 +0000 Subject: [PATCH 31/80] select_answer was missing a version --- proposals/2746-reliable-voip.md | 1 + 1 file changed, 1 insertion(+) diff --git a/proposals/2746-reliable-voip.md b/proposals/2746-reliable-voip.md index 6b85910c08..78dfb56818 100644 --- a/proposals/2746-reliable-voip.md +++ b/proposals/2746-reliable-voip.md @@ -80,6 +80,7 @@ Example: { "type": "m.call.select_answer", "content": { + "version": 1, "call_id": "12345", "party_id": "67890", "selected_party_id": "111213", From 6478f9dc36cb3803415a33fcbbc4622cdcb23f14 Mon Sep 17 00:00:00 2001 From: David Baker Date: Thu, 3 Dec 2020 17:30:35 +0000 Subject: [PATCH 32/80] Fix old type Co-authored-by: Matthew Hodgson --- proposals/2746-reliable-voip.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/2746-reliable-voip.md b/proposals/2746-reliable-voip.md index 78dfb56818..6fc1083363 100644 --- a/proposals/2746-reliable-voip.md +++ b/proposals/2746-reliable-voip.md @@ -291,7 +291,7 @@ The same behaviour applies when a client is looking at historic calls. ## Unstable prefix Since VoIP events already have a 'version' field, we would ideally use a string, namespaced version during -development, but this field is defined to be a string in version 0. This MSC proposes changing the version +development, but this field is defined to be an int in version 0. This MSC proposes changing the version field to a string so that this namespacing can be used for future changes. Since there is no other easy way to namespace events whilst in development and ensure interoperability, we have chosen not to use an unstable prefix for this change, on the understanding that in future we will be able to use the string `version` field From b62f842c0535594ae9efe6fb6db1322f84ab709b Mon Sep 17 00:00:00 2001 From: David Baker Date: Mon, 15 Feb 2021 14:55:11 +0000 Subject: [PATCH 33/80] Clarfy that whatever codecs webrtc say is what goes --- proposals/2746-reliable-voip.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/proposals/2746-reliable-voip.md b/proposals/2746-reliable-voip.md index 6fc1083363..5242f8b8aa 100644 --- a/proposals/2746-reliable-voip.md +++ b/proposals/2746-reliable-voip.md @@ -240,6 +240,13 @@ wish to treat it as a rejection if there are no more users in the room who could The same behaviour applies when a client is looking at historic calls. +### Clarify that supported codecs should follow the WebRTC spec +The Matrix spec does not mandate particular audio or video codecs, but instead defers to the +WebRTC spec. A compliant matrix VoIP client will behave in the same wau as a supported 'browser' +in terms of what codecs it supports and what variants thereof. The latest WebRTC specification +applies, so clients should keeps up to date with new versions of the WebRTC specification whether +or not there have been any changes to the Matrix spec. + ## Potential issues * The ability to call yourself makes the protocol a little more complex for clients to implement, and is somewhat of a special case. However, some of the necessary additions are also required for From 9156c80900270fee2876fdf6cb2a933a4ef203b9 Mon Sep 17 00:00:00 2001 From: David Baker Date: Mon, 15 Feb 2021 15:00:50 +0000 Subject: [PATCH 34/80] Typos Co-authored-by: Brendan Abolivier --- proposals/2746-reliable-voip.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/proposals/2746-reliable-voip.md b/proposals/2746-reliable-voip.md index 5242f8b8aa..8775117ea5 100644 --- a/proposals/2746-reliable-voip.md +++ b/proposals/2746-reliable-voip.md @@ -242,9 +242,9 @@ The same behaviour applies when a client is looking at historic calls. ### Clarify that supported codecs should follow the WebRTC spec The Matrix spec does not mandate particular audio or video codecs, but instead defers to the -WebRTC spec. A compliant matrix VoIP client will behave in the same wau as a supported 'browser' +WebRTC spec. A compliant matrix VoIP client will behave in the same way as a supported 'browser' in terms of what codecs it supports and what variants thereof. The latest WebRTC specification -applies, so clients should keeps up to date with new versions of the WebRTC specification whether +applies, so clients should keep up to date with new versions of the WebRTC specification whether or not there have been any changes to the Matrix spec. ## Potential issues From 834bc3bce93ecc89f1df75936fb96c8cbc4a1a3b Mon Sep 17 00:00:00 2001 From: David Baker Date: Thu, 4 Mar 2021 15:12:16 +0000 Subject: [PATCH 35/80] Typo Co-authored-by: DeepBlueV7.X --- proposals/2746-reliable-voip.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/2746-reliable-voip.md b/proposals/2746-reliable-voip.md index 8775117ea5..d679574c58 100644 --- a/proposals/2746-reliable-voip.md +++ b/proposals/2746-reliable-voip.md @@ -231,7 +231,7 @@ in the RTP payload. `call_id`s and the newly introduced `party_id` are explicitly defined to be up to 32 characters from the set of `A-Z` `a-z` `0-9` `.-_`. -### Specify behhaviour on room leave +### Specify behaviour on room leave If the client sees the party it is in a call with leave the room, the client should treat this as a hangup event for any calls that are in progress. No specific requirement is given for the situation where a client has sent an invite and the invitee leaves the room, but the client may From ec2c7fe9c89db2661625e272be1f0e2c29b7af56 Mon Sep 17 00:00:00 2001 From: David Baker Date: Thu, 4 Mar 2021 18:23:23 +0000 Subject: [PATCH 36/80] only allow the number zero as numeric version --- proposals/2746-reliable-voip.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/2746-reliable-voip.md b/proposals/2746-reliable-voip.md index d679574c58..3cc60210af 100644 --- a/proposals/2746-reliable-voip.md +++ b/proposals/2746-reliable-voip.md @@ -18,7 +18,7 @@ Matrix has basic support for signalling 1:1 WebRTC calls, but has a number of sh ### Change the `version` field in all VoIP events to `1` This will be used to determine whether determine whether devices support this new version of the protocol. If clients see events with `version` other than `0` or `1`, they should treat these the same as if they had -`version` == `1`. In addition, clients must accept either a number or a string for the value of the `version` +`version` == `1`. In addition, clients must accept either the number `0` or a string for the value of the `version` field, in order to allow for namespaced versions in the future. ### Define the configurations of WebRTC streams and tracks in each call type From a572eb83fd35d79591eecefe38f57e92ec880615 Mon Sep 17 00:00:00 2001 From: Matthew Hodgson Date: Tue, 6 Apr 2021 22:23:36 +0100 Subject: [PATCH 37/80] Update 2746-reliable-voip.md clarify tense --- proposals/2746-reliable-voip.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/2746-reliable-voip.md b/proposals/2746-reliable-voip.md index 3cc60210af..7599f09d91 100644 --- a/proposals/2746-reliable-voip.md +++ b/proposals/2746-reliable-voip.md @@ -1,6 +1,6 @@ # MSC2746: Improved Signalling for 1:1 VoIP -Matrix has basic support for signalling 1:1 WebRTC calls, but has a number of shortcomings: +Historically, Matrix has basic support for signalling 1:1 WebRTC calls which suffer a number of shortcomings: * If several devices try to answer the same call, there is no way for them to determine clearly that the caller has set up the call with a different device, and no way for the caller to From 65920236d1b279a7935bae2000dfd205c08c7b51 Mon Sep 17 00:00:00 2001 From: David Baker Date: Wed, 26 May 2021 13:30:25 +0100 Subject: [PATCH 38/80] Add user_busy hangup / reject reason --- proposals/2746-reliable-voip.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/proposals/2746-reliable-voip.md b/proposals/2746-reliable-voip.md index 7599f09d91..f9b007d935 100644 --- a/proposals/2746-reliable-voip.md +++ b/proposals/2746-reliable-voip.md @@ -133,6 +133,9 @@ The client may: `user_hangup`. * `user_media_failed`: The client was unable to start capturing media in such a way as it is unable to continue the call. + * `user_busy`: The user is busy. Note that this exists primarily for bridging to other networks such + as the PSTN. A Matrix client that receives a call whilst already in a call would not generally reject + the new call unless the user had specifically chosen to do so. * `unknown_error`: Some other failure occurred that meant the client was unable to continue the call rather than the user choosing to end it. From 996adabfc2b79430f08c30e65b9d1799e5109d12 Mon Sep 17 00:00:00 2001 From: David Baker Date: Mon, 21 Jun 2021 17:53:32 +0100 Subject: [PATCH 39/80] Add capability for DTMF --- proposals/2746-reliable-voip.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/proposals/2746-reliable-voip.md b/proposals/2746-reliable-voip.md index f9b007d935..17bc57568d 100644 --- a/proposals/2746-reliable-voip.md +++ b/proposals/2746-reliable-voip.md @@ -230,6 +230,11 @@ Add that Matrix clients can send DTMF as specified by WebRTC. The WebRTC standar 2020 does not support receiving DTMF but a Matrix client can receive and interpret the DTMF sent in the RTP payload. +We also add a capability to the `capabilities` section of invites and answers (detailed in +[MSC2747](https://github.com/matrix-org/matrix-doc/pull/2747) called `m.call.dtmf`. Clients +should only display UI for sending DTMF during a call if the other party advertises this +capability (boolean value `true`). + ### Specify exact grammar for VoIP IDs `call_id`s and the newly introduced `party_id` are explicitly defined to be up to 32 characters from the set of `A-Z` `a-z` `0-9` `.-_`. From 91428c83688ef32cbb7a2ff23f27edb2ae4c33e6 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C5=A0imon=20Brandner?= Date: Mon, 11 Jul 2022 11:23:24 +0200 Subject: [PATCH 40/80] Be clear about versions MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Šimon Brandner --- proposals/2746-reliable-voip.md | 17 ++++++++++------- 1 file changed, 10 insertions(+), 7 deletions(-) diff --git a/proposals/2746-reliable-voip.md b/proposals/2746-reliable-voip.md index 17bc57568d..e680e32210 100644 --- a/proposals/2746-reliable-voip.md +++ b/proposals/2746-reliable-voip.md @@ -15,10 +15,10 @@ Historically, Matrix has basic support for signalling 1:1 WebRTC calls which suf on a different device to be hung up. ## Proposal -### Change the `version` field in all VoIP events to `1` +### Change the `version` field in all VoIP events to `"1"` This will be used to determine whether determine whether devices support this new version of the protocol. -If clients see events with `version` other than `0` or `1`, they should treat these the same as if they had -`version` == `1`. In addition, clients must accept either the number `0` or a string for the value of the `version` +If clients see events with `version` other than `0` or `"1"`, they should treat these the same as if they had +`version` == `"1"`. In addition, clients must accept either the number `0` or a string for the value of the `version` field, in order to allow for namespaced versions in the future. ### Define the configurations of WebRTC streams and tracks in each call type @@ -80,7 +80,7 @@ Example: { "type": "m.call.select_answer", "content": { - "version": 1, + "version": "1", "call_id": "12345", "party_id": "67890", "selected_party_id": "111213", @@ -90,7 +90,7 @@ Example: ### Introduce `m.call.reject` - * If the `m.call.invite` event has `version` `1`, a client wishing to reject the call + * If the `m.call.invite` event has `version` `"1"`, a client wishing to reject the call sends an `m.call.reject` event. This rejects the call on all devices, but if the calling device sees an accept, it disregards the reject event and carries on. The reject has a `party_id` just like an answer, and the caller sends a `select_answer` for it just like an @@ -103,7 +103,7 @@ Example: { "type": "m.call.reject", "content" : { - "version": 1, + "version": "1", "call_id": "12345", "party_id": "67890", } @@ -144,7 +144,7 @@ This introduces SDP negotiation semantics for media pause, hold/resume, ICE rest call up/downgrading. Clients should implement & honour hold functionality as per WebRTC's recommendation: https://www.w3.org/TR/webrtc/#hold-functionality -If both the invite event and the accepted answer event have `version` equal to `1`, either party may +If both the invite event and the accepted answer event have `version` equal to `"1"`, either party may send `m.call.negotiate` with a `description` field to offer new SDP to the other party. This event has `call_id` with the ID of the call and `party_id` equal to the client's party ID for that call. The caller ignores any negotiate events with `party_id` + `user_id` tuple not equal to that of the @@ -311,3 +311,6 @@ field to a string so that this namespacing can be used for future changes. Since to namespace events whilst in development and ensure interoperability, we have chosen not to use an unstable prefix for this change, on the understanding that in future we will be able to use the string `version` field for the unstable prefix. + +For backwards compatibility, strongly typed implementations should allow for +`version` to be both integer and string. From e42dd416b4b245d7491e7b8b77d9581c4ec5a349 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C5=A0imon=20Brandner?= Date: Mon, 11 Jul 2022 11:29:21 +0200 Subject: [PATCH 41/80] Clarify clients must respond to `m.call.negotiate` MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Šimon Brandner --- proposals/2746-reliable-voip.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/proposals/2746-reliable-voip.md b/proposals/2746-reliable-voip.md index e680e32210..e8237fc974 100644 --- a/proposals/2746-reliable-voip.md +++ b/proposals/2746-reliable-voip.md @@ -166,7 +166,7 @@ Example: "call_id": "12345", "party_id": "67890", "lifetime": 10000, - "description": { + "description": { "sdp": "[some sdp]", "type": "offer", }, @@ -174,6 +174,9 @@ Example: } ``` +Once an `m.call.negotiate` event is received, the client must respond with an +event of the same type with type with the SDP answer (`type: "answer"`). + This MSC also proposes clarifying the `m.call.invite` and `m.call.answer` events to state that the `offer` and `answer` fields respectively are objects of type `RTCSessionDescriptionInit` (and hence the `type` field, whilst redundant in these events, is included for ease of working From 29485c4e6fe02118317b2739c50ddb7a4daf2b36 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C5=A0imon=20Brandner?= Date: Mon, 11 Jul 2022 11:49:04 +0200 Subject: [PATCH 42/80] Give `m.call.negotiate` a version MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Šimon Brandner --- proposals/2746-reliable-voip.md | 1 + 1 file changed, 1 insertion(+) diff --git a/proposals/2746-reliable-voip.md b/proposals/2746-reliable-voip.md index e8237fc974..0dc3c4c772 100644 --- a/proposals/2746-reliable-voip.md +++ b/proposals/2746-reliable-voip.md @@ -163,6 +163,7 @@ Example: { "type": "m.call.negotiate", "content": { + "version": "1", "call_id": "12345", "party_id": "67890", "lifetime": 10000, From f69ae72f8ec37a7b7af1fcaa854b37cfb6874218 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C5=A0imon=20Brandner?= Date: Mon, 11 Jul 2022 11:50:12 +0200 Subject: [PATCH 43/80] Remove repeated words MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Kévin Commaille <76261501+zecakeh@users.noreply.github.com> --- proposals/2746-reliable-voip.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/2746-reliable-voip.md b/proposals/2746-reliable-voip.md index 0dc3c4c772..56689ee3cc 100644 --- a/proposals/2746-reliable-voip.md +++ b/proposals/2746-reliable-voip.md @@ -16,7 +16,7 @@ Historically, Matrix has basic support for signalling 1:1 WebRTC calls which suf ## Proposal ### Change the `version` field in all VoIP events to `"1"` -This will be used to determine whether determine whether devices support this new version of the protocol. +This will be used to determine whether devices support this new version of the protocol. If clients see events with `version` other than `0` or `"1"`, they should treat these the same as if they had `version` == `"1"`. In addition, clients must accept either the number `0` or a string for the value of the `version` field, in order to allow for namespaced versions in the future. From 51e02b2667ca7edb434948b83561bf89c2ce574d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C5=A0imon=20Brandner?= Date: Mon, 11 Jul 2022 14:21:48 +0200 Subject: [PATCH 44/80] Be clearer about types MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Šimon Brandner --- proposals/2746-reliable-voip.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/2746-reliable-voip.md b/proposals/2746-reliable-voip.md index 56689ee3cc..d18553e7ec 100644 --- a/proposals/2746-reliable-voip.md +++ b/proposals/2746-reliable-voip.md @@ -317,4 +317,4 @@ prefix for this change, on the understanding that in future we will be able to u for the unstable prefix. For backwards compatibility, strongly typed implementations should allow for -`version` to be both integer and string. +`version` to either be a string or the integer `0`. From 312ffe56b5de561f122d79b85ad052202dd019ef Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C5=A0imon=20Brandner?= Date: Mon, 11 Jul 2022 15:42:26 +0200 Subject: [PATCH 45/80] Avoid defining call types MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Šimon Brandner --- proposals/2746-reliable-voip.md | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/proposals/2746-reliable-voip.md b/proposals/2746-reliable-voip.md index d18553e7ec..d986dc03f1 100644 --- a/proposals/2746-reliable-voip.md +++ b/proposals/2746-reliable-voip.md @@ -21,10 +21,12 @@ If clients see events with `version` other than `0` or `"1"`, they should treat `version` == `"1"`. In addition, clients must accept either the number `0` or a string for the value of the `version` field, in order to allow for namespaced versions in the future. -### Define the configurations of WebRTC streams and tracks in each call type -We define that: - * A voice call has at least one track of kind 'audio' in the first stream - * A video call has at least one track of kind 'video' and at least one track of kind 'audio' in the first stream +### Define the configurations of WebRTC streams and tracks + +Clients are expected to send one stream with one track of kind `audio` (creating +voice call). They can optionally send a second track in the same stream of kind +`video` (creating a video call). + Clients implementing this specification use the first stream and will ignore any streamless tracks. Note that in the Javascript WebRTC API, this means `addTrack()` must be passed two parameters: a track and a stream, not just a track, and in a video call the stream must be the same for both audio and video track. From 4617af5befa8b26d4e9daf0bc853006574fbf91c Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C5=A0imon=20Brandner?= Date: Mon, 11 Jul 2022 15:57:20 +0200 Subject: [PATCH 46/80] Specify minimal `lifetime` MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Šimon Brandner --- proposals/2746-reliable-voip.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/proposals/2746-reliable-voip.md b/proposals/2746-reliable-voip.md index d986dc03f1..9ef87542c5 100644 --- a/proposals/2746-reliable-voip.md +++ b/proposals/2746-reliable-voip.md @@ -210,6 +210,9 @@ indicate the call has been hung up, rejected, or answered elsewhere, the client If on startup, after processing locally stored events, the client determines that there is an invite that is still valid, it should still signal it but only after it has completed a sync from the homeserver. +The minimal recommended lifetime is 90 seconds - this should give the user +enough time to actually pick up the call. + ### Introduce recommendations for batching of ICE candidates Clients should aim to send a small number of candidate events, with guidelines: * ICE candidates which can be discovered immediately or almost immediately in the invite/answer From 289fb3f6aa0df4403cfff0b6f6bf422944732d06 Mon Sep 17 00:00:00 2001 From: David Baker Date: Tue, 8 Nov 2022 16:06:02 +0000 Subject: [PATCH 47/80] Use MSC1597 grammar for call / party IDs. --- proposals/2746-reliable-voip.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/proposals/2746-reliable-voip.md b/proposals/2746-reliable-voip.md index 9ef87542c5..07968ae785 100644 --- a/proposals/2746-reliable-voip.md +++ b/proposals/2746-reliable-voip.md @@ -245,8 +245,8 @@ should only display UI for sending DTMF during a call if the other party adverti capability (boolean value `true`). ### Specify exact grammar for VoIP IDs -`call_id`s and the newly introduced `party_id` are explicitly defined to be up to 32 characters -from the set of `A-Z` `a-z` `0-9` `.-_`. +`call_id`s and the newly introduced `party_id` are explicitly defined to be in the grammar of +'opaque IDs' from [MSC1597](https://github.com/matrix-org/matrix-spec-proposals/blob/rav/proposals/id_grammar/proposals/1597-id-grammar.md#opaque-ids). ### Specify behaviour on room leave If the client sees the party it is in a call with leave the room, the client should treat this From 1392eaef62dab5ce458b3385e90c9540f4bb1f53 Mon Sep 17 00:00:00 2001 From: David Baker Date: Mon, 6 Feb 2023 16:54:35 +0000 Subject: [PATCH 48/80] Add more rationale around voip event version --- proposals/2746-reliable-voip.md | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/proposals/2746-reliable-voip.md b/proposals/2746-reliable-voip.md index 07968ae785..849188d32c 100644 --- a/proposals/2746-reliable-voip.md +++ b/proposals/2746-reliable-voip.md @@ -16,10 +16,15 @@ Historically, Matrix has basic support for signalling 1:1 WebRTC calls which suf ## Proposal ### Change the `version` field in all VoIP events to `"1"` -This will be used to determine whether devices support this new version of the protocol. -If clients see events with `version` other than `0` or `"1"`, they should treat these the same as if they had -`version` == `"1"`. In addition, clients must accept either the number `0` or a string for the value of the `version` -field, in order to allow for namespaced versions in the future. +This will be used to determine whether devices support this new version of the protocol. For example, +clients can use this field to know whether to expect an `m.call.select_answer` event from their +opponent. If clients see events with `version` other than `0` or `"1"`, they should treat these the +same as if they had `version` == `"1"`. In addition, clients must accept either the number `0` or a +string for the value of the `version` field, in order to allow for namespaced versions in the future. + +Note that this implies any and all future versions of VoIP events should be backwards-compatible. +If it does become necessary to introduce a non backwards-compatible VoIP spec, the intention would +be for it to simply use a separate set of event types. ### Define the configurations of WebRTC streams and tracks From 46bfbde0da17cfca568a5137b0bc98742b1e9534 Mon Sep 17 00:00:00 2001 From: David Baker Date: Mon, 6 Feb 2023 17:02:42 +0000 Subject: [PATCH 49/80] Change advice for calls in public rooms. --- proposals/2746-reliable-voip.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/proposals/2746-reliable-voip.md b/proposals/2746-reliable-voip.md index 849188d32c..6f99ba1516 100644 --- a/proposals/2746-reliable-voip.md +++ b/proposals/2746-reliable-voip.md @@ -50,10 +50,10 @@ The field should be added for all invites where the target is a specific user. I field are defined to be intended for any member of the room other than the sender of the event. Clients should consider an incoming call if they see a non-expired invite event where the `invitee` field is either absent or equal to their user's Matrix ID, however they should evaluate whether or not to ring based on their -user's trust relationship with the caller, eg. ignoring call invites from users in public rooms that they have -no other connection with. As a starting point, it is suggested that clients ring for any call invite from a user -that they have a direct message room with. It is strongly recommended that when clients do not ring for an -incoming call invite, they still display the invite in the room and annotate that it was ignored. +user's trust relationship with the callers and/or where the call was placed. As a starting point, it is +suggested that clients ignore call invites from users in public rooms. It is strongly recommended that +when clients do not ring for an incoming call invite, they still display the invite in the room and +annotate that it was ignored. ### Add `party_id` to all VoIP events Whenever a client first participates in a new call, it generates a `party_id` for itself to use for the From 9dcda02facf8a529431227c09b10f8095d883255 Mon Sep 17 00:00:00 2001 From: David Baker Date: Mon, 6 Feb 2023 17:03:36 +0000 Subject: [PATCH 50/80] Typo Co-authored-by: Hubert Chathi --- proposals/2746-reliable-voip.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/2746-reliable-voip.md b/proposals/2746-reliable-voip.md index 6f99ba1516..ecc3b8f239 100644 --- a/proposals/2746-reliable-voip.md +++ b/proposals/2746-reliable-voip.md @@ -68,7 +68,7 @@ they can no longer ignore events from their own user. This field also identifies by different clients to an invite, and matches `m.call.candidate` events to their respective answer/invite. A client implementation may choose to use the device ID used in end-to-end cryptography for this purpose, -or it may choose, for example, to use a different one for each call to avoid lekaing information on which +or it may choose, for example, to use a different one for each call to avoid leaking information on which devices were used in a call (in an unencrypted room) or if a single device (ie. access token were used to send signalling for more than one call party. From 6dc85a8f015f1f0e50300a5188f0f2539cbe6a30 Mon Sep 17 00:00:00 2001 From: David Baker Date: Mon, 6 Feb 2023 17:06:51 +0000 Subject: [PATCH 51/80] Clarify reject/hangup sending --- proposals/2746-reliable-voip.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/proposals/2746-reliable-voip.md b/proposals/2746-reliable-voip.md index 6f99ba1516..01ab167520 100644 --- a/proposals/2746-reliable-voip.md +++ b/proposals/2746-reliable-voip.md @@ -103,7 +103,8 @@ Example: `party_id` just like an answer, and the caller sends a `select_answer` for it just like an answer. If the other client that had already sent an answer sees the caller select the reject response instead of its answer, it ends the call. - * If the `m.call.invite` event has `version` `0`, the callee sends an `m.call.hangup` event before. + * If the `m.call.invite` event has `version` `0`, the callee sends an `m.call.hangup` event + as in spec version 0. Example: ``` From 2fd97c91be89dbc5386c88c5828ee79fbf82a2b0 Mon Sep 17 00:00:00 2001 From: David Baker Date: Mon, 6 Feb 2023 17:10:08 +0000 Subject: [PATCH 52/80] Clarify hangup reason backwards compat --- proposals/2746-reliable-voip.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/proposals/2746-reliable-voip.md b/proposals/2746-reliable-voip.md index 312bc400cc..b616d63cb0 100644 --- a/proposals/2746-reliable-voip.md +++ b/proposals/2746-reliable-voip.md @@ -137,8 +137,8 @@ The client may: an ICE renegotiation, a client should be sure to send `ice_timeout` rather than `ice_failed` if media had previously been received successfully, even if the ICE renegotiation itself failed. * `user_hangup`: Clients must now send this code when the user chooses to end the call, although - for backwards compatability, a clients should treat an absence of the `reason` field as - `user_hangup`. + for backwards compatability with version 0, a clients should treat an absence of the `reason` + field as `user_hangup`. * `user_media_failed`: The client was unable to start capturing media in such a way as it is unable to continue the call. * `user_busy`: The user is busy. Note that this exists primarily for bridging to other networks such From 04eaee28c6c0bcb18500a76a901bfe1d5b6c313f Mon Sep 17 00:00:00 2001 From: David Baker Date: Mon, 6 Feb 2023 17:28:22 +0000 Subject: [PATCH 53/80] Clarify party ID --- proposals/2746-reliable-voip.md | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/proposals/2746-reliable-voip.md b/proposals/2746-reliable-voip.md index b616d63cb0..f92c95e73d 100644 --- a/proposals/2746-reliable-voip.md +++ b/proposals/2746-reliable-voip.md @@ -62,10 +62,11 @@ both generating an answer at the same time generating the same party ID is vanis lowercase alphanumeric characters is recommended. Parties in the call are identified by the tuple of `(user_id, party_id)`. -The client adds a `party_id` field containing this ID alongside the `user_id` field to all VoIP events it sends on the -call. Clients use this to identify remote echo of their own events, since a user may now call themselves, -they can no longer ignore events from their own user. This field also identifies different answers sent -by different clients to an invite, and matches `m.call.candidate` events to their respective answer/invite. +The client adds a `party_id` field containing this ID alongside the `user_id` field to all VoIP events +it sends on the call, including `m.call.inbite`. . Clients use this to identify remote echo of their own +events, since a user may now call themselves, they can no longer ignore events from their own user. This +field also identifies different answers sent by different clients to an invite, and matches `m.call.candidate` +events to their respective answer/invite. A client implementation may choose to use the device ID used in end-to-end cryptography for this purpose, or it may choose, for example, to use a different one for each call to avoid leaking information on which From c52a845b8ee53aaee276ff1250e523f58dc18359 Mon Sep 17 00:00:00 2001 From: David Baker Date: Mon, 6 Feb 2023 17:32:29 +0000 Subject: [PATCH 54/80] There is no sender field. Also fix typos in previous commit. --- proposals/2746-reliable-voip.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/proposals/2746-reliable-voip.md b/proposals/2746-reliable-voip.md index f92c95e73d..d5fdcefc8a 100644 --- a/proposals/2746-reliable-voip.md +++ b/proposals/2746-reliable-voip.md @@ -62,8 +62,8 @@ both generating an answer at the same time generating the same party ID is vanis lowercase alphanumeric characters is recommended. Parties in the call are identified by the tuple of `(user_id, party_id)`. -The client adds a `party_id` field containing this ID alongside the `user_id` field to all VoIP events -it sends on the call, including `m.call.inbite`. . Clients use this to identify remote echo of their own +The client adds a `party_id` field containing this ID to the top-level of the content of all VoIP events +it sends on the call, including `m.call.invite`. Clients use this to identify remote echo of their own events, since a user may now call themselves, they can no longer ignore events from their own user. This field also identifies different answers sent by different clients to an invite, and matches `m.call.candidate` events to their respective answer/invite. From 6dcf65c1fb912cabb623219f0d213eb6591096d7 Mon Sep 17 00:00:00 2001 From: David Baker Date: Mon, 6 Feb 2023 17:33:58 +0000 Subject: [PATCH 55/80] Require ignoring negotiates not matching party ID Co-authored-by: Hubert Chathi --- proposals/2746-reliable-voip.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/2746-reliable-voip.md b/proposals/2746-reliable-voip.md index d5fdcefc8a..f77c8a75fa 100644 --- a/proposals/2746-reliable-voip.md +++ b/proposals/2746-reliable-voip.md @@ -157,7 +157,7 @@ If both the invite event and the accepted answer event have `version` equal to ` send `m.call.negotiate` with a `description` field to offer new SDP to the other party. This event has `call_id` with the ID of the call and `party_id` equal to the client's party ID for that call. The caller ignores any negotiate events with `party_id` + `user_id` tuple not equal to that of the -answer it accepted. Clients should use the `party_id` field to ignore the remote echo of their +answer it accepted and the callee ignores any negotiate events with `party_id` + `user_id` tuple not equal to that of the caller. Clients should use the `party_id` field to ignore the remote echo of their own negotiate events. This has a `lifetime` field as in `m.call.invite`, after which the sender of the negotiate event From 859cf6d436e7a414b8289f48b1bb07a8dd8fb0c4 Mon Sep 17 00:00:00 2001 From: David Baker Date: Mon, 6 Feb 2023 17:37:48 +0000 Subject: [PATCH 56/80] Word negotiate events better --- proposals/2746-reliable-voip.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/proposals/2746-reliable-voip.md b/proposals/2746-reliable-voip.md index f77c8a75fa..bae0e07305 100644 --- a/proposals/2746-reliable-voip.md +++ b/proposals/2746-reliable-voip.md @@ -184,8 +184,8 @@ Example: } ``` -Once an `m.call.negotiate` event is received, the client must respond with an -event of the same type with type with the SDP answer (`type: "answer"`). +Once an `m.call.negotiate` event is received, the client must respond with another `m.call.negotiate` +event, with the SDP answer (with `"type"`: `"answer"`) in the description property. This MSC also proposes clarifying the `m.call.invite` and `m.call.answer` events to state that the `offer` and `answer` fields respectively are objects of type `RTCSessionDescriptionInit` From 340e769e27ef1b032c4e487c6c9eaca6ffed5801 Mon Sep 17 00:00:00 2001 From: David Baker Date: Mon, 6 Feb 2023 17:40:55 +0000 Subject: [PATCH 57/80] Don't forget the txn ID is returned by the send call. --- proposals/2746-reliable-voip.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/proposals/2746-reliable-voip.md b/proposals/2746-reliable-voip.md index bae0e07305..813dc1b30c 100644 --- a/proposals/2746-reliable-voip.md +++ b/proposals/2746-reliable-voip.md @@ -286,10 +286,10 @@ or not there have been any changes to the Matrix spec. is problematic with Matrix since any device or user could answer the call, so it is not known which device is going to answer before the user chooses to answer. It would also leak information on which of a user's devices were online. - * We could define that the ID of a call is implcitly the event IDs of the invite event rather than + * We could define that the ID of a call is implcitly the event ID of the invite event rather than having a specific `call_id` field. This would mean that a client would be unable to know the ID of - a call before the remote echo of the invite came back, which could complicate implementations. - There is probably no compelling reason to change this. + a call before the it received the response from sending the invite event, which could complicate + implementations. There is probably no compelling reason to change this. * `m.call.select_answer` was chosen such that its name reflect the intention of the event. `m.call.ack` is more succinct and mirrors SIP, but this MSC opts for the more descriptive name. * This MSC elects to allow invites without an `invitee` field to mean a call for anyone in the room. From 8be57edea56853d40ce8defcfffefbfd887cb3d0 Mon Sep 17 00:00:00 2001 From: David Baker Date: Mon, 6 Feb 2023 17:44:10 +0000 Subject: [PATCH 58/80] Enumerate all the current VoIP events in 'version' section. --- proposals/2746-reliable-voip.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/proposals/2746-reliable-voip.md b/proposals/2746-reliable-voip.md index 813dc1b30c..ab8d7e292a 100644 --- a/proposals/2746-reliable-voip.md +++ b/proposals/2746-reliable-voip.md @@ -16,6 +16,13 @@ Historically, Matrix has basic support for signalling 1:1 WebRTC calls which suf ## Proposal ### Change the `version` field in all VoIP events to `"1"` +The version property is changed to `"`1`" in all existing VoIP events +([`m.call.answer`](https://spec.matrix.org/v1.5/client-server-api/#mcallanswer), +[`m.call.candidates`](https://spec.matrix.org/v1.5/client-server-api/#mcallcandidates), +[`m.call.hangup`](https://spec.matrix.org/v1.5/client-server-api/#mcallhangup) +[`m.call.invite`](https://spec.matrix.org/v1.5/client-server-api/#mcallinvite)). +This will be used to determine whether devices support this new version of the protocol. + This will be used to determine whether devices support this new version of the protocol. For example, clients can use this field to know whether to expect an `m.call.select_answer` event from their opponent. If clients see events with `version` other than `0` or `"1"`, they should treat these the From a09be957737d53782d59b5029439d601831fe5bc Mon Sep 17 00:00:00 2001 From: David Baker Date: Mon, 6 Feb 2023 17:47:08 +0000 Subject: [PATCH 59/80] Clarify treatment of version numeric 1. --- proposals/2746-reliable-voip.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/proposals/2746-reliable-voip.md b/proposals/2746-reliable-voip.md index ab8d7e292a..e9419358f1 100644 --- a/proposals/2746-reliable-voip.md +++ b/proposals/2746-reliable-voip.md @@ -26,8 +26,9 @@ This will be used to determine whether devices support this new version of the p This will be used to determine whether devices support this new version of the protocol. For example, clients can use this field to know whether to expect an `m.call.select_answer` event from their opponent. If clients see events with `version` other than `0` or `"1"`, they should treat these the -same as if they had `version` == `"1"`. In addition, clients must accept either the number `0` or a -string for the value of the `version` field, in order to allow for namespaced versions in the future. +same as if they had `version` == `"1"` (including the numeric value `1`). In addition, clients must +accept either the number `0` or a string for the value of the `version` field, in order to allow for +namespaced versions in the future. Note that this implies any and all future versions of VoIP events should be backwards-compatible. If it does become necessary to introduce a non backwards-compatible VoIP spec, the intention would From 5d38f15ad6926a1f7b26812711c35e1e12bbcacb Mon Sep 17 00:00:00 2001 From: David Baker Date: Mon, 6 Feb 2023 17:52:31 +0000 Subject: [PATCH 60/80] Clarify that track/stream layout is new. Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> --- proposals/2746-reliable-voip.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/proposals/2746-reliable-voip.md b/proposals/2746-reliable-voip.md index e9419358f1..0733b1dab7 100644 --- a/proposals/2746-reliable-voip.md +++ b/proposals/2746-reliable-voip.md @@ -36,7 +36,9 @@ be for it to simply use a separate set of event types. ### Define the configurations of WebRTC streams and tracks -Clients are expected to send one stream with one track of kind `audio` (creating +The [spec](https://spec.matrix.org/v1.5/client-server-api/#voice-over-ip) does not currently define +the WebRTC streams and tracks that should be sent. Under this proposal, +clients are expected to send one stream with one track of kind `audio` (creating a voice call). They can optionally send a second track in the same stream of kind `video` (creating a video call). From 3162911cabdad5f3392114db3dffab7148bb60c9 Mon Sep 17 00:00:00 2001 From: David Baker Date: Mon, 6 Feb 2023 17:53:07 +0000 Subject: [PATCH 61/80] Link to m.call.invite Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> --- proposals/2746-reliable-voip.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/2746-reliable-voip.md b/proposals/2746-reliable-voip.md index 0733b1dab7..95a04de2c1 100644 --- a/proposals/2746-reliable-voip.md +++ b/proposals/2746-reliable-voip.md @@ -51,7 +51,7 @@ such streams and tracks is undefined. This follows the existing known implementations of v0 VoIP. -### Add `invitee` field to `m.call.invite` +### Add `invitee` field to [`m.call.invite`](https://spec.matrix.org/v1.5/client-server-api/#mcallinvite) This allows for the following use cases: * Placing a call to a specifc user in a room where other users are also present. * Placing a call to oneself. From 097fa58184e315dbd47d3f1b9c2f931145e6fba8 Mon Sep 17 00:00:00 2001 From: David Baker Date: Mon, 6 Feb 2023 17:58:20 +0000 Subject: [PATCH 62/80] Suggestions from richvdh Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> --- proposals/2746-reliable-voip.md | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/proposals/2746-reliable-voip.md b/proposals/2746-reliable-voip.md index 95a04de2c1..7b4e29da82 100644 --- a/proposals/2746-reliable-voip.md +++ b/proposals/2746-reliable-voip.md @@ -56,9 +56,11 @@ This allows for the following use cases: * Placing a call to a specifc user in a room where other users are also present. * Placing a call to oneself. -The field should be added for all invites where the target is a specific user. Invites without an `invitee` -field are defined to be intended for any member of the room other than the sender of the event. Clients -should consider an incoming call if they see a non-expired invite event where the `invitee` field is either +The field should be added for all invites where the target is a specific user, and should be set +to the Matrix user ID of that user. Invites without an `invitee` +field are defined to be intended for any member of the room other than the sender of the event. + +Clients should consider an incoming call if they see a non-expired invite event where the `invitee` field is either absent or equal to their user's Matrix ID, however they should evaluate whether or not to ring based on their user's trust relationship with the callers and/or where the call was placed. As a starting point, it is suggested that clients ignore call invites from users in public rooms. It is strongly recommended that @@ -83,6 +85,8 @@ or it may choose, for example, to use a different one for each call to avoid lea devices were used in a call (in an unencrypted room) or if a single device (ie. access token were used to send signalling for more than one call party. +A grammar for `party_id` is defined [below](#specify-exact-grammar-for-voip-ids). + ### Introduce `m.call.select_answer` This event is sent by the caller's client once it has chosen an answer. Its `selected_party_id` field indicates the answer it's chosen (and has `call_id` From 4eaa0b4785c6c609071d5ef6cf1c0169c6810b83 Mon Sep 17 00:00:00 2001 From: David Baker Date: Mon, 6 Feb 2023 17:59:26 +0000 Subject: [PATCH 63/80] More suggestions from richvdh Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> --- proposals/2746-reliable-voip.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/proposals/2746-reliable-voip.md b/proposals/2746-reliable-voip.md index 7b4e29da82..a6b4332378 100644 --- a/proposals/2746-reliable-voip.md +++ b/proposals/2746-reliable-voip.md @@ -106,7 +106,7 @@ Example: "call_id": "12345", "party_id": "67890", "selected_party_id": "111213", - }, + } } ``` @@ -114,9 +114,9 @@ Example: * If the `m.call.invite` event has `version` `"1"`, a client wishing to reject the call sends an `m.call.reject` event. This rejects the call on all devices, but if the calling - device sees an accept, it disregards the reject event and carries on. The reject has a + device sees an `answer` before the `reject`, it disregards the reject event and carries on. The reject has a `party_id` just like an answer, and the caller sends a `select_answer` for it just like an - answer. If the other client that had already sent an answer sees the caller select the + answer. If another client had already sent an answer and sees the caller select the reject response instead of its answer, it ends the call. * If the `m.call.invite` event has `version` `0`, the callee sends an `m.call.hangup` event as in spec version 0. From db5ca80701f21b7e6e274566569dc8b5f197d0e1 Mon Sep 17 00:00:00 2001 From: David Baker Date: Mon, 6 Feb 2023 18:00:16 +0000 Subject: [PATCH 64/80] More suggestions from richvdh Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> --- proposals/2746-reliable-voip.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/proposals/2746-reliable-voip.md b/proposals/2746-reliable-voip.md index a6b4332378..73d1f9811f 100644 --- a/proposals/2746-reliable-voip.md +++ b/proposals/2746-reliable-voip.md @@ -138,15 +138,15 @@ as previously. ### Clarify what actions a client may take in response to an invite The client may: - * Attempt to accept the call by sending an answer - * Actively reject the call everywhere: reject the call as per above, which will stop the call from + * Attempt to accept the call by sending an `m.call.answer`. + * Actively reject the call everywhere: send an `m.call.reject` as per above, which will stop the call from ringing on all the user's devices and the caller's client will inform them that the user has rejected their call. * Ignore the call: send no events, but stop alerting the user about the call. The user's other devices will continue to ring, and the caller's device will continue to indicate that the call is ringing, and will time the call out in the normal way if no other device responds. -### Introduce more reason codes to `m.call.hangup` +### Introduce more reason codes to [`m.call.hangup`](https://spec.matrix.org/v1.5/client-server-api/#mcallhangup) * `ice_timeout`: The connection failed after some media was exchanged (as opposed to current `ice_failed` which means no media connection could be established). Note that, in the case of an ICE renegotiation, a client should be sure to send `ice_timeout` rather than `ice_failed` if @@ -154,7 +154,7 @@ The client may: * `user_hangup`: Clients must now send this code when the user chooses to end the call, although for backwards compatability with version 0, a clients should treat an absence of the `reason` field as `user_hangup`. - * `user_media_failed`: The client was unable to start capturing media in such a way as it is unable + * `user_media_failed`: The client was unable to start capturing media in such a way that it is unable to continue the call. * `user_busy`: The user is busy. Note that this exists primarily for bridging to other networks such as the PSTN. A Matrix client that receives a call whilst already in a call would not generally reject @@ -220,7 +220,7 @@ time since it received the event from the homeserver to determine whether the in The use of the `age` field ensures that incorrect clocks on client devices don't break calls. If the invite is still valid *and will remain valid for long enough for the user to accept the call*, it should signal an incoming call. The amount of time allowed for the user to accept the call may -vary between clients, for example, it may be longer on a locked mobile device than on an unlocked +vary between clients. For example, it may be longer on a locked mobile device than on an unlocked desktop device. The client should only signal an incoming call in a given room once it has completed processing the @@ -261,7 +261,7 @@ Add that Matrix clients can send DTMF as specified by WebRTC. The WebRTC standar in the RTP payload. We also add a capability to the `capabilities` section of invites and answers (detailed in -[MSC2747](https://github.com/matrix-org/matrix-doc/pull/2747) called `m.call.dtmf`. Clients +[MSC2747](https://github.com/matrix-org/matrix-doc/pull/2747)) called `m.call.dtmf`. Clients should only display UI for sending DTMF during a call if the other party advertises this capability (boolean value `true`). @@ -270,7 +270,7 @@ capability (boolean value `true`). 'opaque IDs' from [MSC1597](https://github.com/matrix-org/matrix-spec-proposals/blob/rav/proposals/id_grammar/proposals/1597-id-grammar.md#opaque-ids). ### Specify behaviour on room leave -If the client sees the party it is in a call with leave the room, the client should treat this +If the client sees the user it is in a call with leave the room, the client should treat this as a hangup event for any calls that are in progress. No specific requirement is given for the situation where a client has sent an invite and the invitee leaves the room, but the client may wish to treat it as a rejection if there are no more users in the room who could answer the call From 48527fcb8f3e44e7a56b8b7ab8ccd57b94534781 Mon Sep 17 00:00:00 2001 From: David Baker Date: Mon, 6 Feb 2023 18:01:58 +0000 Subject: [PATCH 65/80] Clarify call invite --- proposals/2746-reliable-voip.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/2746-reliable-voip.md b/proposals/2746-reliable-voip.md index 73d1f9811f..353372224d 100644 --- a/proposals/2746-reliable-voip.md +++ b/proposals/2746-reliable-voip.md @@ -64,7 +64,7 @@ Clients should consider an incoming call if they see a non-expired invite event absent or equal to their user's Matrix ID, however they should evaluate whether or not to ring based on their user's trust relationship with the callers and/or where the call was placed. As a starting point, it is suggested that clients ignore call invites from users in public rooms. It is strongly recommended that -when clients do not ring for an incoming call invite, they still display the invite in the room and +when clients do not ring for an incoming call invite, they still display the call invite in the room and annotate that it was ignored. ### Add `party_id` to all VoIP events From 312cdf727460dff7aaf3e4e4916bf3c593c06def Mon Sep 17 00:00:00 2001 From: David Baker Date: Mon, 6 Feb 2023 18:02:42 +0000 Subject: [PATCH 66/80] Pluralise --- proposals/2746-reliable-voip.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/2746-reliable-voip.md b/proposals/2746-reliable-voip.md index 353372224d..b465a22a99 100644 --- a/proposals/2746-reliable-voip.md +++ b/proposals/2746-reliable-voip.md @@ -77,7 +77,7 @@ lowercase alphanumeric characters is recommended. Parties in the call are identi The client adds a `party_id` field containing this ID to the top-level of the content of all VoIP events it sends on the call, including `m.call.invite`. Clients use this to identify remote echo of their own events, since a user may now call themselves, they can no longer ignore events from their own user. This -field also identifies different answers sent by different clients to an invite, and matches `m.call.candidate` +field also identifies different answers sent by different clients to an invite, and matches `m.call.candidates` events to their respective answer/invite. A client implementation may choose to use the device ID used in end-to-end cryptography for this purpose, From 8286d10cfd7f3e333876baf1eab06ea254a6a493 Mon Sep 17 00:00:00 2001 From: David Baker Date: Mon, 6 Feb 2023 18:04:08 +0000 Subject: [PATCH 67/80] Reflect that MSC1597 hasn't landed yet. Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> --- proposals/2746-reliable-voip.md | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/proposals/2746-reliable-voip.md b/proposals/2746-reliable-voip.md index b465a22a99..0b6485ad95 100644 --- a/proposals/2746-reliable-voip.md +++ b/proposals/2746-reliable-voip.md @@ -266,8 +266,13 @@ should only display UI for sending DTMF during a call if the other party adverti capability (boolean value `true`). ### Specify exact grammar for VoIP IDs -`call_id`s and the newly introduced `party_id` are explicitly defined to be in the grammar of -'opaque IDs' from [MSC1597](https://github.com/matrix-org/matrix-spec-proposals/blob/rav/proposals/id_grammar/proposals/1597-id-grammar.md#opaque-ids). +`call_id`s and the newly introduced `party_id` are explicitly defined to be between 1 +and 255 characters long, consisting of the characters `[0-9a-zA-Z._~-]`. + +(Note that this matches the grammar of 'opaque IDs' from +[MSC1597](https://github.com/matrix-org/matrix-spec-proposals/blob/rav/proposals/id_grammar/proposals/1597-id-grammar.md#opaque-ids), +and that of the `id` property of the + [`m.login.sso` flow schema](https://spec.matrix.org/v1.5/client-server-api/#definition-mloginsso-flow-schema).) ### Specify behaviour on room leave If the client sees the user it is in a call with leave the room, the client should treat this From a5e963f0f9ec611957d148b70f7d2727f10b2c7e Mon Sep 17 00:00:00 2001 From: David Baker Date: Mon, 6 Feb 2023 18:10:38 +0000 Subject: [PATCH 68/80] Politeness only applies to renegotiation --- proposals/2746-reliable-voip.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/proposals/2746-reliable-voip.md b/proposals/2746-reliable-voip.md index 0b6485ad95..1cdfe79c3f 100644 --- a/proposals/2746-reliable-voip.md +++ b/proposals/2746-reliable-voip.md @@ -208,10 +208,10 @@ with the WebRTC API). ### Designate one party as 'polite' In line with WebRTC perfect negotiation (https://w3c.github.io/webrtc-pc/#perfect-negotiation-example) -we introduce rules to establish which party is polite. The callee is always the polite party. In a -glare situation, the politenes of a party is therefore determined by whether the inbound or outbound -call is used: if a client discards its outbound call in favour of an inbound call, it becomes the polite -party. +we introduce rules to establish which party is polite in the process of renegotiation. The callee is +always the polite party. In a glare situation, the politenes of a party is therefore determined by +whether the inbound or outbound call is used: if a client discards its outbound call in favour of +an inbound call, it becomes the polite party. ### Add explicit recommendations for call event liveness. `m.call.invite` contains a `lifetime` field that indicates how long the offer is valid for. When From b09af730df2258cdc6beac600dd4443d69d4f5ec Mon Sep 17 00:00:00 2001 From: David Baker Date: Mon, 6 Feb 2023 18:13:33 +0000 Subject: [PATCH 69/80] s/Mandate/define/ --- proposals/2746-reliable-voip.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/2746-reliable-voip.md b/proposals/2746-reliable-voip.md index 1cdfe79c3f..6e82f71ab3 100644 --- a/proposals/2746-reliable-voip.md +++ b/proposals/2746-reliable-voip.md @@ -246,7 +246,7 @@ Clients should aim to send a small number of candidate events, with guidelines: anyway after the invite whilst the client waits for the user to accept it). ### Mandate the end-of-candidates candidate -Mandate that an ICE candidate whose value is the empty string must be sent in an m.call.candidates +Define that an ICE candidate whose value is the empty string must be sent in an m.call.candidates message to signal that no more ICE candidates will be sent. The WebRTC spec requires browsers to generate such a candidate, however note that at time of writing, not all browsers do (Chrome does not, but does generate an `icegatheringstatechange` event). The client should send any remaining From 1fc6b37f88a2c8063c8218a3c541c3a8af4f85dc Mon Sep 17 00:00:00 2001 From: David Baker Date: Mon, 6 Feb 2023 18:18:24 +0000 Subject: [PATCH 70/80] Remove DTMF capability section to move tov MSC2747. --- proposals/2746-reliable-voip.md | 5 ----- 1 file changed, 5 deletions(-) diff --git a/proposals/2746-reliable-voip.md b/proposals/2746-reliable-voip.md index 6e82f71ab3..63d74543d9 100644 --- a/proposals/2746-reliable-voip.md +++ b/proposals/2746-reliable-voip.md @@ -260,11 +260,6 @@ Add that Matrix clients can send DTMF as specified by WebRTC. The WebRTC standar 2020 does not support receiving DTMF but a Matrix client can receive and interpret the DTMF sent in the RTP payload. -We also add a capability to the `capabilities` section of invites and answers (detailed in -[MSC2747](https://github.com/matrix-org/matrix-doc/pull/2747)) called `m.call.dtmf`. Clients -should only display UI for sending DTMF during a call if the other party advertises this -capability (boolean value `true`). - ### Specify exact grammar for VoIP IDs `call_id`s and the newly introduced `party_id` are explicitly defined to be between 1 and 255 characters long, consisting of the characters `[0-9a-zA-Z._~-]`. From c9f05744a5f9d1126501c1c6e8cc77c5b0fbb497 Mon Sep 17 00:00:00 2001 From: David Baker Date: Tue, 28 Mar 2023 18:40:05 +0100 Subject: [PATCH 71/80] Clarify backwards compat Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> --- proposals/2746-reliable-voip.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/proposals/2746-reliable-voip.md b/proposals/2746-reliable-voip.md index 63d74543d9..98588b6296 100644 --- a/proposals/2746-reliable-voip.md +++ b/proposals/2746-reliable-voip.md @@ -25,8 +25,8 @@ This will be used to determine whether devices support this new version of the p This will be used to determine whether devices support this new version of the protocol. For example, clients can use this field to know whether to expect an `m.call.select_answer` event from their -opponent. If clients see events with `version` other than `0` or `"1"`, they should treat these the -same as if they had `version` == `"1"` (including the numeric value `1`). In addition, clients must +opponent. If clients see events with `version` other than `0` or `"1"` (including, for example, the numeric +value `1`), they should treat these the same as if they had `version` == `"1"`. In addition, clients must accept either the number `0` or a string for the value of the `version` field, in order to allow for namespaced versions in the future. From 0880475308b70c1242c763d70b4025da834d21e5 Mon Sep 17 00:00:00 2001 From: David Baker Date: Tue, 28 Mar 2023 18:40:56 +0100 Subject: [PATCH 72/80] Grammar Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> --- proposals/2746-reliable-voip.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/2746-reliable-voip.md b/proposals/2746-reliable-voip.md index 98588b6296..318404e23a 100644 --- a/proposals/2746-reliable-voip.md +++ b/proposals/2746-reliable-voip.md @@ -76,7 +76,7 @@ lowercase alphanumeric characters is recommended. Parties in the call are identi The client adds a `party_id` field containing this ID to the top-level of the content of all VoIP events it sends on the call, including `m.call.invite`. Clients use this to identify remote echo of their own -events, since a user may now call themselves, they can no longer ignore events from their own user. This +events: since a user may now call themselves, they can no longer ignore events from their own user. This field also identifies different answers sent by different clients to an invite, and matches `m.call.candidates` events to their respective answer/invite. From e45c1e02fbae91b948424bae32acecffa958ecad Mon Sep 17 00:00:00 2001 From: David Baker Date: Tue, 28 Mar 2023 18:41:14 +0100 Subject: [PATCH 73/80] Fix quotes Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> --- proposals/2746-reliable-voip.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/2746-reliable-voip.md b/proposals/2746-reliable-voip.md index 318404e23a..107ade51b6 100644 --- a/proposals/2746-reliable-voip.md +++ b/proposals/2746-reliable-voip.md @@ -199,7 +199,7 @@ Example: ``` Once an `m.call.negotiate` event is received, the client must respond with another `m.call.negotiate` -event, with the SDP answer (with `"type"`: `"answer"`) in the description property. +event, with the SDP answer (with `"type": "answer"`) in the `description` property. This MSC also proposes clarifying the `m.call.invite` and `m.call.answer` events to state that the `offer` and `answer` fields respectively are objects of type `RTCSessionDescriptionInit` From bdf9639100569447ea1ef04b5ed5bbed5b103219 Mon Sep 17 00:00:00 2001 From: David Baker Date: Tue, 28 Mar 2023 18:41:45 +0100 Subject: [PATCH 74/80] Typo Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> --- proposals/2746-reliable-voip.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/2746-reliable-voip.md b/proposals/2746-reliable-voip.md index 107ade51b6..b239af6f9e 100644 --- a/proposals/2746-reliable-voip.md +++ b/proposals/2746-reliable-voip.md @@ -300,7 +300,7 @@ or not there have been any changes to the Matrix spec. is problematic with Matrix since any device or user could answer the call, so it is not known which device is going to answer before the user chooses to answer. It would also leak information on which of a user's devices were online. - * We could define that the ID of a call is implcitly the event ID of the invite event rather than + * We could define that the ID of a call is implicitly the event ID of the invite event rather than having a specific `call_id` field. This would mean that a client would be unable to know the ID of a call before the it received the response from sending the invite event, which could complicate implementations. There is probably no compelling reason to change this. From 082f216e74139226bd9578855e040d6745e16c31 Mon Sep 17 00:00:00 2001 From: David Baker Date: Tue, 28 Mar 2023 18:53:37 +0100 Subject: [PATCH 75/80] Remove sentence that I think is now just redundant --- proposals/2746-reliable-voip.md | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/proposals/2746-reliable-voip.md b/proposals/2746-reliable-voip.md index b239af6f9e..d69a62b57a 100644 --- a/proposals/2746-reliable-voip.md +++ b/proposals/2746-reliable-voip.md @@ -26,9 +26,7 @@ This will be used to determine whether devices support this new version of the p This will be used to determine whether devices support this new version of the protocol. For example, clients can use this field to know whether to expect an `m.call.select_answer` event from their opponent. If clients see events with `version` other than `0` or `"1"` (including, for example, the numeric -value `1`), they should treat these the same as if they had `version` == `"1"`. In addition, clients must -accept either the number `0` or a string for the value of the `version` field, in order to allow for -namespaced versions in the future. +value `1`), they should treat these the same as if they had `version` == `"1"`. Note that this implies any and all future versions of VoIP events should be backwards-compatible. If it does become necessary to introduce a non backwards-compatible VoIP spec, the intention would From ce0e3388f71b9223ecd9eec400d4cc45017faaa6 Mon Sep 17 00:00:00 2001 From: David Baker Date: Tue, 28 Mar 2023 19:00:16 +0100 Subject: [PATCH 76/80] Clarify mre on type field --- proposals/2746-reliable-voip.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/proposals/2746-reliable-voip.md b/proposals/2746-reliable-voip.md index d69a62b57a..ab2a6a234e 100644 --- a/proposals/2746-reliable-voip.md +++ b/proposals/2746-reliable-voip.md @@ -200,9 +200,10 @@ Once an `m.call.negotiate` event is received, the client must respond with anoth event, with the SDP answer (with `"type": "answer"`) in the `description` property. This MSC also proposes clarifying the `m.call.invite` and `m.call.answer` events to state that -the `offer` and `answer` fields respectively are objects of type `RTCSessionDescriptionInit` -(and hence the `type` field, whilst redundant in these events, is included for ease of working -with the WebRTC API). +the `offer` and `answer` fields respectively are objects of type `RTCSessionDescriptionInit`. +Hence the `type` field, whilst redundant in these events, is included for ease of working +with the WebRTC API and is mandatory. Receiving clients should not attempt to validate the `type` field, +but simply pass the object into the WebRTC API. ### Designate one party as 'polite' In line with WebRTC perfect negotiation (https://w3c.github.io/webrtc-pc/#perfect-negotiation-example) From 2919112d7c8cadbf7c7140bc8c816151bc2643e9 Mon Sep 17 00:00:00 2001 From: David Baker Date: Tue, 28 Mar 2023 19:06:07 +0100 Subject: [PATCH 77/80] Clarify end-of-candidates --- proposals/2746-reliable-voip.md | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-) diff --git a/proposals/2746-reliable-voip.md b/proposals/2746-reliable-voip.md index ab2a6a234e..b27632030a 100644 --- a/proposals/2746-reliable-voip.md +++ b/proposals/2746-reliable-voip.md @@ -245,12 +245,11 @@ Clients should aim to send a small number of candidate events, with guidelines: anyway after the invite whilst the client waits for the user to accept it). ### Mandate the end-of-candidates candidate -Define that an ICE candidate whose value is the empty string must be sent in an m.call.candidates -message to signal that no more ICE candidates will be sent. The WebRTC spec requires browsers to -generate such a candidate, however note that at time of writing, not all browsers do (Chrome does -not, but does generate an `icegatheringstatechange` event). The client should send any remaining -candidates once candidate generation finishes, ignoring timeouts above. - +Define that an ICE candidate whose value is the empty string means that no more ICE candidates will +be sent and mandate that clients must send such a candidate in an `m.call.candidates` message. +The WebRTC spec requires browsers to generate such a candidate, however note that at time of writing, +not all browsers do (Chrome does not, but does generate an `icegatheringstatechange` event). The +client should send any remaining candidates once candidate generation finishes, ignoring timeouts above. This allows bridges to batch the candidates together when bridging to protocols that don't support trickle ICE. From c949b32d8bbb25f4f474153187aaceb53146ad7b Mon Sep 17 00:00:00 2001 From: David Baker Date: Wed, 29 Mar 2023 09:59:32 +0100 Subject: [PATCH 78/80] Add comma Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> --- proposals/2746-reliable-voip.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/2746-reliable-voip.md b/proposals/2746-reliable-voip.md index b27632030a..d0735d73ec 100644 --- a/proposals/2746-reliable-voip.md +++ b/proposals/2746-reliable-voip.md @@ -246,7 +246,7 @@ Clients should aim to send a small number of candidate events, with guidelines: ### Mandate the end-of-candidates candidate Define that an ICE candidate whose value is the empty string means that no more ICE candidates will -be sent and mandate that clients must send such a candidate in an `m.call.candidates` message. +be sent, and mandate that clients must send such a candidate in an `m.call.candidates` message. The WebRTC spec requires browsers to generate such a candidate, however note that at time of writing, not all browsers do (Chrome does not, but does generate an `icegatheringstatechange` event). The client should send any remaining candidates once candidate generation finishes, ignoring timeouts above. From 7d8d5274fb35ec5d75ff07fe5fc602b686b828de Mon Sep 17 00:00:00 2001 From: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> Date: Wed, 5 Apr 2023 16:36:15 +0100 Subject: [PATCH 79/80] Update 2746-reliable-voip.md (#3992) --- proposals/2746-reliable-voip.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/2746-reliable-voip.md b/proposals/2746-reliable-voip.md index d0735d73ec..8c682a9bf6 100644 --- a/proposals/2746-reliable-voip.md +++ b/proposals/2746-reliable-voip.md @@ -16,7 +16,7 @@ Historically, Matrix has basic support for signalling 1:1 WebRTC calls which suf ## Proposal ### Change the `version` field in all VoIP events to `"1"` -The version property is changed to `"`1`" in all existing VoIP events +The version property is changed to `"1"` in all existing VoIP events ([`m.call.answer`](https://spec.matrix.org/v1.5/client-server-api/#mcallanswer), [`m.call.candidates`](https://spec.matrix.org/v1.5/client-server-api/#mcallcandidates), [`m.call.hangup`](https://spec.matrix.org/v1.5/client-server-api/#mcallhangup) From 3925586df0d726b22e952d4a58956966268a9fa4 Mon Sep 17 00:00:00 2001 From: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com> Date: Fri, 28 Apr 2023 10:01:11 +0100 Subject: [PATCH 80/80] wording changes --- proposals/2746-reliable-voip.md | 25 +++++++++++++------------ 1 file changed, 13 insertions(+), 12 deletions(-) diff --git a/proposals/2746-reliable-voip.md b/proposals/2746-reliable-voip.md index 8c682a9bf6..579604c9e3 100644 --- a/proposals/2746-reliable-voip.md +++ b/proposals/2746-reliable-voip.md @@ -20,8 +20,9 @@ The version property is changed to `"1"` in all existing VoIP events ([`m.call.answer`](https://spec.matrix.org/v1.5/client-server-api/#mcallanswer), [`m.call.candidates`](https://spec.matrix.org/v1.5/client-server-api/#mcallcandidates), [`m.call.hangup`](https://spec.matrix.org/v1.5/client-server-api/#mcallhangup) -[`m.call.invite`](https://spec.matrix.org/v1.5/client-server-api/#mcallinvite)). -This will be used to determine whether devices support this new version of the protocol. +[`m.call.invite`](https://spec.matrix.org/v1.5/client-server-api/#mcallinvite)). Note +that this changes the type of the `version` field from an integer to a string, as +described in the [Unstable Prefix](#unstable-prefix) section. This will be used to determine whether devices support this new version of the protocol. For example, clients can use this field to know whether to expect an `m.call.select_answer` event from their @@ -51,7 +52,7 @@ This follows the existing known implementations of v0 VoIP. ### Add `invitee` field to [`m.call.invite`](https://spec.matrix.org/v1.5/client-server-api/#mcallinvite) This allows for the following use cases: - * Placing a call to a specifc user in a room where other users are also present. + * Placing a call to a specific user in a room where other users are also present. * Placing a call to oneself. The field should be added for all invites where the target is a specific user, and should be set @@ -80,15 +81,16 @@ events to their respective answer/invite. A client implementation may choose to use the device ID used in end-to-end cryptography for this purpose, or it may choose, for example, to use a different one for each call to avoid leaking information on which -devices were used in a call (in an unencrypted room) or if a single device (ie. access token were used to +devices were used in a call (in an unencrypted room) or if a single device (ie. access token) were used to send signalling for more than one call party. A grammar for `party_id` is defined [below](#specify-exact-grammar-for-voip-ids). ### Introduce `m.call.select_answer` -This event is sent by the caller's client once it has chosen an answer. Its -`selected_party_id` field indicates the answer it's chosen (and has `call_id` -and its own `party_id` too). If the callee's client sees a `select_answer` for an answer +This event is sent by the caller's client once it has decided which other +client to talk to, by selecting one of multiple possible incoming `m.call.answer` +events. Its `selected_party_id` field indicates the answer it's chosen. The `call_id` +and `party_id` of the caller is also included. If the callee's client sees a `select_answer` for an answer with party ID other than the one it sent, it ends the call and informs the user the call was answered elsewhere. It does not send any events. Media can start flowing before this event is seen or even sent. Clients that implement previous @@ -116,8 +118,7 @@ Example: `party_id` just like an answer, and the caller sends a `select_answer` for it just like an answer. If another client had already sent an answer and sees the caller select the reject response instead of its answer, it ends the call. - * If the `m.call.invite` event has `version` `0`, the callee sends an `m.call.hangup` event - as in spec version 0. + * If the `m.call.invite` event has `version` `0`, the callee sends an `m.call.hangup` event. Example: ``` @@ -175,7 +176,7 @@ own negotiate events. This has a `lifetime` field as in `m.call.invite`, after which the sender of the negotiate event should consider the negotiation failed (timed out) and the recipient should ignore it. -The `description` field is the same as the `offer` field in in `m.call.invite` and `answer` +The `description` field is the same as the `offer` field in `m.call.invite` and `answer` field in `m.call.answer` and is an `RTCSessionDescriptionInit` object as per https://www.w3.org/TR/webrtc/#dom-rtcsessiondescriptioninit. @@ -241,7 +242,7 @@ Clients should aim to send a small number of candidate events, with guidelines: suggested as a starting point. * The client should then allow some time for further candidates to be gathered in order to batch them, rather than sending each candidate as it arrives. A starting point of 2 seconds after sending the - invite or 500ms after sending the answer is suggested as starting point (since a delay is natural + invite or 500ms after sending the answer is suggested as a starting point (since a delay is natural anyway after the invite whilst the client waits for the user to accept it). ### Mandate the end-of-candidates candidate @@ -307,7 +308,7 @@ or not there have been any changes to the Matrix spec. * This MSC elects to allow invites without an `invitee` field to mean a call for anyone in the room. This could be useful for hunt group style semantics where an incoming call causes many different users' phones to ring and any one of them may pick up the call. This does mean clients will need - to not blindly ring for any call invites in any room, since this would make unsolicited calls in + to not blindly ring for any call invites in any room, since this would make unsolicited calls easy in public rooms. We could opt to leave this out, or make it more explicit with a specific value for the `invitee` field. * `party_id` is one of many potential solutions: callees could add `answer_id`s to their events and