-
Notifications
You must be signed in to change notification settings - Fork 1.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add support for Simulcast/SVC. #1016
Comments
@cloudwebrtc that would be an amazing feature! Also would be really nice to just have it be on the I will need to think about a little bit though. I need to do some research/API thinking and will come back on this! thanks for filing, this could really open up some cool things. |
@Sean-Der I started studying how adding simulcast will work and doing an initial implementation. Since it'll break various pion packages api (primarily webrtc and srtp) (like also discussed in #1025 for track api) I'd like to propose a design doc to cover and explain the required changes and discuss the possible options. Is the wiki the right place (a page like https://github.com/pion/webrtc/wiki/PlanningV3) to add and discuss such design doc or do you have something better in mind? |
@sgotti Yes the wiki is the best place, and please edit away! I don't think it will be a substantial amount of work, but I am just timid to change the API and not do it right. The |
I'll start writing something here the keep you updated, I'll try to summarize my work until now and explain why we need a different pion I have an initial working implementation of simulcast handling and, to be honest, is a quite big amount of work. I'll clean it up and split in multiple patches and then publish it in the next days so you can take a look at it. It's a big amount of work because it has to change the way tracks are handled. We have to handle mid, rid rtp/rtcp extensions, sdp extmap, rid and simulcast attributes. The biggest issue, is related to the current pion implementation of Additionally currently track is tied to a single SSRC (in webrtc spec API the rtp stream ssrc is completely hidden). Also without simulcast this isn't quite right since in a mediatrack we could receive multiple streams also without simulcast (i.e the repaired stream) that are currently skipped and it's not possible to send them. Additionally on the sdp side, a simulcast media track won't provide ssrc information anymore (since ssrc could change due to different condition explained on the various rfc) and instead you'll receive information on the rtp stream ids ( |
(edit: my issues, and I suspect issues that are going to come up w/ the simulcast impl, are bitrate/bandwidth limitations enforced by Chrome based on their bandwidth estimation which I believe is handled via RTCP...I am still researching and will get back) |
Ok, I hacked together a simulcast example in a branch on my fork. Quick notes:
|
Just as an update, simulcast works correctly on both chrome and firefox. Before opening a PR I'm trying to split all my changes in many small patches so they can be easily reviewed just because there're many changes required to handle tracks with multiple sources and all the features required for simulcast (extmaps negotiation, handling rid based streams etc...). I also have two track proposals to make them multistream. One is compatible with v2 API (but it's not the cleanest one), the other one is a proposal for a future v3 api. |
pc.startReceiver will set the receive flag for Transceivers so no reason to do extra filtering here. Relates to #1016
Summary
Add multiple resolution/rate support for the video track.
Motivation
In a video conference or live broadcast scenario, multiple resolutions need to be generated at the same time on the same PC, and different devices can subscribe to the resolution they want.
SVC can adapt to network transmission by dynamically adjusting resolution and bit rate during the SFU Relay process.
Describe alternatives you've considered
mediasoup already implements similar functionality, but it uses C ++
Additional context
The text was updated successfully, but these errors were encountered: