MSC4195: MatrixRTC using LiveKit backend

[hughns]

Rendered

Dependencies:

• MSC4143: MatrixRTC #4143

To-do:

• Define error responses for POST /sfu/get

[anoadragon453]

I spoke with the Element VoIP team internally about a concern that was raised to me regarding the reliance on LiveKit as part of the Matrix spec.

LiveKit is a protocol and SFU implementation developed by a company with the same name, rather than an open standards body, and as such could develop in any number of ways depending on the company's motives. They also currently gate features such as Cascading SFUs behind their enterprise offering, whereas we would like to have something like this in Matrix (to support our federation model). There is a concern that reliance here stunts Matrix's ability to extend our own protocol, and beholds us to the whims of this other company.

However, there are several key points that the VoIP team shared that lessen the concern for me:

• They already had the same concern when building this solution.
• Matrix has the option to fork the LiveKit protocol (Apache2) as it is today and build something more open if needed.
• They've spoken with the LiveKit team and they've said that Matrix is free to build its own Cascading SFU implementation on top of the open LiveKit protocol - they would not object to this. This would allow us to do away with the current central "elected" SFU per call approach we have today.

The reasoning for the "type" field in this MSC, rather than building a generic Matrix SFU API, was to allow building much smaller and simpler APIs for niche applications such as gaming server protocols - which don't need all the features of VoIP.

The reason behind choosing LiveKit's protocol instead of starting from scratch was a "hack" to get a good implementation working quickly. But it's not painting ourselves into a corner - what is being built now, even if LiveKit switches its license tomorrow, won't be lost in the process.

---

While I would like to see this concern addressed directly by the MSC author(s) (in the "Potential issues" section ideally), the authors acknowledged this MSC has been lagging behind the implementation and is in need of an update.

[networkException]

From my understanding there's no API that allows room_id validation. Without that implementing access control in a federated environment seems infeasible however.

Given that all members in a session use one SFU via oldest_membership, just validating against matrix_server_name (authenticated using TLS / OpenID) would require allowlisting any server that users might want to join from.

Ideally access control could check either for an allowlisted server or for an existing session with the oldest member being on an allowlisted server, gating an SFU to sessions being started by their "own" users.

[networkException]

The implementation in element-hq/lk-jwt-service#67 does allow limiting creation of new SFU rooms based on servers in a given allowlist, but still allows arbitrary users to join