webrtcH4cKS: ~ How Zoom’s web client avoids using WebRTC (DataChannel Update)

Editor’s Note: This post was originally published on October 23, 2018. Zoom recently started using WebRTC’s DataChannels so we have added some new details at the end in the DataChannels section.

Zoom has a web client that allows a participant to join meetings without downloading their app. Chris Koehncke was excited to see how this worked (watch him at the upcoming KrankyGeek event!) so we gave it a try. It worked, removing the download barrier. The quality was acceptable and we had a good chat for half an hour.

Opening chrome://webrtc-internals showed only getUserMedia being used for accessing camera and microphone but no RTCPeerConnection like a WebRTC call should have. This got me very interested – how are they making calls without WebRTC?

Why don’t they use WebRTC?

The relationship between Zoom and WebRTC is a difficult one as shown in this statement from their website:

The Jitsi folks just did a comparison of the quality recently in response to that. Tsahi Levent-Levi had some useful comments on that as well.

So let’s take a quick look at the “excellent features” under quite interesting circumstances — running in Chrome.

The Zoom Web client

Chromes network developer tools quickly showed two things:

WebSockets are used for data transfer

there are workers loading WebAssembly (wasm) files

The WebAssembly file names quickly lead to a GitHub repository where those files, including some of the other JavaScript components are hosted. The files are mostly the same as the ones used in production.

Media over WebSockets

The overall design is quite interesting. It uses WebSockets to transfer the media which is certainly not an optimal choice. It is similar to using TURN/TCP in WebRTC — it has a quality impact and will not work well in quite a number of cases. The general problem of doing realtime media over TCP is that packet loss can lead to resends and increased delay. Tsahi described this over at TestRTC a while ago, showing the impact on bitrate and other things.

The primary advantage of using media over WebSockets is that it might pass firewalls where even TURN/TCP and TURN/TLS could be blocked. And it certainly avoids the issue of WebRTC TURN connections not getting past authenticated proxies. That was a long-standing issue in Chrome’s WebRTC implementation that was only resolved last year.

Data received on the WebSockets goes into a WebAssembly (WASM) based decoder. Audio is fed to an AudioWorklet in browsers that support that. From there the decoded audio is played using the WebAudio “magic” destination node.

Video is painted to a canvas. This is surprisingly smooth and the quality is quite high.

In the other direction, WebAudio captures media from the getUserMedia call and is sent to a WebAssembly encoder worker and then delivered via WebSocket. Video capture happens with a resolution of 640×360 and, unsurprisingly, is grabbed from a canvas before being sent to the WebAssembly encoder.

The WASM files seem to contain the same encoders and decoders as Zooms native client, meaning the gateway doesn’t have to do transcoding. Instead it is probably little more than a websocket-to-RTP relay, similar to a TURN server. The encoded video is somewhat pixelated at times and Mr. Kranky even complained about staircase artifacts. While the CPU usage of the encoder is rather high (at 640×360 resolution) this might not matter as the user will simply blame Chrome and use the native client the next time.

H.264

Delivering the media engine as WebAssembly is quite interesting, it allows for supporting codecs not supported by Chrome/WebRTC. This is not entirely novel – FFmpeg compiled with emscripten has been done many times before and emscripten seems to have been used here as well. Delivering the encoded bytes via WebSockets allowed inspecting their content using Chrome’s excellent debugging tools and showed a H264 payload ...