Realtime Videos · Concepts
Connecting media
How a client negotiates WebRTC with a session, receives frames, and sends commands.
The shape
- Signaling is HTTPS through our proxy at
/v1/realtime/sessions/{id}/transport/webrtc/*. We authenticate it with your API key and forward it to the pod. - Media and commands flow directly between your client and the GPU pod over WebRTC. The proxy never carries frames.
- The pod publishes one outbound video track,
main_video. Commands go over thedatadata channel; keep-alive and track control use thecontroldata channel.
Negotiation sequence
All paths below are relative to the session's /transport/webrtc prefix.
| # | Request | Purpose |
|---|---|---|
| 1 | GET /ice_servers | STUN/TURN servers the pod expects you to use. |
| 2 | POST /connections | Register a connection. Returns connection_id and a track_map of the pod's tracks. |
| 3 | POST /connections/{cid}/sdp_params | Send your SDP offer and a track_mapping (one receive-only entry per outbound track). |
| 4 | GET /connections/{cid}/sdp_params | Poll until 200; the body holds the sdp_answer. |
| 5 | POST /connections/{cid}/ice_candidates | Trickle your local ICE candidates until the control channel opens. |
Then
- Create two data channels,
dataandcontrol, before making the offer. - When
controlopens, send aresume_tracknotification for each track you want to receive. - Send a
pingoncontrolabout once a second while connected (the reference viewer does). - Send commands on
data. See Commands.
Messages on both channels are protobufs from the reactor_wire.v1 package, and the WebRTC peer uses the reactor_webrtc package. The Python viewer imports both and is the canonical working example.
Frames
- The pod paces output to the model's playout rate (16 fps for
longlive-1.3b), so a slow model shows as a held last frame, not an error. - Each frame can carry metadata; the reference models tag frames with a step counter.
- On loopback the test model streams about 16 fps with a time to first frame near 60 ms. Numbers from rented GPUs are not published yet.
Network requirements
- If the control channel never opens, ICE failed: the client and pod could not find a working path.
- Corporate networks that block UDP will need the TURN relay path.