Skip to content

Live streaming

This page follows live video from the camera to your screen. It covers how nvrd registers each camera in go2rtc, how the browser’s WebSocket is authorized and proxied, how the player chooses WebRTC or MSE and recovers from drops, and where snapshots come from. It is written for developers and for anyone debugging live view; the everyday side is in Live view.

go2rtc (aperture-go2rtc.service)
camera ──RTSP──────────> producer per stream: cam_<id>, cam_<id>_sub
│
├── RTSP restream :8554 ──> recorder, motion, audio, external players
├── HTTP API 127.0.0.1:1984 <── nvrd only (register streams, frame.jpeg, /api/ws)
└── WebRTC media :8555 <══════════════> browser (when ICE connects)
browser ──ws(s)──> nvrd GET /api/v1/cameras/{id}/ws?token=… ──ws──> go2rtc /api/ws?src=cam_<id>
(resolve camera, check token, drop Origin) (signalling, MSE data)
  • go2rtc holds the only connection to each camera stream and fans it out. A camera sees one client for its main stream and, when a sub stream is set and something reads it, one for that, however many people watch.
  • The browser never reaches go2rtc’s HTTP API. Every live socket goes to nvrd, which authorizes it and proxies it to go2rtc.
  • Only the signalling passes through nvrd. With WebRTC, the video itself flows directly between the browser and go2rtc’s WebRTC port 8555. With MSE, the video travels inside the WebSocket, through nvrd.

nvrd addresses go2rtc streams by name. The names come from StreamName and SubStreamName in internal/stream/manager.go. A camera id is 32 hex characters (store.NewID).

Stream Source Registered when Read by
cam_<id> the camera’s main link (rtsp_main_url) the camera is enabled and has a main link the recorder, audio detection, motion detection and local object recognition when there is no sub stream, live view at Auto and High (HD), snapshots, the RTSP restream for other players
cam_<id>_sub the camera’s sub link the camera is enabled and has a sub link live view at Low (SD), motion detection and local object recognition (which receives its frames from the motion decoder), the recorder when the recording quality preference asks for the sub stream
test_<random id> a link being tested for one frame grab, then deleted Test connection in the camera editor

The recorder, motion and audio detection read the RTSP restream at APERTURE_GO2RTC_RTSP (RTSPURL, RTSPURLSub), so they are consumers of go2rtc like any browser. Other players use the same restream at rtsp://<host>:8554/cam_<id>, where the host is APERTURE_STREAM_HOST if set and otherwise the host the request came in on (advertisedRTSPHost in internal/api/ha_handlers.go). See ONVIF and RTSP for using it.

Manager.Sync reconciles go2rtc with the camera table:

  1. It reads go2rtc’s current streams with GET /api/streams.
  2. For every enabled camera with a main link it registers cam_<id>, and cam_<id>_sub when a sub link is set, with PUT /api/streams?name=<name>&src=<link>.
  3. It skips a stream whose registered producer URLs already match (sameSources). Re-registering an unchanged stream drops and recreates the producer, which would interrupt every viewer, so renaming a camera or changing its retention leaves go2rtc alone.
  4. It deletes every stream whose name starts with cam_ that is no longer wanted: a disabled or deleted camera, or a sub link that was removed (DELETE /api/streams?src=<name>). Streams with other names are not touched.

Sync runs at startup, every 60 seconds (the reconcile loop in cmd/nvrd/main.go, each pass bounded to 15 seconds), and in the background after each camera change (syncStreams in internal/api/camera_handlers.go). If go2rtc restarts and loses its streams, live view comes back within a minute without restarting nvrd.

Manager.Restart re-registers a camera’s main stream even when nothing changed, which tears down a wedged producer. The camera watchdog (internal/watchdog/watchdog.go, every 15 seconds) calls it when a camera that should be recording has written no segment for 90 seconds (store.RecordingGraceMs) and the recorder has stopped writing, at most once per camera per minute. It restarts the camera’s recorder as well. Live viewers of that stream lose their connection and the player reconnects (see Reconnecting and Limits).

Links are normalized before they are saved (NormalizeSource in internal/stream/protect.go). A UniFi Protect link on port 7441, rtsps://<console>:7441/<key>?enableSrtp, becomes rtspx://<console>:7441/<key>: go2rtc’s RTSPS without a certificate check, because the console’s certificate is self-signed, and without enableSrtp, which only Protect’s own apps use. See UniFi Protect.

Test connection calls POST /api/v1/cameras/test (admin; cameras:write for a token). Manager.TestStream accepts only rtsp://, rtsps:// and rtspx:// links. It registers the link as test_<random id>, waits 800 ms for go2rtc to dial the camera, fetches one JPEG from /api/frame.jpeg (at most 8 MiB) and deletes the temporary stream, even when the request was cancelled. The handler allows 12 seconds in total. When an existing camera’s link arrives with its login masked, the saved login is filled back in on the server; errors are scrubbed of credentials before they are returned.

The installer (deploy/install.sh) seeds /var/lib/aperture/go2rtc.yaml with:

api:
listen: "127.0.0.1:1984"
rtsp:
listen: ":8554"
webrtc:
listen: ":8555"
candidates:
- stun:8555
log:
level: info
streams: error
Port Bound to Purpose
1984 loopback only go2rtc’s HTTP API. It hands every camera’s source link, password included, to whoever can reach it, so only nvrd on the same machine uses it.
8554 all interfaces The RTSP restream used by the recorder, the detectors and external players.
8555 all interfaces WebRTC media between go2rtc and browsers.

go2rtc writes the streams nvrd registers back into that file, which is why it holds camera passwords and is mode 0600, owned by the aperture-go2rtc user. The unit aperture-go2rtc.service runs /usr/lib/aperture/bin/go2rtc -config /var/lib/aperture/go2rtc.yaml with Restart=always, RestartSec=2 and MemoryMax=1G. The streams log level is kept at error because go2rtc’s reconnect warnings include the full source link.

The web app opens a camera in two steps.

  1. It asks for a stream token: GET /api/v1/cameras/{id}/stream-token. The route requires media:live for an API token and access to that camera (scopedCam(ScopeMediaLive) in internal/api/server.go). The reply:

    {
    "expiresIn": 300,
    "hasSub": true,
    "token": "<stream token>",
    "wsUrl": "/api/v1/cameras/<id>/ws?token=<stream token>"
    }
  2. It opens ws:// or wss:// (matching the page) on its own host with that wsUrl, appending &q=sub for the sub stream.

hasSub says whether the camera has a sub link, so the app can offer low quality. q=sub on a camera without one falls back to the main stream. expiresIn is how many seconds the token can still open a socket. A socket that is already open stays open after that.

Every new socket needs a token that is still valid: the first one, the MSE fallback after WebRTC fails, and every reconnect. liveUrlSource in web/src/lib/streamToken.ts supplies the address before each socket. It reuses a cached token only while more than 60 seconds of its expiresIn are left, and asks for a new one otherwise. Concurrent requests for one camera share a fetch. A socket refused before it opened makes the next one ask for a new token. The whole cache is dropped when the app returns to the foreground.

GET /api/v1/cameras/{id}/ws is declared pub in the route table: the generic access check does nothing, and handleStreamWS in internal/api/stream_handlers.go does its own authorization. It answers 404 (camera not found) for an unknown camera and 401 (invalid stream token) for a bad credential.

A browser cannot set an Authorization header on a WebSocket handshake, so the socket takes its credential from the token query parameter, and only from there. A session cookie or a bearer header on the handshake is not consulted. authorizeStreamWS accepts three kinds of credential, tried in this order:

Credential What it is Grants
Stream token An HS256 JWT minted by /stream-token (auth.Manager.StreamToken), signed with the stream secret, carrying the camera id, issued-at and an expiry 5 minutes later (streamTokenTTL). That one camera, and only if the id in the token matches the camera in the URL.
Live key A single server-managed secret kept in the settings table as integrations.live_key (prefix aplv_). It is created the first time an admin reads it and can be rerolled, which ends the old value at once. Compared in constant time. Live view of every camera, and nothing else.
API token with media:live A scoped token (apt_…) passed as token. Its claims are attached to the request. Cameras the token may see: its owner’s cameras, narrowed to the token’s camera grants if it has any (allowedCamera).

The stream secret comes from APERTURE_STREAM_SECRET, or is generated once and kept in secrets.json in the data directory (loadSecrets in internal/config/config.go). The expiry is checked when the socket opens; an open socket is not cut when its token expires.

The {id} in the socket URL may be the camera id or a slug of its name (resolveCameraID in internal/api/integrations_live.go). The id is tried first, as an indexed lookup. Otherwise the value is slugified and compared against each camera’s slugified name, in camera list order (sort_order, then name); the first match wins. slugify lowercases the name, collapses each run of characters other than a–z and 0–9 into one underscore, and trims underscores: “Front door” becomes front_door.

The slug is resolved on every request, so it always follows the camera’s current name; renaming a camera changes its slug. Only the socket route accepts slugs; /stream-token and the other camera routes take the id.

After authorization, handleStreamWS:

  1. Picks the stream: cam_<id>, or cam_<id>_sub for q=sub when the camera has a sub link.
  2. Registers a viewer with object recognition (detect.Engine.Watch) for as long as the socket is open. For a camera that uses local object recognition, frames are then examined even while nothing moves: every 700 ms on a CPU, every 200 ms on CUDA (watchInterval in internal/detect/detect.go), so a person standing still in view gets a box. An open live view therefore costs recognition time. See Detection pipeline.
  3. Hands the request to Manager.ProxyWS, which rewrites the path to go2rtc’s /api/ws, replaces the whole query with src=<stream name> (the credential never reaches go2rtc), sets the host to go2rtc’s, deletes the Origin header and passes the request to an httputil.ReverseProxy.

go2rtc’s WebSocket upgrader rejects cross-origin requests, and the browser’s origin is nvrd’s public address, not go2rtc’s. Deleting Origin presents the proxied request as same-origin; nvrd has already decided who may connect.

The HTTP server sets no read or write timeout (NewHTTPServer in internal/api/shutdown.go), so a socket can stay open as long as both ends keep it. A reverse proxy in front needs WebSocket upgrades enabled for this route and its own idle timeout set long enough; see Reverse proxy. The proxied socket is a hijacked connection, which http.Server.Shutdown does not wait for: on a restart of nvrd the sockets close and the players reconnect, within the limits below.

nvrd sends no Access-Control-* headers on any route. It does not need them, and their absence is part of the security model:

  • The web app is served by nvrd itself from the embedded bundle, so every call it makes is same-origin.
  • A page on another origin can make the browser send a request, but cannot read the response. It can trigger a /stream-token request, and never sees the token.
  • Writes (POST, PUT, PATCH, DELETE) from a cookie session whose Origin or Referer names another host are refused with 403 (csrfGuard in internal/api/ext_broker.go).
  • WebSockets are not covered by CORS; any page may open one to any host. That is why the live socket ignores the cookie and requires a credential in the URL. A first-party page gets a stream token through a response other origins cannot read. A client on another origin that has its own credential, a media:live API token or the live key, connects to the socket directly and needs no readable cross-origin call first.

The player in web/src/lib/livestream.ts speaks go2rtc’s WebSocket protocol directly, with no third-party player library. Text frames are JSON objects with a type and a value:

Direction type value
browser to go2rtc webrtc/offer the SDP offer
browser to go2rtc webrtc/candidate one ICE candidate string, sent as the browser gathers it
go2rtc to browser webrtc/answer the SDP answer
go2rtc to browser webrtc/candidate one of go2rtc’s ICE candidates
browser to go2rtc mse the comma-separated codecs this browser’s MediaSource supports
go2rtc to browser mse the MIME type of the stream it will send
go2rtc to browser error a reason; ends the attempt

In MSE mode, go2rtc then sends the video as binary frames of fragmented MP4.

{"type": "webrtc/offer", "value": "v=0\r\no=- …"}
{"type": "mse", "value": "avc1.640029,avc1.64002A,avc1.640033,mp4a.40.2,mp4a.40.5,opus"}

connectWebRTC creates an RTCPeerConnection with iceServers: [] and bundlePolicy: "max-bundle", and adds one receive-only video and one receive-only audio transceiver. With no STUN or TURN servers, the browser offers only its own local (host) candidates. go2rtc’s candidates come from the webrtc.candidates list in its configuration; the installer seeds stun:8555, which lets go2rtc discover a routable address for its port 8555.

When the socket opens, the player creates the offer, sets it locally and sends it. Candidates trickle both ways. The connection counts as playing when connectionState reaches connected. The player falls back to MSE when, before that:

  • connectionState becomes failed,
  • the socket errors or go2rtc sends error,
  • creating the offer or applying the answer throws, or
  • 5 seconds pass without reaching connected.

There is no TURN relay. If the browser and go2rtc cannot reach each other directly, for example from outside your network through a reverse proxy that forwards only HTTP, WebRTC does not connect and the player uses MSE.

connectMSE opens a new WebSocket to the same URL, so the token must still be valid. Once it is open, the player creates a ManagedMediaSource when the browser provides one and a MediaSource otherwise, and on sourceopen sends the codecs it supports, probed from this list with isTypeSupported:

Codec string Meaning
avc1.640029, avc1.64002A, avc1.640033 H.264 High
hvc1.1.6.L153.B0 H.265
mp4a.40.2, mp4a.40.5 AAC
flac, opus FLAC, Opus

go2rtc answers with the MIME type; the player adds a SourceBuffer in segments mode and appends each binary frame through a queue. The first binary frame marks the stream as playing. If no data arrives within 9 seconds, or the socket fails before any data, the player gives up and the tile shows No signal.

A LiveStream heals a dropped or frozen feed without help from the page:

Mechanism Value Behavior
Freeze watchdog checks every 2.5 s, STALL_MS = 6000 ms If the video is playing, not paused, and currentTime has not advanced for 6 seconds, reconnect. A video you paused is left alone.
WebRTC drop connectionState failed, or closed after playing Reconnect. disconnected is left to the watchdog, because ICE often recovers by itself.
Backoff min(400 × 2^retries, 15000) ms 400, 800, 1600, 3200, 6400, 12800, then 15000 ms between attempts. A successful connection resets it.
Restart order Every reconnect starts with WebRTC again, then falls back to MSE.
Failed reconnect WebRTC and then MSE fail Once the player has shown video, it schedules the next reconnect and keeps trying for as long as it is open. Only a first connection that fails ends in No signal.
Stream token before every socket Each socket opens with a token that is still valid (see Opening a socket), so a reconnect long after the first connection is not refused.

A reconnect closes the old socket and peer connection first and ignores the events they fire while closing. The last picture stays on the element, under the placeholder, so the tile does not flash black.

Mobile browsers and installed web apps freeze WebRTC and MSE connections in the background. onLiveResume in web/src/lib/liveResume.ts fires its subscribers when the app comes back: on visibilitychange to visible, pageshow (back-forward cache and iOS resume), window focus, and online. A burst of these events within 60 ms is collapsed into one call.

On that call, each LiveStream calls play() and checks again after 700 ms; if the picture still has not moved, it reconnects at once instead of waiting 6 seconds for the watchdog. A player showing No signal tries again. The stream-token cache is cleared on the same call, because a device that slept may have slept past a token’s expiry. The recorded-playback player in Playback subscribes to the same coordinator and rebuilds its HLS player when it has not recovered after 700 ms.

web/src/lib/streamCache.ts keeps the connected WebRTC MediaStream of each camera, keyed by camera id and quality (<id>:main, <id>:sub). One MediaStream can feed several <video> elements, so a second view of the same camera, such as the pop-out over a dashboard tile, attaches the existing stream for an immediate picture and opens no second connection.

  • Only WebRTC streams are cached; an MSE stream has no shareable MediaStream.
  • A stream whose video tracks have all ended is dropped when it is read.
  • A borrowed stream has no owner to heal it, and after a background suspend a track can report live and deliver nothing. On resume, LiveVideo checks a borrowed stream after 700 ms; if it has not advanced, it clears that camera from the cache and opens a fresh connection.

While Playback is at the live edge, warmLive in web/src/lib/liveWarm.ts keeps one WebRTC connection open for the selected camera in a hidden 2 px video element and registers it in the stream cache. Returning from recorded footage to live then attaches that stream instead of negotiating again. The element is transparent, not display: none, because a hidden element stops decoding. Warming stops while you watch recorded footage, so it does not compete with HLS playback for CPU and bandwidth.

LiveVideo in web/src/components/LiveVideo.tsx draws, from bottom to top:

  • the <video> element;
  • until a real frame is presented (requestVideoFrameCallback, or 80 ms after loadeddata or playing), the camera’s recent snapshot as a placeholder, which then fades out over 300 ms; connecting dots appear only when no snapshot is available;
  • No signal when both transports failed, or Disabled for a disabled camera;
  • Offline over everything when the server reports the camera offline.

Offline comes from the server’s watchdog, not the player. A camera is online while go2rtc’s received-byte count for its streams grows between checks. A camera that something reads counts as offline after 30 seconds with nothing arriving; one that nothing reads is asked with an RTSP DESCRIBE through go2rtc every minute and counts as offline after two failures in a row (observe and runProbes in internal/watchdog/watchdog.go). The overlay is shown because a frozen last frame or a snapshot looks live when it is not.

Boxes around recognized objects are not part of the video. They come from objects.live messages on the change feed (GET /api/v1/feed) and are drawn over the tile; each box disappears 1.5 seconds after its last update (OBJECT_TTL in web/src/lib/liveState.ts).

The quality of a LiveVideo decides the stream: low asks for cam_<id>_sub, and auto and high both ask for cam_<id>. Auto therefore currently plays the main stream, the same as High (HD).

Where Stream
Desktop Dashboard tiles, the pop-out, Devices main
Playback at the live edge and its multi-camera grid the choice in the quality menu in the control bar: High (HD), Auto or Low (SD)
Phone dashboard, List View main
Phone dashboard, Grid View and Compact Grid sub

Watching the main stream usually adds no load on the camera: when the camera records its main stream, the recorder already keeps that producer running. The sub stream is opened lazily: go2rtc dials the camera only when a client asks for it. Motion detection reads the sub stream when there is one, so on a camera with motion detection it is usually running already. Without a sub link, every view gets the main stream, and a grid of high-resolution main streams decodes a lot in the browser; adding sub links is the fix (see Cameras).

GET /api/v1/cameras/{id}/snapshot.jpg (media:read for a token, plus camera access) returns one JPEG. It is served by Manager.SnapshotCached:

  • A frame cached less than 20 seconds ago is returned at once.
  • Otherwise nvrd fetches a fresh one from go2rtc’s /api/frame.jpeg?src=cam_<id>, which go2rtc decodes from the stream, and caches it.
  • If go2rtc fails and an older frame is cached, the old frame is returned rather than an error. With nothing cached, the route answers 503 (snapshot unavailable).

A background loop in cmd/nvrd/main.go refreshes the cache for every enabled camera every 15 seconds (each round bounded to 10 seconds), so a frame is almost always ready. Responses carry Cache-Control: no-store.

Terminal window
curl -H "Authorization: Bearer apt_…" -o front_door.jpg \
http://192.168.1.10:8971/api/v1/cameras/<id>/snapshot.jpg

The same cached frame is used for the live placeholder, the Home Assistant snapshot entity (internal/ha), the ONVIF snapshot (internal/onvif), and motion and audio event thumbnails (captureThumb). The frame burst sent to an external detection server takes fresh frames from the main stream instead (Snapshot). SnapshotSub grabs a frame from cam_<id>_sub; the detection test falls back between the two streams, because some main streams, such as dual-lens cameras with two video tracks, cannot be turned into a single picture while their sub stream can.

GET /api/v1/cameras/{id}/frame?t=<unix ms> is not a live snapshot. It extracts a 320 px wide JPEG from recorded footage for the hover previews on the timeline (handleFrame in internal/api/recordings_handlers.go). See Recording pipeline.

  • WebRTC has the lowest latency. The player adds no buffering of its own.
  • MSE keeps at most about 5 seconds buffered. After each append the player removes everything older than 5 seconds behind the newest buffered data, and if the playhead has fallen behind that point it jumps to 0.4 seconds before the end. Expect more delay than WebRTC, and a jump forward after a stall.
  • Start-up is hidden rather than removed: the cached snapshot appears at once, a stream already open in the same tab is reused, and Playback keeps the live stream of the selected camera warm. A camera whose producer is idle, such as a sub stream nobody reads, needs go2rtc to connect to the camera first.
  • A player whose first connection fails (WebRTC and then MSE) shows No signal. It tries again when the app comes back to the foreground or the network returns, or when you open it again. A player that has shown video keeps reconnecting by itself.
  • WebRTC has no TURN relay. Outside your network it connects only when the browser can reach go2rtc’s port 8555; otherwise MSE carries the video through nvrd, with more delay and more load on the server.
  • The RTSP restream on port 8554 has no login in the seeded configuration.
  • A slug depends on the camera’s name. Two cameras whose names slugify alike resolve to the first in list order, and renaming a camera breaks any address that used its old slug.

Related: Live view, How it works, Reverse proxy, API scopes.