Live streaming
This page follows live video from the camera to your screen. It covers how nvrd registers each camera in go2rtc, how
the browser’s WebSocket is authorized and proxied, how the player chooses WebRTC or MSE and recovers from drops, and
where snapshots come from. It is written for developers and for anyone debugging live view; the everyday side is in
Live view.
The path at a glance
Section titled “The path at a glance” go2rtc (aperture-go2rtc.service)camera ──RTSP──────────> producer per stream: cam_<id>, cam_<id>_sub │ ├── RTSP restream :8554 ──> recorder, motion, audio, external players ├── HTTP API 127.0.0.1:1984 <── nvrd only (register streams, frame.jpeg, /api/ws) └── WebRTC media :8555 <══════════════> browser (when ICE connects)
browser ──ws(s)──> nvrd GET /api/v1/cameras/{id}/ws?token=… ──ws──> go2rtc /api/ws?src=cam_<id> (resolve camera, check token, drop Origin) (signalling, MSE data)- go2rtc holds the only connection to each camera stream and fans it out. A camera sees one client for its main stream and, when a sub stream is set and something reads it, one for that, however many people watch.
- The browser never reaches go2rtc’s HTTP API. Every live socket goes to
nvrd, which authorizes it and proxies it to go2rtc. - Only the signalling passes through
nvrd. With WebRTC, the video itself flows directly between the browser and go2rtc’s WebRTC port 8555. With MSE, the video travels inside the WebSocket, throughnvrd.
Streams in go2rtc
Section titled “Streams in go2rtc”nvrd addresses go2rtc streams by name. The names come from StreamName and SubStreamName in
internal/stream/manager.go. A camera
id is 32 hex characters (store.NewID).
| Stream | Source | Registered when | Read by |
|---|---|---|---|
cam_<id> |
the camera’s main link (rtsp_main_url) |
the camera is enabled and has a main link | the recorder, audio detection, motion detection and local object recognition when there is no sub stream, live view at Auto and High (HD), snapshots, the RTSP restream for other players |
cam_<id>_sub |
the camera’s sub link | the camera is enabled and has a sub link | live view at Low (SD), motion detection and local object recognition (which receives its frames from the motion decoder), the recorder when the recording quality preference asks for the sub stream |
test_<random id> |
a link being tested | for one frame grab, then deleted | Test connection in the camera editor |
The recorder, motion and audio detection read the RTSP restream at APERTURE_GO2RTC_RTSP (RTSPURL, RTSPURLSub), so
they are consumers of go2rtc like any browser. Other players use the same restream at rtsp://<host>:8554/cam_<id>,
where the host is APERTURE_STREAM_HOST if set and otherwise the host the request came in on (advertisedRTSPHost in
internal/api/ha_handlers.go). See ONVIF and RTSP for using it.
Registration and reconciliation
Section titled “Registration and reconciliation”Manager.Sync reconciles go2rtc with the camera table:
- It reads go2rtc’s current streams with
GET /api/streams. - For every enabled camera with a main link it registers
cam_<id>, andcam_<id>_subwhen a sub link is set, withPUT /api/streams?name=<name>&src=<link>. - It skips a stream whose registered producer URLs already match (
sameSources). Re-registering an unchanged stream drops and recreates the producer, which would interrupt every viewer, so renaming a camera or changing its retention leaves go2rtc alone. - It deletes every stream whose name starts with
cam_that is no longer wanted: a disabled or deleted camera, or a sub link that was removed (DELETE /api/streams?src=<name>). Streams with other names are not touched.
Sync runs at startup, every 60 seconds (the reconcile loop in cmd/nvrd/main.go, each pass bounded to 15 seconds),
and in the background after each camera change (syncStreams in internal/api/camera_handlers.go). If go2rtc restarts
and loses its streams, live view comes back within a minute without restarting nvrd.
Manager.Restart re-registers a camera’s main stream even when nothing changed, which tears down a wedged producer.
The camera watchdog (internal/watchdog/watchdog.go, every 15 seconds) calls it when a camera that should be recording
has written no segment for 90 seconds (store.RecordingGraceMs) and the recorder has stopped writing, at most once per
camera per minute. It restarts the camera’s recorder as well. Live viewers of that stream lose their connection and
the player reconnects (see Reconnecting and Limits).
Links are normalized before they are saved (NormalizeSource in internal/stream/protect.go). A UniFi Protect link on
port 7441, rtsps://<console>:7441/<key>?enableSrtp, becomes rtspx://<console>:7441/<key>: go2rtc’s RTSPS without a
certificate check, because the console’s certificate is self-signed, and without enableSrtp, which only Protect’s own
apps use. See UniFi Protect.
Testing a link before it is saved
Section titled “Testing a link before it is saved”Test connection calls POST /api/v1/cameras/test (admin; cameras:write for a token). Manager.TestStream
accepts only rtsp://, rtsps:// and rtspx:// links. It registers the link as test_<random id>, waits 800 ms for
go2rtc to dial the camera, fetches one JPEG from /api/frame.jpeg (at most 8 MiB) and deletes the temporary stream,
even when the request was cancelled. The handler allows 12 seconds in total. When an existing camera’s link arrives with
its login masked, the saved login is filled back in on the server; errors are scrubbed of credentials before they are
returned.
Where go2rtc listens
Section titled “Where go2rtc listens”The installer (deploy/install.sh) seeds /var/lib/aperture/go2rtc.yaml with:
api: listen: "127.0.0.1:1984"rtsp: listen: ":8554"webrtc: listen: ":8555" candidates: - stun:8555log: level: info streams: error| Port | Bound to | Purpose |
|---|---|---|
| 1984 | loopback only | go2rtc’s HTTP API. It hands every camera’s source link, password included, to whoever can reach it, so only nvrd on the same machine uses it. |
| 8554 | all interfaces | The RTSP restream used by the recorder, the detectors and external players. |
| 8555 | all interfaces | WebRTC media between go2rtc and browsers. |
go2rtc writes the streams nvrd registers back into that file, which is why it holds camera passwords and is mode
0600, owned by the aperture-go2rtc user. The unit aperture-go2rtc.service runs
/usr/lib/aperture/bin/go2rtc -config /var/lib/aperture/go2rtc.yaml with Restart=always, RestartSec=2 and
MemoryMax=1G. The streams log level is kept at error because go2rtc’s reconnect warnings include the full source
link.
The live WebSocket
Section titled “The live WebSocket”Opening a socket
Section titled “Opening a socket”The web app opens a camera in two steps.
-
It asks for a stream token:
GET /api/v1/cameras/{id}/stream-token. The route requiresmedia:livefor an API token and access to that camera (scopedCam(ScopeMediaLive)ininternal/api/server.go). The reply:{"expiresIn": 300,"hasSub": true,"token": "<stream token>","wsUrl": "/api/v1/cameras/<id>/ws?token=<stream token>"} -
It opens
ws://orwss://(matching the page) on its own host with thatwsUrl, appending&q=subfor the sub stream.
hasSub says whether the camera has a sub link, so the app can offer low quality. q=sub on a camera without one falls
back to the main stream. expiresIn is how many seconds the token can still open a socket. A socket that is already
open stays open after that.
Every new socket needs a token that is still valid: the first one, the MSE fallback after WebRTC fails, and every
reconnect. liveUrlSource in
web/src/lib/streamToken.ts supplies
the address before each socket. It reuses a cached token only while more than 60 seconds of its expiresIn are left,
and asks for a new one otherwise. Concurrent requests for one camera share a fetch. A socket refused before it opened
makes the next one ask for a new token. The whole cache is dropped when the app returns to the foreground.
GET /api/v1/cameras/{id}/ws is declared pub in the route table: the generic access check does nothing, and
handleStreamWS in internal/api/stream_handlers.go
does its own authorization. It answers 404 (camera not found) for an unknown camera and 401
(invalid stream token) for a bad credential.
Authorization
Section titled “Authorization”A browser cannot set an Authorization header on a WebSocket handshake, so the socket takes its credential from the
token query parameter, and only from there. A session cookie or a bearer header on the handshake is not consulted.
authorizeStreamWS accepts three kinds of credential, tried in this order:
| Credential | What it is | Grants |
|---|---|---|
| Stream token | An HS256 JWT minted by /stream-token (auth.Manager.StreamToken), signed with the stream secret, carrying the camera id, issued-at and an expiry 5 minutes later (streamTokenTTL). |
That one camera, and only if the id in the token matches the camera in the URL. |
| Live key | A single server-managed secret kept in the settings table as integrations.live_key (prefix aplv_). It is created the first time an admin reads it and can be rerolled, which ends the old value at once. Compared in constant time. |
Live view of every camera, and nothing else. |
API token with media:live |
A scoped token (apt_…) passed as token. Its claims are attached to the request. |
Cameras the token may see: its owner’s cameras, narrowed to the token’s camera grants if it has any (allowedCamera). |
The stream secret comes from APERTURE_STREAM_SECRET, or is generated once and kept in secrets.json in the data
directory (loadSecrets in internal/config/config.go). The expiry is checked when the socket opens; an open socket is
not cut when its token expires.
Camera id or slug
Section titled “Camera id or slug”The {id} in the socket URL may be the camera id or a slug of its name (resolveCameraID in
internal/api/integrations_live.go). The id is tried first, as an indexed lookup. Otherwise the value is slugified and
compared against each camera’s slugified name, in camera list order (sort_order, then name); the first match wins.
slugify lowercases the name, collapses each run of characters other than a–z and 0–9 into one underscore,
and trims underscores: “Front door” becomes front_door.
The slug is resolved on every request, so it always follows the camera’s current name; renaming a camera changes its
slug. Only the socket route accepts slugs; /stream-token and the other camera routes take the id.
What nvrd does with the socket
Section titled “What nvrd does with the socket”After authorization, handleStreamWS:
- Picks the stream:
cam_<id>, orcam_<id>_subforq=subwhen the camera has a sub link. - Registers a viewer with object recognition (
detect.Engine.Watch) for as long as the socket is open. For a camera that uses local object recognition, frames are then examined even while nothing moves: every 700 ms on a CPU, every 200 ms on CUDA (watchIntervalininternal/detect/detect.go), so a person standing still in view gets a box. An open live view therefore costs recognition time. See Detection pipeline. - Hands the request to
Manager.ProxyWS, which rewrites the path to go2rtc’s/api/ws, replaces the whole query withsrc=<stream name>(the credential never reaches go2rtc), sets the host to go2rtc’s, deletes theOriginheader and passes the request to anhttputil.ReverseProxy.
go2rtc’s WebSocket upgrader rejects cross-origin requests, and the browser’s origin is nvrd’s public address, not
go2rtc’s. Deleting Origin presents the proxied request as same-origin; nvrd has already decided who may connect.
The HTTP server sets no read or write timeout (NewHTTPServer in internal/api/shutdown.go), so a socket can stay open
as long as both ends keep it. A reverse proxy in front needs WebSocket upgrades enabled for this route and its own idle
timeout set long enough; see Reverse proxy. The proxied socket is a hijacked connection, which
http.Server.Shutdown does not wait for: on a restart of nvrd the sockets close and the players reconnect, within
the limits below.
Why there is no CORS
Section titled “Why there is no CORS”nvrd sends no Access-Control-* headers on any route. It does not need them, and their absence is part of the
security model:
- The web app is served by
nvrditself from the embedded bundle, so every call it makes is same-origin. - A page on another origin can make the browser send a request, but cannot read the response. It can trigger a
/stream-tokenrequest, and never sees the token. - Writes (
POST,PUT,PATCH,DELETE) from a cookie session whoseOriginorReferernames another host are refused with403(csrfGuardininternal/api/ext_broker.go). - WebSockets are not covered by CORS; any page may open one to any host. That is why the live socket ignores the cookie
and requires a credential in the URL. A first-party page gets a stream token through a response other origins cannot
read. A client on another origin that has its own credential, a
media:liveAPI token or the live key, connects to the socket directly and needs no readable cross-origin call first.
Signalling and transports
Section titled “Signalling and transports”Messages
Section titled “Messages”The player in web/src/lib/livestream.ts
speaks go2rtc’s WebSocket protocol directly, with no third-party player library. Text frames are JSON objects with a
type and a value:
| Direction | type |
value |
|---|---|---|
| browser to go2rtc | webrtc/offer |
the SDP offer |
| browser to go2rtc | webrtc/candidate |
one ICE candidate string, sent as the browser gathers it |
| go2rtc to browser | webrtc/answer |
the SDP answer |
| go2rtc to browser | webrtc/candidate |
one of go2rtc’s ICE candidates |
| browser to go2rtc | mse |
the comma-separated codecs this browser’s MediaSource supports |
| go2rtc to browser | mse |
the MIME type of the stream it will send |
| go2rtc to browser | error |
a reason; ends the attempt |
In MSE mode, go2rtc then sends the video as binary frames of fragmented MP4.
{"type": "webrtc/offer", "value": "v=0\r\no=- …"}{"type": "mse", "value": "avc1.640029,avc1.64002A,avc1.640033,mp4a.40.2,mp4a.40.5,opus"}WebRTC first
Section titled “WebRTC first”connectWebRTC creates an RTCPeerConnection with iceServers: [] and bundlePolicy: "max-bundle", and adds one
receive-only video and one receive-only audio transceiver. With no STUN or TURN servers, the browser offers only its own
local (host) candidates. go2rtc’s candidates come from the webrtc.candidates list in its configuration; the installer
seeds stun:8555, which lets go2rtc discover a routable address for its port 8555.
When the socket opens, the player creates the offer, sets it locally and sends it. Candidates trickle both ways. The
connection counts as playing when connectionState reaches connected. The player falls back to MSE when, before that:
connectionStatebecomesfailed,- the socket errors or go2rtc sends
error, - creating the offer or applying the answer throws, or
- 5 seconds pass without reaching
connected.
There is no TURN relay. If the browser and go2rtc cannot reach each other directly, for example from outside your network through a reverse proxy that forwards only HTTP, WebRTC does not connect and the player uses MSE.
MSE fallback
Section titled “MSE fallback”connectMSE opens a new WebSocket to the same URL, so the token must still be valid. Once it is open, the player
creates a ManagedMediaSource when the browser provides one and a MediaSource otherwise, and on sourceopen sends
the codecs it supports, probed from this list with isTypeSupported:
| Codec string | Meaning |
|---|---|
avc1.640029, avc1.64002A, avc1.640033 |
H.264 High |
hvc1.1.6.L153.B0 |
H.265 |
mp4a.40.2, mp4a.40.5 |
AAC |
flac, opus |
FLAC, Opus |
go2rtc answers with the MIME type; the player adds a SourceBuffer in segments mode and appends each binary frame
through a queue. The first binary frame marks the stream as playing. If no data arrives within 9 seconds, or the socket
fails before any data, the player gives up and the tile shows No signal.
The player in the browser
Section titled “The player in the browser”Reconnecting
Section titled “Reconnecting”A LiveStream heals a dropped or frozen feed without help from the page:
| Mechanism | Value | Behavior |
|---|---|---|
| Freeze watchdog | checks every 2.5 s, STALL_MS = 6000 ms |
If the video is playing, not paused, and currentTime has not advanced for 6 seconds, reconnect. A video you paused is left alone. |
| WebRTC drop | connectionState failed, or closed after playing |
Reconnect. disconnected is left to the watchdog, because ICE often recovers by itself. |
| Backoff | min(400 × 2^retries, 15000) ms |
400, 800, 1600, 3200, 6400, 12800, then 15000 ms between attempts. A successful connection resets it. |
| Restart order | Every reconnect starts with WebRTC again, then falls back to MSE. | |
| Failed reconnect | WebRTC and then MSE fail | Once the player has shown video, it schedules the next reconnect and keeps trying for as long as it is open. Only a first connection that fails ends in No signal. |
| Stream token | before every socket | Each socket opens with a token that is still valid (see Opening a socket), so a reconnect long after the first connection is not refused. |
A reconnect closes the old socket and peer connection first and ignores the events they fire while closing. The last picture stays on the element, under the placeholder, so the tile does not flash black.
Resume coordinator
Section titled “Resume coordinator”Mobile browsers and installed web apps freeze WebRTC and MSE connections in the background. onLiveResume in
web/src/lib/liveResume.ts fires its
subscribers when the app comes back: on visibilitychange to visible, pageshow (back-forward cache and iOS resume),
window focus, and online. A burst of these events within 60 ms is collapsed into one call.
On that call, each LiveStream calls play() and checks again after 700 ms; if the picture still has not moved, it
reconnects at once instead of waiting 6 seconds for the watchdog. A player showing No signal tries again. The
stream-token cache is cleared on the same call, because a device that slept may have slept past a token’s expiry. The recorded-playback player in Playback
subscribes to the same coordinator and rebuilds its HLS player when it has not recovered after 700 ms.
Shared streams
Section titled “Shared streams”web/src/lib/streamCache.ts keeps the
connected WebRTC MediaStream of each camera, keyed by camera id and quality (<id>:main, <id>:sub). One
MediaStream can feed several <video> elements, so a second view of the same camera, such as the pop-out over a
dashboard tile, attaches the existing stream for an immediate picture and opens no second connection.
- Only WebRTC streams are cached; an MSE stream has no shareable
MediaStream. - A stream whose video tracks have all ended is dropped when it is read.
- A borrowed stream has no owner to heal it, and after a background suspend a track can report
liveand deliver nothing. On resume,LiveVideochecks a borrowed stream after 700 ms; if it has not advanced, it clears that camera from the cache and opens a fresh connection.
The warm stream in Playback
Section titled “The warm stream in Playback”While Playback is at the live edge, warmLive in web/src/lib/liveWarm.ts keeps one WebRTC connection open for the
selected camera in a hidden 2 px video element and registers it in the stream cache. Returning from recorded footage to
live then attaches that stream instead of negotiating again. The element is transparent, not display: none, because
a hidden element stops decoding. Warming stops while you watch recorded footage, so it does not compete with HLS
playback for CPU and bandwidth.
What a tile shows
Section titled “What a tile shows”LiveVideo in
web/src/components/LiveVideo.tsx
draws, from bottom to top:
- the
<video>element; - until a real frame is presented (
requestVideoFrameCallback, or 80 ms afterloadeddataorplaying), the camera’s recent snapshot as a placeholder, which then fades out over 300 ms; connecting dots appear only when no snapshot is available; - No signal when both transports failed, or Disabled for a disabled camera;
- Offline over everything when the server reports the camera offline.
Offline comes from the server’s watchdog, not the player. A camera is online while go2rtc’s received-byte count for its
streams grows between checks. A camera that something reads counts as offline after 30 seconds with nothing arriving;
one that nothing reads is asked with an RTSP DESCRIBE through go2rtc every minute and counts as offline after two
failures in a row (observe and runProbes in internal/watchdog/watchdog.go). The overlay is shown because a frozen
last frame or a snapshot looks live when it is not.
Boxes around recognized objects are not part of the video. They come from objects.live messages on the change feed
(GET /api/v1/feed) and are drawn over the tile; each box disappears 1.5 seconds after its last update (OBJECT_TTL in
web/src/lib/liveState.ts).
Main or sub stream
Section titled “Main or sub stream”The quality of a LiveVideo decides the stream: low asks for cam_<id>_sub, and auto and high both ask for
cam_<id>. Auto therefore currently plays the main stream, the same as High (HD).
| Where | Stream |
|---|---|
| Desktop Dashboard tiles, the pop-out, Devices | main |
| Playback at the live edge and its multi-camera grid | the choice in the quality menu in the control bar: High (HD), Auto or Low (SD) |
| Phone dashboard, List View | main |
| Phone dashboard, Grid View and Compact Grid | sub |
Watching the main stream usually adds no load on the camera: when the camera records its main stream, the recorder already keeps that producer running. The sub stream is opened lazily: go2rtc dials the camera only when a client asks for it. Motion detection reads the sub stream when there is one, so on a camera with motion detection it is usually running already. Without a sub link, every view gets the main stream, and a grid of high-resolution main streams decodes a lot in the browser; adding sub links is the fix (see Cameras).
Snapshots
Section titled “Snapshots”The snapshot route
Section titled “The snapshot route”GET /api/v1/cameras/{id}/snapshot.jpg (media:read for a token, plus camera access) returns one JPEG. It is served by
Manager.SnapshotCached:
- A frame cached less than 20 seconds ago is returned at once.
- Otherwise
nvrdfetches a fresh one from go2rtc’s/api/frame.jpeg?src=cam_<id>, which go2rtc decodes from the stream, and caches it. - If go2rtc fails and an older frame is cached, the old frame is returned rather than an error. With nothing cached,
the route answers
503(snapshot unavailable).
A background loop in cmd/nvrd/main.go refreshes the cache for every enabled camera every 15 seconds (each round bounded
to 10 seconds), so a frame is almost always ready. Responses carry Cache-Control: no-store.
curl -H "Authorization: Bearer apt_…" -o front_door.jpg \ http://192.168.1.10:8971/api/v1/cameras/<id>/snapshot.jpgOther consumers
Section titled “Other consumers”The same cached frame is used for the live placeholder, the Home Assistant snapshot entity (internal/ha), the ONVIF
snapshot (internal/onvif), and motion and audio event thumbnails (captureThumb). The frame burst sent to an external
detection server takes fresh frames from the main stream instead (Snapshot). SnapshotSub grabs a frame from
cam_<id>_sub; the detection test falls back between the two streams, because some main streams, such as dual-lens
cameras with two video tracks, cannot be turned into a single picture while their sub stream can.
Recorded frames are different
Section titled “Recorded frames are different”GET /api/v1/cameras/{id}/frame?t=<unix ms> is not a live snapshot. It extracts a 320 px wide JPEG from recorded footage
for the hover previews on the timeline (handleFrame in internal/api/recordings_handlers.go). See
Recording pipeline.
Latency
Section titled “Latency”- WebRTC has the lowest latency. The player adds no buffering of its own.
- MSE keeps at most about 5 seconds buffered. After each append the player removes everything older than 5 seconds behind the newest buffered data, and if the playhead has fallen behind that point it jumps to 0.4 seconds before the end. Expect more delay than WebRTC, and a jump forward after a stall.
- Start-up is hidden rather than removed: the cached snapshot appears at once, a stream already open in the same tab is reused, and Playback keeps the live stream of the selected camera warm. A camera whose producer is idle, such as a sub stream nobody reads, needs go2rtc to connect to the camera first.
Limits
Section titled “Limits”- A player whose first connection fails (WebRTC and then MSE) shows No signal. It tries again when the app comes back to the foreground or the network returns, or when you open it again. A player that has shown video keeps reconnecting by itself.
- WebRTC has no TURN relay. Outside your network it connects only when the browser can reach go2rtc’s port 8555;
otherwise MSE carries the video through
nvrd, with more delay and more load on the server. - The RTSP restream on port 8554 has no login in the seeded configuration.
- A slug depends on the camera’s name. Two cameras whose names slugify alike resolve to the first in list order, and renaming a camera breaks any address that used its old slug.
Related: Live view, How it works, Reverse proxy, API scopes.