# SFU / LiveKit smoke test — S0 report

> **Date:** 13.11.2026 · **Scope:** S0 spike (`docs/sfu-plan.md` §4) ·
> **Status:** signaling/REST **PASS**, media/ICE **not exercised**, UDP
> reachability spot-checked.
>
> Test host: `root@95.182.94.7` (Ubuntu 24.04, kernel 6.8.0-35, 1 vCPU,
> 1968 MiB RAM, 19 GB disk / 3.0 GB free, Docker 29.1.3, Compose 2.40.3).
> The test ran in a **separate directory** `/opt/agil-sfu`; the production app
> `/opt/agil-calls` and its compose were not modified. No secrets from this run
> are stored in the repo or in this report.

## 0. TL;DR

- LiveKit **v1.13.11** + Redis **7-alpine** started and became healthy in a
  separate compose project; **10/10 smoke checks passed** (health, HS256 token,
  token validation, room create/list, two WebSocket participants, participant
  list, ping/pong, room delete).
- Localhost signaling timings are low (WS join 2–4 ms, ping RTT ~5 ms);
  external runs are dominated by ~160–170 ms WAN RTT.
- TCP **7880/7881** are reachable externally. UDP ports **50000, 50001, 50049,
  50050, 50100** answered an external UDP probe individually; a rapid
  20-port/burst probe was dropped (likely anti-scan/rate limiting at the
  provider edge). **Actual ICE/DTLS/SRTP media was not tested** (no browser or
  WebRTC client on the box).
- Resource cost with two idle signaling participants: LiveKit **~37.6 MiB RAM /
  ~1% CPU**, Redis **~7.1 MiB**. No media was flowing, so this is not a
  capacity number.
- Stack was removed with `docker compose down -v`; `/opt/agil-sfu` was deleted,
  no containers/volumes/networks remain, app health is still `200`.

## 1. What was deployed

| Item | Value |
|---|---|
| Compose project | `agil-sfu` (separate from `agil-calls`) |
| Compose file | `deploy/livekit/docker-compose.yml` |
| LiveKit | `livekit/livekit-server:v1.13.11` · `sha256:25cd902de7cacbd5f1f779f6a6ac1eae20ba8f56b0935934ce6b0ab80d1e3d20` · 41.4 MB |
| Redis | `redis:7-alpine` · `sha256:7f7d30b7fdd09467bac58198a1f166dd69789d348d4fe859cccb20e0cf287fbe` · 16.4 MB |
| Egress | `livekit/egress:v1.15.0` pinned in the file, **profile `egress`, not pulled/run** |
| LiveKit node | `ND_9HT23Srrhmww`, node IP `95.182.94.7` |
| External IP | STUN (`global.stun.twilio.com:3478`) → `95.182.94.7`; `use_external_ip: true` worked |
| ICE range | `50000–50100/udp`, TCP fallback `7881` |
| Redis endpoint | `redis:6379` inside compose; loopback `127.0.0.1:6380` for host-networked egress |
| Metrics | `:6789` internal to compose, `prometheus.port` (new key), not published |

Port mappings actually created by Docker:

```
0.0.0.0:7880-7881->7880-7881/tcp
0.0.0.0:50000-50100->50000-50100/udp
127.0.0.1:6380->6379/tcp
```

Relevant compose excerpt (full file: `deploy/livekit/docker-compose.yml`):

```yaml
services:
  livekit:
    image: livekit/livekit-server:v1.13.11
    command: ["--config", "/etc/livekit.yaml"]
    environment:
      LIVEKIT_KEYS: "${LIVEKIT_API_KEY}: ${LIVEKIT_API_SECRET}"
    ports:
      - "7880:7880/tcp"
      - "7881:7881/tcp"
      - "50000-50100:50000-50100/udp"
  redis:
    image: redis:7-alpine
    ports: ["127.0.0.1:6380:6379"]
```

```bash
# only the media plane; egress stays off unless the profile is requested
docker compose up -d livekit redis
docker compose --profile egress up -d egress
docker compose down -v
```

Keys were generated for this test only: a random 32-byte hex secret, injected
as `LIVEKIT_KEYS` via `LIVEKIT_API_KEY` / `LIVEKIT_API_SECRET`, and removed with
`/opt/agil-sfu`. The repo scaffold contains no real key; `livekit.yaml` has no
`keys:` block at all.

## 2. Commands used

```bash
# 1) separate directory (app compose untouched)
mkdir -p /opt/agil-sfu
tar -xzf /tmp/agil-sfu-deploy.tgz -C /opt/agil-sfu

# 2) throwaway keys, env only
KEY=agil_sfu_smoke
SECRET=$(openssl rand -hex 32)
printf 'LIVEKIT_API_KEY=%s\nLIVEKIT_API_SECRET=%s\n' "$KEY" "$SECRET" > /opt/agil-sfu/.env
chmod 600 /opt/agil-sfu/.env

# 3) start only livekit + redis (egress is behind a profile)
cd /opt/agil-sfu
docker compose up -d
docker compose ps
docker compose logs --tail=100 livekit

# 4) smoke (Node is not installed on the host; script ran from the laptop
#    against the public port, and in a one-off node:23-alpine container with
#    host networking for localhost timings)
LIVEKIT_URL=http://95.182.94.7:7880 \
LIVEKIT_API_KEY="$KEY" LIVEKIT_API_SECRET="$SECRET" \
node scripts/sfu-smoke.mjs

docker run --rm --network host --env-file /opt/agil-sfu/.env \
  -e LIVEKIT_URL=http://127.0.0.1:7880 \
  -v /opt/agil-sfu/tools:/tools -w /tools \
  node:23-alpine node scripts/sfu-smoke.mjs

# 5) resource sample while two participants were held open
docker run -d --rm --name agil-sfu-smoke-hold --network host \
  --env-file /opt/agil-sfu/.env -e LIVEKIT_URL=http://127.0.0.1:7880 \
  -v /opt/agil-sfu/tools:/tools -w /tools \
  node:23-alpine node scripts/sfu-smoke.mjs --hold=15000
docker stats --no-stream

# 6) teardown
docker compose down -v
rm -rf /opt/agil-sfu
```

The smoke script is dependency-free apart from the repo's existing `ws`:
it creates HS256 JWTs with `node:crypto`, calls the Twirp REST API, opens two
WebSocket signaling clients at `/rtc?access_token=…`, inspects the initial
protobuf `JoinResponse`, confirms both identities via
`RoomService/ListParticipants`, does a JSON `ping`/`pong` round-trip, then
deletes the room. It never prints a token.

## 3. Smoke results

### 3.1 Checks (10/10 PASS)

| # | Check | Result |
|---|---|---|
| 1 | `GET /` health | HTTP 200 `OK` |
| 2 | HS256 admin token | 509 chars, 600 s TTL, `iss`/`sub`/`video` grants |
| 3 | `GET /rtc/validate` (participant token) | HTTP 200 `success` |
| 4 | `RoomService/CreateRoom` | room created, `sid` returned |
| 5 | `RoomService/ListRooms` | room present |
| 6 | WS participant A (`smoke-a-…`) | open + protobuf join frame (623 B) |
| 7 | WS participant B (`smoke-b-…`) | open + protobuf join frame (695 B) |
| 8 | `RoomService/ListParticipants` | exactly 2 identities in the room |
| 9 | signaling ping/pong | JSON text `pong` received |
| 10 | `RoomService/DeleteRoom` + list | deleted, list no longer contains it |

LiveKit logs independently confirm two `starting RTC session` entries with
`numParticipants: 0` then `1`, `ListParticipants` status 200, both
`CLIENT_REQUEST_LEAVE` removals, and `API_DELETE` closing the room.

### 3.2 Latency (ms)

| Step | Localhost | External (public IP) |
|---|---:|---:|
| health | 201 (cold first fetch) | 364–368 |
| token generation | <1 | <1 |
| token validate | 257 | 322–862 |
| room create | 73 | 188–192 |
| room list | 34 | 161–163 |
| WS participant A | 81 (open→join 4 ms) | 339–367 (open→join 1 ms) |
| WS participant B | 29 (open→join 2 ms) | 328–361 (open→join 16 ms) |
| ListParticipants (2) | 15 | 164–171 |
| ping/pong RTT | 5.1 | 158–171 |
| room delete | 360 | 656–661 |
| **whole smoke** | **1230** | **2802–3424** |

The external numbers are dominated by the ~160–170 ms WAN RTT (each REST/WS
step is a new TCP handshake). Server-side signaling work is small:
WS join 2–4 ms, ping 5 ms, REST after warm-up 15–73 ms.

## 4. Firewall / UDP findings

Host firewall is not the filter: `iptables -S` has `-P INPUT ACCEPT`, `nft
list ruleset` is empty, `ufw` is not active. The provider/edge firewall is the
authority and was probed from outside (`87.117.185.55`):

| Probe | Result |
|---|---|
| TCP 7880 | open (`Test-NetConnection` + full smoke) |
| TCP 7881 | open |
| UDP 50000 | echo received/replied |
| UDP 50001 | echo received/replied |
| UDP 50049 | echo received/replied (individual probe) |
| UDP 50050 | echo received/replied (individual probe) |
| UDP 50100 | echo received/replied (individual probe) |
| 20-port burst scan (50000–50010 + selected ports) | **no packets received** |

Conclusion: the UDP range is reachable when probed one port at a time, but
rapid multi-port probing was filtered/timed out — consistent with anti-scan or
rate limiting at the provider edge. This is a **risk for ICE**, which opens
several ports quickly, and it means **media over the full range is not proven**
by the echo test. Re-test with a real WebRTC client and provider firewall
rules in place; if the range stays unreliable, switch to the single-port mux
(`rtc.udp_port`, e.g. 7882/udp or 50000/udp) and keep `7881/tcp` as fallback.

Not tested: ICE connectivity, DTLS/SRTP, actual audio/video, TURN/TLS, packet
loss/jitter. The smoke only proves signaling, auth, room lifecycle and L3/L4
reachability of the ports.

## 5. Resource usage

`docker stats --no-stream` samples, all containers on the shared 1 vCPU host:

| State | LiveKit CPU | LiveKit RAM | Redis CPU | Redis RAM | Host used / available |
|---|---:|---:|---:|---:|---:|
| Idle, no rooms | 0.07% | 15.94 MiB | 1.00% | 7.11 MiB | 1108 / 859 MiB |
| 2 WS participants held 15 s | 1.00% | 37.62 MiB | 6.10%* | 7.14 MiB | 1150 / 817 MiB |
| After leave + room delete | 0.20% | 37.63 MiB | 1.20% | 7.14 MiB | 1137 / 831 MiB |

\* single snapshot; Redis CPU fluctuates (1–6%) and is not a media number.
LiveKit keeps its Go heap after rooms close (37.6 MiB vs 15.9 MiB idle) — not a
leak conclusion, just observed behaviour.

There was **no media** in this test. CPU/RAM for publishing/subscribing tracks
must be measured in S1/S7; `docs/scaling.md` §7 still gives the planning
figures (≈1–2k subscriptions per SFU node, CPU-bound). This host (1 vCPU,
1.9 GiB, already running 8 unrelated containers) is adequate for signaling
experiments only, not for media load or egress.

## 6. Blockers and risks

1. **No TLS / `wss://`.** LiveKit is plain HTTP/WS on 7880. Public browser
   WebRTC requires a secure context, so media cannot be validated from a
   browser until a TLS reverse proxy exists (`sfu.example.com` → 127.0.0.1:7880).
2. **TURN disabled.** No certificate/DNS was available; embedded TURN/TLS
   (5349 or 443) and its firewall rule remain untested. Corporate/Symmetric-NAT
   clients are not covered yet.
3. **Media/ICE not exercised.** No browser or `livekit-cli` on the host;
   Firefox/Chrome media automation was out of scope for S0. UDP echo only proves
   port reachability.
4. **UDP burst filtering.** A multi-port scan was dropped even though
   individual ports answered. This may affect ICE; verify with a real client
   and, if needed, use a single UDP mux port.
5. **Egress untested.** The image is pinned but not pulled; this host has only
   ~1.9 GiB RAM and 1 vCPU. Egress (Chrome + GStreamer) needs ≥4 GB per
   concurrent recording and a separate worker.
6. **No webhooks (S2), no Redis multi-node, no room-node routing.** The
   scaffold has Redis and a pinned egress, but orchestration is intentionally
   out of scope.
7. **Provider firewall is external.** Nothing on the host can change it; the
   UDP range must be opened/verified by the infrastructure owner before S1
   acceptance.
8. **Shared host.** The test box runs unrelated production containers; resource
   numbers and any packet filtering are affected by that environment.

## 7. Recommended next step for S1 (SFU media integration)

Order matters — do the TLS/UDP/media gate **before** writing the client engine:

1. **Network/TLS gate:** add `sfu.calls.example` DNS + Caddy/Nginx TLS in front
   of `127.0.0.1:7880`; open `7880/tcp`, `7881/tcp`, `50000–50100/udp` at the
   provider; verify `wss://sfu.../rtc` and ICE from a browser outside the host.
   Enable embedded TURN/TLS (5349/443) once a TURN cert exists.
2. **Media proof:** connect two real participants (browser or `livekit-cli`
   load-test) and confirm `chrome://webrtc-internals` shows a UDP candidate
   pair and non-zero `bytesReceived`; then repeat over a mobile/VPN network.
   If the range is filtered, test the single-port mux before changing app code.
3. **Server token contract:** in `POST /v1/rooms/:id/join` / `welcome`, add
   `media: { mode:'sfu', url, token, room, canPublish }`; mint a LiveKit JWT
   (HS256, `identity` = participantId, `room` = `tenantId_roomId`, TTL ≤ AGIL
   token TTL, `canPublish` by role) from `AGIL_SFU_API_KEY/SECRET`; add an SFU
   health check (`GET /` + Twirp) and gate it by `scale.mediaMode`.
4. **Client:** add `livekit-client` and a `SfuEngine` behind the current
   `WebRtcEngine` interface; switch on `mediaMode`; keep mesh fallback ≤8 and a
   clear “media server unavailable” state for >8.
5. **Then** S2 webhooks (`/v1/sfu/webhook`), and S3 egress on a larger,
   separate node with S3/MinIO in the required region.
6. Add this smoke to CI as the cheap REST/token/WS gate; add the real
   media/ICE test to S7 load/acceptance, not to this script.

## 8. Cleanup evidence

```
docker compose down -v   → containers, network agil-sfu_default, volume
                            agil-sfu_redis-data removed
docker ps -a --filter name=agil-sfu → empty
docker network/volume ls --filter name=agil-sfu → empty
rm -rf /opt/agil-sfu     → removed (including the temporary .env)
app: agil-calls-agil-calls-1 Up (healthy), /v1/health → 200
/opt/agil-calls mtime unchanged (2026-11-13 18:21:14 UTC)
```

Remaining on the host: the pinned `livekit/livekit-server:v1.13.11` and
`redis:7-alpine` images (~58 MB total), kept for the next S1 iteration; no
containers, volumes or networks are running. The `node:23-alpine` test-only
image was removed.

**Repro entry point:** `deploy/livekit/README.md` (quick start, ports,
host-network alternative, egress). Smoke: `scripts/sfu-smoke.mjs`.