Doppler

A small demo that makes a Pexip Infinity video call using the Pexip Pulse C API, with Dear ImGui as the application layer.

The goal of this repository is to show — with as little glue as possible — how to drive Pulse from your own application. Beyond placing a call, the demo also doubles as a tiny reference for three slightly more advanced Pulse building blocks lifted from pexninja/pexninja.cpp:

• In-window rendering of the incoming MAIN video and the self-view via pulse_data_session_connect_output + an OpenGL texture.
• An RTMP ingest listener via pulse_rtmp_session_connect_input.
• A "Twitch streaming" video mix that composites the local camera on top of the RTMP feed via pulse_video_mix_connect, with optional camera segmentation (PULSE_VIDEO_PROCESS_TYPE_SEGMENTATION).

The whole thing still fits in a single, heavily-commented src/main.cpp plus one small CMakeLists.txt.

┌─────────────────────────────────────────────┐
│         ImGui (GLFW + OpenGL3)              │
│                                             │
│  Server / Conference / ...   [Connect]      │
│  RTMP path / port            [Start RTMP]   │
│  [x] Camera segmentation     [Twitch mix]   │
│                                             │
│  ┌──────── Remote ────────┐  ┌─ Self-view ─┐│
│  │  (pulled from MAIN     │  │ (pulled    ││
│  │   data-session)        │  │  from      ││
│  │                        │  │  SELFVIEW) ││
│  └────────────────────────┘  └────────────┘│
└──────────────────────┬──────────────────────┘
                       │ pulse_connect_with_rest_async()
                       │ pulse_rtmp_session_connect_input()
                       │ pulse_video_mix_connect()
                       ▼
                ┌───────────────┐
                │  libpexpulse  │
                └───────────────┘

What's in the box

File	Purpose
src/main.cpp	The whole application — heavily commented.
src/gateway.cpp	A sibling app (pulse_gateway) that runs two Pulse instances and bridges raw audio + video between two conferences.
CMakeLists.txt	Build glue. Fetches Dear ImGui via FetchContent.
debs/	Pre-built Pulse .deb packages for Ubuntu 24.04.
opt/, usr/	The extracted contents of the Pulse .debs.
pexninja/	The much larger reference Pulse client (optional, see below).

Build

1. Install the Pexip Pulse runtime + headers

The Pulse packages need a couple of system libs (libpulse0, libspeex1, libxv1, ...). apt -f install will pull them in for you:

sudo dpkg -i debs/libpexcommon_*.deb \
             debs/libpexpulse_*.deb  \
             debs/libpexpulse-dev_*.deb
sudo apt-get install -f

This installs the headers under /opt/pexip/include and the shared library under /opt/pexip/lib.

2. Install the build dependencies

sudo apt-get install -y cmake build-essential \
                        libglfw3-dev libgl1-mesa-dev

3. Configure and build

cmake -S . -B build
cmake --build build -j

On the first run CMake fetches Dear ImGui from GitHub (≈ 5 MB).

4. Run

./build/run-doppler.sh

The launcher script (generated by CMake) just sets LD_LIBRARY_PATH to /opt/pexip/lib and execs the binary — this is needed because libpexpulse.so's private siblings (libpexlgpl.so, libimf.so, libonnxruntime.so.1, …) are installed alongside it in /opt/pexip/lib and have to be on the dynamic-linker search path. If you'd rather not use the wrapper, either:

LD_LIBRARY_PATH=/opt/pexip/lib ./build/doppler
# or, system-wide:
echo /opt/pexip/lib | sudo tee /etc/ld.so.conf.d/pexip.conf && sudo ldconfig

Fill in the Server (your Pexip Infinity node, e.g. vc.example.com), the Conference alias and a Display name, then press Connect. The far-end video and your own self-view will be rendered directly into the ImGui window (Pulse's auto-spawn windows are disabled via pulse_options_set_*_window_handle(NULL)).

RTMP ingest + "Twitch mix"

Open the RTMP ingest + Twitch mix section in the UI, pick a port + path (defaults: 1935 / live), and press Start RTMP server. Then publish a stream to it from OBS or ffmpeg, e.g.:

ffmpeg -re -i some-video.mp4 -c:v libx264 -c:a aac -f flv \
       rtmp://localhost:1935/live

Press Enable Twitch mix and Pulse will composite your camera as a PIP on top of the RTMP feed and send the result as the MAIN outgoing video. Toggle Camera segmentation to key out your camera background for the full Twitch streaming look.

The binary has its RPATH set to /opt/pexip/lib, so the direct dependency libpexpulse.so is found without help. The launcher script handles the transitive deps for you.

Code tour

src/main.cpp walks you through the lifecycle steps inline:

1. pulse_new() — create a Pulse instance.
2. pulse_options_set_*() — register a small set of callbacks (conference state, application user-agent) and pin the video window handles to NULL so Pulse won't auto-spawn its own windows.
3. pulse_data_session_connect_output() — open RGBA "pull" sessions for MAIN and SELFVIEW so we can render the frames ourselves.
4. pulse_connect_with_rest_async() — kick off the call. The progress and async-result callbacks feed our status panel.
5. pulse_disconnect_async() — tear it down.
6. pulse_free() — release the handle.

On top of that, the demo wires in the RTMP ingest + video MIX building blocks: pulse_rtmp_session_connect_input() opens an RTMP listener on the PRESENTATION slot, and pulse_video_mix_input_from_device + pulse_video_mix_input_from_rtmp_session + pulse_video_mix_connect composite the camera on top of the RTMP feed (optionally with PULSE_VIDEO_PROCESS_TYPE_SEGMENTATION) and ship the result out on MAIN.

Everything else (the GLFW window, the ImGui form, the status text, the video tiles, the RTMP + mix wiring) is just plumbing around those calls. The bigger sibling pexninja/pexninja.cpp is where all of these patterns were lifted from — it's the place to look when you outgrow the demo.

The pulse_gateway sibling demo

src/gateway.cpp builds into a second binary, pulse_gateway, that demonstrates how to use the Pulse data-session input and output APIs together to build a "two-Pulse bridge".

Two independent Pulse instances each place their own call:

        ┌─────────┐     OUTPUT (raw I420 / F32LE)     ┌─────────┐
        │ pulse 1 │ ────────────────────────────────► │ pulse 2 │
Conf A ─┤         │                                   │         ├─ Conf B
        │         │ ◄──────────────────────────────── │         │
        └─────────┘     OUTPUT (raw I420 / F32LE)     └─────────┘
                       (pushed into the other leg's INPUT)

A small background thread pumps pulse_data_session_pull_frame_data from one leg's OUTPUT straight into pulse_data_session_push_frame on the other leg's INPUT, both for audio (F32LE PCM, 48 kHz, mono) and video (I420 planar YUV). No system mic, camera or speaker is ever touched — the data sessions take their place entirely.

For the on-screen preview tiles the pump thread converts I420 to RGBA on the CPU (BT.601 limited-range) before handing the buffer to the UI thread, so the GL upload path stays trivial.

Because everything that crosses the gap is raw, decoded samples — no container, no signalling, no metadata — the bridge is the kind of "guard" you'd put at the boundary of a cross-domain solution: by construction the only thing that can pass through is honest audio and video.

The ImGui UI has two side-by-side panels (one per leg) with their own Call / Hang up buttons, plus live counters of audio/video bytes and frames being forwarded in each direction, and a small RGBA preview of what each leg is relaying.

Build + run:

cmake --build build -j --target pulse_gateway
./build/run-pulse-gateway.sh

pexninja/ contains a much fuller Pulse client (~12k LoC, lifted from another repo). It uses the same libpexpulse but layers in ImPlot, gl3w, ImGui-Addons (file browser) and GStreamer. It's gated behind a CMake option so the default doppler build stays lean:

sudo apt-get install -y libglib2.0-dev libgstreamer1.0-dev \
                        libgstreamer-plugins-base1.0-dev libx11-dev
cmake -S . -B build -DBUILD_PEXNINJA=ON
cmake --build build -j --target pexninja
./build/run-pexninja.sh

The extra dependencies (ImPlot, gl3w, ImGui-Addons) are fetched at configure time via FetchContent. The Dear ImGui tag is the -docking variant because pexninja uses multi-viewport + docking features; the simpler doppler target keeps building unchanged against the same superset.