Skip to main content

Rendering Streams

Language:

The render sender is how your application's pixels reach the room. You create one sender per target, then drive a simple loop: begin frame → render into the provided textures → show frame. QAROS takes care of encoding, transport, and — via the mixer — warping and compositing your content into every viewer's personal view.

Accepting render stream requests​

There is no API for pulling a peer into your stream by address — instead, a peer that wants your content (a Visualizer app volume, a headset viewer, ...) sends a render stream request, and your application subscribes to be notified of these:

static void on_render_request(QarRenderStreamRequest* request, void* user_state)
{
    QarPeerId target_peer_id = qar_peer_id_default();
    qar_result_log_if_error(
        qar_render_request_get_target_peer_id(request, &target_peer_id));

    QarStreamId stream_id = qar_stream_id_default();
    qar_result_log_if_error(
        qar_render_request_get_stream_id(request, &stream_id));

    /* Create the sender for this peer, e.g. by handing target_peer_id off
       to your render thread — see "Creating a sender" below. */

    qar_render_request_handle_destroy(request);
}

qar_result_log_if_error(
    qar_render_sender_subscribe_requests(session, on_render_request, NULL, NULL));

The callback fires once per incoming request for as long as the subscription is alive (until the session is destroyed or you cancel it via the optional QarCancelToken). Each request carries the requesting peer's ID — pass it as init.peer_id when you create the matching sender — and a QarStreamId you can use to correlate multiple requests. Always destroy the request handle once you are done reading it.

Creating a sender​

QarRenderSenderInit init = qar_render_sender_init_default();
init.peer_id = target_peer_id;          /* who receives this stream        */
init.enable_auto_reconnects = true;     /* survive network hiccups         */
init.app_volume_id = &volume_id;        /* composite into our app volume   */
init.graphics_api = QAR_GRAPHICS_API_CPU;
init.color_format = QAR_PIXEL_FORMAT_R8G8B8A8;
init.depth_format = QAR_PIXEL_FORMAT_R32_FLOAT;

/* Describe the views inside one video frame: stereo color + depth. */
init.texture_layout = QAR_FRAME_LAYOUT_SIDE_BY_SIDE;
init.frame_views[0] = (QarRenderFrameView){
    .data_type = QAR_VIDEO_FRAME_VIEW_TYPE_COLOR,
    .eye = QAR_VIDEO_FRAME_VIEW_EYE_LEFT,
    .texture_index = 0 };
/* ... one entry per view (right eye, depth views, ...) ... */
init.frame_views_count = 4;

QarRenderSender* sender = NULL;
qar_result_log_if_error(
    qar_render_sender_create(session, &init, &sender));
info

The C# snippet above is illustrative — it tracks the Qar binding surface but is not yet compiled from a published sample project.

Key choices:

  • Target peer — a sender streams to one peer: the requesting peer's ID from the render stream request above, or one obtained through peer enumeration.
  • App volume — pass your volume's ID so the mixer composites the stream inside that box; see App Volumes.
  • Graphics API — QAR_GRAPHICS_API_CPU (portable, always available) or QAR_GRAPHICS_API_D3D11 (Windows, zero-copy GPU textures; chain a QarStreamParamsD3D11 extension with your device/context into init.header.next).
  • Frame layout — how views pack into textures: SIDE_BY_SIDE (both eyes in one texture), SEPARATED_TEXTURES, or LAYERED (texture array). The negotiated layout can be queried (qar_render_sender_layout) and changed at runtime (qar_render_sender_change_layout).
  • Depth matters. Sending real depth (not just color) is what enables correct occlusion between app volumes and low-latency warping. Send it whenever you can.

The frame loop​

for (;;)
{
    /* 1. Begin: get the poses/FOVs you must render with. */
    QarRenderFrameInfo* info = NULL;
    QarResult r = qar_render_sender_begin_frame(sender, NULL, &info);
    if (qar_result_is_error(r)) { /* stream gone? reconnecting? */ continue; }

    QarPose pose;  QarFov fov;
    qar_render_frame_info_get_view_pose(info, 0, &pose);
    qar_render_frame_info_get_view_fov(info, 0, &fov);
    /* ... same for the other views ... */

    /* 2. Get the frame's textures and render into them. */
    QarVideoFrameCpu frame;
    qar_render_sender_frame_cpu(sender, &frame);
    /* frame.textures[i].texture_data / .pitch / .size — write your pixels;
       frame.texture_views map views (eye, color/depth) to textures.   */
    render_my_scene(&frame, &pose, &fov);

    /* 3. Show: submit with the projection metadata you used. */
    QarRenderFrameShow show = qar_render_frame_show_default();
    show.rendered_near_far.near_plane = 0.1f;
    show.rendered_near_far.far_plane = 100.0f;
    show.depth_scale = 1.0f;
    qar_result_log_if_error(qar_render_sender_show_frame(sender, &show));

    qar_render_frame_info_handle_destroy(info);
}

The contract to respect: render with the pose and FOV that begin_frame gives you, and report the near/far you actually used in show_frame. The mixer uses exactly this metadata (plus your depth buffer) to re-project your frame to the viewer's latest head pose — wrong metadata shows up as swimming or misplaced content, not as an error code.

Per-frame overrides are possible when you must deviate: chain a QarRenderFrameShowViewOverridesExt into show.header.next with per-view pose/FOV overrides.

begin_frame also has an async variant (qar_render_sender_begin_frame_async) so render threads can pipeline instead of blocking.

The D3D11 path​

On Windows, request QAR_GRAPHICS_API_D3D11 and chain QarStreamParamsD3D11:

QarStreamParamsD3D11 d3d = qar_stream_params_d3d11_default();
d3d.d3d11_device = my_device1;
d3d.d3d11_context = my_context1;
d3d.color_bind_flags = D3D11_BIND_RENDER_TARGET | D3D11_BIND_SHADER_RESOURCE;
d3d.depth_bind_flags = D3D11_BIND_DEPTH_STENCIL;
d3d.acquire_keyed_mutex_sync = true; /* let QAROS manage keyed-mutex sync */
init.header.next = &d3d.header;

qar_render_sender_frame_d3d11 then hands you shared D3D11 textures to render into directly — no CPU copy. Set acquire_keyed_mutex_sync = false only if you need to manage the keyed-mutex acquire/release chain yourself (e.g. when copying through a chain of shared textures).

Reading back the viewer: hand tracking​

The stream is bidirectional: the target device reports input, and you can read the viewer's hands each frame:

QarDeviceHandsWithJoints hands;
if (qar_result_is_success(qar_render_sender_last_hands(sender, &hands)))
{
    /* 26 joints per hand, with locations and velocities, in room space */
}

This is how a source application implements grabbing, pointing, and touch against its own content (combine with gesture events for higher-level interactions).

Lifecycle and failure​

  • enable_auto_reconnects = true makes the sender survive network drops and target restarts transparently — frame calls fail while disconnected and recover on their own.
  • Destroy with qar_render_stream_handle_destroy(sender).
  • Senders are per-target: to serve several viewers directly you create several senders — but in a Hub deployment you typically stream once into your app volume and let the per-user mixers fan the content out.

Compiled tutorial​

This page is backed by the compiled example cpu_rendering_visualizer.c, which obtains a session, subscribes to render stream requests, and streams a CPU-rendered gradient to the requesting peer: