Vulkan Rendering

What you build

The Rust port of QAROS's Vulkan cube: a rotating cube with one flat color per face, rendered on the GPU in stereo color and depth for every target app in the session. It is the reference for a GPU source app in Rust. It runs on the Hub machine and joins the Hub as a local app, creates its own Vulkan device on the GPU the session uses, and hands each frame to QAROS through the Vulkan render sender without a copy.

Take this path when your renderer already runs on Vulkan. The API uses the types of the ash crate, version 0.38. If your renderer uses ash 0.38 too (as wgpu-hal 30, the Vulkan backend of wgpu 30, does), pass its vk::PhysicalDevice and vk::Device to the sender and use the frame's vk::Images directly, with no conversion. Otherwise convert your raw Vulkan handles with ash's Handle::from_raw and as_raw.

What you will learn:

• how to join as a local app, and how to cancel the approval wait on a stop request;
• how to create your Vulkan device on the session's GPU with the extensions QAROS needs;
• the safety contract of Session::create_vulkan_render_sender;
• the frame hand-off: acquire each image from the state QAROS reports, render, release it to the external queue family, signal the frame's semaphore, write the state back, show;
• why the device must outlive the session, not only the senders.

The program is examples/src/bin/vulkan_cube.rs in the Rust binding, behind the vulkan feature. This page includes its code; the file is the authority. The CPU Rendering tutorial explains peers, app volumes and the frame cycle; this one builds on it. The Vulkan API of the binding is summarized in docs/vulkan-api.md.

Before you start

• Platform. Windows 10 or 11 for the real runtime. Joining as a local app is available on Windows; elsewhere the runtime refuses it. Without QAROS, the example and its test run against qaros-stub, which emulates the Vulkan path on your device.
• GPU. A GPU whose Vulkan driver offers VK_KHR_external_memory_win32 and VK_KHR_external_semaphore_win32 on Windows. QAROS supports only NVIDIA GPUs (Choosing the GPU). The Vulkan loader comes with the GPU driver.
• QAROS. Installed on the Hub machine, where the example runs. When the Hub is not running, joining as a local app starts it, and the Hub keeps running after the app exits. The example takes no path: it finds the runtime with the default lookup, so the installed QAROS, unless a runtime sits beside the executable or QAROS_RUNTIME_DIR names one.
• What you do. On the first run, click Allow in the QAROS approval prompt, in the Visualizer or in a tray window, within 120 s (The approval prompt). An unsigned build asks again after every rebuild.
• A target. Open the Visualizer, or start a headset.
• The latest stable Rust. No shader compiler: the shaders are QAROS's GLSL, checked in as SPIR-V.

To use the Vulkan path in your own crate, add qaros with the vulkan feature to the [dependencies] section of your crate's Cargo.toml. The crates are not on crates.io, so the dependency names the QAROS repository, and Cargo finds the crate inside it:

[dependencies]
qaros = { git = "https://github.com/Quaternar/qaros", features = ["vulkan"] }

With a clone of the repository, point a path at its qar-streaming-rust/crates/qaros folder instead, relative to your Cargo.toml:

[dependencies]
qaros = { path = "../qaros/qar-streaming-rust/crates/qaros", features = ["vulkan"] }

For your own ash types to be the binding's, depend on ash 0.38 as well.

Run it

From the qar-streaming-rust folder of the repository:

cargo run --release -p qaros-examples --features vulkan --bin vulkan_cube

The example takes no arguments; --help prints its usage, and any other argument prints it to stderr and exits with 1. Stop it with Ctrl+C. When its stdin is a pipe, the end of stdin stops it too: that is how QAROS stops an app it started.

With QAROS installed and the Visualizer open, a run that is later stopped prints this (the runtime may add log lines of its own; the GPU, the target and the peer id are those of your session):

Loaded <QAROS folder>\bin\qar-streaming-c.dll (the installed QAROS)
Joining the QAROS on this PC. On the first run, click Allow in the QAROS approval prompt.
Connecting to the local QAROS hub.
Certificate signed.
Onboarding complete.
Rendering on <GPU name> (LUID <adapter LUID>), the session's GPU
Connected. Waiting for rendering targets. Ctrl+C to quit.
Rendering for target <target name> (peer <peer id>)
Stopping.

It exits with 0. The lines between Joining and Onboarding complete. are the runtime's progress messages. While the approval prompt is open, one more appears, Waiting for approval in QAROS hub. In the Visualizer, the cube appears in the "Vulkan Cube" app volume.

Without QAROS, run the example's own test. It renders one frame through the stub on the first supported GPU and reads both eyes back:

cargo build -p qaros-stub
QAROS_REQUIRE_VULKAN=1 cargo test -p qaros-examples --features vulkan --bin vulkan_cube

It needs a Vulkan loader and driver. Without one it prints skipped and passes, unless QAROS_REQUIRE_VULKAN is set. On macOS the loader is not on the library path the test searches, and a DYLD_LIBRARY_PATH set for cargo test does not reach the test. Build the test binary, then run it yourself with DYLD_LIBRARY_PATH naming the folder of libvulkan.dylib, /opt/homebrew/lib for the Homebrew Vulkan loader:

cargo test -p qaros-examples --features vulkan --bin vulkan_cube --no-run
# The last line names the binary: Executable unittests src/bin/vulkan_cube.rs (target/debug/deps/vulkan_cube-<hash>)
QAROS_REQUIRE_VULKAN=1 DYLD_LIBRARY_PATH=/opt/homebrew/lib target/debug/deps/vulkan_cube-<hash>

The example itself also runs against the stub. It takes no library path, so put a copy of the stub under the runtime's file name in a folder and name that folder with QAROS_RUNTIME_DIR. target/stub-runtime keeps the copy inside the build folder, which git ignores. On macOS, with the Homebrew Vulkan loader and MoltenVK:

cargo build -p qaros-stub
cargo build -p qaros-examples --features vulkan --bin vulkan_cube
mkdir -p target/stub-runtime
cp target/debug/libqaros_stub.dylib target/stub-runtime/libqar-streaming-c.dylib
QAROS_RUNTIME_DIR="$PWD/target/stub-runtime" DYLD_LIBRARY_PATH=/opt/homebrew/lib target/debug/vulkan_cube

Stopped by closing a piped stdin, it prints (with your own path and GPU):

Loaded .../target/stub-runtime/libqar-streaming-c.dylib (QAROS_RUNTIME_DIR)
Joining the QAROS on this PC. On the first run, click Allow in the QAROS approval prompt.
stub onboarding complete
Rendering on <GPU name> (LUID <adapter LUID>), the first supported GPU
Connected. Waiting for rendering targets. Ctrl+C to quit.
Rendering for target Stub Visualizer (peer 56:56:56:56:56:56:56:56:56:56:56:56:56:56:56:56)
Stopping.

The stub accepts the local-app join without a prompt and names no GPU, so the example takes the first supported one.

Walkthrough

1. Start, load the runtime, map failures to exit codes

fn main() -> ExitCode {
    if let Some(argument) = std::env::args().nth(1) {
        if argument == "--help" || argument == "-h" {
            print!("{USAGE}");
            return ExitCode::SUCCESS;
        }
        eprint!("{USAGE}");
        return ExitCode::from(1);
    }
    stop::watch_for_stop_requests();
    // SAFETY: the runtime found is the installed QAROS, or one the user chose with
    // QAROS_RUNTIME_DIR or put beside this executable, which matches these bindings.
    let library = match unsafe { Library::load_default(&LibraryConfig::default()) } {
        Ok(library) => library,
        Err(qaros::Error::Load(load_error)) => {
            eprintln!(
                "Could not load QAROS. Install QAROS on this PC, then start this app again.\n{load_error}"
            );
            return ExitCode::from(2);
        }
        Err(error) => {
            eprintln!("The QAROS runtime failed to initialize: {error}");
            return ExitCode::from(3);
        }
    };
    println!("Loaded {}", describe_loaded_runtime(&library));
    let outcome = run_app(&library);
    // The wrapper logs a failing qar_library_destroy; the library is never unloaded.
    drop(library);
    match outcome {
        Ok(()) => ExitCode::SUCCESS,
        Err(error) => {
            eprintln!("error: {error}");
            ExitCode::from(4)
        }
    }
}

stop::watch_for_stop_requests installs a Ctrl+C handler (SIGINT and SIGTERM outside Windows) and, when stdin is a pipe, a thread that waits for its end. A signal handler may safely do nothing but write an atomic, so all QAROS and GPU cleanup stays on the main thread, which polls stop::requested().

Library::load_default is the C qar_library_load(NULL): the runtime beside the executable, the folder QAROS_RUNTIME_DIR names, the installed QAROS, then PATH. It is unsafe because a library file that does not match these bindings is undefined behavior. The exit codes: 0 after a stop, 2 when no runtime loads, 3 when the library fails to initialize, 4 for any other failure.

2. The order of the whole program

/// Everything between loading the library and destroying it. A stop request is not a failure.
fn run_app(library: &Library) -> ExampleResult {
    // runtime_binaries_directory stays empty: the runtime uses the folder it was loaded from.
    let runtime = library.create_runtime(&RuntimeConfig::default())?;
    println!(
        "Joining the QAROS on this PC. On the first run, click Allow in the QAROS approval prompt."
    );
    let Some(session) = join_as_local_app(library, &runtime)? else {
        return Ok(());
    };
    // Onboard first: the session decides the GPU, and the cube creates its device there.
    let gpu = CubeDevice::create(session.graphics_adapter()?)?;
    let pipeline = CubePipeline::create(&gpu)?;
    println!("Connected. Waiting for rendering targets. Ctrl+C to quit.");
    let served = serve_targets(library, &session, &gpu, &pipeline);
    // The session ends before the device its senders streamed from, as in the C sample, whose
    // device outlives the session; destroying the device first crashes the runtime's teardown.
    drop(pipeline);
    drop(session);
    drop(gpu);
    served
}

Join first: the session decides the GPU, and the device is created there. Read the drop order at the end now; Ownership and teardown explains it.

3. Join as a local app

A source app on the Hub machine joins as a local app: the user approves it once in QAROS (Local app). Nothing is stored; every start onboards again.

/// Joins the QAROS on this PC as a local app: no code, the user approves the app once in QAROS.
/// No adapter is chosen: the session decides the GPU. A stop request cancels the call, which can
/// wait minutes for the approval; `None` then.
fn join_as_local_app(library: &Library, runtime: &Runtime) -> ExampleResult<Option<Session>> {
    let token = library.create_cancel_token(None)?;
    let cancel = token.cancel_handle();
    let joined = Arc::new(AtomicBool::new(false));
    let canceller = {
        let joined = Arc::clone(&joined);
        std::thread::spawn(move || {
            while !joined.load(Ordering::Acquire) && !stop::requested() {
                std::thread::sleep(STOP_POLL_INTERVAL);
            }
            if !joined.load(Ordering::Acquire)
                && let Err(error) = cancel.cancel()
            {
                eprintln!("Failed to cancel joining QAROS: {error}");
            }
        })
    };
    let (progress, progress_reports) = std::sync::mpsc::channel::<Progress>();
    let printer = std::thread::spawn(move || {
        for report in progress_reports {
            if !report.message.is_empty() {
                println!("{}", report.message);
            }
        }
    });
    let presentation = PeerPresentation {
        display_name: Some("Vulkan Cube Source".to_string()),
        ..PeerPresentation::default()
    };
    let onboarded = runtime.onboard(
        &OnboardMethod::LocalApp {
            approval_timeout: None,
        },
        &presentation,
        &BlockingOptions {
            progress: Some(progress),
            cancel: Some(&token),
        },
    );
    joined.store(true, Ordering::Release);
    if canceller.join().is_err() || printer.join().is_err() {
        eprintln!("a helper thread of the join panicked");
    }
    match onboarded {
        Ok(onboarded) if !stop::requested() => Ok(Some(onboarded.session)),
        Ok(_) => Ok(None),
        Err(_) if stop::requested() => Ok(None),
        Err(error) => {
            explain_onboarding_failure(&error);
            Err("could not join QAROS".into())
        }
    }
}

The call can wait minutes for the approval, so it takes a cancel token. CancelToken::cancel_handle gives a Send handle that another thread can cancel with while the token lives; here a thread cancels when a stop is requested. Progress messages, such as Waiting for approval in QAROS hub, arrive on a library thread and go through a channel to a printing thread. approval_timeout: None keeps the Hub's default of 120 s.

When joining fails, the example tells the user what happened and what to do, by status code:

/// Tells the user, in plain words, why joining failed and what to do.
fn explain_onboarding_failure(error: &qaros::Error) {
    let explanations = [
        (
            StatusCode::ONBOARDING_APPROVAL_DENIED,
            "QAROS did not allow this app to join: the approval was denied, or revoked. Start the app again to be asked again.",
        ),
        (
            StatusCode::ONBOARDING_APPROVAL_TIMEOUT,
            "Nobody answered the approval prompt in QAROS in time. It opens in the QAROS Visualizer, or as a QAROS tray window. Start the app again.",
        ),
        (
            StatusCode::ONBOARDING_HUB_NOT_AUTHENTIC,
            "The program answering on this PC is not the installed QAROS, so this app does not trust it and stops. Reinstall QAROS or ask your administrator.",
        ),
        (
            StatusCode::ONBOARDING_APP_IDENTITY_MISMATCH,
            "This app no longer matches what QAROS approved: it was moved, copied or re-signed. In the QAROS Visualizer, revoke it under Connection > Approved apps, then start it again to approve it anew.",
        ),
        (
            StatusCode::ONBOARDING_HUB_UNREACHABLE,
            "QAROS is not running on this PC. Start QAROS from the Start menu, then start this app again.",
        ),
    ];
    let explanation = explanations
        .iter()
        .find(|(code, _)| error.has_code(*code))
        .map_or("Could not join QAROS.", |(_, explanation)| explanation);
    eprintln!("{explanation}");
    eprintln!("Details: {error}");
}

These are the codes of Status codes and what to tell the user. Retry none of them silently.

4. Create the device on the session's GPU

QAROS creates the frame images on your device, and the receiver reads them on the session's GPU. Render on that GPU, so frames need no copy between GPUs. Session::graphics_adapter() returns the adapter the session uses: the one chained at onboarding, else the one QAROS or the library selected. The example matches it against each physical device's VkPhysicalDeviceIDProperties:

/// The first physical device that is on `adapter` (when given), supports the extensions QAROS
/// needs, and has a graphics queue family.
fn pick_physical_device(
    instance: &ash::Instance,
    adapter: Option<GraphicsAdapter>,
) -> ExampleResult<Option<Candidate>> {
    // SAFETY: live instance.
    let physical_devices =
        unsafe { instance.enumerate_physical_devices() }.map_err(failed("enumerate GPUs"))?;
    for physical_device in physical_devices {
        let mut identifiers = vk::PhysicalDeviceIDProperties::default();
        let mut properties = vk::PhysicalDeviceProperties2::default().push_next(&mut identifiers);
        // SAFETY: live physical device of a Vulkan 1.1 instance.
        unsafe { instance.get_physical_device_properties2(physical_device, &mut properties) };
        let name = properties
            .properties
            .device_name_as_c_str()
            .map(|text| text.to_string_lossy().into_owned())
            .unwrap_or_default();
        let luid = (identifiers.device_luid_valid == vk::TRUE).then_some(identifiers.device_luid);
        if adapter.is_some_and(|wanted| !wanted.matches(luid, identifiers.device_uuid)) {
            continue;
        }
        let Some(extensions) = device_extensions(instance, physical_device)? else {
            continue;
        };
        let Some(queue_family) = graphics_queue_family(instance, physical_device) else {
            continue;
        };
        return Ok(Some(Candidate {
            physical_device,
            name,
            luid,
            queue_family,
            extensions,
        }));
    }
    Ok(None)
}

/// The device extensions to enable: QAROS's, plus `VK_KHR_portability_subset` where the device
/// has it (the Vulkan spec requires enabling it then) and, on Linux, `VK_KHR_external_semaphore_fd`
/// where the device has it, as the C sample does. `None` when QAROS's are missing.
fn device_extensions(
    instance: &ash::Instance,
    physical_device: vk::PhysicalDevice,
) -> ExampleResult<Option<Vec<*const c_char>>> {
    // SAFETY: live physical device.
    let available = unsafe { instance.enumerate_device_extension_properties(physical_device) }
        .map_err(failed("enumerate extensions"))?;
    let has = |wanted: &CStr| {
        available
            .iter()
            .any(|extension| extension.extension_name_as_c_str() == Ok(wanted))
    };
    let required = qaros::vulkan::required_device_extensions();
    if !required.iter().all(|name| has(name)) {
        return Ok(None);
    }
    let mut extensions: Vec<*const c_char> = required.iter().map(|name| name.as_ptr()).collect();
    if has(ash::khr::portability_subset::NAME) {
        extensions.push(ash::khr::portability_subset::NAME.as_ptr());
    }
    if cfg!(target_os = "linux") && has(ash::khr::external_semaphore_fd::NAME) {
        extensions.push(ash::khr::external_semaphore_fd::NAME.as_ptr());
    }
    Ok(Some(extensions))
}

GraphicsAdapter::matches compares a LUID on Windows and a UUID elsewhere. qaros::vulkan::required_device_extensions() lists what QAROS needs on your device: Win32 external memory and semaphores on Windows, file-descriptor external memory on Linux. When the session names an adapter and no physical device matches it with those extensions, the example stops with an error rather than render on another GPU. When the session names none, it takes the first supported GPU.

The sender gets the device as two raw handles:

fn stream_params(&self) -> VulkanStreamParams {
    VulkanStreamParams {
        physical_device: self.physical_device,
        device: self.device.handle(),
    }
}

5. One sender and one camera per target

The cube declares one texture per view: each eye gets a color and a depth image of its own, the SeparatedTextures layout.

/// The C sample's views: per eye a color and a depth texture of their own.
const FRAME_VIEWS: [FrameViewSpec; 4] = [
    FrameViewSpec {
        kind: ViewKind::Color,
        eye: Eye::Left,
        texture_index: 0,
    },
    FrameViewSpec {
        kind: ViewKind::Depth,
        eye: Eye::Left,
        texture_index: 1,
    },
    FrameViewSpec {
        kind: ViewKind::Color,
        eye: Eye::Right,
        texture_index: 2,
    },
    FrameViewSpec {
        kind: ViewKind::Depth,
        eye: Eye::Right,
        texture_index: 3,
    },
];

The app volume is the cube's own; QAROS registers none for an app:

fn create_volume(session: &Session) -> qaros::Result<AppVolumeId> {
    let mut volume_config = AppVolumeConfig::new("vulkan-cube");
    volume_config.display_name = Some("Vulkan Cube".to_string());
    volume_config.pose = Pose::from_position(Vec3::new(0.0, 0.0, -1.5));
    volume_config.size = AppVolumeSize {
        width: 1.2,
        length: 1.2,
        height: 1.2,
    };
    session.app_volumes().get_or_create(&volume_config)
}

Each target gets a Target: a sender and a camera, the per-target GPU state.

/// One target app: its camera and the sender rendering for it. The camera is declared first so
/// that it drains its GPU work before the sender's images go.
struct Target {
    peer: PeerId,
    label: String,
    camera: Camera,
    sender: RenderSender,
}

impl Target {
    fn create(
        library: &Library,
        session: &Session,
        volume: AppVolumeId,
        gpu: &CubeDevice,
        peer: PeerId,
        label: String,
    ) -> ExampleResult<Self> {
        let config = RenderSenderConfig {
            views: FRAME_VIEWS.to_vec(),
            app_volume: Some(volume),
            color_format: PixelFormat::R8G8B8A8,
            depth_format: PixelFormat::D32Float,
            layout: FrameLayout::SeparatedTextures,
            ..RenderSenderConfig::new(peer)
        };
        let timeout = library.create_cancel_token(Some(SENDER_CREATION_TIMEOUT))?;
        // SAFETY: the device was created on the session's adapter with
        // required_device_extensions(), and `run` drops every target before the device.
        let sender = unsafe {
            session.create_vulkan_render_sender(
                &config,
                gpu.stream_params(),
                &BlockingOptions {
                    cancel: Some(&timeout),
                    ..BlockingOptions::default()
                },
            )
        }?;
        Ok(Self {
            peer,
            label,
            camera: Camera::create(gpu)?,
            sender,
        })
    }
}

create_vulkan_render_sender is unsafe. You promise:

• the physical device and device are valid, and the device was created from that physical device;
• the device is on the adapter the session was onboarded with, and enables required_device_extensions();
• the device outlives the sender and every handle taken from it.

The wrapper cannot check any of it. The sender is the same RenderSender type as on the CPU path.

The serving loop follows the peer list as the CPU example does, with one addition: a target whose sender cannot be created is tried again after a delay that starts at 3 s and doubles up to 60 s (RetryPlan). Creating a sender blocks the loop until the target answers, so a target that never answers would otherwise stall every other target.

/// Follows the targets and renders for them until a stop request.
fn serve_targets(
    library: &Library,
    session: &Session,
    gpu: &CubeDevice,
    pipeline: &CubePipeline,
) -> ExampleResult {
    let volume = create_volume(session)?;
    // Subscribe first, then list, so a target joining in between is not missed.
    let (forward, peer_updates) = qaros::channel();
    let _peer_subscription = session.subscribe_peer_updates(forward)?;
    let started = Instant::now();
    let mut targets: Vec<Target> = Vec::new();
    let mut retries = RetryPlan::default();
    let mut last_listing: Option<Instant> = None;

    while !stop::requested() {
        let updates_arrived = peer_updates.try_iter().count() > 0;
        if updates_arrived || last_listing.is_none_or(|last| last.elapsed() >= PEER_LIST_INTERVAL) {
            last_listing = Some(Instant::now());
            follow_targets(
                library,
                session,
                volume,
                gpu,
                &session.peers()?,
                &mut targets,
                &mut retries,
            );
        }
        let seconds = started.elapsed().as_secs_f32();
        let mut showed_a_frame = false;
        targets.retain_mut(
            |target| match render_next_frame(target, pipeline, seconds) {
                Ok(showed) => {
                    showed_a_frame |= showed;
                    true
                }
                Err(error) => {
                    println!("Stopped streaming to {}: {error}", target.label);
                    false
                }
            },
        );
        if !showed_a_frame {
            std::thread::sleep(IDLE_WAIT);
        }
    }
    println!("Stopping.");
    // Cameras drain their GPU work and senders close before the pipeline and the device go.
    drop(targets);
    Ok(())
}

The C cube serves each target on a thread of its own. qaros handles are not Send, so this port serves all targets from one loop with the non-blocking try_begin_owned_frame, as the CPU example does.

6. One frame

/// Renders and shows the target's next frame when its sender has one; false while none is
/// ready or the stream is waiting.
fn render_next_frame(
    target: &mut Target,
    pipeline: &CubePipeline,
    seconds: f32,
) -> ExampleResult<bool> {
    let mut frame = match target.sender.try_begin_owned_frame() {
        Ok(Some(frame)) => frame,
        Ok(None) => return Ok(false),
        Err(error) if is_waiting(&error) => return Ok(false),
        Err(error) => return Err(error.into()),
    };
    let semaphore = frame.vulkan_frame()?.semaphore();
    let rendered = eye_views(&mut frame)
        .and_then(|eyes| render_frame(&mut frame, &mut target.camera, pipeline, &eyes, seconds));
    if let Err(error) = rendered {
        // A begun frame that is not rendered still signals its semaphore before it is dropped.
        if let Err(discard_error) = target.camera.discard(semaphore) {
            eprintln!("Failed to discard the camera frame: {discard_error}");
        }
        return Err(error);
    }
    match frame.show(&ShowParams {
        near: NEAR_PLANE,
        far: FAR_PLANE,
        depth_scale: 1.0,
    }) {
        Ok(()) => Ok(true),
        Err(error) if is_waiting(&error) => Ok(false),
        Err(error) => Err(error.into()),
    }
}

Every begun frame must signal its semaphore before it is dropped, even one that is not rendered: QAROS waits for that signal. So when rendering fails, Camera::discard submits an empty batch that only signals it.

The pose and field of view come from the view showing each eye's color. view_index finds the view by eye and kind; the same index selects the pose with frame.view(index):

/// Each eye's pose and field of view, from the frame view showing that eye's color.
fn eye_views(frame: &mut OwnedFrame) -> ExampleResult<[ViewPose; 2]> {
    let color_views = {
        let vulkan_frame = frame.vulkan_frame()?;
        [Eye::Left, Eye::Right].map(|eye| vulkan_frame.view_index(eye, ViewKind::Color))
    };
    let [Some(left), Some(right)] = color_views else {
        return Err("the frame lacks a color view for an eye".into());
    };
    Ok([frame.view(left)?, frame.view(right)?])
}

The poses are in the app volume's app content space, the space the cube renders in. The projection uses the OpenXR field-of-view angles and forward Vulkan depth, 0 at the near plane and 1 at the far plane:

/// Projection times view for one eye: OpenXR field-of-view signs, forward Vulkan depth (0 at
/// near, 1 at far). The pose is already in app content space.
fn view_projection(eye: &ViewPose) -> Mat4 {
    let left = eye.fov.angle_left.tan();
    let right = eye.fov.angle_right.tan();
    let down = eye.fov.angle_down.tan();
    let up = eye.fov.angle_up.tan();
    let projection = Mat4::from_cols(
        Vec4::new(2.0 / (right - left), 0.0, 0.0, 0.0),
        Vec4::new(0.0, 2.0 / (up - down), 0.0, 0.0),
        Vec4::new(
            (right + left) / (right - left),
            (up + down) / (up - down),
            -FAR_PLANE / (FAR_PLANE - NEAR_PLANE),
            -1.0,
        ),
        Vec4::new(
            0.0,
            0.0,
            -FAR_PLANE * NEAR_PLANE / (FAR_PLANE - NEAR_PLANE),
            0.0,
        ),
    );
    let orientation = eye.pose.orientation;
    let position = eye.pose.position;
    let camera = Mat4::from_rotation_translation(
        glam::Quat::from_xyzw(orientation.x, orientation.y, orientation.z, orientation.w),
        glam::Vec3::new(position.x, position.y, position.z),
    );
    projection * camera.inverse()
}

The hand-off itself, in the order the safety rules require: submit the GPU work that signals the frame's semaphore, write the images' new state back, then show.

/// Renders both eyes into the frame's images, signals its semaphore, and writes the images'
/// new state back. The frame is then ready to show.
fn render_frame(
    frame: &mut OwnedFrame,
    camera: &mut Camera,
    pipeline: &CubePipeline,
    eyes: &[ViewPose; 2],
    seconds: f32,
) -> ExampleResult {
    let (views, images, semaphore) = {
        let vulkan_frame = frame.vulkan_frame()?;
        (
            vulkan_frame.views(),
            vulkan_frame.images(),
            vulkan_frame.semaphore(),
        )
    };
    camera.render(pipeline, &views, &images, semaphore, eyes, seconds)?;
    let mut vulkan_frame = frame.vulkan_frame()?;
    vulkan_frame.release_to_external(SHOWN_LAYOUT);
    vulkan_frame.set_fence(vk::Fence::null());
    Ok(())
}

vulkan_frame() borrows the open frame mutably, so the frame record cannot change while you read it, and the write-back cannot happen after show. The submission is outside the wrapper's view: submitting after show is a bug it cannot catch. set_fence(vk::Fence::null()) is what the C cube writes.

7. Acquire and release the images

QAROS owns the images, their memory and the semaphore; never destroy, free or re-bind them. Each frame reports every image's current layout and owning queue family. Transition from exactly that state: the real runtime reports TRANSFER_SRC_OPTIMAL owned by VK_QUEUE_FAMILY_EXTERNAL from the first frame on, the stub UNDEFINED and VK_QUEUE_FAMILY_IGNORED at first. After rendering, leave every image in SHOWN_LAYOUT (TRANSFER_SRC_OPTIMAL), released to VK_QUEUE_FAMILY_EXTERNAL.

/// Records and submits one stereo frame into the sender's images and signals `semaphore`
/// when the GPU is done with it, leaving every image in [`SHOWN_LAYOUT`], released to the
/// external queue family.
fn render(
    &mut self,
    pipeline: &CubePipeline,
    views: &[VideoFrameView],
    images: &[VulkanImage],
    semaphore: vk::Semaphore,
    eyes: &[ViewPose; 2],
    seconds: f32,
) -> ExampleResult {
    if views.len() != FRAME_VIEWS.len() || images.len() != FRAME_VIEWS.len() {
        return Err("expected the separated stereo color and depth layout".into());
    }
    let slot = self.next_slot()?;
    let commands = self.slots[slot].commands;
    let begin = vk::CommandBufferBeginInfo::default()
        .flags(vk::CommandBufferUsageFlags::ONE_TIME_SUBMIT);
    // SAFETY: the slot's previous submission finished (next_slot).
    unsafe {
        self.device
            .reset_command_buffer(commands, vk::CommandBufferResetFlags::empty())
            .map_err(failed("reset camera commands"))?;
        self.device
            .begin_command_buffer(commands, &begin)
            .map_err(failed("begin cube commands"))?;
    }
    let acquire: Vec<_> = images.iter().map(|image| self.acquire(image)).collect();
    // SAFETY: recording; the barriers name live sender images.
    unsafe {
        self.device.cmd_pipeline_barrier(
            commands,
            vk::PipelineStageFlags::ALL_COMMANDS,
            vk::PipelineStageFlags::COLOR_ATTACHMENT_OUTPUT
                | vk::PipelineStageFlags::EARLY_FRAGMENT_TESTS,
            vk::DependencyFlags::empty(),
            &[],
            &[],
            &acquire,
        );
    }
    let model = cube_model(seconds);
    for (eye, eye_view) in [Eye::Left, Eye::Right].into_iter().zip(eyes) {
        let mvp = view_projection(eye_view) * model;
        self.record_eye(commands, pipeline, views, images, eye, &mvp)?;
    }
    let release: Vec<_> = images.iter().map(|image| self.release(image)).collect();
    // SAFETY: recording; the barriers name live sender images.
    unsafe {
        self.device.cmd_pipeline_barrier(
            commands,
            vk::PipelineStageFlags::COLOR_ATTACHMENT_OUTPUT
                | vk::PipelineStageFlags::EARLY_FRAGMENT_TESTS
                | vk::PipelineStageFlags::LATE_FRAGMENT_TESTS,
            vk::PipelineStageFlags::BOTTOM_OF_PIPE,
            vk::DependencyFlags::empty(),
            &[],
            &[],
            &release,
        );
        self.device
            .end_command_buffer(commands)
            .map_err(failed("end cube commands"))?;
    }
    self.submit(slot, &[commands], semaphore)
}

/// From the reported state to the attachment layout on this queue family, with an
/// ownership transfer only when another family (the external one) owns the image.
fn acquire(&self, image: &VulkanImage) -> vk::ImageMemoryBarrier<'static> {
    let (layout, access, aspect) = attachment_of(image);
    let unowned = image.queue_family_index == vk::QUEUE_FAMILY_IGNORED
        || image.queue_family_index == self.queue_family;
    let (source_family, destination_family) = if unowned {
        (vk::QUEUE_FAMILY_IGNORED, vk::QUEUE_FAMILY_IGNORED)
    } else {
        (image.queue_family_index, self.queue_family)
    };
    vk::ImageMemoryBarrier::default()
        .image(image.image)
        .old_layout(image.layout)
        .new_layout(layout)
        .src_queue_family_index(source_family)
        .dst_queue_family_index(destination_family)
        .dst_access_mask(access)
        .subresource_range(whole_image(aspect))
}

/// From the attachment layout to [`SHOWN_LAYOUT`], released to the external family.
fn release(&self, image: &VulkanImage) -> vk::ImageMemoryBarrier<'static> {
    let (layout, access, aspect) = attachment_of(image);
    vk::ImageMemoryBarrier::default()
        .image(image.image)
        .old_layout(layout)
        .new_layout(SHOWN_LAYOUT)
        .src_queue_family_index(self.queue_family)
        .dst_queue_family_index(vk::QUEUE_FAMILY_EXTERNAL)
        .src_access_mask(access)
        .subresource_range(whole_image(aspect))
}

/// Signals the semaphore of a begun frame that could not be rendered, so that the sender
/// can still consume it.
fn discard(&mut self, semaphore: vk::Semaphore) -> ExampleResult {
    let slot = self.next_slot()?;
    self.submit(slot, &[], semaphore)
}

Image handles can change between frames, after a layout change for example. Read the image records every frame. The camera caches its image views and framebuffers by image handle, so a new handle gets new ones.

The camera keeps two frames in flight. It waits on a slot's fence only when it reuses the slot, never for the frame just submitted: the frame's semaphore hands that work to QAROS, which waits for it on the GPU.

8. Keep the sender, or end it

/// A stream that is paused, reconnecting or has no transfer yet is not broken: keep the sender
/// and try the next frame. Like the C cube's `IsWaiting` this matches a wrapped cause too
/// (begin-frame reports these inside `RENDERING_UNABLE_TO_BEGIN_FRAME`). An app volume that
/// does not resolve yet is waiting as well: the header calls it recoverable. Any other error
/// ends the sender.
pub fn is_waiting(error: &qaros::Error) -> bool {
    [
        StatusCode::MEDIA_STREAMING_STREAM_IS_PAUSED,
        StatusCode::MEDIA_STREAMING_STREAM_IS_RECONNECTING,
        StatusCode::MEDIA_STREAMING_NO_ACTIVE_TRANSFERS,
        StatusCode::RENDERING_APP_VOLUME_UNRESOLVED,
    ]
    .into_iter()
    .any(|code| error.has_cause(code))
}

This is the CPU tutorial's helper, shared in examples/src/lib.rs: paused, reconnecting, transfer-less streams and an unresolved app volume keep the sender; anything else, RENDERING_STREAM_IS_CLOSED included, drops the target, and the next listing creates a new one when the target still runs (Troubleshooting). The C cube's IsWaiting keeps the sender for the unresolved app volume too, as the header calls that status recoverable.

Ownership and teardown

The qaros handles keep each other alive as on the CPU path: a sender keeps its session, a session its runtime, a runtime the library. The Vulkan objects are different: the wrapper does not know your device, so their order is yours.

• Camera before sender. Target declares the camera before the sender, and Rust drops struct fields in declaration order. The camera's Drop waits for its submissions, then destroys its views and framebuffers over the sender's images, while those images still exist.
• Targets before the pipeline and the device. serve_targets drops every target before it returns.
• Session before device. run_app drops the pipeline, then the session, then the device. QAROS keeps using your device until the session is gone, not only until the senders are: a device destroyed before the session can crash the runtime's teardown (exit 0xC0000005). The C cube keeps its device until after the session too.
• Device last. CubeDevice's Drop waits for the device to go idle, then destroys the device and the instance:

impl Drop for CubeDevice {
    fn drop(&mut self) {
        // SAFETY: pipelines, cameras and senders on this device are dropped before it.
        unsafe {
            if let Err(error) = self.device.device_wait_idle() {
                eprintln!("vkDeviceWaitIdle failed at teardown: {error}");
            }
            self.device.destroy_device(None);
            self.instance.destroy_instance(None);
        }
    }
}

• Library. main drops the Library after run_app. It destroys the library but never unloads the DLL.

What the wrapper guarantees on this path: the frame record is borrowed, so it cannot be used after another frame call or the sender's destruction; writes to it happen only inside a frame cycle; one frame is open per sender. What it cannot see: your device's lifetime, your submissions, the semaphore signal, and the barriers.

Troubleshooting

The example says	Cause	What to do
Could not load QAROS. Install QAROS on this PC, then start this app again. (exit 2)	the next line says which: no runtime found by the default lookup, one that does not open, or one that lacks functions these bindings need	install QAROS, or set QAROS_RUNTIME_DIR to the bin folder of a runtime that matches the binding's header
The QAROS runtime failed to initialize: ... (exit 3)	the runtime loaded, but qar_library_init failed	read the message; check the QAROS installation
QAROS is not running on this PC. ...	ONBOARDING_HUB_UNREACHABLE: no Hub answered, and none could be started	start QAROS from the Start menu, then the example
QAROS did not allow this app to join ...	ONBOARDING_APPROVAL_DENIED: Deny, a closed prompt, or a revoked approval	start it again and click Allow
Nobody answered the approval prompt in QAROS in time. ...	ONBOARDING_APPROVAL_TIMEOUT	start it again and answer within 120 s
The program answering on this PC is not the installed QAROS ...	ONBOARDING_HUB_NOT_AUTHENTIC	reinstall QAROS; do not retry
This app no longer matches what QAROS approved ...	ONBOARDING_APP_IDENTITY_MISMATCH: the executable moved, was copied or re-signed	revoke it in the Visualizer, then start it again
the session's GPU (...) names no graphics GPU with the external memory and semaphore support QAROS needs, or the same without the session's GPU	no Vulkan device on the session's adapter (or on this machine) has a graphics queue and the required extensions	update the GPU driver; check that the GPU is supported
No sender for <target> yet (...); trying again in N s	the target did not answer the sender creation within 10 s	wait; the delay doubles up to 60 s
Stopped streaming to <target>: ...	a failure outside the normal flow, a closed stream included	nothing; the next listing serves the target again if it still runs
the sender reports ... where the cube renders ...	QAROS reported a pixel format the cube's pipeline does not render	the header reports no VkFormat; assumed_vk_format maps QAROS formats to UNORM and SFLOAT ones
your own app crashes at exit (0xC0000005)	the Vulkan device was destroyed before the session	drop the session first, the device last

From C

The C tutorial is Vulkan Rendering; the C original is the vulkan_cube source, and Getting Started builds and runs it. What corresponds:

C cube (main.cpp)	Rust
qar_library_load(NULL)	Library::load_default
JoinAsLocalApp, CancelOnStop	join_as_local_app, CancelToken::cancel_handle
ExplainOnboardingFailure	explain_onboarding_failure
exit codes 0, 1 (a bad argument, a failed qar_library_init, any other failure), 2, and 3 to 7 for a refused join	exit codes 0, 1 (a bad argument), 2, 3 (a failed qar_library_init), 4 for any other failure
SessionAdapter (qar_session_get_graphics_device_id)	Session::graphics_adapter
QarStreamParamsVulkan chained into QarRenderSenderInit	VulkanStreamParams, unsafe Session::create_vulkan_render_sender
qar_peer_spec_get_role is QAR_PEER_ROLE_TARGET	PeerInfo::is_target_app (qar_peer_spec_is_target_app, defined as the same test)
IsReachable: qar_peer_spec_get_presence is ONLINE or UNKNOWN	PeerInfo::app_state is AppState::Running (is_running_target): the deprecated app state, computed from the presence, so the same targets for every current peer (CPU Rendering)
one thread per target, blocking qar_render_sender_begin_frame	one loop, RenderSender::try_begin_owned_frame
qar_render_sender_frame_vulkan	OwnedFrame::vulkan_frame
FindFrameView	VulkanFrameState::view_index
writing layout, queue_family_index and fence into the record	VulkanFrame::release_to_external, set_image_state, set_fence
IsWaiting, the unresolved app volume included	is_waiting
frame info kept until show_frame (a std::unique_ptr with a deleter)	kept until the frame is shown or dropped
retry a failed target after 3 s	after 3 s, doubling up to 60 s
qar_session_handle_destroy, then qar_runtime_handle_destroy, the device after the session	drop the session, then the device; the runtime goes when run_app returns, the library when main drops it

Next

• docs/vulkan-api.md: every type of the Vulkan path, its safety rules, and how the stub emulates it.
• Rendering Streams: layouts, hand tracking, the lifecycle of a stream.
• Launched by QAROS: the contract this example follows, so an operator can add it to the Visualizer as a QAROS API app.