Vulkan Rendering
Source: qar-streaming-c/examples/vulkan_cube/main.cpp, with the renderer in renderer.cpp. Conceptual background: Rendering Streams.
Vulkan Rendering
What you build
The Vulkan cube: a rotating cube with one flat color per face, rendered on the GPU in stereo color and depth for every target app in the session. Each target, a Visualizer or a headset, gets its own sender and its own camera. The cube runs on the Hub machine, joins the Hub as a local app, creates its own Vulkan device on the GPU the session uses, and hands each frame to QAROS through the Vulkan render sender without a copy.
Take this path when your renderer already runs on Vulkan. QAROS creates the frame images on your device; you render into them and hand them back.
What you will learn:
- how to join as a local app, and how to cancel the approval wait on a stop request
- how to create your Vulkan device on the session's GPU with the extensions QAROS needs
- how to find the target apps and serve each on a thread of its own
- the frame hand-off: take each image from the state QAROS reports, render, release it to the external queue family, signal the frame's semaphore, write the new state back, show
- which handle lives how long, and why the device outlives the session
The cube is C++20, not C: it uses threads, a mutex and std::unique_ptr for the parts that are not QAROS. Every QAROS call is the C API. The program is qar-streaming-c/examples/vulkan_cube/: main.cpp holds everything QAROS, renderer.cpp the Vulkan device and the camera. This page shows their code. The CPU Rendering tutorial explains peers, app volumes and the frame cycle; this one builds on it.
Before you start
- Windows 10 or 11, x64. The cube is built for Windows only.
- A GPU whose Vulkan driver offers VK_KHR_external_memory_win32 and VK_KHR_external_semaphore_win32. QAROS supports NVIDIA GPUs (Choosing the GPU).
- QAROS installed on the Hub machine, where the cube runs. When the Hub does not run, the join starts it, and the Hub keeps running after the cube exits.
- Visual Studio 2022 Build Tools ("Desktop development with C++"), CMake 3.29 or newer and Git. The Vulkan headers, the loader, glm and the shader compiler come from vcpkg.
- What you do: on the first run, click Allow in the QAROS approval prompt, in the Visualizer or in a tray window, within 120 s (The approval prompt). The cube is not code-signed, so QAROS asks again after every rebuild.
- A target: open the Visualizer, or start a headset that joined the Hub.
Run it
Build from the root of this repository, in a Developer PowerShell for VS 2022. The first configure downloads vcpkg and builds the dependencies, which takes several minutes:
The cube takes no arguments. --help prints its usage; any other argument prints the usage to stderr and exits with 1. Stop it with Ctrl+C. When its stdin is a pipe, the end of stdin stops it too: that is how QAROS stops an app it started.
Expected output, with the Visualizer open, for a run that you stop later (your GPU and peer id differ; with console logging on, the runtime adds log lines of its own):
The three lines after Joining are the runtime's progress messages. While the approval prompt is open, one more appears: Waiting for approval in QAROS hub. In the Visualizer, the cube appears in the "Vulkan Cube" app volume, 1.5 m in front of the room origin.
Exit codes:
| Code | Meaning |
|---|---|
| 0 | stopped by Ctrl+C or the end of stdin |
| 1 | an argument other than --help or -h, qar_library_init or qar_runtime_create failed, or any other failure: a join failure not listed below, the GPU, the app volume, the peer list |
| 2 | no runtime loaded |
| 3 | the join could neither start nor reach the Hub (QAR_STATUS_ONBOARDING_HUB_UNREACHABLE) |
| 4 | the approval was denied or revoked (QAR_STATUS_ONBOARDING_APPROVAL_DENIED) |
| 5 | nobody answered the approval prompt in time (QAR_STATUS_ONBOARDING_APPROVAL_TIMEOUT) |
| 6 | the program answering on this machine is not the installed QAROS (QAR_STATUS_ONBOARDING_HUB_NOT_AUTHENTIC) |
| 7 | the executable no longer matches what QAROS approved (QAR_STATUS_ONBOARDING_APP_IDENTITY_MISMATCH) |
Walkthrough
1. Start, load the runtime, map failures to exit codes
A signal handler may safely do nothing but write a lock-free atomic, so Ctrl+C only sets g_stopRequested. Every QAROS and GPU cleanup stays on the threads that own the objects. When stdin is a pipe, a thread waits for its end and sets the same flag.
qar_library_load(NULL) runs the default lookup: the runtime beside the executable, the folder QAROS_RUNTIME_DIR names, the installed QAROS, then PATH. It prints where it looked when it finds nothing. The Dynamic Loading tutorial explains each step.
2. The order of the whole program
Join first: the session decides the GPU, and the device is created there. Note the order of the declarations. gpu is declared first, so it is destroyed last, after the session. Ownership and teardown explains why.
3. Join as a local app
A source app on the Hub machine joins as a local app: the user approves it once in QAROS, and there is no code to type. Nothing is stored; every start joins again.
The call can wait minutes for the approval, so it takes a cancel token. A second thread watches the stop flag and cancels the token when a stop arrives first. A zeroed QarOnboardLocalAppExt::approval_timeout_ms keeps the Hub's default of 120 s. No QarGraphicsDeviceId is chained: QAROS selects the GPU. The progress callback prints the runtime's messages, such as Waiting for approval in QAROS hub.
When joining fails, the cube tells the user what happened and what to do, by status code, and exits with that refusal's own code, the exit codes of Local App Onboarding:
These are the codes of Status codes and what to tell the user. Retry none of them silently. A stop during the wait is not a failure: RunApp returns 0 without an explanation.
4. Create the device on the session's GPU
QAROS creates the frame images on your device, and the receiver reads them on the session's GPU. Render on that GPU, so frames need no copy between GPUs. qar_session_get_graphics_device_id returns the adapter the session uses: the one a chained QarGraphicsDeviceId named, else the one QAROS selected, else the one the library picked.
VulkanDevice::Create in renderer.cpp takes the first physical device whose VkPhysicalDeviceIDProperties match that LUID or UUID, that has a graphics queue family, and that offers these device extensions:
When the session names an adapter and no physical device on it qualifies, the cube stops with an error rather than render on another GPU. When the session names none, it takes the first supported GPU and says so.
5. Find the targets
The app volume is the cube's own. A sender attaches its target to it, and that is what makes the target ask for the content:
Which targets to serve is the source app's choice. The cube serves every target app that is not gone. The peer callback runs on a library thread, so it only queues ids under a lock:
Subscribe first, then list the peers already there, so a target that joins in between is not missed. The main thread then manages targets only: it starts one for each new target app, restarts one that failed after 3 s, and forgets one whose stream closed for good.
6. One sender per target
Each Target runs on a thread of its own. It creates its sender, then renders until it is stopped or its stream fails. A target that never answers blocks only its own thread.
QarStreamParamsVulkan, chained into the init, hands QAROS your physical device and device. Each eye gets a color and a depth texture of its own, the QAR_FRAME_LAYOUT_SEPARATED_TEXTURES layout, four views in all. qar_render_sender_create blocks until the target answers; RequestStop cancels it through the target's token.
7. One frame
qar_render_sender_begin_frame paces the thread: it returns when this target wants its next frame, so each target runs at its own display rate, and a slow or paused one never delays another. The frame path sleeps only after a waiting status, 2 ms before the next begin.
The frame info it returns describes the begun frame. Keep it until qar_render_sender_show_frame has shown that frame. The FrameInfo type, a std::unique_ptr whose deleter calls qar_render_frame_info_handle_destroy, destroys it on every path out of RenderNext, after the show.
A begun frame is always shown, as in the CPU and D3D11 programs: the show closes the frame cycle that begin opened. When rendering fails, the camera first discards the frame by signaling its semaphore, the cube shows it, and the target ends. So a stop request that arrives after a successful begin waits for that one frame, and the thread stops at the next turn of its loop. One case differs, for a Vulkan reason: the show waits for the frame's semaphore, which comes with the frame record. When reading the record fails, nothing can signal it, so the target ends without a show and its sender is destroyed with it.
The pose and the field of view of each eye come from the view showing that eye's color. The poses are in the app volume's app content space, the space the cube renders in:
A failed begin-frame or show is not always the end of the target:
Reconnecting and transfer-less streams keep their sender, and so does an app volume that does not resolve yet; begin-frame holds the unresolved volume about 16 ms before it reports it. A paused stream does not show up here: begin-frame waits through the pause. Any other failure ends the target, QAR_STATUS_RENDERING_PRODUCER_STREAM_IS_CLOSED included; a closed stream is forgotten until the peer list offers the target again (Status codes that are part of normal flow).
8. Hand the images back
qar_render_sender_frame_vulkan gives the sender's frame: its views, its images with their current layout and owning queue family, and the frame's semaphore. The pointer stays valid until the next call to it or the sender's destruction. QAROS owns the images, their memory and the semaphore: never destroy, free or re-bind them.
The camera takes each image from exactly the state the frame reports, and leaves it in VK_IMAGE_LAYOUT_TRANSFER_SRC_OPTIMAL, released to VK_QUEUE_FAMILY_EXTERNAL:
Then, in this order: submit the work that signals the frame's semaphore, write the images' new state back into the frame, and only then show:
The camera keeps two frames in flight. It waits on a slot's fence only when it reuses that slot, never for the frame it just submitted: the semaphore hands that work to QAROS, which waits for it on the GPU.
A begun frame that could not be rendered still signals its semaphore, so the show that closes its cycle has something to wait for:
Image handles can change between frames. Read the frame every frame. The camera caches its image views and framebuffers by image handle, so a new handle gets new ones.
Ownership and teardown
| Object | Created by | Lives until |
|---|---|---|
| QarRenderFrameInfo* | qar_render_sender_begin_frame | the frame is shown or discarded; FrameInfo destroys it |
| QarVideoFrameVulkan* | qar_render_sender_frame_vulkan | the next call to it, or the sender's destruction; never freed by you |
| frame images, memory, semaphore | QAROS | the sender; never destroyed by you |
| camera (views, framebuffers, fences) | CubeCamera::Create | the Target, before its sender |
| QarRenderSender* | qar_render_sender_create | the Target; qar_render_stream_handle_destroy |
| target token | qar_cancel_token_create | the Target, after its sender |
| subscription token | qar_cancel_token_create | the end of Run, after every target |
| QarSession* | qar_runtime_onboard | RunApp, before the runtime and the device |
| VulkanDevice | VulkanDevice::Create | RunApp, last |
- Stop before join. Run asks every target to stop first, so all their waits end together, then destroys them. ~Target stops the thread and joins it before it touches anything the thread uses.
- Camera before sender. The camera waits for its own GPU work, then destroys its views and framebuffers over the sender's images while those images still exist. Then the sender goes, then its token.
- Subscription before its state. Targets lives in RunApp, longer than the subscription that writes into it. Cancelling the subscription's token waits for a running callback, so nothing writes into Targets afterwards.
- Session before device. QAROS keeps using your device until the session is gone, not only until the senders are. RunApp declares the device first, so C++ destroys it after the session and the runtime.
- Library last. main calls qar_library_destroy, then qar_library_unload.
Troubleshooting
| The cube says | Cause | What to do |
|---|---|---|
| Could not load QAROS. Install QAROS on this machine, then start this app again. (exit 2) | no runtime in the default lookup; the line before says where it looked | install QAROS, or fix QAROS_RUNTIME_DIR |
| an error logged by qar_library_init (exit 1) | the runtime loaded but did not initialize | read the message; check the QAROS installation |
| Could not start or reach QAROS on this machine. Start QAROS from the Start menu, then start this app again. (exit 3) | QAR_STATUS_ONBOARDING_HUB_UNREACHABLE: the join could not start the Hub, or not reach it | start QAROS from the Start menu, then the cube |
| QAROS did not allow this app to join ... (exit 4) | QAR_STATUS_ONBOARDING_APPROVAL_DENIED: Deny, or a revoked approval | start it again and click Allow |
| Nobody answered the approval prompt in QAROS in time. ... (exit 5) | QAR_STATUS_ONBOARDING_APPROVAL_TIMEOUT | start it again and answer within 120 s |
| The program answering on this machine is not the installed QAROS ... (exit 6) | QAR_STATUS_ONBOARDING_HUB_NOT_AUTHENTIC | reinstall QAROS; do not retry |
| This app no longer matches what QAROS approved ... (exit 7) | QAR_STATUS_ONBOARDING_APP_IDENTITY_MISMATCH: the executable moved, was copied or re-signed | revoke it in the Visualizer, then start it again |
| The session's GPU (...) names no graphics GPU with external memory/semaphore support (exit 1) | no Vulkan device on the session's adapter has a graphics queue and both extensions | update the GPU driver; check that the GPU is supported |
| Connected. Waiting for rendering targets. and nothing more | no target app runs in the session, or none answered its sender yet | open the Visualizer, or start a headset |
| Expected separated stereo color/depth layout | the sender reported another layout than the cube asked for | none in your setup; the cube ends that target and tries it again after 3 s |
| no more frames after the "Vulkan Cube" volume was removed | begin-frame reports QAR_STATUS_RENDERING_PRODUCER_APP_VOLUME_UNRESOLVED; a removed volume never resolves, and the cube does not recreate it | start the cube again |
| your own app crashes at exit | the Vulkan device was destroyed before the session | destroy the session first, the device last |
Next
- Rendering Streams: layouts, hand tracking, the lifecycle of a stream.
- Launched by QAROS: the contract the cube follows, so an operator can add it to the Visualizer as a QAROS API app.
- Vulkan Rendering in Rust: the same cube with the Rust binding.
Generated via doxygen2docusaurus 2.2.1 by Doxygen 1.9.8.