Real-Time QR Code Pose Estimation in Unreal Engine (Android Camera2 → UE | Quest 3)
This video showcases the first stage of my QR Code Pose Estimation pipeline in Unreal Engine 5, powered by a custom Android Camera2 → UE integration. The demo maps camera pixels to world rays using intrinsics and lens pose from Camera2, and validates poses against the eye view in XR.
What’s inside
Camera model: LENS_INTRINSIC_CALIBRATION (+ crop/scale to stream)
Extrinsics: LENS_POSE_ROTATION / LENS_POSE_TRANSLATION aligned to HMD tracking
Per-eye projection match in UE (CalculateStereoViewOffset, real projection matrices)
End-to-end: pixel → world ray → QR PnP → reproject back to UE view
Repo / branch
QR Code Pose Estimation – First Stage
➡️ GitHub (branch): QRCodePoseEstimation_FirstStage
https://github.com/yesidfonseca/Andro...
Hardware/Software
Unreal Engine 5.6, OpenXR (Quest 3)
Android Camera2 API (NDK r25b, API ≥ 28 recommended)
QR detection + PnP for 6DoF pose
Why this matters
Unlike compositor passthrough overlays, this path exposes true camera geometry (intrinsics/extrinsics), enabling metric-accurate alignment of real QR markers with virtual content—key for AR/MR calibration, anchoring, and robotic vision in UE.
#UnrealEngine #UE5 #OpenXR #Quest3 #Android #Camera2 #AugmentedReality #MixedReality #ComputerVision #QRCode #Intrinsics #Extrinsics #VRDev #ARDev #Passthrough #MetaQuest