In this demo I create two scenes where I access them through two UI buttons. A Raycaster were created on the right hand and used for pointing at objects as well selecting/clicking on them. Two buttons on the VR controller were in play: one for the Raycaster and one for clicking.
I used Unity Editor 2022.1.0 Version and oculus Quest 2 for this demo.