-
Notifications
You must be signed in to change notification settings - Fork 258
Add experimental support for WebXR based experiences. #73
Comments
Re-opening as we need to utilise multiple streamers to improve the experience |
@Belchy06 Super excited to see this! I can report that it's working well for me on Quest 2, but on Quest Pro, it seems like the HMD's IPD is wrong, or somehow looks super crosseyed. Any suggestions on settings I can try to fix it? Sorry if I should have started a seperate thread for this. Edit: and also, on the Quest 2, the FOV in webxr viewer seems to be much wider(a bit unnatural feeling) compared to the FOV of a native compiled apk app running in the headset. |
@SaibotC The IPD is currently a hard-coded value in the PixelStreamingHMD module of the PixelStreaming plugin. From my understanding, the WebXR API doesn't provide information regarding the device's IPD, so for systems with a variable IPD (like the HTC Vive) it may be difficult to properly propagate this information back to the application. For systems with a fixed IPD, I'm sure we could modify the C++ code in the plugin to check for the active system and modify the IPD accordingly. Edit: Regarding your edit, the FOV is also hard-coded. Something that should definitely be sent back to the application. |
@Belchy06 testing this version I'm currently getting the following message in my Browser-Log when I'm in Browser in WebXR Mode: "Msg: Attempted to send a message to the streamer with message type: XRHMDTransform, but the frontend hasn't been configured to send such a message. Check you've added the message type in your cpp" I suppose I need to implement a message Listener in my Unreal Project do you have any hint or example on how to do this? Mouse and keyboard inputs work fine. Thanks a lot beforehand |
Thanks for the fast reply, ok thats it we used 5.1 will try build from source and add your parameter. Thanks alot for your help :) |
We tried to build Unreal 5.2 from source on Windows
but received the following errors:
Furthermore there was a DirectX which could be fixed by providing the correct dll path. It is the same error as descibed here https://forums.unrealengine.com/t/errors-building-ue5-2-and-5-2-preview-1/775605/3 We are setting up a linux build to see if we can build there sucessfully. -> Issue is now fixed and it builds fine for VS2022 when applying the fixes mentioned in https://forums.unrealengine.com/t/errors-building-ue5-2-and-5-2-preview-1/775605/3 Merge request is in progress |
Okay, let us know how you go with the VR feature now you have a working build. |
But what about required hardwares ? But how about Nvidia Tesla hardwares ? If we rent Azure or AWS GPU instances, can we If answer is yes, this will be a game changer. |
The idea with this feature is that you'll be able to use your standard GPU enabled instances (eg g4dn.xlarge or similar) to stream out your XR experiences without the need for CloudXR. |
For those that have tested this so far you may have noticed that if you tilted your heard side-to-side (roll), the image would have had some weird distortion. This has since been fixed. |
Closing this issue as multi streamer support has landed in 5.2. See https://youtu.be/OHZ7rZc90ig on how you can setup multiple cameras. If you're looking to stream the spectator camera from the VR template, you want to follow everything Alex is doing, but use the SpectatorCamera instead of adding a new SceneCapture2D. |
Hi @Belchy06 and @lukehb as the Prerelease 5.2 is now available I tested the feature again with that release against Pixelstreaming Tag UE5.2-0.6.0 on Oculus Quest 2. The streaming and activation of xr mode is working and moving with the controllers as well. The movement of the headset and the controller position is not transmitted with that combination do you have any hints to get it working again? My "Additional Launch Parameters" argument for local testing currently looks like this:
Also building the current ue5-main branch again and will test there again in debugging. Engine/UE5/Plugins/Media/PixelStreaming/PixelStreamingHMD is the startingpoint i suppose. In any case thank you both for the feature |
If you're going to be debugging, I'd suggest chucking a breakpoint in this line https://github.com/EpicGames/UnrealEngine/blob/ue5-main/Engine/Plugins/Media/PixelStreaming/Source/PixelStreamingInput/Private/PixelStreamingInputHandler.cpp#L874 If your breakpoint hits, then you can be confident that the browser is transmitting the HMD position and orientation to UE. If the breakpoint doesn't hit, let me know as it may end up being an issue on the frontend. |
Hi guys ! Trying this new feature on the Meta quest 2 I still have the issue described earlier by @Belchy06. In the webXR experience the 2 images don't blend perfectly and it's giving the impression of squinting. Any idea of what could be wrong ? |
As of this commit, Pixel Streaming supports VR streamed experiences.
However, as of 7th of February 2023, the latest reference Epic Games implementation doesn't support these experiences.
Adding support for these experiences will require:
Known limitations:
The text was updated successfully, but these errors were encountered: