Thicket data repository for the EEG
anil.recoil.org
1{
2 "id": "https://ryan.freumh.org/vr-ar.html",
3 "title": "Virtual and Augmented Reality",
4 "link": "https://ryan.freumh.org/vr-ar.html",
5 "updated": "2022-04-19T00:00:00",
6 "published": "2022-04-18T00:00:00",
7 "summary": "<div>\n \n <span>Published 18 Apr 2022.</span>\n \n \n <span>Last update 19 Apr 2022.</span>\n \n </div>\n \n \n\n <p><span>I’ve been interested in virtual reality (VR) and\naugmented reality (AR) recently as the use case for the system(s) I’m\nworking on in my MPhil project. This blog post will cover my development\nexperience in porting my <a href=\"../cubes/\">cubes</a> game to both VR\nand AR, and give a small demo for the latter.</span></p>\n<p><span>Microsoft Windows was used for ease of development\nwith <a href=\"https://unity.com/\">Unity</a> and as it’s a hard\nrequirement for the AR headset made by Microsoft.</span></p>\n<h2>VR</h2>\n<p><img src=\"./images/quest2.jpg\"></p>\n<p><span>The <a href=\"https://www.oculus.com/quest-2/\">Oculus Quest 2</a> from Meta\n(formally Facebook) is a VR headset running an Android derivative. This\nmeans you can do cool stuff like run <a href=\"https://www.reddit.com/r/cyberDeck/comments/fc5sfr/oculus_quest_termux_selfcontained_hmd_cyberdeck/\">Termux</a>\non it (an Android terminal emulator and unvirtualised Linux\nenvironment). It’s nowhere near as flexible as something like what <a href=\"https://simulavr.com/\">SimulaVR</a> are doing, which features a\nneat <a href=\"https://github.com/SimulaVR/Simula\">Haskell Wayland\ncompositor</a>.</span></p>\n<p><span>With access to the Oculus Quest 2, I was first\nable to explore the world of free games on the <a href=\"https://www.oculus.com/experiences/quest/\">Oculus\nstore</a>.</span></p>\n<p><span>I quickly found it outlived its novelty,\nhowever, and as I’ve always enjoyed making things more than using them I\nmoved on to try and develop something for the headset. Instead of\ncreating something brand new, I decided to try and port my existing\ncubes game created back at the end of my final undergrad\nyear.</span></p>\n<p><span>Development for the Quest is quite a polished\nexperience, although I may only be saying this in hindsight after\ndeveloping for the Hololens. Bar some linkrot, the <a href=\"https://developer.oculus.com/documentation/unity/unity-gs-overview/\">documentation</a>\ncovers setting up a development environment and creating an app. The\nbasic gist is to install Unity with some Android and <a href=\"https://docs.unity3d.com/Manual/XRPluginArchitecture.html\">VR\nsupport</a> plugins. You also need either developer mode to sideload\napps, a compatible USB-C cable, or <a href=\"https://support.oculus.com/airlink/\">Oculus Air Link</a> with\nWi-Fi that plays nice. I opted for the latter. With some modifications\nto the <a href=\"https://github.com/RyanGibb/cubes/compare/vr#diff-4fa152eddce941e5e1dba548d9551e10a7055f8cc4ceabf8085efce55054a12eL149\">scene\ncamera</a> and the <a href=\"https://github.com/RyanGibb/cubes/compare/vr#diff-6559439996a27afac6044cabe35d54d1d69bed7fc62874eb3d86bfaf3370d1a3R1\">movement\nscript</a> it was good to go.</span></p>\n<p><span>UX for VR is a whole different ball game - but\nit didn’t give nearly as much nausea as it might have.</span></p>\n<p><span>Many thanks to <a href=\"https://pjohnsonportfolio.wordpress.com/\">Paul Johnson</a> for\nlending me this device.</span></p>\n<p><span>See the source code <a href=\"https://github.com/RyanGibb/cubes/tree/vr\">here</a>.</span></p>\n<h2>AR</h2>\n<p><img src=\"./images/hololens2.jpg\"></p>\n<p><span>The <a href=\"https://www.microsoft.com/en-us/hololens/\">Microsoft Hololens\n2</a> from Microsoft is an AR headset running the “Windows Holographic”\nOS - essentially a 64bit ARM version of windows with a spatial window\nmanager. This was a real pain as it turns out it doesn’t support Win32\napplications - which killed my dreams of running the <a href=\"https://github.com/microsoft/terminal\">Windows Terminal</a> on it.\nSee <a href=\"https://github.com/microsoft/terminal/issues/12219\">this</a>\nGitHub issue for more info</span></p>\n<p><span>Aside from being hostile to hacking, the\ndevelopment experience was quite horrible. Recalling from memory, the\ndevelopment environment setup steps were something like:</span></p>\n<ul>\n<li>Install Unity Hub</li>\n<li>Install Unity Editor through Unity Hub</li>\n<li>Install Windows 10 SDK 10.0.18362.0 or later</li>\n<li>Install Microsoft Mixed Reality Toolkit\n<ul>\n<li>Purportedly requires <a href=\"https://dotnet.microsoft.com/en-us/download/dotnet/5.0\">.NET 5.0\nruntime</a>, but actuality requires <a href=\"https://dotnet.microsoft.com/en-us/download/dotnet/thank-you/runtime-desktop-5.0.13-windows-x64-installer\">.NET\nDesktop Runtime 5.0</a></li>\n</ul></li>\n<li>Visual Studio requires a restart</li>\n<li>Install workloads:\n<ul>\n<li>‘Desktop development with C++’</li>\n<li>‘Universal Windows Platform (UWP) development’</li>\n<li>‘Game development with Unity’</li>\n<li>C++ (v142) Universal Windows Platform tools (required when using\nUnity)</li>\n</ul></li>\n<li>Compile the Unity project to a Visual Studio project</li>\n<li>Change Visual Studio startup project to Cubes from\nIl2CppOutputProject</li>\n<li>And run on the Hololens from there (which also installs the\nApp)</li>\n</ul>\n<p><span>Having started to <a href=\"https://github.com/RyanGibb/nixos/\">use</a> the <a href=\"https://nixos.org/\">nix</a> package manager recently this is\nexactly the dependency and reproducibility headache I’m trying to\navoid.</span></p>\n<p><span>To its credit, the Visual Studio remote machine\nexecution over the local network worked very well once it was set up.\nThe <a href=\"https://docs.microsoft.com/en-us/windows/mixed-reality/develop/advanced-concepts/device-portal-api-reference\">device\nportal API</a> also proved very useful in watching a real-time feed of\nthe user’s POV.</span></p>\n<p><span>This <a href=\"https://docs.microsoft.com/en-us/learn/modules/learn-mrtk-tutorials/\">article</a>\nproved the most useful in setting up the development environment.\nNavigating the Microsoft documentation proved challenging in\nitself.</span></p>\n<p><span>The modifications to the project were more\nsignificant than for the Quest 2 but mainly boiled down to getting\nthings working the way the Microsoft Mixed Reality Toolkit (MRTK)\nwanted.</span></p>\n<p><span>While this game isn’t taking full advantage of\nthe possibilties of AR, the main advantage I found was a reduced fear of\nsmacking your head against the wall while dodging some red\ncubes.</span></p>\n<p><span>Many thanks to <a href=\"https://anil.recoil.org/\">Anil Madhavapeddy</a> for lending me\nthis device.</span></p>\n<p><span>See the source code <a href=\"https://github.com/RyanGibb/cubes/tree/ar\">here</a>.</span></p>\n<h2>Demo</h2>\n<p><span>See a demo of this game on the Hololens 2 with\nsome willing participants below.</span></p>",
8 "content": "<div>\n \n <span>Published 18 Apr 2022.</span>\n \n \n <span>Last update 19 Apr 2022.</span>\n \n </div>\n \n \n\n <p><span>I’ve been interested in virtual reality (VR) and\naugmented reality (AR) recently as the use case for the system(s) I’m\nworking on in my MPhil project. This blog post will cover my development\nexperience in porting my <a href=\"../cubes/\">cubes</a> game to both VR\nand AR, and give a small demo for the latter.</span></p>\n<p><span>Microsoft Windows was used for ease of development\nwith <a href=\"https://unity.com/\">Unity</a> and as it’s a hard\nrequirement for the AR headset made by Microsoft.</span></p>\n<h2>VR</h2>\n<p><img src=\"./images/quest2.jpg\"></p>\n<p><span>The <a href=\"https://www.oculus.com/quest-2/\">Oculus Quest 2</a> from Meta\n(formally Facebook) is a VR headset running an Android derivative. This\nmeans you can do cool stuff like run <a href=\"https://www.reddit.com/r/cyberDeck/comments/fc5sfr/oculus_quest_termux_selfcontained_hmd_cyberdeck/\">Termux</a>\non it (an Android terminal emulator and unvirtualised Linux\nenvironment). It’s nowhere near as flexible as something like what <a href=\"https://simulavr.com/\">SimulaVR</a> are doing, which features a\nneat <a href=\"https://github.com/SimulaVR/Simula\">Haskell Wayland\ncompositor</a>.</span></p>\n<p><span>With access to the Oculus Quest 2, I was first\nable to explore the world of free games on the <a href=\"https://www.oculus.com/experiences/quest/\">Oculus\nstore</a>.</span></p>\n<p><span>I quickly found it outlived its novelty,\nhowever, and as I’ve always enjoyed making things more than using them I\nmoved on to try and develop something for the headset. Instead of\ncreating something brand new, I decided to try and port my existing\ncubes game created back at the end of my final undergrad\nyear.</span></p>\n<p><span>Development for the Quest is quite a polished\nexperience, although I may only be saying this in hindsight after\ndeveloping for the Hololens. Bar some linkrot, the <a href=\"https://developer.oculus.com/documentation/unity/unity-gs-overview/\">documentation</a>\ncovers setting up a development environment and creating an app. The\nbasic gist is to install Unity with some Android and <a href=\"https://docs.unity3d.com/Manual/XRPluginArchitecture.html\">VR\nsupport</a> plugins. You also need either developer mode to sideload\napps, a compatible USB-C cable, or <a href=\"https://support.oculus.com/airlink/\">Oculus Air Link</a> with\nWi-Fi that plays nice. I opted for the latter. With some modifications\nto the <a href=\"https://github.com/RyanGibb/cubes/compare/vr#diff-4fa152eddce941e5e1dba548d9551e10a7055f8cc4ceabf8085efce55054a12eL149\">scene\ncamera</a> and the <a href=\"https://github.com/RyanGibb/cubes/compare/vr#diff-6559439996a27afac6044cabe35d54d1d69bed7fc62874eb3d86bfaf3370d1a3R1\">movement\nscript</a> it was good to go.</span></p>\n<p><span>UX for VR is a whole different ball game - but\nit didn’t give nearly as much nausea as it might have.</span></p>\n<p><span>Many thanks to <a href=\"https://pjohnsonportfolio.wordpress.com/\">Paul Johnson</a> for\nlending me this device.</span></p>\n<p><span>See the source code <a href=\"https://github.com/RyanGibb/cubes/tree/vr\">here</a>.</span></p>\n<h2>AR</h2>\n<p><img src=\"./images/hololens2.jpg\"></p>\n<p><span>The <a href=\"https://www.microsoft.com/en-us/hololens/\">Microsoft Hololens\n2</a> from Microsoft is an AR headset running the “Windows Holographic”\nOS - essentially a 64bit ARM version of windows with a spatial window\nmanager. This was a real pain as it turns out it doesn’t support Win32\napplications - which killed my dreams of running the <a href=\"https://github.com/microsoft/terminal\">Windows Terminal</a> on it.\nSee <a href=\"https://github.com/microsoft/terminal/issues/12219\">this</a>\nGitHub issue for more info</span></p>\n<p><span>Aside from being hostile to hacking, the\ndevelopment experience was quite horrible. Recalling from memory, the\ndevelopment environment setup steps were something like:</span></p>\n<ul>\n<li>Install Unity Hub</li>\n<li>Install Unity Editor through Unity Hub</li>\n<li>Install Windows 10 SDK 10.0.18362.0 or later</li>\n<li>Install Microsoft Mixed Reality Toolkit\n<ul>\n<li>Purportedly requires <a href=\"https://dotnet.microsoft.com/en-us/download/dotnet/5.0\">.NET 5.0\nruntime</a>, but actuality requires <a href=\"https://dotnet.microsoft.com/en-us/download/dotnet/thank-you/runtime-desktop-5.0.13-windows-x64-installer\">.NET\nDesktop Runtime 5.0</a></li>\n</ul></li>\n<li>Visual Studio requires a restart</li>\n<li>Install workloads:\n<ul>\n<li>‘Desktop development with C++’</li>\n<li>‘Universal Windows Platform (UWP) development’</li>\n<li>‘Game development with Unity’</li>\n<li>C++ (v142) Universal Windows Platform tools (required when using\nUnity)</li>\n</ul></li>\n<li>Compile the Unity project to a Visual Studio project</li>\n<li>Change Visual Studio startup project to Cubes from\nIl2CppOutputProject</li>\n<li>And run on the Hololens from there (which also installs the\nApp)</li>\n</ul>\n<p><span>Having started to <a href=\"https://github.com/RyanGibb/nixos/\">use</a> the <a href=\"https://nixos.org/\">nix</a> package manager recently this is\nexactly the dependency and reproducibility headache I’m trying to\navoid.</span></p>\n<p><span>To its credit, the Visual Studio remote machine\nexecution over the local network worked very well once it was set up.\nThe <a href=\"https://docs.microsoft.com/en-us/windows/mixed-reality/develop/advanced-concepts/device-portal-api-reference\">device\nportal API</a> also proved very useful in watching a real-time feed of\nthe user’s POV.</span></p>\n<p><span>This <a href=\"https://docs.microsoft.com/en-us/learn/modules/learn-mrtk-tutorials/\">article</a>\nproved the most useful in setting up the development environment.\nNavigating the Microsoft documentation proved challenging in\nitself.</span></p>\n<p><span>The modifications to the project were more\nsignificant than for the Quest 2 but mainly boiled down to getting\nthings working the way the Microsoft Mixed Reality Toolkit (MRTK)\nwanted.</span></p>\n<p><span>While this game isn’t taking full advantage of\nthe possibilties of AR, the main advantage I found was a reduced fear of\nsmacking your head against the wall while dodging some red\ncubes.</span></p>\n<p><span>Many thanks to <a href=\"https://anil.recoil.org/\">Anil Madhavapeddy</a> for lending me\nthis device.</span></p>\n<p><span>See the source code <a href=\"https://github.com/RyanGibb/cubes/tree/ar\">here</a>.</span></p>\n<h2>Demo</h2>\n<p><span>See a demo of this game on the Hololens 2 with\nsome willing participants below.</span></p>",
9 "content_type": "html",
10 "categories": [],
11 "source": "https://ryan.freumh.org/atom.xml"
12}