Today I worked on recovering the mesh that I lost last year. I actually made progress! I found a high definition mesh of the middle room that George and I made last year. I am now in the process of simplifying its geometry, as it is about 109KB right now, and Unity will not process it. I also worked on it after school after I had finished my sports.
Independent Science Research
Project Expanded Reality
Project Expanded Reality focuses on using virtual reality technology to transport users to locations that they would ordinarily be unable to travel to, or to give them a semi-firsthand experience within an environment- namely, HPA.
Introduction and Goals
I started working on Project Expanded Reality during the second semester of my junior year with George Donev, who began the project. I was motivated to continue this project because it presents prospective students with the opportunity to experience the HPA environment without physically traveling to the school. As a student who was unable to visit HPA before enrolling, I understand the importance of experiencing a potential living environment firsthand (or in this case, semi-firsthand). My goals for Project Expanded Reality this semester were to enable prospective HPA students to experience HPA in virtual reality and to develop connections with outside organizations dappling in VR, such as Keck and Canada France Hawaii observatories. I seek an answer to the following: how closely can I imitate the likeness of Hawaii Preparatory Academy in virtual reality?
Planning and Implementation
This semester, I intended to improve the existing virtual reality model of the energy lab by using higher quality photogrammetry and modeling programs such as Capturing Reality and Pix4D. I had also planned to expand the HPA virtual environment by creating new models of different rooms in the Energy Lab, along with the campus landscape and facilities. In my work on the project this semester, I gained a lot of first-hand experience in terms of virtual reality troubleshooting and development. I had anticipated some difficulty in gathering quality images from which the scene would be compiled (because I have little photography background and had only taken pictures for the purpose of environmental compilation once before). However, this proved to be less of an issue than I had originally predicted, as Daniel and Ilan have been very accommodating with their aerial photography.
I also had Pikoi, a sophomore from British Columbia, who is now my trainee/partner in Project Expanded Reality. He has experience with photography, and is almost as enthusiastic as I am. He was unable to take ISR this year, but sat in on most classes during F period to help me during the first quarter. Due to some scheduling conflicts, he had difficulty coming during F period second quarter, but will be back next semester. He has been helpful in the photogrammetry process and has been doing a lot to optimize the models by establishing ground control points.
A watershed moment in Project Expanded Reality was when I got the mesh of the middle room in the Energy Lab working again with the touch controllers and Oculus headset. By being creative and pulling ideas from many different tutorials and developer guides online, I figured out how to use C# in Unity. This now enables me to give presentations to tours and visitors; it was a relief to finally have a product to show for all my work this semester.
Challenges and Next Steps
My biggest challenge throughout this project has been recovering/regenerating the model of the middle room of the Energy Lab that George and I made last year, that I somehow deleted from Unity and saved over. I have tried regenerating it, but have only just now acquired a license for Capturing Reality, and consequently did not have time. I tried to recover it, but there are no safety nets on Unity for accidental missaving. However, I have gone on to work on other parts of the Energy Lab and have recovered the 3D meshes of the outside of the Energy Lab. I have worked to integrate those models with Unity; my efforts have been fruitful. I recently presented my project to some visitors from Punahou who came specifically to sit in on our ISR class, and had the model working well enough to give them a virtual tour.
Next semester, I plan to focus more on the outdoor environment of HPA. I will be working with Ilan and Daniel again to take aerial shots of the track, the pool, etc., and showcase the campus facilities. I also plan to finally finish the model of the Energy Lab.
Appendix A: Project Documentation
Elab main room: point cloud
This image shows the point cloud compiled in Capturing Reality of the main room in the Elab. Daniel took aerial photos with the drone, which allowed me to get an extremely accurate indication as to the location of the points.
Scene view of Unity mesh (left) and character view (right)
The image above shows the meshes featured in the Unity scene (featured on the left) and lists the components to the left of it. The image on the right is what an active viewer would see through the Oculus Rift.
Simplified scene view of Unity mesh (left) and character view (right)
This is an untextured version of the mesh in Unity shown in the previous image. Again, it shows the meshes featured in the Unity scene (featured on the left) and lists the components to the left of it. The image on the right is what an active viewer would see through the Oculus Rift.
Play mode: Oculus view of building mesh
The image above depicts the view of the mesh through the Oculus Rift. This is the refined view of the mesh featured in the polished version.
Recovered mesh of Elab from aerial footage
This is an image of the Elab mesh compiled from aerial footage from last year. As you can see, it needs some work.
Appendix B: Key Resources
This was one tutorial detailing how to set up the Oculus touch in Unity by making virtual hands out of cubes.
Other invaluable resources came via Ilan and Daniel and their assistance in aerial photography.
Another resource I constantly made use of was George and his weblog. I referenced the weblog to find where he had saved files I needed, and often had him send me information he had saved on his personal computer.