Vuforia and LiDAR Scanner
Vuforia is a mobile augmented reality software development kit (SDK) designed to enable developers to create augmented reality (AR) applications. It utilizes advanced computer vision technologies to recognize and track planar images and 3D objects in real time. This image registration functionality allows developers to position and orient virtual objects, such as 3D models and media, in relation to real-world objects when viewed through a mobile device's camera.
The SDK ensures that the virtual object continuously tracks the position and orientation of the image target, aligning the viewer's perspective with that of the target. This creates the illusion that the virtual object is seamlessly integrated into the real-world scene.
Vuforia supports a variety of 2D and 3D target types, including "markerless" Image Targets, 3D Model Targets, and VuMarks, which are addressable Fiducial Markers. Additional features include six degrees of freedom device localization, localized occlusion detection via "Virtual Buttons," runtime image target selection, and the ability to programmatically create and modify target sets during runtime. These capabilities make Vuforia a versatile tool for building engaging AR experiences.
Light Detection and Ranging (LiDAR) is a technology that has been in use for a considerable time. It employs lasers to emit pulses of light that reflect off objects and return to the source, measuring the distance based on the time it takes for the light pulse to complete its journey. This method is a form of time-of-flight measurement. Unlike some smartphones that use a single light pulse to determine depth, LiDAR-enabled devices emit waves of light pulses in a spread of infrared dots. Each pulse is measured by a sensor, creating a field of points that map out distances and generate a "mesh" of the space and objects within it.
These light pulses are invisible to the human eye but can be detected with a night vision camera. In one of the game modes for iOS devices, LiDAR technology is utilized to scan the environment and create a mesh. This mesh allows the in-game car to interact with real-world objects, enabling realistic collisions with the surrounding environment.
Gameplay
The initial project plan focused on creating an RC Car game using ARCore and ARKit plugins. However, these technologies have since been integrated into ARFoundation. A car model was downloaded following a YouTube tutorial to test the functionality and mechanics of the car. The game was designed to be launched on both iOS and Android devices, with the game mode accessible on both platforms.
This game mode allows players to control a 3D car using a virtual joystick, scale it in the real world, and place it on any surface. Players can move around in the real-world environment to interact with the car, change its colors, turn its lights on and off, and open or close its doors, offering an immersive augmented reality experience.
Version 2
In this version, the integration of the AR Car with LiDAR has been implemented. Currently, this game mode is available exclusively for iOS devices, as only iPhone 12 Pro, 13 Pro + models, and iPad Pro models are equipped with the LiDAR sensor. This exciting feature allows players to scan their surroundings and generate a mesh. The car in this feature will interact with the real-world environment, colliding with scanned objects, which enhances the immersive experience.
Additionally, obstacles have been added to the environment, providing more challenge and making the gameplay more engaging, rather than simply controlling the car. The LiDAR sensor continuously scans the surroundings and updates the game scene with the mesh, allowing the car to interact with real-world objects in real-time.
Design Process
The car was created using Unity's ProBuilder package, while Photoshop was used to design the UI and controls. The controls include left and right buttons for steering and accelerate and brake buttons for moving the car forward and reversing.
In the development phase, a Unity camera was used instead of the original AR camera to provide a clearer understanding of how the game mechanics work. This setup allowed for more precise adjustments and testing during the creation process.
The script RearWheelDrive allows the player to control the car in the AR Scene in this, Max steer angle and maxTorque are initialised and which can later be changed in the inspector. This controls the wheel colliders that are attached to the wheels of the car.
GameOptions.cs will allow the player to press the meshing on button which will allow the LiDAR scanner to scan the surroundings and create a mesh. That will later allow the player to turn it off when the mesh has been scanned.
In LiDAR RC Car game mode, the reticle will be used to place objects in the game scene and the PlayerMissionManager.cs is Responsible for that, the objects get placed in the game scene and when the car collides with that object, the object gets picked up.
The initial phase of the project laid the groundwork for development by addressing the project's context, objectives, and potential benefits. This phase highlighted the importance of understanding the project's scope and purpose to establish a solid foundation for subsequent stages. A comprehensive literature review provided insights into Augmented Reality (AR) technologies and their impact on daily life, while a competitor analysis helped identify market trends and gaps.
Key advantages of the project include cost-effectiveness due to the use of free software and targeting widely accessible mobile devices, eliminating the need for expensive AR hardware. The project scope was clearly defined to guide planning and execution. A survey involving 45 target users helped gather feedback to refine the development approach. Tools like Unity 3D, Visual Studio, and Photon Unity Networking were chosen for game development, with Android and iOS as target platforms.
Limitations
The project encountered several challenges:
Outdated Tutorials: Many online tutorials for AR car games were based on deprecated Unity packages, complicating the transition to AR Foundation.
Testing Constraints: iOS testing required XCode, unavailable on Windows PCs. Without access to an updated Mac or lab resources, iOS 15+ compatibility tests were hindered.
LiDAR Accessibility: Limited availability of LiDAR-equipped devices (iPhone 12+ and iPad Pro) required borrowing suitable hardware for testing.
Future Improvements
Future enhancements could include:
Improved Graphics: Creating realistic car models, textures, and shaders to blur the line between virtual and real objects.
Custom Assets: Designing original assets to enhance the visual appeal and uniqueness of the game.
Expanded Platform Support: Optimizing tools and workflows for better cross-platform testing and compatibility.
In conclusion, while challenges existed, the project's initial phase effectively established a strong foundation. The development process provided valuable learning experiences, and future improvements could elevate the game’s realism and accessibility.
