Expop - Our Augmented Reality (AR) Application

One of our esteemed Client in Education industry approached us with an idea of developing an infotainment mobile app for their students. The requirement was to engage students to answer a set of questions and the answers these questions were placed in virtual balloons which will pop-up near a location within a  specific area which will be related to the question.

We develop an AR (Augmented Reality) mobile app is in native iOS and Android and me the clients requirement to their satisfaction. This AR app will be showcased as a innovative idea and learning solution in Education sector in upcoming 2020 Expo.

The summary of the Augmented Reality Mobile App and other technical details of development are as follows,

Augmented Reality in iOS integrates iOS device camera and motion features to produce augmented reality experiences in our app or game. AR describes user experiences that add 2D or 3D elements to the live view from a device’s camera, in a way that makes those elements appear to inhabit the real world. ARKit combines device motion tracking, camera scene capture, advanced scene processing, and display conveniences to simplify the task of building an AR experience, using either the back camera or front camera of an iOS device. AR consists of combining the real world image with an image which is computer generated. AR is now into all aspects of business including the travel and tourism, real estate, online retail etc. AR provides a new way for business requirement in the form information sharing and interaction capabilities. The information may be in the form of video, graphics, sound and more. Augmented Reality has some benefits for business which includes improved information sharing, easy visualization, unique user experience, improve brand awareness and competitive differentiation.

During the initial stages of ARKit, we’ve received a proposal from one of our clients to develop an AR app.

Requirements

  • Create an AR Application ‘Expop’ for the Kids in School
  • They can open the application and view the surroundings via the camera
  • Random balloons will pop up at various locations around them at random time intervals and they have to find it through the camera
  • The kids can pop ‘em up and score points during limited game time
  • Once the game is over, their score will be calculated and displayed against their current high score
  • The students with the highest score can display it to the teacher for rewards

Plan & implementation

In the beginning, it was very difficult due to the unawareness and the limited time period. We took it as a challenge and tried.

Since our projects were in native XCode’s single view, basic Obj-C style, most of our developers were not familiar with the ARKit Coding style and gaming. It took 3 days to complete the project and test it (including R&D)

During the development, we’ve got the chance to learn more about the ARKit. It has:

  • World Tracking
  • Face Tracking
  • React to people
  • Image Tracking
  • Object Tracking
  • Orientation Tracking
  • Positional Tracking
  • Custom Recognition
  • Custom Display

And so on…

  • World tracking:

It is the ability to create and track a correspondence between the real-world space the user inhabits and a virtual space where you can model visual content. When your app displays that content together with a live camera image, the user experiences augmented reality: the illusion that your virtual content is part of the real world.

  • Face tracking:

Track faces that appear in the front camera feed. Face Tracking allows the users of the app to try out the products that best fits them such as eyewear, jewellery etc. LensKart is an example of AR face tracking which helps the customers to choose their lens that best suits them.

  • React to people:

  React to people that ARKit identifies in the camera feed.

  • Image Tracking: 

Recognize images in the physical environment and track their position and orient ation. AR apps are designed in such a way that they can track real time images at any anytime and also can display 2D/3D images, videos, audios etc. This helps to match out thousands of images within the smartphone without any internet connection with the help of real time image recognition using Augmented Reality SDK. It match the images within seconds with the help of the smartphone hardware.

  • Object Tracking: 

Recognize known objects at run-time by first scanning them with the scanner app. AR apps also track and match the 3D objects in polygonal shapes such as cylinder, cube, sphere, prism etc. The app is equipped with such features that helps for fast object tracking and matching.

  • Orientation Tracking: 

A configuration used when you just want to track the device’s orientation using the device’s back camera.

  • Positional Tracking: 

A configuration used when you just want to track the device’s position in space.

  • Custom Recognition: 

Create anchors that track objects you recognize in the camera feed, using a custom optical-recognition algorithm.

  • Custom Display: 

Create a full-featured AR experience by implementing your own renderer.

  • Body Tracking:

AR apps helps to track and match the entire human body by using the technology like Microsoft Kinect. Retailers can make use of this feature for a better experience. As the customers can look how they look in their favorite apparel and dresses at the time of purchase using the AR app that gives them more immersive experience.

  • Hand Tracking:

The AR app can better track and detect ten fingers in the hand at utmost precision using the Leap Motion Technology. The users can play games, paint pictures and much more by connecting the leap motion device to their laptop or PC. Leap Motion Controller is a small USB peripheral device which can be connected to a desktop or laptop.