Image tracking in Web AR: what it is and how to use it

There are several types of tracking in augmented reality that allow us to place digital content in different environments. These environments are the markers or trackers of the AR experience. They can be trackers based on surfaces or planes, on GPS positions, on complete spaces, on our own face or body, etc. Although one of the most used and widespread techniques in web AR in this technological field is the one we are going to discuss in this post: images.

What is Image tracking?

First of all, it is important to be clear about what we mean by image tracking when we talk about augmented reality experiences. With this type of technology what we are looking for is to place AR content on an image (tracker or trigger of the experience), on which the content will be fixed at all times. Our trigger can be any kind of visual element of our daily life: a poster, a magazine cover, a business card, a Christmas toy catalog, or even a picture of our vacations. Any image can be a potential marker of an augmented reality experience.

There are several relevant steps or phases when implementing an AR experience: the training or learning phase, the detection phase and the tracking phase. The training and detection phases will be discussed in detail below. As for the tracking phase, it is the one that allows us to place the content on the image, in its correct position, regardless of the camera’s perspective changing throughout the execution. That is, even if we move our phone, the content will remain fixed in its position, giving the feeling of being attached to that image. 

Therefore, this image tracking phase is the one that allows us to see the content anchored to the image at all times. In the example above, it is as if our 3D robot and the gifs of offers were completely associated with the packaging of that phone.

Training and image recognition

A fundamental phase for the augmented reality experience is the initial detection or recognition phase. This is the process by which our device is able to make a match between the image in front of it and the image that triggers the AR experience. 

To achieve this there is a previous phase of training, classification or learning (also known as machine learning). During this phase, the technology is able to analyze a photograph and recognize its key points: areas with contrast, changes in shape, changes in intensity, lines, etc. In essence, it is like marking the parts of the image that make it recognizable and distinguishable from other images. With this, our classifier generates what we call a descriptor for each image, which gathers an interpretation of all those key points.

The greater the number of key points in an image, the greater its ability to be recognized and, therefore, the greater its visual quality for use in an AR experience. This will make the final tracking better, more stable and realistic.

Image Tracking with Onirix

Onirix incorporates this type of AR experiences on images also adapted to the web environment, or what is known as web AR. This allows you to generate an experience on the image (or set of images) in a matter of minutes. All phases are covered automatically within the platform, so it is very simple, intuitive and transparent to generate a web AR experience from a specific image.

By incorporating this type of technology in web environments, anyone is able to create, customize and finally share or publish this type of experience (directly with a QR code or a web link). 

A lot of contextualized and relevant information can be provided starting from a specific image, adding different contents and events between the elements of the experience so that they interact with each other and with the user. Subsequently, corporate elements can be included to fully customize the experience (colors, fonts, logos, integration with an external system, etc.) and finally share it or publish it as part of a project.

Benefits of using AR with image tracking

The use of AR on images allows us to generate another level of interaction with content that surrounds us constantly. For example, any product has a packaging or any advertisement we see on the street is represented in an image such as a booth, a poster or supermarket or toy catalogs. Even elements such as instruction manuals, board games, or works of art in a museum. There are many types of content in our daily lives on which we can include AR. 

But what does this type of content bring us? This question will depend a lot on the sector or the objective of each experience. Perhaps it is something aimed at achieving greater engagement with an ad or product and increasing the rate of interaction with the brand. Or, to put another case, it may be focused on providing time savings through a mechanism that helps to reach a solution with a visual guide, for example in an installation process (a boiler, a coffee maker, a router, etc.). Let’s take a look at some of the most outstanding benefits:

• Time saving in the generation of graphic resources (catalogs, restaurant menus, signage in establishments). Thanks to the digitization of these supports, their life can be extended and reused, updating the associated digital content. Even through AR they can be designed taking into account this new life cycle.
• Generate a communication channel with the user, buyer or visitor. This is to establish an interaction tool through elements that are in our daily life and to be able to offer and obtain valuable information.
• Generate a visual interface mechanism that simplifies complex processes: installation, industrial maintenance, operations. To offer a quick way to access valuable information about something we have in front of us. In short: saving time and improving the user experience.
• Provide contextualized and high-value information, such as in tourist topics (facade of an emblematic building) or in cultural topics (explanations about relevant parts of a work of art) automatically.
• Include a system of analytics and traceability on images that already exist in our environment (advertisements, magazines, campaigns). To be able to measure the interaction with these elements.
• Include segmented and differentiated information in the same physical element without the need to redesign or have different versions of these resources (maps in shopping malls, informative posters, etc.).

With this example of a museum experience with image tracking you can test how AR can provide you with relevant information about the artwork and its most interesting parts, at the touch of a camera.

Recognition of multiple images: modes of use

Within the scope of image tracking it is convenient to explain how to use the experiences depending on the amount of images that may be required. In this section, we can present some cases and all of them are covered with the AR web module existing in Onirix:

Unitary experience: single image

Here we have a single image on which we want to place a content, so the image will be the trigger of the experience. For example, if we create an AR experience on the map of a shopping mall where we want to show the stores with offers or even a visual guide to shoppers. The same image will trigger the same experience, even though the image is located in different locations of the mall.

Experience with multiple images: no order set

This would be the case in which we have a series of images where any of them has an associated content. The user can choose which image to view and switch between them freely. This could be the case of a museum. As a visitor, you can enter a specific room where a series of works are located and, with a web AR experience, you can obtain information about each work. There is no specific order, but you can freely choose the next content to view.

Experience with multiple nested images: guidance order

A next level of complexity allows us to provide image recognition experiences where there is an established order. In this type of experience we can nest the execution of each simple experience and at the end it leads to the next one. That is, these would be experiences where to continue in the process, the user must first complete a challenge that allows him to discover the next step. This type of experience can be very useful to create treasure hunts or complete puzzles that give us clues at each step. 

Experience with a large number of images

Normally when we refer to multiple images we are talking about projects with a small number of images: about 10-15. But there are cases where it is required to use a very large image bank, with hundreds or even thousands of possible images on which to launch AR web experiences. Onirix is prepared for these cases, having the possibility to generate classifiers with thousands of images.

How to use image tracking with Onirix

Within Onirix it is very easy to create image tracking experiences for web AR by following these steps:

1. Scene and content creation:

• Create a project with an image type scene in Onirix and select the image that will be the marker.

• Include contents in the scene (animated 3D models, videos, audios, gifs, images).
• Add events and actions in the elements to generate interaction between the scene and the user.
• Include (optionally) customization in your scene to adapt it to a client’s brand, such as corporate colors, etc; through the online code editor.

1.1. Download existing experience from the Onirix library.

Fastest option that combines all of the above:

• Download to your Onirix account an already created experience from the Onirix experience library, marked as Image tracking.

2. Test and adjust the scene

Through the AR mode preview menu you can test your image tracking scene in a simple way, with your cell phone. Simply open the previewer and you will see two key elements: a QR that launches the AR web url from which you can start the experience (detection) and the experience marker that you will have to recognize. With this you can check the full flow of the experience and make the necessary adjustments to complete it.

In our example it would look something like this:

3. Sharing and validating the experience

Once your experience is fully defined you can easily share it through the scene editor menu. In essence, you can simply send a web URL via email, networks or messaging, which in turn will open a page with all the elements described above and allow anyone to view the experience. 

The recipient will have to open the web page on a desktop computer to get the marker reference and then pull out their cell phone to start enjoying your creation. 

Here is an example of a url of an Onirix experience: 

• If you open it from the desktop you will see all the elements as in the previous image.
• If you open it from the phone the experience will be launched waiting to detect the corresponding image.

Examples and use cases

As we have already advanced, image tracking experiences apply to any type of physical graphic content that surrounds us today. In this section we are going to give three examples of use cases, although dozens could be found.

Shopper loyalty: interactive experiences in the retail sector

Whether with loyalty cards, catalogs, flyers, posters or point-of-sale assemblies. By means of web AR experiences on images, interactive scenes can be included in any of these media. Let’s imagine a loyalty card on which instead of obtaining points in a classic way, we could play a game of can tossing, as in a fair. 

The cans you manage to knock down, for example associated with a brand of milk, could give the buyer a discount for that brand. In this way each supermarket can generate its own dynamic, and give it a personalization for each of the brands participating in the campaign.

Web AR in packaging: products that come to life

One of the most demanded use cases is the use of AR to give life to existing physical products. Through this type of experience we can include virtual and interactive content in almost any type of product. Everything we buy today has a well-defined brand image. For example, a box of cereal. Through an AR web experience we can add a dynamic associated with that box and create a communication with the final consumer. 

Dynamics like the one we see in this example could be perfectly embedded in a campaign for a specific product. We can include a visual experience or even add some gamification and generate an experience that helps to improve the brand image. In addition, we can obtain very valuable and segmented information about what type of users interact, how long they stay in the experience, and even promote their purchase through loyalty systems.

Image tracking experiences in user manuals

Imagine that before installing a router, the new coffee machine you have bought, or a boiler, you had a visual and quick way to access the most relevant information: the commissioning manual.

Through the AR web, this type of experience can be added to the different physical products and provide the user (or even the maintenance operator) with a tool that allows him to save time and access the information he needs. Here is a complete example of a boiler installation. 

For more information on these use cases and many more, please visit our library of template experiences.

How to design or choose a good image for your experience

The reference image or experience tracker is a key element. It must have sufficiently good visual characteristics so that the detection and loading time is fast, and the experience is stable (low vibration). There are also other external conditions that can affect the performance of the experience: brightness, reflective materials, weather conditions.

Design or choice of the image

It is important that the image is clearly identifiable. Especially if we use it in a multiple image experience, it is essential that there is differentiation between them to prevent the system from finding false positives (i.e. confusing one image for another). For this, in our documentation we explain in detail the characteristics of a good image. 

In Onirix, when generating an image type scene, the system itself generates the image trainer and gives it a percentage of goodness or quality. When we create a scene of this type, in a first step Onirix generates an empty canvas on which we can start adding content. This canvas also provides us with information about the quality of the image we have selected or designed. The ideal is to try to use images that are always above 50% (the higher the quality, the better the experience):

If you want to check the quality of any image before you start defining your experience you can use this checking service: https://marker-quality.onirix.com/ 

External factors to the image

It is important to take into account where your experience is going to be consumed. It is not the same if it is placed in a magazine than in a street poster where it can affect the change of daylight or weather conditions. Depending on the needs of your experience, you can even consider generating a classifier with several images that lead to the same content. The important thing is that your audience is able to interact with the content in a comfortable, intuitive and fast way. That way you can ensure that your experience is a success.

Conclusions

Image tracking is one of the main techniques for using augmented reality. It helps us to bring to life a multitude of visual elements that surround us in our daily lives and thus extend the life cycle of these elements. From advertisements, product packages, museum artwork or even user manuals. Any image can become a digital support to generate a bidirectional communication channel with our audience: buyers, museum visitors, or children choosing their next toy for Christmas. Thanks to its use through web AR, it allows us to generate very complex experiences in a very short time, including personalization to adapt to multiple customers and share its use with a massive audience. 

It is therefore a key piece in future digitalization, where everything 2D will gradually become 3D. Because our imagination is not configured in two dimensions, thanks to AR we can make our visual content acquire a new dimension.