SensiMARDinofelisAR: mobile AR for Cultural Heritage
For two days, Conimbriga’s visitors were able to view, in a 360 degrees angle, a majestic reconstruction of a Forum from the Roman Era superimposed over its current ruins.
The use of Augmented Reality (AR) has a positive impact in users’ perspective and was recognized as a good solution to use in diversified areas.
DinofelisAR, an AR app, aimed to evaluate a mobile augmented reality application created for outdoors in order to represent an ancient building - arising from the Roman civilization - which is, currently, in ruins. This application was tested by visitors from the Conimbriga’s Ruins, managed by the Conimbriga Monographic Museum-National Museum, Portugal.
Before Implementing the Final Prototye
Techniques for AR recognition and tracking were primarily explored, in order to predict the most effective ones for overlaying 3D models in real-time over real images using the smartphone’s camera in outdoor environments.
3D Object Recognition
Using Vuforia, objects were scaned to later be used in the recognition process to trigger the AR experience. Several objects were tested and the better results were acquired with objects with a geometry made of well defined faces. In order to reach better results when exploring the virtual models – and because it is intended to explore a large-scale environment – they should be scanned with large image targets maps, thus ensure more stability while exploring the virtual element. The following picture shows the difference between the size of the 3D virtual element added to the real scene, with the same real object scanned using different image map target sizes.
figure 1 - Demonstration of the virtual size difference on the same object, when scanned with fifferent target sizes: Format 297 cm × 420 cm (left), and 1340 cm x 950 cm (right).
The extended tracking provided by Vuforia can work well, especially in environments without many artifacts. However, the user must be aware that he/she can’t move too fast or the tracking will lose some references and change the position of the virtual object. The tests made by changing the lighting conditions, disclosed great sensitivity to any changes, causing failed AR experiences.
The overall conclusion for this demonstration emphasizes the successful results when using extended tracking but also a critical remark for the object recognition in consideration of lighting conditions. This determined that the use of real objects to recognition and tracking should not be used for uncontrolled lighting conditions, such as outdoors experiences.
SDKs for mobile AR
Due to a large quantity and diversity of frameworks available to support the development of AR systems, a research study was made in order to identify which were the frameworks that would fulfil the requirements needed for this project.
Considering this prototype requirements, three frameworks seemed comply the requisites: Vuforia SDK, Kudan SDK and, Wikitude SDK. The referred study revealed that, for the specific tests made, besides the fact that all presented good results (two examples of the severeal tests made are showen in the follwoing two figures), Vuforia SDK was the one which best fulfilled the needs for this prototype.
Figure 2 - Minimum distance needed in relation the target image, with brightness changes of the environment, comparing the three SDKs evaluated.
Figure 3 - Comparison of the three tested SDK, regarding to oscillation and to error of positioning. Two different marks were used: a 2D image and a real image which is generated by capturing a picture of the real scene and use this captured image as mark to recognition and tracking.
In addition to the good performance tests achieved by Vuforia SDK, this framework was the only one that, in the free version for developers, allowed to track the camera with natural features when the marker gets out of the field of view with a feature named by Vuforia as Extended Tracking. Although this technique appeared to be less stable when compared to the marker-based tracking, especially with restricted light conditions, this feature enhanced the application flexibility and freedom to navigate in space.
DinofelisAR, the prototype created to be tested in-situ, was developed in Unity 3D and using a Vuforia package for Unity. An image was used as mark (figure 4) and the functionality Extended Tracking was activated in order to allow users to explore the surroundings even if the image used as mark, got out of the field of view.
Figure 4 - Image used as mark for the experiment using the AR app DinofelisAR in-situ.
Prototype Tests in-situ
DinofelisAR prototype was outlined relying on previous studies focused on identifying the more suitable technique and framework to implement an AR system in an archaeological site - bearing in mind the user tests in loco.
This prototype was created for Android devices using Vuforia SDK. An image - known as a mark - was used to detect the position and orientation of the user. This technique is used to place a virtual building - The Forum - in the proper position when overlapped to its ruins, perceived trough the smartphone’s camera.
Local of the Experiment
The intention to evaluate DinofelisAR app by users when used in cultural heritage contexts, took us to test the prototype in Conimbriga. Conimbriga is a Roman provincial city which was abandoned during the medieval age and it was inhabited between centuries IX B.C. and A.D. VII–VIII.
DinofelisAR app was tested in the location where used to be edified the Forum – a public monument with large dimensions which was virtually reconstructed – and its position corresponds to the centre of the ancient city (Figure 5). Its dimensions were approximately 48 x 96 m and it would be around 9 m high.
Figure 5 - Identified by number 23, highlighted in red, the Forum’s location is stressed (image created based on the information available at www.conimbriga.pt)
Users’ Experiment
The tests occurred between 17th and 19th of May of 2017 in the Ruins of the Conimbriga Monographic Museum-National Museum.
For the AR experience, a smartphone was handed to each participant (Motorola 3rd Generation 5.0’’ ) and they were invited to lauch the application DinofelisAR, then to explore the surroundings with the smartphone to visualize the virtual model over its ruins, and to fill-in a questionnaire.
Figure 6 - View of the place before the AR experience (left) occurs and, during the AR experience with DinofelisAR, with the virtual model visible over Forum’s ruins (right).
Results
A total of 90 participants accepted to be part of this study.
The results collected disclosed that the participants considered that AR contributed for a more dynamic and active visit to the Forum (94%) and more pleasant as well (94%). Moreover, 92% of the participants agreed or strongly agreed with the allegation of being more involved with this tour zone when using the AR app.
97% of the participants agreed or strongly agreed that AR applications can contribute to cultural enrichment during their visits. Considering the Forum features acquired, 91% of participants stated that became able to describe better the Forum characteristics – e.g. size, colours, architecture.
An interesting fact about the use of AR technology, is that even people who never used this technology before this experiment (61%), considered it easy to use, as well as those who already had tried it before (figure 6).
The majority (84%) didn’t feel lost during the experience, not knowing very well where to point the smartphone in order to see the virtual model. Furthermore, most of the participants (88%) assumed that after some short moments using AR technology, they already were comfortable using it and understanding how to use it correctly.
Conclusions
The results emphasize that AR technology brings added value to CH by benefiting visitors and institutions as well. Despite performance issues of this technology didn’t hamper visitors’ experience, there are some improvements that may be drawn up. Therefore, this study expresses that AR technology has great potential and motivates opportune and useful new lines of research for scientific community, for cultural legacy, and also for society in general.
[1] A. Marto, A. A. de Sousa, and A. Gonçalves, «DinofelisAR Demo Augmented Reality Based on Natural Features», in 12a Conferência Ibérica de Sistemas e Tecnologias de Informação (CISTI’17), 2017, vol. 64, pp. 852–861.
[2] A. Marto, A. A. de Sousa, and A. Gonçalves, «Mobile Augmented Reality in Cultural Heritage Context: Current Technologies», in Proceedings of the 24o Encontro Português de Computação Gráfica e Interação (EPCGI 2017), 2017, pp. 39–46.
[3] A. Marto and A. A. de Sousa, «Mobile AR Performance Issues in a Cultural Heritage Environment», International Journal of Creative Interfaces and Computer Graphics (IJCICG), vol. 9, no. 1, pp. 15–31, 2018.
