SensiMAR Results: Sense of Presence in the Experience

November 25, 2020

Long time no see! Due to some deadlines regarding my PhD, it’s been a while since I last updated the SensiMAR page.

But this does not mean that the work on SensiMAR has stopped! First results are out!

While audiovisual approaches appear to be the most commonly used solutions in previous implementations and studies, multisensory approaches have arisen and indicated to further enrich users’ experience in cultural heritage (CH) contexts.

However, little is known about the impact of the addition of different stimuli in multisensory augmented reality experiences in cultural heritage contexts. Thus, we studied the impact of different sensory conditions on a user’s sense of presence when visiting the cultural site.

Introduction

Feeling present in virtual environments has been demonstrated to affect the success of virtual heritage applications mainly in Virtual Reality (VR) approaches, but it has been also defended as a central aspect of immersive Augmented Reality (AR), in particular, from the consumer perspective.

Certain studies support VR methods to evaluate presence in AR environments. However, the disconnection from the real world that VR usually pursues is not intended in AR. In addition to this, the challenges of evaluating an outdoor multisensory system based on AR with end-users are vast.

Following the challenges of evaluating presence in AR, the main goal of this study is to investigate the impact of different sensory conditions by combining visual, audio, and smell stimuli on measures of the user’s presence. As a secondary objective, we intend to identify correlations between the referred variables and certain individual characteristics of participants, such as age, gender, previous AR experience, and country of origin.

Methods of the research

Here, we briefly describe the participants of the study, the variables at issue, the materials and apparatus used, the followed procedure, and the instruments used for evaluation.

Participants of the study

All participants in this study were random visitors who were visiting an archaeological site and were invited to take part in the experiment as volunteers. A between-subjects design was adopted, making use of the non-probabilistic convenience sampling procedure. Each condition was evaluated by 12 participants, with a total of 48 participants.

SensiMAR participant

Figure 1 - Overview on the place where participants tried the SensiMAR application.

Variables of the study

The independent variable in this study was the stimuli combination of the experience (i.e., the condition), namely, Visual, Visual+Audio, Visual+Smell, and Visual+Audio+Smell. The dependent variables are spatial presence, involvement, experienced realism, overall presence, and behavioral presence.

Materials and Apparatus

For this study, the SensiMAR prototype was used. Thus, the stimuli provided for the experiment were as follows:

Visual: A 3D reconstruction of a wealthy house, the Cantaber House, some animated characters, and a passing wagon.
Audio: A soundscape track was available at the beginning of the experience, (with sounds like the water fountains, the blacksmith working). Another soundtrack started as the AR experience began to have sounds synchronized with the animations.
Smell: During a small guided visit, the participants were invited to smell the garum since it was very often used in Roman cuisine. This same smell was later released during the AR experience.

The experimental condition in the archaeological site, comprehensively discussed here, is briefly illustrated in figure 2. The system allowed a 360-degree exploration by having the user in the center of this experience and accessing AR visual contents with a mobile device (indicated in figure 2 with letter A), the sound through four speakers (indicated by the letter B) and the smell released in his/her vicinity (specified with the letter C).

SensiMAR Scema

Figure 2 - Schema of SensiMAR implementation for user evaluation.

Procedure

The experience took place at the Monographic Museum of Conimbriga-National Museum (Portugal), more specifically in the ruins of a wealthy house–the House of Cantaber.

The first step was to provide a guided visit in the vestibule of this house, as illustrated in figure 3. The information given to visitors in this short-guided visit was the same for all participants, being provided in Portuguese, English, or French, depending on the language that they preferred.

No Condition

Figure 3 - Photograph taken during one of the experiments, while the guided visit was occurring.

Instruments for Evaluation

To measure the sense of presence, the igroup presence questionnaire (IPQ) was used, which was composed of the subscales spatial presence, involvement, experienced realism, and overall presence.

Following the literature suggestions for awareness of evaluating and analyzing the sense of presence in AR environments, we included ethnographic research in our study to analyze behavioral presence.

The baseline for direct observation took into consideration active social presence as an indicator of users’ sensitivity to media content, combined with body responses as indicators of presence and people’s reactions as an objective measure of presence.

Presence Results

The post hoc analysis with spatial presence revealed statistically significant differences between Visual and Visual+Smell, with a stronger influence in this last condition.

Regarding involvement, significant differences were found between Visual+Smell and two other conditions, namely, a stronger influence for Visual and for Visual+Audio.

Considering experienced realism, significant differences were found between Visual and Visual+Smell, with a stronger influence in this last condition. Significant differences were also found between Visual+Audio and Visual+Smell, with a stronger influence in this last condition.

Behavioural Presence

It was possible to identify the number of reactions registered for each group of participants, divided by condition. Note that the reaction defined as “interacted” indicates a behaviour related to making a sound out loud or pointing at the screen.

Conclusion

The lack of AR studies related to multisensory implementation does not allow us to relate the results with results obtained using other AR systems. Considering VR multisensory systems, the results obtained in the current study are not exactly in line with the literature, which highlighted that users stated that sense of presence was improved when using the multisensory approach. The results demonstrated differences among presence-related variables but did not show overall presence being influenced by the different stimuli in the experience.

Direct observation records showed evidence to support the multisensory AR approach instead of the traditional AR. The knowledge acquired also proved to increase with the multisensory AR solution, since the participants were more dynamic and interacted more, e.g., by raising further questions and discussing the Roman culture and architecture more.

Overall, we concluded that each stimulus has a differential impact on the studied variables, demonstrating that its usage should depend on the goal of the experience: smell should be used to privilege realism and spatial presence, while audio should be adopted when the goal is to elicit involvement.

The dualities found in the different subscales that represent the sense of presence suggest that different stimuli evoke specific impressions but, together, do not confirm an increased sense of presence–since different stimuli have distinct impacts on the different subscales. This evinces the distinct impact that each stimulus can cause, and a deeper understanding of how we can combine them to enhance a specific feeling remains.