Interactive 3D Simulations in Measuring
Consumer Preferences:
Friend or Foe to Test Results?
Alma Berneburg
University of Applied Sciences in Merseburg, Germany
Abstract
With the inexorable progression of technological development, more and more technological solutions are gaining access to market research processes (e.g., online panels, mobile surveys). A promising tool in this context involves interactive, three-dimensional, virtual reality techniques to measure consumer preferences. Many benefits are conceivable: Artificial lab environments can use more realistic designs and improve cost efficiency and “time to market” factors. However, the increasing degree of reality and respondent immersion into the interactive three-dimensional (3D) environment could cause a market research task to fall prey to a tense virtual reality adventure. This study tests an interactive 3D technique empirically in terms of its usability in a choice-based conjoint study. The 3D simulation does not create bias as a result of immersion, and it provides much better test results in terms of estimated utilities and conjoint importance than simple two-dimensional stimuli.
Introduction
Adopting virtual reality (VR) techniques could offer many advantages for market research: Artificial lab environments could be designed in a more realistic manner to enhance the validity of test results. Time to market factors could be considered more accurately, and test results would appear faster and with less expense, because instead of expensive dummies and real products, the VR approach could use highly flexible virtual products and point-of-sale simulations (Burke 1996; Horst and Berneburg 2006).
Yet such new techniques also potentially introduce new concerns, which could threaten the quality of survey results. The increasing degree of reality provided by VR technology likely increases the level of submersion in the lab survey (Isgrò et al. 2004). In this case, the quality of a respondent’s answers might improve or rather deteriorate, posing a sizeable threat to the quality of the test results. When the respondent is immersed in the VR environment, the focal market research task could fall victim to the tension-filled adventure of a VR experience. In turn, the objects of investigation (e.g., buying decisions, consumer preferences) could suffer because the respondent’s exaggerated concentration on the task would differ substantially from the often habitual decisions consumers make in their day-to-day lives. To avoid either thoughtlessly ignoring a promising new survey tool or, worse, adopting a biased new instrument in market research, a comparative analysis seems necessary.
This study uses an interactive three-dimensional (3D) screen to build a VR test environment. The test objects appear to float spatially in front of the screen. To generate this effect, the technology identifies respondents’ eyes, with the help of a head tracker, and projects a different perspective of the test object for each eye. The 3D effect thus occurs without the application of further technical additives (e.g., 3D glasses), which could enforce the artificiality of the survey. The respondent just sits in front of a screen and sees the test objects three-dimensionally (Renault and Stachel 2004). In addition, the respondent can actively control and navigate through the survey by pointing at the displayed tasks and objects (e.g., picking up products from the shelf and putting them back with just a fingertip). This avoidance of further technical additives, like a stylus, is possible through the usage of a hand tracker that scans the respondent’s fingertips (de la Barré 2004; Hopf and de la Barré 2004). The result is a virtual 3D touchscreen. (For more details, see the technical appendix)
With this technique, virtual objects seem more real to a respondent than they would with alternative artificial stimulus presentations, such as computer-based two-dimensional (2D) stimuli or older 3D techniques that rely on technical additives, such as a helmet, to achieve the spatial effect. (Many so-called 3D online research tools actually consist of 2D visual effects that offer the possibility of interaction with the environment, such as moving and turning objects, but do not achieve real 3D with depth and perspective.)
To analyze the indicated 3D technology for its usability in market research, this study applies the following test:
- A choice-based conjoint analysis (CBC) of consumer preferences occurs in three alternative versions, in which the test objects are presented via computer-based 2D pictures, the 3D simulation, and real physical stimuli (dummies).
- In a preliminary interview, each respondent indicates his or her immersive tendencies to ensure homogenous predispositions in the three test groups.
- In a postsurvey interview, respondents indicate the intensity of their experiences during the CBC-survey, which reveals the varying degrees of submersion prompted by the three different stimulus presentations.
Using these methods, this study attempts to determine the level of presence generated by the different stimulus presentations and any potential biases of a 3D test environment. If comparable immersive tendencies occur across the three homogenous samples, any different intensities of the experienced submersion in the survey environment must result from the variant stimuli. Assuming the 3D stimuli generate a superior degree of submersion in the lab environment and the survey than previous artificial stimuli, the investigation of potential biases in the 3D sample requires that the 3D test results be compared with the results of former types of surveys (i.e., 2D survey) and the benchmark of reality (i.e., real physical stimuli)
The article is structured as follows: A brief literature review discusses the main concepts of VR and previous studies of different stimulus presentation formats. Following the development of three fundamental hypotheses, a section introduces the methods and the test design of the study. Next, this article presents the statistical techniques and results, as well as discusses the findings. Finally, the implications of these results for the leading hypotheses are provided, and a summary section concludes the article.
Literature Review
To establish the study’s underlying hypotheses, relevant literature pertaining to basic concepts of VR and previous insights into different stimulus presentation formats provides a foundation.
Virtual reality can be defined in many very different ways; even the names of the different approaches are unique. Virtual environments, virtual worlds, artificial realities, simulated realities, and synthetic environments all try to describe the same phenomenon (Biocca 1992; Biocca, Kim, and Levy 1995; Carr 1995; Ebersole 1997). This study subsumes the variety of definitions within the most commonly used term, “virtual reality.” Corresponding to the patchwork of definitions, the techniques used to create VR are also manifold. Biocca (1992, p. 25) provides a useful description of the technical development: “There will be no single type of VR system and no paradigmatic virtual environment. We are more likely to see tailored combinations of components and applications, each capable of producing various types of experience.” Several years later, this statement still has not lost any accuracy (Kosfeld 2003). A great array of different so-called VR techniques continues to exist, intensified by the ongoing evolution of VR.
This research adopts Lanier’s definition, which states that VR describes a new level of experience, which is generated with the help of technological solutions that synthesize a new reality (see Heilbrun and Stacks 1991). In his original definition, Lanier used the idea of data suits as the gate to a new, virtual reality, but modern advances require a loosening of this definition to focus on the essence of the idea: a technology-based gate to another reality. This definition implies two main components of VR: the help of computer-based technological solutions and a new level of experience based on an illusion, which actually generates a real experience.
To apply these aspects to the current problem, this study considers the two main components of VR and analyzes their occurrence in the used 3D test environment. Clearly, the test environment fulfils Lanier’s technology requirement, as detailed in the technical appendix. The second component of Lanier’s definition is not as self-evident but nevertheless exists; that is, the test environment generates a new level of experience because the respondent is captivated by a virtual buying situation while still physically sitting in front of an experimental stimulus in a lab environment. Thus, in accordance with Lanier’s definition, the 3D screen generates a virtual reality.
Existing VR created with used 3D technology also implies some side effects. An important phenomenon in this context is the degree of “presence” generated by the VR. Presence in this sense can be traced back to Sheridan (1992), who shaped the term “virtual presence” to describe the notional attendance to a simulated, synthetically generated place while physically residing in a totally different situation. When considering presence, researchers face the same diversity of definitions as exist for VR; this study follows Steuer (1992, p. 75), who defines presence as “the sense of being in an environment,” and Biocca (1997), who makes the definition more concrete by asserting, “The shorter and more common term, presence, has been generalized to the illusion of ‘being there’ whether or not ‘there’ exists in physical space.” In line with these definitions, three aspects constitute presence, as substantiated by Slater and Wilbur (1997):
1. The sense of being in a special situation presented by the virtual environment.
2. The degree to which the virtual environment dominates the real environment.
3. The degree to which the respondent remembers the virtual environment as “real”.
These three points can be difficult to grasp, so Sheridan’s (1996) efforts complement these definitions. Specifically, Sheridan (1996, p. 243) argues the degree of presence depends on the
- “information content of the stimulus independent of the observer,”
- “ability of the observer to freely modify the ‘viewerpoint,” and
- “ability of the observer to modify the configuration of the environment.”
Thus, in applying these comments to the focal 3D technology, several results emerge. First, the virtual environment created by the 3D technology presents the sense and illusion of being at the point of sale and making a buying decision. Second, during survey, the respondent’s reality is dominated by the virtual environment, a fact that is intensified because he or she may move freely and observe the virtual environment from different angles. Third, the environment offers the possibility to interact with the test environment (the respondent may “grab” products from the simulated shelves, put them back and grab new ones, or put the products into the shopping basket). Thus, the 3D screen appears to create presence; the empirical study will determine the actual level of presence.
Immersion relates closely to the concept of presence. Again, relevant literature provides multiple views and definitions; over the years, two main opposing concepts have evolved. Witmer and Singer (1998, p. 227) regard immersion as “a psychological state characterized by perceiving oneself to be enveloped by, included in, and interacting with an environment that provides a continuous stream of stimuli and experiences,” whereas Slater and Wilbur (1997, p. 604ff.) state: “Immersion is a description of a technology, and describes the extent to which the computer displays are capable of delivering an inclusive, extensive, surrounding, and vivid illusion of reality to the senses of a human participant.” That is, immersion connects directly with the technological solution that creates the VR. This research applies the first definition to the level of individual differences but the second definition to discuss the abilities of the technology itself: A highly immersive technology creates a virtual reality that provides a strong presence. The focal point here remains the development and analysis of technological components and applications that carry the respondent to a virtual environment with the aim of getting that respondent to accept, for the time being, the virtual environment as reality. Again, the 3D screen clearly represents a technology-based solution that can deliver an inclusive, surrounding, and vivid illusion of a new reality, as required by Slater and Wilbur (1997).
When measuring consumer preferences, respondents likely react differently to a test product depending on the stimulus presentation format and its degree of realism (de Bont 1992; Loosschilder and Orrt 1994). Furthermore, these differences probably have consequences for the quality of the test results. In response, many studies have tried to research this assumption and gain further insight into the importance of stimulus presentation formats, especially in conjoint analyses.
Table 1 provides an overview of existing studies that consider the convergence of test results obtained through different stimulus presentation formats. Although not all of the studies are based on conjoint analyses, they nevertheless are pertinent to the research question at hand.
TABLE 1 Overview of Previous Studies Comparing Different Stimulus Presentation Formats
In most cases, these existing studies find differences when they compare alternative forms of stimulus presentation. Overall, their results hint at different respondent reactions, depending on the stimulus presentation format. However, these studies generally compare verbal against visual or real stimuli and almost exclusively employ 2D visual stimuli (e.g., sketches, pictures). In terms of reactions to 3D stimuli, no clear statements can be made.
Hypotheses
The first hypothesis proposes a relationship between the dimensionality of the artificial stimuli involved in the survey and the degree of presence generated. In this sense, the 3D test environment should create a higher degree of presence for respondents than the 2D stimulus presentation.
H1: The dimensionality of the stimulus presentation influences the degree of presence created for a respondent, such that a 3D test environment creates a higher degree of presence than a 2D test environment.
For validation purposes, this research compares the CBC results of the 3D test environment with those derived from the 2D presentation as a lower limit and against those from the real dummy stimuli as an upper limit. No artificial stimuli would ever likely beat the realistic impressions offered by a physical test, but if the alternative 3D stimuli deliver comparable results, the generalizability and validity of the more flexible and less expensive 3D test results would appear sufficient and promising.
H2: The test results derived with the 3D technique are comparable to the test results reached through a classical test involving physical stimuli.
H3: The test results derived with the 3D technique are better than those reached through a test involving 2D stimuli.
Research Method
To test the innovative, interactive 3D technology in terms of its usability for marketing research, the study uses the design described next.
In November 2005, an overall sample of 181 respondents was drawn from a homogeneous survey population consisting of students of a medium-sized German university. This convenience sample does not distort the survey’s results, because the survey does not aim to measure consumers’ attitudes toward the test product but rather to compare the results from the 3D VR technology with those from alternative 2D and dummy stimulus presentations. In other words, convenience sampling does not pose a threat to these early test results (Orme and King 1998). Furthermore, the between-subjects design minimizes distorting learning effects and prevents overwhelming demands on the respondents’ readiness and patience (Agarwal and Green 1991; Huber et al. 1993).
Respondents were randomly assigned to one of the three samples; therefore, the results compare the responses of 54 persons in the classical dummy study (physical stimulus material in the CBC analysis) with those of 48 respondents in the 3D and 79 in the 2D studies. The uneven distribution occurs because the set up of the 3D environment and the performance of the dummy tasks are somewhat more time consuming than those of the 2D technique. Therefore, the 2D survey started slightly earlier, and more respondents could be recruited for this sample.
The actual survey consists of three consecutive steps.
Preliminary Interview. The actual survey began with a preliminary interview that follows the immersive tendencies questionnaire (ITQ) of Witmer and Singer (1998) to ensure homogenous predispositions of the respondents in all three groups (see Table 2). The questionnaire determines respondents’ “immersive tendencies” as their tendencies to plunge into a virtual environment and experience some degree of presence; as the authors note, “The ITQ was developed to measure the capability or tendency of individuals to be involved or immersed” (Witmer and Singer 1998, p. 230).
All respondents participated in the preliminary interview before the actual CBC tasks, regardless of their subsequent stimulus presentation format.
TABLE 2 Questions from the Preliminary Interview
Choice-Based Conjoint Study. Following the preliminary interview, a CBC analysis measures respondents’ product preferences. Choice-based conjoint analysis, initiated by Louviere and Woodworth (1983), is the most commonly applied version of traditional conjoint analysis (Hartmann and Sattler 2002). The goal of CBC is to determine consumers’ product preferences and express these preferences in terms of partworth utilities, similar to traditional conjoint analysis. Choice-based conjoint analysis, however, offers some advantages over traditional conjoint analysis, in that it enhances the degree of reality of the survey and thus the external validity of the results. CBC surveys allow consumers to express their preferences by choosing their preferred single product concept from a variety of concepts rather than rating or ranking them. Therefore, the task is much closer to a real point-of-sale buying decision, because choosing a preferred concept is similar to what consumers actually do in the market on a daily basis (Orme 2006). Because this study similarly tries to enhance the degree of reality in an experimental lab environment, using a CBC design seems logical.
To validate the 3D test environment, three comparative CBC studies use identical test designs, as follows:
- A computer-based 2D study (lower benchmark).
- A study using real physical stimuli (upper benchmark).
- An analysis representing the products three-dimensionally through the 3D screen.
In each sample, 10 randomized CBC tasks with three alternative concepts each are specified, along with one holdout task (similar for every respondent) to ensure comparable predictive validity (11 choice tasks overall). The focal product is shower gel with a fixed brand and package size. The varying attributes are packaging and price, as detailed in Table 3. The few attributes allow for a consistent test design in all three comparative surveys and thus ensure comparable test results. Although it would not have been difficult to simulate products with varying brand, size, packaging, and price in the 3D and 2D simulations, implementing that many attributes in a dummy test with physical stimulus material would have been problematic. In a CBC study, respondents express their preferences by choosing one of several product concepts on the basis of its attributes and levels. To use real stimuli, the researcher must physically build each potential attribute combination that might occur-a nearly impossible task. Because of the limitations of this CBC test design, this study includes a relatively small number of varying attributes and attribute levels.
TABLE 3 Attributes and Levels Used in the CBC Analysis
The prices vary only over a small range, because the small and sensitive product differences generated by the packaging should not be dominated by massive price differences, which would reduce consumers’ buying decision to only the price attribute.
The test objects chosen already existed in the market with varying packaging colors and shapes at that time, due to a change of packages by the manufacturer (see Figure 1). Therefore, it was not necessary to build physical stimuli for the dummy sample, because they already existed:
FIGURE1 Visual Specifications of the Test Objects Used in the CBC Analysis
Post-Interview. In an additional interview after the main CBC tasks, respondents in each of the three sample conditions provided information about their virtual experience to indicate their achieved degree of presence. As detailed in Table 4, extracts from Witmer and Singer’s presence questionnaire serve to measure “the degree to which individuals experience presence in a VE” (Witmer and Singer 1998, p. 230).
TABLE 4 Questions from the Post-Interview Regarding Presence
Although the actual phrasing of some questions required rearranging to suit the particular stimulus of the survey and some questions were useful only for one or two of the tests, depending on the stimulus, the overall meaning of the questions remains unchanged.
The post-interview also includes questions about the degree of entertainment that the survey provided to evaluate the indirect validity of the tests (Hartmann and Sattler 2004), as listed in Table 5.
TABLE 5 Questions from the Post-Interview Regarding Validity and Presence
If a survey is of special interest to a respondent and generally enjoyable, the quality of the answers, and therefore of the whole study and its results, likely is better than that in a survey in which the respondent feels compelled to take part. Indirect validity therefore can be measured according to criteria such as simplicity, length, entertainment value, diversification, or interestingness, which reveal the respondents’ motivation and therefore the validity of the method (Ernst 2001).
In addition to measuring the validity of the different surveys, the preceding five questions indicate the level of presence that respondents experienced, because a survey that is of greater interest likely creates deeper submersion into the test environment.
Findings
A comparison of the respective means of answers to the immersive tendencies questionnaire, on a scale from 1 (very strong/very much) to 7 (very weak/not at all), indicates whether respondents’ immersive tendencies are homogenous across samples. The null hypothesis states that all groups of data are sampled from distributions with the same mean, which is tested by a one-way ANOVA with a 95% level of confidence. According to the preliminary interviews, respondents in the three samples show the same immersive tendencies, with no significant differences in the means in most cases (details are available from the author). Only items it22, it24, and it27 seem to reject the null hypothesis because of their p-values lower than 5%, though in the case of it22, it is only marginally lower.
Rejecting the null hypothesis with the one-way ANOVA does not mean that the means of every subgroup differ, because an ANOVA only indicates whether a difference exists between two or more groups, not exactly whence the differences result. To determine the source of these differences, this study therefore uses a post hoc multiple comparison text. Because similar variances appear in the three different samples, the Student-Newman-Keuls (SNK) test is applicable for comparing all pairs of means. This test compares the differences among means to the critical points of the studentized range. In the existing case, SNK can fuse the ANOVA decisions and identify the sources of significant differences in the means.
For most items, SNK validates the outcomes of the ANOVA (details are available from the author). In the case of item it22, ANOVA suggests rejecting the null hypothesis; SNK supports retaining it (p = .054). It thereby is possible to get significant results from a post hoc test, even when the overall ANOVA is not significant, because ANOVA tests the null hypothesis of identical means in all of the groups, whereas the post hoc test analyzes the null hypothesis that two particular means are identical. Because the latter is more focused, it can identify differences between groups, even when ANOVA does not. In this case, in line with the SNK results, the null hypothesis of identical means is not rejected. For items it24 and it27, SNK supports significant differences in the means. First, for it24, two subgroups emerge that indicate in pairwise comparison that the 2D and 3D test subsamples show no significant differences in their means (M2D = 3.73, M3D = 3.92, p = .604). Similarly, the 3D and dummy test subsamples show no significant differences (M3D = 3.92, MDummy = 4.59, p = .056). When comparing the mean of the 2D sample with the mean of the dummy sample however, significant differences result. Therefore, the null hypothesis must be rejected in this case. Second, for it27, the SNK results imply no significant differences in the means of the 3D and dummy samples (M3D = 1.67, MDummy = 1.74, p = .407), but both differ significantly from the 2D group (M2D = 1.47). Therefore, the null hypothesis of similar means in the three subgroups is rejected in the case of it27.
When considering the relevance of these findings for the question of homogeneous immersive tendencies, the items’ importance in determining the level of immersive tendencies must be taken into account; that importance is quite low for it24 and it27, because these items appear in the survey mainly as control questions that help identify potential distorting external influences. Because their absolute values indicate no excessive external influence on the immersive tendencies, the differences in their means are of little consequence.
Furthermore, 83% of the items show no significant differences in their means; therefore, homogeneous immersive tendencies exist in the three subsamples.
After determining homogenous immersive tendencies, the next step involves analyzing respondents’ product preferences on the basis of the choice tasks. The utility estimations obtained with the three different stimulus presentations appear in Table 6.
TABLE 6 Estimated Partworth Utilities of the Three Samples
For price, the comparison of the utility estimations across the three samples paints a uniform picture. Each group reveals a declining preference as price increases, which coincides with prior expectations, because price is an ordered attribute for which low normally is preferred to high. This accordance of partworth utilities with a priori expectations provides confirmation of reliability (Orme and King 1998).
The comparison of consumers’ preferences in terms of packaging, however, paints a different picture, as summarized in Table 7.
TABLE 7 Rank Order of Consumer Preferences in the Three Samples
Whereas the 3D and dummy utilities result in the same rank order, the utilities in the 2D case suggest a different ranking. In other words, the stimulus presentation in the 2D case seems to deliver biased results when testing an attribute that depends strongly on visual impressions; however, the 3D stimulus presentation does not seem biased compared with the real physical stimuli that represent the benchmark (i.e., test results nearest to real consumer preferences).
Although the 3D and dummy utilities result in the same rankings, the 3D scenario’s utilities for ranks 2 and 3 (packages B and C) are very close (-.05590 and -.05964), whereas the distinction between the two is clearer in the dummy case. Nevertheless, the rank order in the 3D test overall is clearly similar to the dummy benchmark, whereas the 2D survey results in different stated preferences.
The interpretation can go much further than just stating a different order of the measured partworth utilities. Even considering the different order of preferences, the results suggest that the color of the packages is judged similarly. In all three cases, respondents prefer green over black; the hypothetical subattribute “color” thus appears independent of the dimensionality of the stimulus in the survey and can be evaluated by consumers even when they only view a simple 2D stimulus, like a picture. In contrast, the subattribute “form” is judged rather differently; that is, in the 2D test, form is judged in an opposite fashion. This result clearly appears to depend on the dimension of the stimulus presentation.
This phenomenon is easy to understand, considering that a color will not change its appearance when presented in two or three dimensions; it does not depend on dimensionality. A consumer does not necessarily need a 3D simulation or physical dummy of a product to give a statement of his or her color preferences. However, the spatial impression of a real product or a 3D simulation contains more information about form than does a 2D picture. When the form is a deciding factor, a 3D impression, whether in the form of a dummy or a 3D simulation, is superior to a simple 2D impression.
To further analyze the differences among the answers provided by respondents in the three samples, this research calculates conjoint importance by taking the percentages of the differ-ences between the best and the worst utility for each attribute on an aggregate level, as de-picted in Table 8. This calculation provides a set of attribute importance values that equal 100% and describe the impact, for a given range of levels, of each attribute on the consumers’ decisions. Conjoint importance depends on the respective attribute levels in the particular study, such that an even narrower range of price could cause it to forfeit its importance (Orme 2006). However, the comparability of importance among these samples remains unchanged, because the same attribute levels appear in all three samples.
TABLE 8 Conjoint Importance in the Three Samples
In the dummy sample benchmark, the relative importance of price is greater than the relative importance of packaging; the 3D sample offers a similar importance distribution. In both cases, price has a greater impact (60%) on respondents’ decisions than does packaging. Yet in the 2D sample, the results indicate that both attributes have nearly the same level of importance (50/50) and a comparable impact on consumers’ decisions.
An existing explanation of this phenomenon is difficult to find. However, it appears that a higher degree of reality in stimulus presentation causes the impact of the price attribute to increase. A possible explanation for this increased importance could be that a broad, lucid, undistorted visual impression of the stimulus (e.g., 3D or dummy presentation) reduces the need to inspect the product, so respondents do not have to concentrate as much on observing the displayed product. A realistic representation instead allows consumers to “understand” the product much more quickly and thus behave as they would at the real point of sale. With only a 2D stimulus to evaluate, respondents may artificially increase their concentration on the visual impulse to confirm their product preferences. The impact of the price attribute therefore recedes in a way that does not reflect reality.
Although this explanation certainly is debatable, the finding that the 3D results resemble the dummy benchmark much more than the 2D results indicates that the 3D stimulus produces more realistic results, whereas the 2D results seem biased.
Generally, the data thus tell a comforting story; 3D surveys perform almost as well as dummy surveys, with equal reliability and validity and similar utilities and importance. In contrast, the 2D test results are not as good and seem seriously biased.
Post Interview
The final step analyzes the role of VR specifically to determine the actual degree of presence generated by the 3D technique. Therefore, this portion of the analysis considers only those items that either show significant differences among the three samples or for which the lack of differences is of special interest for this study (further details are available from the author). Responses to the presence questionnaire (Witmer and Singer 1998) only allow a comparison between the 2D and 3D samples, because the concept of presence in a virtual environment cannot be applied to the dummy survey. Moreover, because the goal of this study is to analyze the effects of an enhanced degree of reality in VR market research, the dummy survey is of minor interest at this point; dummy comparisons mainly serve to validate the 3D screen as a research instrument. Thus, the focus in the following section centers on the very simple but virtual 2D versus the improved and virtual 3D stimulus presentation.
In the postsurvey presence interview, respondents rated their experiences in the virtual test environment on a scale from 1 (very much) to 7 (not at all). Most cases show no significant differences in the means, but item p23 (“How involved were you in the virtual environment experience?”) is of a special interest. According to a t-test with a 95% confidence level, the respective means are significantly different (M2D = 3.43, M3D = 2.96). The null hypothesis that both groups of data come from distributions with the same mean thus must be rejected (further details are available from the author). In other words, for this scale, respondents in the 3D study feel significantly more involved in the virtual experience than do those in the 2D study; therefore, the supposition of a higher degree of presence in the 3D VR environment cannot be rejected.
Answers to the statements in Table 5 provide a similar impression. First, the one-way ANOVA indicates that all three surveys are equally easy to deal with (item Val1), at p = .334 (SNK supports this result). This finding is especially interesting, because older 3D techniques using a data helmet resulted in considerable distortions in the test results because of problems handling the technique. Second, for item Val2 (“the survey was too long”), both the ANOVA and a post hoc SNK test confirm no significant differences among the three surveys.
Third, the null hypothesis must be rejected for items Val3 and Val4, according to the one-way ANOVA. The SNK test, conducted to identify the sources of the significant differences, indicates that the mean for item Val3 (“the survey was interesting” from 1 [I agree very much] to 5 [I do not agree at all]) in the 3D sample is 2.40 and in the dummy sample is 2.63, which suggest significantly more interest than in the 2D test, with a mean of 3.19. That is, the more realistic 3D and dummy stimuli improve the quality of the test results, because respondents remain more focused on the survey. Moreover, an increased degree of interest in the survey should increase the validity of the results. In addition, item Val4 (“the survey was enjoyable” from 1 [I agree very much] to 5 [I do not agree at all]) suggests the 3D study (M3D = 2.21) is significantly more fun than the dummy study (MDummy = 2.56) or the 2D test (M2D = 3.10). This finding not only implies that the quality of the answers, and consequently the validity of the survey, are enhanced by the VR technique but also the level of achieved presence. When experiencing significantly more fun in a survey than in alternative methods, submersion into the tasks likely is deeper; that is, concentration on the tasks becomes both simplified and intensified at the same time.
Discussion
This study attempts to (1) verify whether the 3D environment generates a higher degree of presence than a simple, standard 2D environment and (2) compare the 3D test results with those of a 2D survey and those of a survey using real physical stimuli (dummy). The first hypothesis therefore proposes a relationship between the dimensionality of the artificial stimuli and the degree of presence generated. The results of the analyses suggest support for H1: The 3D survey creates a higher degree of presence, and respondents feel more involved than they would in a 2D survey.
The next two hypotheses deal with the quality of the test results, to determine whether biases occur. Specifically, they posit that the results of a survey using 3D stimuli and test environments would be comparable to those using physical stimuli and better than those using 2D stimuli. The analyses show that the test results of the 3D survey are comparable to the benchmark and better than the 2D results; again, the hypotheses cannot be rejected.
Summary and Outlook
Nonconvergent behavior already appears in most research comparing verbal with alternative stimuli, but different visual stimuli with varying dimensions have not been objects of investigation. No comparison of virtual 3D stimuli, 2D stimuli, and physical dummies has previously been performed in a conjoint analysis framework. This study therefore analyzes the effects of a VR 3D technique in terms of the degree of presence it generates and its influence on CBC test results in a market research environment. The increased submersion into the lab survey, as assumed herein, can be confirmed for the new 3D technique compared with the older, simpler artificial 2D stimuli. In turn, the 3D technique appears to deliver unbiased CBC test results that outperform the 2D test results in terms of quality and comparability to the dummy benchmark.
The main insight gained from this study therefore pertains to the different dimensions of the stimulus, which make a crucial difference in the quality of a study’s results. Thus, the question for marketing research may not be whether to employ verbal or visual stimuli to measure consumer preferences but rather which visual stimulus performs best. It might not always be necessary to select 3D simulations, but when dealing with packaging matters or product innovations-that is, when respondents rely strongly on the visual appearance of the test object-a third dimension can add significant quality to the test results and the market predictions they indicate.
The findings also indicate that the new 3D monitor can be used to measure consumer preferences and that it represents an improvement in terms of achieving a flexible and realistic lab environment that offers better than sufficient quality of the test results. In addition, 3D simulations might be useful not only in market research but also in advertising. The deeper submersion of an observer, created by the 3D environment, could influence recall or recognition processes, as well as emotional appeals. These suggestions naturally should be analyzed in-depth in further research.
More comparative studies will have to come, further investigating the kind and degree of presence and the quality of the test results delivered by the 3D test environment. A next step therefore will be a comparative analysis of test results gained from 2D and 3D studies in collaboration with a large market research institute conducting an actual packaging test with representative samples. For such tests, 3D test environments should provide substantial advantages, because they diminish the need to build expensive physical stimuli while still including many different levels for attributes such as color, form, and size in the survey, with visualization superior to that of simple 2D packaging tests. Therefore, especially in tests containing objects that do not yet exist or that strongly depend on visual impressions, the operational appeal of this new technique should be confirmed carefully.
Finally, the influence of presence in a market research environment should be analyzed more broadly before companies engage in actual commercial use of these techniques in a lab environment. However, this study offers an initial contribution to promoting the use of VR techniques in market research.
References
Agarwal, Manoj K. and Paul E. Green (1991), “Adaptive Conjoint Analysis versus Self Explicated Models: Some Empirical Results,” International Journal of Research in Marketing, 8, 141-146.
Anderson, James C. (1987), “The Effect of Type of Representation on Judgments of New Product Acceptance,” Industrial Marketing & Purchasing, 2, 29-46.
Biocca, Frank (1992), “Virtual Reality Technology: A Tutorial,” Journal of Communication, 42 (4), 23-72.
Biocca, Frank (1997), “The Cyborg’s Dilemma-Progressive Embodiment in Virtual Environments,” Journals of Computer-Mediated Communication, (3) 2, available at <http://jcmc.huji.ac.il/vol3/issue2/biocca2.html> (accessed 5/2/2006).
Biocca, Frank, Taejong Kim, and Mark L. Levy (1995), “The Vision of Virtual Reality,” in Communication in the Age of Virtual Reality, F. Biocca, and M. L. Levy, eds. Hillsdale, NJ: Lawrence Erlbaum, 3-14.
Burke, Raymond R. (1996), “Virtual Shopping: Breakthrough in Marketing Research,” Harvard Business Review, 74 (March-April), 120-131.
Carr, Karen (1995), “Introduction,” in Simulated and Virtual Realities-Elements of Perception, K. Carr and R. England, eds. London, Bristol: Taylor & Francis, 1-10.
de la Barré, René (2004), “Der Griff zum virtuellen Objekt-Interaktive 3D-Kiosksysteme mit Gesteninterpretation [Grabbing the Virtual Object-Interactive 3D-Kiosksystems with Gesture Interpretation],” Proc. Electronic Displays 2004, 19. Konferenz für Bildschirme und Anzeigesysteme, ihre Bauelement und Baugruppen, Hagenburg (Germany), on CD [in German].
de Bont, C. (1992). Consumer Evaluations of Early Product-Concepts.Delft: Delft University Press.
Domzal, Teresa J. and Lynette S. Unger (1985), “Judgments of Verbal versus Pictorial Presentations of a Product with Functional and Aesthetic Features,” Advances in Consumer Research, 12, 268-272.
Ebersole, Samuel (1997), A Brief History of Virtual Reality and Its Social Applications. Colorado: University of Southern Colorado.
Ernst, Olaf (2001), Multimediale versus abstrakte Produktpräsentationsformen bei der Adaptiven Conjoint-Analyse: Ein empirischer Validitätsvergleich [Multi-Medial Versus Abstract Forms of Product Presentation in Adaptive Conjoint-Analysis: An Empirical Comparison of Validity]. Frankfurt am Main: Peter Lang (in German).
Ernst, Olaf and Henrik Sattler (2000), “Validität multimedialer Conjoint-Analysen. Ein empirischer Vergleich alternativer Produktpräsentationsformen [Validity of Multi-Medial Conjoint-Analyses: an Empirical Comparison of Alternative Forms of Product Presentation],” Marketing ZFP, 22(2), 161-172 (in German).
Fraunhofer Heinrich-Hertz-Institut (2007), “3D Media Center-Description,” available at (accessed 9/14/2007).
Hartmann, Adriane and Henrik Sattler (2002), “Commercial Use of Conjoint Analysis in Germany, Austria and Switzerland,” Working Paper, University of Hamburg.
Hartmann, Adriane and Henrik Sattler (2004), “Wie robust sind Methoden zur Präferenzmessung? [How Robust are Methods for Measuring Consumer Preferences?],” zfbf, 56 (February), 3-22 (in German).
Heilbrun, Adam and Barbara Stacks (1991), “Was heißt virtuelle Realität? Ein Interview mit Jaron Lanier [An Interview with Jaron Lanier: Virtual Reality],” in Cyberspace-Ausflüge in virtuelle Wirklichkeiten, M. Waffender, ed. Hamburg: Rowohlt, 67-87 (in German).
Holbrook, Morris B. and William L. Moore (1981), “Feature Interactions in Consumer Judgments of Verbal versus Pictorial Presentations,” Journal of Consumer Research, 8 (June), 103-113.
Hopf, Klaus and René de la Barré (2004), “3D-Kiosk mit berührungsloser Interaktion [3D-Kiosk with Contact-Free Interaction],” Proc. EVA Conference2004, Berlin (D), 161-162.
Horst, Bruno and Alma Berneburg (2006), “Virtuelle Welten virtuos genutzt-3D-Verpackungstests zur optimalen Nutzung knapper Ressourcen [The Virtuosic Use of Virtual Worlds-3D-Packaging Tests for an Optimal Use of Limited Resources],” in Recht-Personal-Ökologie-Unternehmung, K.-H. Horst and U. Schindler, eds. Aachen: Shaker, 287-308 (in German).
Huber, Joel C., Dick R. Wittink, John A. Fiedler, and Richard Miller (1993), “The Effectiveness of Alternative Preference Elicitation Procedures in Predicting Choice,” Journal of Marketing Research, 30 (February), 105-114.
Huisman, Dirk (1997), “Creating End User Value with Multi-Media Interviewing Systems,” Proceedings of the Sawtooth Software Conference, Seattle, WA, 49-55.
Isgrò, Francesco, Emanuele Trucco, Oliver Schreer, and Peter Kauff (2004), “Three-Dimensional Image Processing in the Future of Immersive Media,” IEEE Trans. on Circuits, Systems and Video Technology, 14 (3), 288-303.
Kosfeld, Christian (2003), Eintauchen in mediale Welten [Immersing into Medial Worlds]. Wiesbaden: Deutscher Universitäts-Verlag (in German).
Loosschilder, Gerard H. and Roland F. Orrt (1994) “The Effect of the Realism of Product Representations on the Validity of Consumer Evaluations,” 47th ESOMAR Marketing Research Congress, 353-375.
Louviere, Jordan J., Herbert W. Schroeder, Cathy H. Louviere, and George C. Woodworth (1987), “Do the Parameters of Choice Models Depend on Differences in Stimulus Presentation: Visual versus Verbal Presentation?” Advances in Consumer Research, 14, 79-82.
Louviere, Jordan J. and George Woodworth (1983), “Design and Analysis of Simulated Consumer Choice or Allocation Experiments: An Approach Based on Aggregate Data,” Journal of Marketing Research, 20 (November), 350-367.
MacKay, David B., Mark Ellis, and Joseph L. Zinnes (1986), “Graphic and Verbal Presentation of Stimuli: A Probabilistic MDS Analysis,” Advances in Consumer Research, 13, 529-533.
Orme, Bryan (2006), Getting Started with Conjoint Analysis: Strategies for Product Design and Pricing Research. Madison, WI: Research Publishers LLC.
Orme, Bryan and W. Christopher King (1998), “Conducting Full-Profile Conjoint Analysis over the Internet,” Sawtooth Software Research Paper Series 1998, available at (accessed 5/31/2006).
Renault, Sylvain and Oliver Stachel (2004), “Unterstützung multimodaler und natürlicher Interaktionen durch die WORKBENCH3D bei der Modellierung und Visualisierung von VR-Umgebungen auf autostereoskopischen Endgeräten [Supporting Multimodal and Natural Interactions with the Help of WORKBENCH3D for Modeling and Visualizing VR-Environments on Autostereoscopic Terminals],” Internetpräsenz des 3D-Display-Innovationsforums, available at (in German).
Sheridan, Thomas B. (1992), “Musings on Telepresence and Virtual Presence,” Presence – Teleoperators and Virtual Environments (MIT-Press), 1 (1), 120-125.
Sheridan, Thomas B. (1996), “Further Musings on the Psychophysics of Presence,” Presence – Teleoperators and Virtual Environments (MIT-Press), 5 (2), 241-246.
Slater, Mel and Sylvia Wilbur (1997), “A Framework for Immersive Virtual Environments (FIVE): Speculations on the Role of Presence in Virtual Environments,” Presence: Teleoperators and Virtual Environments (MIT-Press), 6 (6), 603-616.
Söderman, Mikael (2005), “Virtual Reality in Product Evaluations with Potential Customers: An Exploratory Study Comparing Virtual Reality with Conventional Product Representations,” Journal of Engineering Design, 16 (3), 311-328.
Steuer, Jonathan (1992), “Defining Virtual Reality-Dimensions Determining Telepresence,” Journal of Communication, 42 (4), 73-93.
Vriens, Marco (1995), Conjoint-Analysis in Marketing-Developments in Stimulus Representation and Segmentation Methods, Capelle a/d Ijsssel: Labyrint Publication.
Weisenfeld, Ursula (1989). Die Einflüsse von Verfahrensvariationen und der Art des Kaufentscheidungsprozesses auf die Reliabilität der Conjoint-Analyse [Influences of Variations in Process and Form of Buying Decision on the Reliability of a Conjoint-Analysis]. Berlin: Duncker & Humblot (in German).
Witmer, Bob G. and Michael J. Singer (1998), “Measuring Presence in Virtual Environments: A Presence Questionnaire,” Presence: Teleoperators and Virtual Environments (MIT-Press), 7 (3), 225-240.
Technical Appendix
In the near future, illustrations on a screen will be conveyed spatially, and traditional computer input devices like the keyboard and the mouse will be supplemented or replaced by more natural interfaces. To explain one of these new approaches, this appendix introduces some important components (Fraunhofer HHI 2007):
The Fraunhofer HHI 3D Kiosk system offers simple intuitive handling because it presents any kind of object in a virtual manner on a large screen in photorealistic 3D quality. The 3D objects seem to float in front of the display, without the need for additional technical devices, like 3D glasses or a helmet. In addition, the user does not need special aids for data input (e.g., stylus, data glove) but instead interacts through simple gestures to point at objects in virtual space.
Head tracking is an essential component of creating 3D content with autostereoscopic displays. The head tracking module constantly monitors the user’s eye position and enables the three-dimensional projection, even if the user moves in front of the screen. No glasses or helmets are needed to achieve the 3D impression, because the monitor projects a single picture separately to each eye to produce a stereovision effect.
A new and very natural pointing tool combines the head (pupil) tracker with a hand (finger tip) tracker. A camera-based hand tracker detects hand gestures and recognizes the position of the finger tip, which can be used to point at or move virtual objects represented by the stereoscopic display. The user in front of the screen can touch, pick up, and turn the three-dimensional objects as if they were real, without any further support of a stylus or glove (virtual 3D touchscreen). In this manner, the user can easily control the position of a marker on the screen by pointing in the desired direction.
About the Author
Alma Berneburg is a Ph.D. candidate in the Department of E-Business at the Otto-von-Guericke-University in Magdeburg, Germany, and a lecturer in the Department of Marketing at the University of Applied Sciences in Merseburg, Germany. Her research interests lie at the interface of market research, consumer decision-making, and the application of new technological solutions in market research processes.
