In Craig Montgomerie and Jarmo Viteli
(Eds.) Proceedings of ED-MEDIA 2001.
Norfolk, VA: AACE, 2001.
The Topic
Interactive Panoramic Imagery (IPI) provides a way for users to simulate the exploration of a real-world space. Based in photography, the technology offers an experience that is visually compelling, and spatially realistic. The specific application built for the purpose of this work (shown in figure 1) provides the user with a self-guided tour of the Cathedral of St. John the Divine. The user is able to move through the entire cathedral and manipulate their point of view in any direction at any one of 39 specific locations. This spatial freedom is then combined with textual descriptions of the areas being visited and an overhead map that is updated as the user moves through the environment.
The advent of tools such as this "virtual cathedral" brings about diverse questions around the nature of experience and cognition. The two primary questions that this work addresses are: does text design affect the exploration of computer-mediated spatial experiences and how does interactive panoramic imagery change the way in which text is read and remembered.
Figure 1 - The IPI cathedral interface - the interface contains a floorplan on the right side of the interface, an interactive panoramic image on the top left, and text describing the area being visited at the bottom left
|
Motivation for the Work and the Major Questions Addressed
The primary motivation for the work is to explore how a simulated spatial environment affects cognitive activity in an educational context. Additionally, we sought to explore how the design of text that accompanies such an environment might also affect cognition.
While the variety of educational content that is being mediated through computer interfaces is tremendously diverse the interfaces that hold such information are less so. Since the mid-1980s, the WIMP (window, icon, mouse, and pointing device) desktop configuration has dominated interface design (Apple Computer, 1992). Alternatives to this interface have been suggested, researched, and developed (Card, Mackinlay, & Shneiderman, 1999) but infrequently adopted.
As increasing amounts of information are mediated via computer interfaces the need to ensure that those interfaces are taking full advantage of computer's presentation abilities - as well as the population's complete cognitive abilities - arises. One particularly ripe avenue for research and development surrounds the computer's ability to simulate spatial environments and the human perceptual system's ability to think and move in three dimensions.
Currently, the majority of psychological research that has looked at computer-based spatial simulations is based in how computer-based experience transfers to real world spatial experience (Witmer, Bailey, Knerr, & Parsons, 1996). However, there is a need to examine how such computer applications might help to affect cognition in ways that one is more accustomed to seeing in a classroom. Spatial simulations may help users to identify architectural features, interpret drawings and descriptions of buildings, or better understand structural forces. A primary goal of this work is to set out a framework for evaluating the cognitive effects of such simulated spatial experience.
Computers can, of course, provide information in modes that do not simulate spatial environments. Researchers, however, have thus far not looked to examine the effects of computer based media such as text, video or dynamic imagery when used in combination with spatial simulations. When scholars do consider the effects of additional media on virtual reality-l experiences they almost exclusively focus on improving navigation within those environments through the use of devices such as compasses (Ruddle, Payne, & Jones, 1998). Another primary goal of the research, then, is to examine how ancillary media, such as text and imagery, affects the user's experience in a computer-based spatial environment.
The General Process and Conceptual Framework with References to the Literature
Knowledge Representation Approach
The knowledge representation approach to cognitive psychology which asserts that the mind works on internal representations of knowledge, provides the foundation for these studies. By examining ways that multiple types of knowledge representations change with experience in various interfaces, the cathedral studies aim to detect meaningful trends around the effects of interfaces on the recall of information. The three types of knowledge representation that the study assesses are factual knowledge, imagistic knowledge, and spatial relations knowledge. While these are not the only types of knowledge representations in effect, these are the three primary knowledge representations in service for the understanding of space.
Explanative Illustrations
As the focus of research on simulated spatial environments has not centered on this type of research, it has been useful to look elsewhere for studies that look at the combination of multiple media types. Mayer and Gallini (1990) wrote about the effects of illustrations on textual descriptions. The work of these researchers focused primarily on the function of explanatory illustrations for scientific devices. Among their concerns is what effects do texts have on participant's abilities to develop "runnable mental models" of devices. The cathedral software research examines the relationship between multiple media forms in establishing such runnable mental models. In each case, the term runnable mental model refers to point-of-view independent mental representations that provide information about how a physical system works and how state changes affect that system(Bower & Morrow, 1990; Mayer & Gallini, 1990; Tversky, 1993). While similarities exist, however, it is necessary to distinguish ways in which the devices learned in Mayer's studies differ from the spatial environments with which I am concerned
Mayer describes two primary ways in which illustrations can aid learners in developing runnable mental models: they can provide system topology as well as indications of component behavior. System topology refers to a map that labels the primary parts of system. Component behavior describes changes among portions of a system or device. Mayer dubs illustrations meeting these criteria "explanative illustrations." The combination between text and explanative illustrations provides adequate mental representations so that a learner can manipulate the machine components mentally, without the aid of a physical model.
Mayer's work has helped us conceptualize and operationalize the term "Spatially Explanatory Media". Spatially explanatory media (SEM) are media that place emphasis on part-to-part or part-to-whole relationships within spatial environments. That emphasis helps users to make spatial inferences both between and within different types of spatial media, allows users to better move between different knowledge representations that reflect their experience with the space, and helps users overcome orientation difficulties that often plague computer-based simulations of spatial relations.
Examples of SEM include overhead maps that make apparent hierarchical distinctions within a space. Such maps highlight various part-to-part and part-to-whole spatial relationships. Interactive Panoramic Imagery (IPI) interfaces are also excellent examples of spatially explanatory media. Users manipulate these media to identify relationships between their location and the various objects within their dynamic field of view. Still photographs offer less in the way of spatial explanations. Still photographs limit viewers to a singular point of view, potentially rendering viewers unable to integrate that point of view within a larger frame of reference.
Text can be written to provide spatial explanations. For example, spatial metaphors that liken spatial layout to a known structure highlight whole/part relationships. Spatially explanative text demands that the reader coordinate two or more spatial locations. In addition, text can be void of spatial explanation. For instance, descriptive text offers information about singular materials along non-spatial dimensions; descriptions of dates, persons, colors, and the like offer no spatially explanatory information.
The overall goal of our work, then, is to examine how different forms of spatially explanatory media combine to bring about different levels of distinct knowledge representations.
The Concrete Method, with Sufficient Detail on Instruments and Procedures
The experiment made use of a computer based application that allowed participants to explore the Cathedral of St. John the Divine. Modifications were made to the program to allow testing to be done that would assess the effects of the various interface components.
Participants
Fifteen participants were run in each of four groups - producing a two by two factorial design. The first factor was presence or absence of interactive panoramic imagery. The second factor was the type of text that participants received.
Participants were all college graduates who were not art historians, architects, or religion experts. This is an important choice as research into expertise (Chase & Simon, 1973) has shown that experts see structure in materials with which they have familiarity. Novices need time in order to perceive those same structures. Participants were paid $12.00 for participating in the study.
Materials and Procedure
The experiment made use of two different texts: a "spatially explanatory" text and a "descriptive text". The spatially explanatory text incorporated spatial metaphors among other devices to bring attention to the cathedral's layout. An example of this text design occurs in the Ambulatory description that mentions the chapels are named for saints and that they "cushion" the symbolic head of Christ. Care was taken to ensure that the two texts had the same readability scores and were of the same length. All factual information needed to answer questions was included in the texts. Additionally, 84.5% of the words and phrases in the texts were identical to each other.
The computer interface was programmed to either deliver interactive panoramic imagery along with the text in the manner described above, or to provide the text without the IPI. In either case, users moved through the program by clicking on areas on the map located on the right side of the interface.
After moving through the entire cathedral, participants were given three different tasks. A factual knowledge task, consisting of short answer questions about the reading, a spatial layout task that asked participants to identify the names of the cathedral's seventeen distinct sections, and a spatial relations task that asked them to provide a compass direction when asked about the relative location between two sections of the cathedral. In addition to these tests, participants were also asked to rate how much they agreed with the statement "I enjoyed using the program" on a scale of 1 to 7. The computer program also kept track of the amount of time that the users spent exploring the cathedral.
Major Points or Results
The Effectiveness of Interactive Panoramic Imagery
The interactive panoramic imagery proved to be effective in helping participants to build factual knowledge representations and also mental representations of the cathedral's spatial layout. The IPI factor was statistically significant (F(1,56)=4.317, p.=042) for questions of factual information, where the IPI group scored higher, and bordered significance for spatial layout (F(1,56)=3.676, p.=.060), which also saw the IPI group perform in a superior fashion. While the IPI group also outperformed the text group on the spatial relations task, this difference was not significant (F(1,56)=1.476 p.=.229).
While these scores do point to the overall effectiveness of interactive panoramic imagery as a tool for providing architectural information, the question of time spent with materials must also be raised. Participants that received interactive panoramic imagery spent significantly longer in the cathedral interface than did the text groups (F(1,56)=31.177, p= <.001). While this does not discount the effectiveness of the IPI, it may point to more motivational effects of the medium than effects that would magnify the ability to assimilate and remember spatial and factual aspects of the experience. Indeed, the groups that received interactive panoramic imagery rated their enjoyment of the experience higher than did the text groups, and that difference was statistically significant (F(1,55)=11.88 ,p.=.001).
The Effectiveness of Spatially Explanatory Text
Unlike the IPI factor, the groups that received the two different types of text spent essentially the same amount of time with the cathedral materials, with the spatially explanatory text group spending slightly more time in the interface(F(1,56)=.014, p.=.91). The differences found between the descriptive text and spatially explanatory text groups on the various measures, then, can be more easily attributed to the nature of the materials themselves, rather than the time that it took to work with the materials.
Participants receiving spatially explanatory text scored higher than those receiving descriptive text on the factual knowledge task. That difference borders on statistical significance (F(1,56)=3.361, p.=.072). We do see a significant difference in the spatially explanatory text group's ability to recall spatial layout (F(1,56)=4.462, p.=.039), but no difference in spatial relations knowledge (F(1,56)=1.786 p.=.187). Though the spatially explanatory text groups rated their experiences more positively, the difference did not approach significance (F(1,55)=.34,p.=.56).
Implications
We see, then, that slight differences in text design can bring about significant changes in the way in which participants come to remember spatial layout. The results of the factual knowledge task with respect to text type also warrants further consideration. When designing multimedia interfaces with the goal of providing learners with persistent knowledge of spatial information, it appears that text design can have an important role. Additionally, interactive panoramic imagery has clear motivational effects, and is potentially of aid in ways beyond motivation. Further studies are needed in order to tease out the particular effects of that medium.
References
Apple Computer, I. (1992). Macintosh Human Interface Guidelines. Reading, MA: Addison-Wesley.
Bower, G. H., & Morrow, D. G. (1990). Mental Models in Narrative Comprehension. Science, 47, 44-48.
Card, S. K., Mackinlay, J. D., & Shneiderman, B. (1999). Readings in Information Visualization: Using Vision to Think. San Francisco: Morgan Kaufmann.
Chase, W. G., & Simon, H. A. (1973). The Mind's Eye in Chess. In W. G. Chase (Ed.), Visual Information Processing (pp. 215-281). New York, NY: Academic Press.
Mayer, R. E., & Gallini, J. K. (1990). When in an Illustration Worth Ten Thousand Words? Journal of Educational Psychology, 82(4), 715-726.
Ruddle, R. A., Payne, S. J., & Jones, D. M. (1998). Navigating large-scale "desk-top" virtual buildings: Effects of orientation aids and familiarity. Presence: Teleoperators and Virtual Environments, 7, 179-192.
Tversky, B. (1993). Cognitive Maps, Cognitive Collages, and Spatial Mental Models. Paper presented at COSIT '93, 14-24.
Witmer, B. G., Bailey, J. H., Knerr, B. W., & Parsons, K. C. (1996). Virtual spaces and real world places: transfer of route knowledge. International Journal of Human-Computer Studies, 45, 413-428.
|
