We investigated to what extent 12-13 year old students would use mental model thinking when designing an artifact, and how we mihgt increase that kind of thinking during design. We examined whether a constructivist design task would facilitate sophisticated mental-model thinking (Hmelo et al., 2000; Jacobson & Archodidou, 2000) and whether there were gender differences in mental-model development (Honey et al., 1991). From this study we drew implications for the creation of a computer-mediated learning space that would foster mental-model thinking during design. Our specific questions were: 1. What types of think do thse students do when designing and how do these types of thinking change with experience and instructional support? 2. Are there gender differences in types of thinking during design and are thinking differences when designing in different domains?
Mental Models
Most cognitive theories about mental models concur that they consist of objects and their relationships (Johnson-Laird, 1983; Kearney and Kaplan, 1997; Jonassen, 1999). However, this definition is limited in that it does not address the greatest power of mental models: the ability to mentally simulate mechanisms to infer rules or make predictions about the operation of a system (Schwartz and Black, 1996; Koffijberg, 1996). From this mental-simulation perspective, a mental model is a dynamic mental structure whose behavior can be used to show how the systems modeled will function under different circumstances (Gentner and Stevens, 1983; Aronson, 1997). Thus, in the fullest sense, mental models consist of the structural components of the system, knowledge of the inter-relatedness of those components and a causal mechanism describing and predicting the performance of the system (Jonassen, 1999).
Therefore, an adequate mental representation (components and relationships) of a system is not sufficient for understanding. There must be the capacity to reformulate or restructure the model and incorporate the consequences of this manipulation (Newton, 1996). This transformation or running of a mental model pertains to adjustments made through the application of the causal reciprocity found in the system. Our definition of mental models involve images of interacting parts that are animated (Schwartz and Black, 1996b). Thus, transformations involve the delineation of kinesmatic information into mental spatial representations (Newton, 1996; Aronson, 1997; Koffijberg, 1996). It is this act that allows the thinker to visually infer how or why a complex system works. A prevalent way of expressing this transformation is as mechanistic.
Understanding is more than the regurgitation of superficial facts and procedures. It is a product of mental processes that infer dependence between elements of information (Newton, 1996). Koffijberg (1996) avers that understanding may even equate with acquiring an accurate mental model. Therefore, research that focuses on mental-model development is essential to facilitate software instructional design aimed at developing student understanding of complex systems.
Constructivist Design Tasks
Design tasks, as instructional tools, have been proposed as facilitating the development of mechanism thinking, and thus, understanding (Honey, Moeller, Brunner and Bennet, 1991; Jonassen, Peck and Wilson, 1999; Hmelo, Holton and Kolodner, 2000). The act of design concentrates student efforts on the elements needed to build robust and sophisticated mental models: that is, student designers focus on acquiring the necessary information, its underlying structure, generating model examples and using the foundation entailed by the subject matter to justify the design (Jonassen et al., 1999). Therefore, the purpose of a design task is to allow the student to determine the relevant components and the ways they influence each other, and allows for the exploration of how systems work (Koffijberg, 1996; Hmelo et al., 2000). However, the context of the constructuvist design environment itself influences the kind of knowledge acquisition and the potential for the formation of a mental model. (Wood, 1995, Koffijberg, 1996, Jonassen et al., 1999). Therefore, analysis of the levels of mental-model development that occur during design tasks is necessary to provide guidance in devising the kinds of scaffolding, modeling and effective modes of transmitting information that need to be present in technology to support understanding.
Gender Issues
Psychological and sociological research suggests that females approach, interpret and understand various aspects of life differently than males (Honey et al., 1991; Gilligan, 1982). Two studies (Hawkins, Brunner, Clements, Honey and Moeller, 1990; and Brunner,1990) pointed to gender differences in domain content concentration during design tasks performed by adults and children. The researchers found that females focused on the Social domain (communication and affective aspects) while male designs tended to focus on the Physical domain (transportation). Further, females were less concerned with describing the internal mechanisms of their designs than males. These findings recommend an examination of gender differences in types of mental-model thinking across differing domains of content and their consideration in a computer-based design environment.
The Design Study
We investigated student thinking during design by having them design a Mars Colony. In using this particularly task we were able to utilize the Mars Millenium Project (see www.mars2030.org) which a number of organizations (e.g., NASA, NEH, etc.) were conducting in order to inspire students to be more interested in science at a time when several space probes would be reaching Mars. Thus, were able to use the information on Mars that this project put on the WWW and exploit the excitement that had been generated. Also, designing a Mars Colony had the advantage of being multidimensional (unlike previous design studies) involving physical, biological and social systems. In the study, we had students design the colony twice on different days, then take place in a trouble-shooting task with the Mars colony.
Method
Participants
Twenty-seven eighth grade students, nineteen females and eight males, participated in this study. The students attended a model middle school in the South Bronx section of New York City. Entrance to the school was selective and students were conscripted from surrounding neighborhoods, consisting of African-American and Hispanic populations. This middle school was lodged within a high school and was characterized by reduced class size (between 20-28 students). All students were involved in a daily science curriculum.
Procedure
This study took place during the regularly scheduled, eighth grade, science hour. Two researchers, along with the science teacher, were in the classroom for three days (Monday, Wednesday and Friday), henceforth referred to as Time 1, Time 2 and Time 3. Each time involved a general introduction, content share and presentation of a variation of the Mars Community design task, and an individual, student problem-solving period. Content for the Mars 2030 design task was obtained through the Mars Millennium Project at www.nasa.gov.
Based on Newton's (1996) outline of Model Processing Failure, we investigated three, slightly modified design tasks. The first task (Time 1) involved the least amount of instructional direction. Students were instructed on minimal base knowledge regarding the intent of the task. Further, there was a purposeful withholding of a conceptual model, which allowed students to formulate their design from only their mental resources. The second task (Time 2) entailed more instructional direction than the first design task. In addition to the previous base knowledge, 10 essential concepts in the complex system were highlighted and students were instructed to focus on the "how and why" of their design (Newton, 1996). Finally, the third task (time 3) contained explicit instructional direction. While still maintaining a strong design element, focused questioning on the mechanistic aspect of the student's mental model and forced prediction were employed.
Following the introduction and content sections of Time 1, the students engaged in approximately 30 minutes of individual work. Each student was supplied with an 8 ½ by 11 piece of white paper and a pencil. (More paper was provided based on individual requests.) The class was then told: "Based on everything that we've just talked about, we now want you to design a community for 100 humans on Mars, for 1 Martian year, in the year 2030. We want your design to be as clear and understandable as possible, so if you need to explain things, please do." The researchers stressed the need for the designs to be comprehendible to outside viewers and gave the analogy of the designs as being comparable to a blueprint. Further, the students were encouraged to complete their work on their own, in an attempt to cut down on collaboration.
A reiteration of the introduction and content sections from Time 1 initiated Time 2. In addition to Time 1 content, the researchers highlighted 10, pre-established categories as important to community design: culture, food, communication, shelter, government, water, transportation, air, health & safety and sanitation. This was followed by 30 minutes of individual work creating another paper and pencil design. The class was instructed: "We want you to design a community for 100 people on Mars again. However, this time, please consider the 10 categories we just talked about as you design your community. You can incorporate any of your first design into your new design." . The students did not receive the design they had created during Time 1.
For Time 3, after the introduction and content sections, the researchers posed the following situation to the students: "An emergency situation has arisen within the context of your second community. A third of your community members have become seriously ill." This was followed with the following assignment: "Your job is to answer the following questions: (1) What do you think happened in your community that affected the members? (2) Why do you think it happened like this? Justify and explain your answer. Then complete the following tasks: (3) Show us how you think this happened. (4) Make a prediction of how this will effect the future of your community. (5) Design a research plan to investigate whether or not your theories/ideas about what caused a change in the community are true." Each student received a pencil and a stapled packet of five pages (8 ½ by 11). Each page had a question or task at the top, with the rest of the page blank. All of the packets were ordered as shown above and the researchers read through the packet with the class before they began the task. Further, each student also received a copy of his/her Time 2 design to be used as a reference. The students were given approximately 30 minutes to design their answers.
Data Analysis
The raw data for this study are the pencil and paper designs created by the students each day. The student's mental models (what we ultimately want to investigate) are just that-mental. One method of externalizing these models is to have subjects draw or write them out (Wood, 1995). Thus, we view the student's designs as cognitive maps (Kearney & Kaplan, 1997) or concrete models (Getner & Stevens, 1983), which characterizes each student's general knowledge regarding his/her Mars community. Specifically, we use the students' designs to assess the level of component (i.e., objects), relational and mechanistic thinking found in their renderings, with the intent that this reflects the level of processing happening in the students' heads.
The coding of each of the three levels of thinking were couched in 10 researcher, pre-determined categories, which we later classified in four domains. These 10 categories where chosen by the researchers to promote coding consistency, because we believe they are all necessary for community survival, and also because they cover physical, biological and social aspects of community development. Further, the 10 categories remained constant for the coding of Time 1, 2 & 3 designs. For Time 1, students during the design task were completely unaware of the pre-established categories. For time 2, the 10 categories were outlined and stressed at the beginning of the design task. During Time 3, students were aware of the categories because of their Time 2 experience, however, the 10 categories were not reiterated or highlighted before the design task began.
Within each of the 10 categories, three scores were tabulated for each student. First, the components within each category, whether pictorial or written, were counted. Second, scores were gathered for relationships among components. Relationships were identified by explicit pictorial or written connections between components. To promote coding consistency, spatial distance was not taken into consideration as indication of a relationship. Thus, components bunched close together were not counted as connected unless specifically indicated in some way by the student. Finally, scores were collected, within each of the 10 categories, for the depictions of mechanisms. Mechanistic representations were coded as being present when objects were pictorially or written with explicit, directional relations that conveyed movement. That is, the relationship between objects where depicted as causal and directional. Based on the coding method explained above, each of the 27 students received 90 scores (3 levels of thinking X 10 categories X 3 sessions).
During the analysis process, the 10 category scores in each level were reduced, through summation, into four domains. The Physical domain consists of the categories of Transportation and Shelter. The Biological domain consists of the categories of Food, Air and Water. The sum of Communication, Culture and Government make up the Social domain. Health & Safety and Sanitation, key elements of communities, did not fall into any one of the three previous categories, so their scores were summed into a Multi-dimensional domain.
Results and Implications
For brevity, we are only including information on results with specific implications for the instructional design of a Mars 2030 computer environment. We conducted three (one for Time 1, Time 2 and Time 3) GML mulivariate tests in the four domains, for each type of thinking, with Gender as a fixed factor. For Time 1 and Time 2, there were no significant differences, at any type of thinking, in any domain, between females and males. Time 3 results indicate non-significant differences between females and males, with the four domains, for relational and mechanistic thinking. There was a significant difference between females and males in component thinking in the social domain (.000) [all other component-domains were non-significant]. However, further evaluation of the direction revealed that males had more components, which counters what Honey et al. (1991) might have expected. Thus, our research indicated no instructional implications with for domain design with gender as a factor.
Next, we conducted individual 2-Tailed T-test for each type of thinking (components, relational and mechanistic) on each day (Time 1, Time2 and Time 3). All levels on each day were significant (null hypothesis = 0) at the .05 level, thus in general, these types of thinking did occur during the design process. This was followed by Time 1 T-tests for each domain. In Time 1 mechanistic-type thinking, all domains were non-significant except physical-mechanistic (.043). This may imply that a computer environment that initially focused on the physical domain and gradually moves to more abstract domains would foster learning. Further, Time 2 T-tests for Social-mechanistic (.327), Time 3 Social-relational (.327) and Social-mechanistic (sd=0) thinking yielded non-significant results. These results may indicate that more structured or supported instruction than what we provided over the three-day experience should be included for this domain in a computer design environment.
Next, we summed domains and conducted 2-Tailed Paired-Sample T-test comparing Times within types of thinking. (Note: we did not conduct a repeated measure because although all the measures are designs, Time 3's measure was altered compared to Time 1 and Time 2). Results indicated significant differences between Time 2-component and Time 3-component thinking (p.000), Time 1-relational and Time 2-relational (.000) and Time 2-relational and Time 3-relational thinking (.001). No significant differences were indicated for mechanistic-type thinking from Time 1 to time 2 or Time 2 to Time 3. These results indicate design practice combined with a progression of more structured tasks affected componental and relational thinking, while having no significant effect on mechanistic thinking. Several reasons could be responsible for these results. First, the students only had 3 experiences designing and this may not have been enough practice, although the time-frame was sufficient to effect componental and relational thinking. Second, the static nature of the medium of instruction and student design (paper and pencil) may have inhibited both the mechanistic thinking of the students and our assessment of their work. This indicates that repeated exposure to a dynamic design environment (one where students could run simulations on their creations), rather than a computerized version of the paper and pencil task (such as a drawing tool) might be the best format for a computerized instructional and assessment tool of complex systems.
Bibliography
Aronson , J.L. (1997) Mental Models and Deduction. The American Behavioral Scientist. 40, 782-797.
Brunner, C. (1990) "Gender and Technological Imagination" Paper presented at the annual meeting of the American Educational Research Association. Boston, MA
Gentner, D and Stevens, A. (ed) (1983) Mental Models. Hillsdale, NJ: Lawrence Erlbaum Associates.
Hawkins, J. Brunner, C., Clements, P., Honey, M. and Moeller, B. (1990) "Women and Technology: A New Basis for Understanding-Final report to the Spencer Foundation". CCT Technical Report, www.edc.org/CCT/
Honey, M., Moeller, B., Brunner, C., and Bennet, B.T.(1991) Girls and Design: Exploring the Question of Technical Imagination. CCT Technical Report, No.17, www.edc.org/CCT.
Hmelo, C.E., Holton, D.L. and Kolodner, J.L. (2000) Designing to Learn about Complex Systems. Journal of Learning Sciences. 9, 247-297.
Jacobson, M.J. and Archodidou, A. (2000) The Design of Hypermedia Tools for Learning: Fostering Conceptual Change and Transfer of Complex Scientific Knoweldge. Journal of Learning Sciences. 9, 145-199.
Jonassen, D.H., Peck, K.L. and Wilson, B.G. (1999) Learning with Technology: A Constructivist Perspective. Columbus, Ohio: Prentice Hall.
Jonassen, D.H. (1999) "Operationalizing Mental Models: Strategies for Assessing Mental Models to Support Meaningful Learning and Design-Supportive Learning Environments". Online Article: http://www-csc195.indiana.edu/csc195/jonassen.html
Johnson-Laird (1983) Mental Models: Towards a Cognitive Science of Language, Inference, and Consciousness. Cambridge, MA: Harvard University Press.
Kearney, A.R. & Kaplan, S. (1997) "Toward a Methodology for the Measurement of Knowledge Structures of Ordinary People: The Conceptual Content Cognitive Map (3CM). Environment and Behavior. 29, 579-617.
Koffijberg, M. (1996) Acquisition of a Mental Model by means of Computer Simulation Zeist/Enschede, http://projects.edte.utwente.n1/pi/Papers/MentalModel.html
Newton, D.P. (1996) Causal Situations in Science: A Model for Supporting Understanding". Learning and Instruction. 6, 201-217.
Schwartz, D.L. and Black, J.B. (1996a) Shuttling Between Depictive Models and Abstract Rules: Induction and Fallback. Cognitive Science, 20, 457-497.
Schwartz, D.L. and Black, J.B. (1996b) Analog imagery in mental model reasoning. Cognitive Psychology, 30, 154-219.
Wood, W. (ed) (1995) Cognitive Strategy Instruction for Middle and High Schools. Cambridge, MA: Brookline Books.
|
