This dissertation research investigates how different concept mapping forms embedded in a collaborative technology-enhanced learning environment can support students’ integration of evolution ideas using case studies of human evolution. p. 3

Knowledge Integration (KI) (Linn et al., 2000; Linn et al., 2004) is used as the operational framework to explore concept maps as knowledge integration tools to elicit, add, critically distinguish, group, connect, and sort out alternative evolution ideas. Concept maps are a form of node-link diagram for organizing and representing connections between ideas as a semantic network (Novak & Gowin, 1984). This dissertation research describes the iterative development of a novel biology-specific form of concept map, called Knowledge Integration Map (KIM), which aims to help learners connect ideas across levels (for example, genotype and phenotype levels) towards an integrated understanding of evolution. p. 3

Study 1 investigates concept maps as generative assessment tools. Study 1A compares the concept map generation and critique process of biology novices and experts. Findings suggest that concept maps are sensitive to different levels of knowledge integration but require scaffolding and revision. Study 1B investigates the implementation of concept maps as summative assessment tools in a WISE evolution module. Results indicate that concept maps can reveal connections between students’ alternative ideas of evolution. p. 4

Study 2 introduces KIMs as embedded collaborative learning tools. After generating KIMs, student dyads revise KIMs through two different critique activities (comparison against an expert or peer generated KIM). Findings indicate that different critique activities can promote the use of different criteria for critique. Results suggest that the combination of generating and critiquing KIMs can support integrating evolution ideas but can be time-consuming. p. 4

As time in biology classrooms is limited, study 3 distinguishes the learning effects from either generating or critiquing KIMs as more time efficient embedded learning tools. Findings suggest that critiquing KIMs can be more time efficient than generating KIMs. Using KIMs that include common alternative ideas for critique activities can create genuine opportunities for students to critically reflect on new and existing ideas. Critiquing KIMs can encourage knowledge integration by fostering self-monitoring of students’ learning progress, identifying knowledge gaps, and distinguishing alternative evolution ideas. p. 4

This dissertation research explores the use of concept maps as Knowledge Integration (KI) (Linn et al., 2000) tools to elicit existing ideas and connections, add new ideas, critically distinguish ideas, and sort out alternative ideas. Concept maps are a form of node-link diagram for organizing and representing connections between ideas as a semantic network (Novak & Gowin, 1984). This study describes the iterative development of novel scaffolded biology- specific forms of concept maps, called Knowledge Integration Maps (KIM) that aim to help learners connect ideas across levels towards an integrated understanding of evolution. This p. 19

The curriculum includes scaffolded inquiry activities using dynamic visualizations that allow learners to investigate the relationships between genetic and evolution ideas. p. 20

Purpose of concept map Concept map as posttest assessment tool Concept map as learning tool: Generation and critique; Concept maps as pre/post- assessment tool Concept map as learning tool: Generation or critique; Concept maps as pre/post- assessment tool p. 20

“How can concept maps be used to assess gains in knowledge integration of evolution ideas?” p. 21

Study 1A investigates concept maps as a generative assessment tools to elicit alternative ideas of evolution. The study compares the concept map generation and revision process of biology experts and novices (students) using a talk-aloud protocol. p. 21

Study 1A aims to answer the research questions: 1) How do biology novices and experts differ in their concept map generation and revision process? 2) How do novices of different academic performance levels differ in their concept map construction? 3) How does verbal reasoning (talk aloud) match with concept map construction? p. 21

Study 1B p. 21

compares the two generative assessments - concept maps and essays - as posttests to capture students’ understanding of given genetic, cell biological, and evolution ideas. Generation activities can promote learning (van Amelsvoort, Andriessen, & Kanselaar, 2005). p. 21

By generating, students articulate and represent their knowledge, apply their representations to solve scientific problems, realize gaps in their knowledge, reorganize ideas, and strengthen connections among ideas. The generation effect of giving explanations (to oneself or others) has been found more beneficial for learning than receiving explanations (Chi, 2000a; Chi, De Leeuw, Chiu, & LaVancher, 1994). Based on the Knowledge Integration framework, a new concept map analysis rubric has been developed that focuses on evaluating connections between core ideas. p. 21

Study 1B aims to answer the research questions: 1) How did students’ integration of evolution ideas change after using the WISE module Meiosis - the next generation? 2) How do the generative summative assessments methods concept mapping and essays differ in describing students' understanding of the connections between evolution ideas after the WISE module Meiosis - the next generation? 3) How can quantitative and qualitative concept map analysis methods (concept map topology, concept map accuracy score, and concept map convergence score) be used to distinguish different levels of knowledge integration? 4) How does the dynamic visualization BioLogica support knowledge integration of evolution ideas? p. 22

In addition to using concept maps as assessment tools, the second study embeds concept maps as collaborative knowledge integration tools. Study 2 extends the concept map generation activity with a subsequent revision and critique step to foster students’ critical reflection and revision of their concept maps. Instead of using concept maps as a one-shot posttest activity as in the first study, students were given the opportunity to critique and revise their concept maps. Asking students to critique has been found to support the development of more coherent and generative criteria (Lehrer & Schauble, 2004). Critique activities require students to use or develop criteria to reflect, elaborate their ideas, revise their ideas, and self-monitor their learning progress, which supports the development of skills for lifelong autonomous learning (Chi, 2000b). In traditional classrooms, students are often given only limited opportunity to apply critique as scientific knowledge is frequently taught as given facts and delivered by textbooks or teachers who represent authority (Shen, 2010). Generation and critique activities can encourage students to actively use dynamic visualizations and facilitate integration of ideas from the visualizations (Buckley, 2000). p. 22

To elicit cross-connections between different levels, a novel form of concept map, called a Knowledge Integration map (KIM), has been developed that divides the drawing area into biology-specific levels: DNA, cell biology, and organism. Students were instructed to place ideas in the corresponding level and generate connections within and across levels. p. 22

a novel form of concept map, called a Knowledge Integration map (KIM), has been developed that divides the drawing area into biology-specific levels p. 22

a novel form of concept map, called a Knowledge Integration map (KIM), has been developed that divides the drawing area into biology-specific levels: DNA, cell biology, and organism p. 22

Similar to study 1, students generated paper-and-pencil concept maps from a given list of genetic, cell biology, and evolution ideas. After the generation phase, students reviewed and revised their maps by comparing them against another map: One group compared their concept map against an expert- made concept map while the other group provided feedback for a peer-generated concept map. Students were required to develop their own criteria for their critique. p. 22

The goal of study 2 is to evaluate the effects on student learning through a technology- supported learning environment with multiple interactive visualizations and critique-enhanced concept maps designed to support a more coherent understanding of evolution. p. 23

The general research question of this study is: How did students’ integration of evolution ideas change after collaboratively generating and critiquing KIMs embedded in an inquiry- focused technology-enhanced learning environment? p. 23

The specific research questions this study addresses are: 1) Did the two treatment groups differ in their integration of evolution ideas after using the WISE module Space Colony? 2) How did students in each treatment group place the given ideas into the corresponding level in their KIMs? 3) What connections did students in each treatment group generate in their KIMs? 4) How did students in each treatment group generate criteria when critiquing expert or peer KIMs? 5) How did students use the critique activity to revise their KIMs? p. 23

C. Overview Study 3: Knowledge Integration Maps as learning tools: Distinguishing generation and critique p. 23

Findings from study 2 indicated that a combination of generating and critiquing KIMs supported students’ knowledge integration but was also time intensive. As time in science classrooms is very limited, study 3 aims to reduce time demands by distinguishing the learning effects from either generating or critiquing Knowledge Integration Maps. These findings will allow the design of more efficient concept mapping activities. Instead of reviewing peer maps that vary widely in quality, students in the critique group received a pre-made KIM with deliberate commonly found alternative ideas, introduced as the creation of a fictional student. The number of given ideas was reduced to limit complexity and time requirements. Study 3 used a revised version of Knowledge Integration Maps that distinguish two biology-specific levels: Genotype and phenotype. It was hoped that these levels allowed for a clearer distinction of ideas p. 23

than the three levels in the second study. To facilitate concept map revisions, an electronic concept mapping tool, Cmap (Canas, 2004), was used instead of a paper-and-pencil format. p. 24

The general research question of this study is: How do students integrate evolution ideas through critiquing or generating concept mapping activities embedded in an inquiry-based technology-enhanced learning environment? p. 24

Specific research questions: A) Overall changes in students’ knowledge integration of evolution ideas 1) How does the WISE module Gene Pool Explorer change students’ integration of evolution ideas? 2) How does the WISE module Gene Pool Explorer help students to integrate evolution ideas across contexts (plants and humans)? B) Changes in knowledge integration of evolution ideas of treatment groups 3) How do treatment groups (critique and generation) differ in generating KIMs after the WISE module Gene Pool Explorer? 4) How do treatment groups (critique and generation) differ in cross-links between genotype and phenotype level ideas in their KIMs after the WISE module Gene Pool Explorer? 5) How do treatment groups (critique and generation) differ in qualitative changes of connecting ideas in their KIMs after the WISE module Gene Pool Explorer? 6) What variables can track changes in students’ evolution ideas in KIMs? 7) How do treatment groups (critique and generation) differ in integrating core evolution ideas in their KIMs after the WISE module Gene Pool Explorer? 8) How do treatment groups (critique and generation) differ in changes of the topology of their KIMs after the WISE module Gene Pool Explorer? p. 24

9) How do treatment groups (critique and generation) differ in critiquing KIMs after the WISE module Gene Pool Explorer? 10) Is generating or critiquing KIMs a more time efficient knowledge integration activity to learn about evolution ideas? p. 25

Constructivism emphasizes that knowledge is actively constructed by the learner through connecting new ideas to prior existing ideas. Effective teaching and learning needs to address learners’ existing ideas and scaffold building connections between new and existing ideas. Within science education, it is widely accepted that prior existing ideas are a key element that influences learning, as summarized by Clifton and Slowiaczek (1981): “Our ability to understand and remember new information critically depends upon what we already know and how our knowledge is organized.” p. 28

A Knowledge Integration learning environment aims to provide learners with a variety of scaffolded opportunities to build criteria to distinguish ideas. By frequently revisiting and selecting alternative ideas, builders can learn to select normative ideas of evolution over non- normative ideas more frequently and in different contexts. When builders select non-normative stones over normative ones, the constructed understanding will be only partial and less strong. The building is never quite finished. The construction of the building is dynamic and the builder should be encouraged to frequently revisit to add or replace stones and add or change connections between stones. In addition to adding and removing stones, builders can also change the shape of stone (change meaning of ideas). p. 32

This applies to a field where we know the outcome - non-normative vs normative stones… How does this apply to "softer" topics? p. 32

This research views learning as the process of integrating ideas through the cognitive processes of eliciting, adding, connecting, critiquing, distinguishing, sorting out, refining and applying ideas in a broad range of contexts (Bransford, Brown, & Crocking, 2000; Linn & Eylon, 2006; Piaget, 1971a; Smith, 1994; Vygotsky, 1962). To encourage students to build and revise connections between related ideas, this study uses the knowledge integration (KI) framework, which focuses on making connections among ideas (Computer As Learning Partner: Revised Annual Report, 1995; Linn, 1996; Linn, 2008; Linn & Hsi, 2000; Linn et al., 2004; Linn, Lee, Tinker, Husic, & Chiu, 2006). p. 33

Conceptual change research indicates that learners hold a rich repertoire of dynamically connected co-existing, and often conflicting, alternative ideas about the world around them (Davis, 2003; Davis & Linn, 2000; diSessa, 2000; Linn, 2002; Slotta, 1995; Songer, 2006) rather than internally consistent scientific theories, and that students often fail to connect ideas from one context to another (diSessa, 1988). Piaget showed that even an infant’s mind contains complex cognitive structures (Piaget, 1971b). p. 33

Research suggests that in order to form coherent understanding of evolution, students need to add and distinguish new ideas and connections to their existing repertoire of ideas rather than replacing existing ideas (Demastes et al., 1995; Linn, 2008; Strike & Posner, 1992). Rather than seeing existing old ideas as obstacles that need to be replaced, knowledge integration seeks to add new ideas, and through application in different contexts, help students develop criteria to distinguish when ideas are relevant (Linn, 2008). Prior ideas are not simply exchanged for a new idea because ideas are embedded in a dynamic network where they define and constrain each other (Demastes et al., 1995; diSessa, 2002; Park, 2007). Adding new evolution ideas to one’s repertoire is influenced by existing alternative ideas, view of the biological world, development of evolutionary theory, epistemological commitments (the nature of knowledge, the nature of learning, and the nature of conceptions), metaphysical beliefs, and emotional aspects (Demastes et al., 1995; diSessa, 2002; Kinchin, 2000b; Park, 2007). The knowledge integration approach encourages adding new normative ideas through carefully designed instruction that supports p. 33

students to revisit their initial ideas, such as powerful pivotal cases (Linn, 2005)[See chapter 2: Human Evolution as a Pivotal Case]. p. 34

Processes that encourage knowledge integration include eliciting students’ existing ideas (for example by explaining or predicting generation activities)[See chapter 2: Generation], adding ideas to build understanding (for example through dynamic population genetics inquiry activities), helping learners refine and sort their repertoire of ideas (for example by asking for explanations about how the molecular view relates to their observations), and developing criteria for distinguishing ideas depending on context (for example by critiquing concept maps generated by peers) [See chapter 2: Critique]. Research suggests that students should frequently revisit and revise their ideas and the connections between them. Ideas should be presented in various formats and contexts, for example text, pictures, dynamic visualizations, symbolic representations, or everyday situations (Linn et al., 2006). By engaging students in knowledge integration processes, they can learn to self-monitor their learning progress and take an active role in refining their knowledge. Developing self-monitoring skills for their own understanding can help students to become lifelong learners for biology-related topics. p. 34

Generating artifacts, such as explanations or concept maps, can promote conceptual learning (van Amelsvoort et al., 2005). Osborne (1983) described learning science as a generative process. Slamecka and Graf (1978) found that learners could remember self- generated words better than when they were simply presented the words to be read. This “generation effect” has been well documented in a variety of different settings. For example, Foss (1995) reported evidence that generating summaries when studying texts can improve recall of ideas. Chi (1994; 2000a) found that generating explanations of a text or diagram, whether for oneself or for others, can be more effective for learning than receiving explanations. By generating explanations and concept maps, students articulate and represent their knowledge in new forms, apply their representations to solve scientific problems, realize gaps in their knowledge, and reorganize ideas. Using concept maps repeatedly as an embedded learning tool may allow learners to collect and connect ideas from different contexts. Traditionally, most concept mapping activities consist of generation from scratch. On the other hand, generating concept maps from scratch is often time-consuming and cognitively demanding. p. 35

Critique is an essential step in the knowledge integration process of distinguishing alternative ideas. This dissertation research explores an alternative to generating concept maps from scratch: Critiquing and revising a provided concept map with deliberate flaws might provide more scaffolding to promote knowledge integration of evolution ideas. p. 35

Critique activities require students to use or develop criteria to reflect, revise their work, and self-monitor their learning progress (Chi, 2000b) that can foster the development of metacognitive skills for lifelong autonomous learning. Critique activities encourage the elaboration of ideas and conjectures. Asking students to critique has been found to lead to the development of coherent and generative criteria (Slotta & Linn, 2000). p. 35

Learners’ views of the nature of science influence their willingness to critique (Schwarz & White, 2005; Tabak, Weinstock, & Zvilling-Beiser, 2009). Many students seem to hold the objectivist view that scientific knowledge is discovered and static (Marcum, 2008) rather than consisting of constructed tentative models. When scientific ideas are understood as immutable p. 35

products there is little reason to critique. p. 36

Learning as argumentation - IBIS? p. 36

In science education, collaboratively critiquing ideas requires learners to argue, negotiate, and make informed decisions (Berland & Reiser, 2009). Finding common ground can be a driving force for critique. To reach such common ground, students need to pose questions, make revisions, accept propositions, defend against criticism, and improve their criteria (Shen & Confrey, 2007). p. 36

DiSessa (2002; 2004) found that students are able to develop their own criteria to critique representations. A meta-study by Falchikov and Goldfinch (2000) found that student-generated criteria work better for peer assessment than using a set of given criteria. p. 36

a) Critiquing one’s own ideas: Research has shown the difficulty of critiquing one's own work, for both experts and novices (Guindon, 1990). People tend to discount ideas that contradict their existing ideas (Chinn & Brewer, 2001; Kuhn, 1962; Schauble, Glaser, Duschl, Schulze, & John, 1995). For example, students as well as professional engineers often stick to their initial design strategies and resist alternative ideas (Cuthbert & Slotta, 2004). p. 36

b) Critiquing a peer’s ideas: Analyzing a peer’s work may be easier than evaluating expert generated work. Critiquing peer work can help to motivate students to improve their own work and better understand what might be refined. Comparing one’s own ideas against those of a peer, can help students to value their own ideas while developing criteria to critically review them. However, critiquing peers can be socially difficult as students tend to give overly generous or overly critical feedback (Hoadley & Kirby, 2004) [See study 2]. p. 36

c) Critiquing common alternative ideas: Providing students common alternative ideas can serve as a starting point for critique [See chapter 2: Students’ Alternative Ideas of Evolution]. Critiquing and revising concept maps with deliberate flaws are partial solutions that require a completion strategy (Chang, Chiao, Chen, & Hsiao, 2000; Sweller, Van Merrienboer, & Paas, 1998; Van Merriënboer, 1990). Giving all students the same artifact equalizes conditions for all students, compared to a peer-critique activity where each student receives very different ideas from peers. Critiquing a generic students’ work (instead of a real classmate) could reduce discrimination issues. When giving peer critique, students are often too generous or too critical due to personal bias. On the negative side, having to compare, critique, and select ideas from p. 36

three different sources (for example two collaborating group members and a given concept map) could increase cognitive load in some students. [See study 3]. p. 37

d) Comparing one’s own ideas to expert ideas could help students identify gaps in their understanding and non-normative ideas. Previous studies using expert-made concept maps often presented the maps to students as a form of summary to be studied (O'Donnell, Dansereau, & Hall, 2002). In these settings, students did not actively generate their own connections or critically evaluate presented propositions. Expert work should not be presented as absolute solutions but as one of many possible solutions. A meta-analysis (Horton et al., 1993) found that studying expert-made and student-generated concept maps seemed to have an equally positive effect on improving students' achievement. p. 37

On the other hand, Cliburn (1990) noted that teacher- generated concept maps could support integrative understanding. O’Donnell et al. (2002) found that students could recall more central ideas when they learned from expert-made knowledge maps than when they learned from texts. Students with low verbal ability or low prior knowledge often benefited the most. Chang et al. (Chang, Sung, & Chen, 2001) compared generating concept maps [See table 5 in chapter 2: Types of Concept Mapping Tasks: Task type #2] to critiquing them [task type #11] using a computer-based tool that provided students feedback by comparing student-generated maps to an expert-generated benchmark map. Generating and critiquing concept maps led to similar results, both better than a control group that did not use concept maps. However, Novak (1980b) observed that studying pre-made expert maps in genetics instruction could be confusing to some students as expert-generated concept maps could be seen by students as the one correct solution [See study 2]. p. 37

History of Mapping Ideas p. 38

Maps are simplified symbolic representations of information that highlight aspects important to the mapmaker. Harley and Woodward define maps as “graphic representations that facilitate a spatial understanding of things, concepts, conditions, processes, or events in the human world” (Harley & Woodward, 1987) (p. xvi). p. 38

Maps are always subjective representations as they only show selective information (Tversky, Franklin, Taylor, & Bryant, 1994). “To map is to construct a bounded graphic representation that corresponds to a perceived reality” (Wandersee, 1990) (p. 923). Maps are the products of compromises, omissions, and interpretations (Wilford, 1998). p. 38

The earliest maps were topological maps, showing hunting grounds, water holes, and shelter. However, as early as the 3rd century AD, people began making maps of abstract ideas (Sowa, 1992; Sowa, 2006). Maps of ideas adopted principles from topographic maps to show associated ideas, represented as words or images, in spatial arrangement. Similar to topological maps, maps of ideas are not comprehensive “windows into the mind” (Shavelson, Ruiz-Primo, & Wiley, 2005) but subjective selections of what is important to that person to include in the map. One of the oldest known maps of ideas is the “Tree of Porphyry” that is named after the 3rd century AD Greek philosopher Porphyry who visualized Aristotle’s ontology of the beings (Sowa, 1992; Sowa, 2006) [See figure 3: Tree of Porphyry]. The “Tree of Porphyry” was translated into Latin by Boethius and used in philosophical textbooks throughout the Middle Ages (Medieval Theories of Categories, 2010). p. 38

Node-Link Diagrams to Elicit Ideas p. 39

The term graphic organizer commonly describes two-dimensional visual knowledge representations, including for example node-link diagrams, timelines, and tables, that show relationships among ideas by means of spatial position, connecting lines, and intersecting figures (Alvermann, 1981; Ives & Hoy, 2003; Winn, 1991) (for a review of the range of graphic organizers specific to science education see (Hamer, Allmark, Chapman, & Jackson, 1998)). p. 39

Besides phylogenetic tree diagrams, other frequently used node-link diagrams in biology include flow charts in ecology, (for example carbon cycle or food webs) and biochemistry pathways (for example cellular metabolic pathways). Node-link diagrams are also used in other fields, for example entity-relationship diagrams (Chen, 1976) are a form of node-link diagrams used in computer science to show the flow of information in relational database systems. Social network theory uses node-link diagrams to represent the interactions and adjacency between people. [See chapter 2: Concept maps as Learning tools]. p. 39

A variety of node-link diagrams have been developed, for example mindmaps (bubble maps, spider maps) (Buzan & Buzan, 1996; Goodnough & Long, 2002), concept maps (Novak & Gowin, 1984), knowledge maps (O'Donnell et al., 2002), argument maps (van Amelsvoort et al., 2005), flow charts (Gilbreth & Gilbreth, 1921), fishbone diagrams (Ishikawa & Loftus, 1990), double-bubble diagrams (Hyerle, 2000), causal maps (Cheng, 2001), and semantic network diagrams (Chi & Koeske, 1983; Fisher, 1990). p. 39

Frequently found forms of node-link diagrams are, for example [see figure 4: Common types of node-link diagrams]: p. 39

[A] Mindmaps are arranged around a central concept. Connections represent non- specified associations. p. 39

[B] Concept maps consist of semantic propositions. The relationships between ideas are represented by labels (usually a verb) and arrows. The label describes the character of the relationship between two ideas (usually nouns). The connected propositions form a semantic p. 39

network. Different than mindmaps or flow charts, the labels in a concept map can represent different forms of relationships, for example temporal, procedural, subset, superset, or causal. The spatial arrangement of ideas in a concept map is not constrained by a central hub (mindmap) or input/output flow (flowchart) p. 40

[C] Flow charts show the intermediate steps between input and output of a system. Flow chart connections are usually ontologically of one kind and represent the quantified flow of information, energy, time, or material. p. 40

[D] Fishbone diagrams represent multiple causes that can affect a certain outcome. p. 40

[E] Double-bubble diagrams show similarities and differences between two ideas. p. 40

[D] Semantic network diagrams include multiple semantic relationships connecting to specific ideas. p. 40

Node-link diagrams can be described and distinguished by four different properties [see figure 4]: p. 41

Really useful way of analyzing graphical elements of functions of a node-link diagram. How can this relate to Suthers' work? p. 41

1) Nodes. -What does the node represent? Nodes can represent a single idea (for example in concept maps or mindmaps), a whole argument (for example in argument maps), or a process (for example in flowcharts). Nodes can have different shapes or colors to represent different types of nodes (as in flowcharts). p. 41

2) Link. -Are there constraints regarding the types of relationships the links can represent? Links can be unspecified associations (for example in mindmaps); constrained to one type of relationship (for example temporal order in flowcharts); or allow any kind of relationship (for example concept maps). -If links can represent different types of relationships, what kind of relationship can be used? Node-link diagrams relationships can include a wide variety of relationships, for example causal, flow (such as time, information, mass, energy), associations, structural (such as hierarchies), etc. p. 42

3) Topology. -How is the topology of the node-link diagram constrained? There are different levels of topology constraints, from unconstrained (free form) to constrained (given network structure). Constrained topology can include a central hub (for example mindmap), chain (for example flow chart), circle, hierarchical tree, or decentralized network (Yin, Vanides, Ruiz-Primo, Ayala, & Shavelson, 2005). Concept maps can take on any network structure. -Are the nodes arranged in groups (clusters) according to field-specific properties (for example Knowledge Integration maps)? The map can have node-clusters in theory-driven levels, for example micro/macro or genotype/phenotype. p. 42

4) Construction. -Node-link diagrams can be generated individually or collaboratively (in pairs or larger groups). -Node-link diagrams can be created by teachers, experts, or students. -Does the node-link diagram construction include a subsequent critique and revision step? -Do node-link diagram authors receive feedback? Authors might receive no feedback, peerfeedback, teacher/expert feedback, or automated feedback (based on a benchmark map). -Does the node-link diagram include deliberate errors for the purpose of a critique exercise? -How do nodes get chosen? Nodes can be given; a given list of nodes to choose from (forced choice); a given list of nodes to choose from with the option to add own nodes (semi- forced choice); or entirely free choice of nodes. p. 42

Node-link diagrams are to varying degrees constrained along one or more of these four properties. Looking at typical forms of node-link diagrams can illustrate different variations of constraints. For example, flow charts use looped chain topologies with only one form of relationship (for example flow of information, energy, or material). Knowledge maps constrain the number of link labels and are often created by teachers/experts as advanced graphical organizers (O'Donnell et al., 2002). Entity-relationship diagrams represent the flow of information in computer databases (Chen, 1976). Mindmaps use hub topologies with connections that represent unspecified associations (Buzan & Buzan, 1996). Fishbone diagrams use a chain topology with cause-effect relationships (Ishikawa & Loftus, 1990). p. 42

Based on the four properties nodes, links, topology, and construction, this dissertation research identified concept maps as one of the most versatile node-link diagrams. Concept maps are multi-purpose tools to visualize connections in complex systems in a wide variety of fields. p. 42

Concept maps can include any form of relationship (for example, temporal, procedural, functional, subset, superset, causal, etc.) and topological arrangement (for example, hierarchical, hub, decentralized network, circular, etc.) (Yin et al., 2005). p. 43

Concept maps can be defined as a form of node-link diagram for organizing and representing semantic relations among ideas. Like other node-link diagrams, concept maps consist of visuo-spatially arranged nodes and links, but additionally they also present semantic information in the link relationships. A concept map consists of nodes (ideas), directional linking lines, and linking labels that describe the relationship between nodes [See figure 5: Concept map of a concept map]. Two nodes connected with a labeled line are called a proposition (Canas, 2003). The relational cognitive model assumes semantic propositions, consisting of nouns (nodes) and verbs (linking labels), can represent ideas and relationships between ideas. These properties define concept maps as versatile graphic organizers that can represent many different forms of relationships between ideas [See figure 6: Concept map example]. Graphic organizers, such as concept maps, can change students’ understanding beyond remembering isolated ideas to constructing meaningful connections of organized knowledge (Bransford, 2000b). Mason (1992) observed that students exposed to ‘mapping’ during instruction demonstrated “insight into the interrelatedness of concepts” (p. 60), instead of seeing scientific knowledge as a collection of isolated facts. In 1972, Josef Novak and colleagues developed a concept mapping method to visually represent changes in children’s knowledge assessed through clinical interviews (Novak & Gowin, 1984; Novak & Canas, 2006). With its emphasis on actively engaging learners in eliciting and connecting existing and new ideas, concept mapping is considered to be consistent with constructivist epistemology (Edmondson, 2000). p. 43

Concept map activities can support eliciting existing ideas and connections (stored in long-term memory) through the process of visualizing them as node-link diagrams. The explicitness and compactness of concept maps can help keeping a big picture overview (Kommers & Lanzing, 1997). The ‘gestalt effect’ of concept maps allows viewing many ideas at once, increasing the probability of identifying gaps and making new connections. Generating concept maps requires learners to represent ideas in a new form which can pose desirable difficulties (Bjork & Linn, 2006; Linn, Chang, Chiu, Zhang, & McElhaney, 2010) - a condition that introduces difficulties for the learner which slow down the rate of the learning and can enhance long-term learning outcomes, retention and transfer. The process of translating ideas from texts and images to a node-link format may foster deeper reflection about ideas and their connections (Weinstein & Mayer, 1983) and prevent rote memorization (Scaife & Rogers, 1996). Throughout a curriculum, learners can add new ideas to their existing concept map. Unlike textbooks, concept maps have no fixed order and may thereby encourage knowledge integration strategies. For example, a student may decide to add the most important or most central idea first. Developing criteria to select ideas requires deeper processing than the student might normally exercise when reading text. p. 44

Clustering related ideas in spatial proximity can support learners’ reflections on shared properties of and relationships between ideas. Cross-links between related p. 44

Difference between KF and CMaps is that CMaps are a few words, everything seen at a glance - KF contains long texts, which you have to open to read. p. 45

Has there been any research looking at the difference between groups using the graphical KF interface, and the linear one? KF embodies a lot (teacher approach etc), how about isolating the effect of the graphical interface? p. 45

ideas can be seen is indication for knowledge integration across different contexts. Concept maps may support making sense of ideas by eliciting semantic relationships between ideas p. 45

To learn a subject is to have actively integrated key ideas and the relationships between them p. 45

Knowledge integration activities are designed to help learners construct more coherent understanding by developing criteria for the ideas that they encounter. Research suggests that concept mapping is especially efficient, in comparison to other interventions such as outlining or defining ideas, for the learning of relationships among ideas (Canas, 2003). p. 45

The visual format of concept maps can foster critical distinctions between alternative ideas and relationships, either individually or through collaboration in communities of learners. p. 45

Multiple representations of ideas (for example dynamic visualizations, animations, pictures, diagrams) can facilitate learning and performance supporting different accounts of scientific phenomena (Ainsworth, 2006; Pallant & Tinker, 2004), for example by complementing each other or constraining interpretations (Ainsworth, 1999). However, having learners make connections between different representations can be challenging as they are connected through multiple relationships that are often not intuitively obvious to the learner (Duncan & Reiser, 2005). p. 45

Concept maps can be used for a variety of different purposes [See figure 7] [See Canas et al. for a literature review of different usages of concept maps (Canas, 2003)]. Initially, concept maps were used by researchers to elicit core ideas and relationships from student interviews p. 46

As illustrated in figure 7, concept maps can be generated by teachers or researchers to identify core ideas and knowledge structures when designing or revising curricula (for example see (Edmondson, 1995; Martin, 1994; Starr & Krajcik, 1990). Concept maps are frequently used as assessment tools: Concept maps as pretests or formative assessment can identify students’ prior ideas which can be used to design curricula that connect to existing alternative ideas and provide feedback. Concept maps can be used as summative assessments, either alone or in comparison to pretest concept maps, to evaluate learning outcomes [See chapter 2: Concept Maps as Assessment Tools]. Concept maps can be used as advance organizers to provide an overview of core ideas prior to instruction (for example see (Mistades, 2009)). In technology- enhanced learning environments, concept maps can serve as dynamic user interfaces to navigate through activities (for example see (Puntambekar, Stylianou, & Huebscher, 2003)). p. 46

The visual knowledge representation of concept maps can support collaborative learning (Canas, Suri, Sanchez, Gallo, & Brenes, 2003; Cicagnani, 2000; Gaines & Shaw, 1995) p. 46

Several meta-analyses reviewed the effects of concept maps as learning tools. Horton et al. (1993) compared the effects of concept mapping reported in 19 classroom-implemented quantitative studies. The meta-analysis found that concept maps as learning tools produced generally medium-sized positive effects on student achievement and large positive effects on student attitudes. Improved achievement was stronger in biology classrooms than chemistry or physics classrooms. The mean effect size for studies using pre-made maps was 0.59. Concept maps generated by students in groups produced a mean effect size of 0.88. Nesbit and Adesope (2006) conducted a meta-analysis of fifty-five experimental and quasi-experimental studies in which students learned using concept maps. The study included 5,818 students ranging from fourth grade to postsecondary in fields such as science, psychology, statistics, and nursing. Across different conditions and settings, the study found that the use of concept maps was associated with increased knowledge retention, with mean effect sizes varying from small to large depending on how concept maps were used. Canas et al. (2003) found concept maps to be p. 48

effective learning tools with generally positive effects on knowledge acquisition. Concept mapping is especially good, in comparison to other interventions, for the learning of relationships among ideas. Concept maps can foster students’ active generation of relationships between ideas presented in different contexts. p. 49

Concept mapping research has mainly focused on science classrooms but has been extended to include a wide variety of disciplines and contexts, for example language, p. 49

mathematics, and history education (Kinchin & Hay, 2007). Study participants have ranged from elementary to college students and pre-service teachers, including middle school students (Coleman, 1998; Sizmur & Osborne, 1997), high school students (Stensvold & Wilson, 1990), college students (Heinze-Fry & Novak, 1990; Pearsall et al., 1997), and teacher credential students (Mason, 1992). Concept maps can represent very simple partial ideas to complex connected networks of ideas, which makes them usable with a wide range of learners. p. 50

Concept maps can also be used as metacognitive tools that support learners by eliciting existing connections and reveal missing connections between ideas, especially cross-connections p. 50

(Shavelson et al., 2005). This can help students to reflect and contrast their existing ideas with new ideas in the learning material. It can encourage students to build on their own ideas, rather than isolate new ideas from existing knowledge. p. 51

Eliciting one’s understanding can promote student self-monitoring of their learning progress and support generation of self-explanations. Self-explanations as an attempt to make sense of new ideas have been found beneficial for the integration of ideas (Chi, 2000b). Ritchard et al. (2009) found that concept maps as a metacognitive tool can support student self-reflection about their conceptions of thinking and thinking processes. p. 51

Concept maps can not only be seen as a cognitive tool that helps to elicit ideas and a meta-cognitive tool that help to support the generation of self-explanations, but also as social artifacts through which students communicate (Roth & Roychoudhury, 1993). The spatial arrangement of concept maps allows for fast information retrieval (Hook & Boerner, 2005), which can support social interaction. The high degree of explicitness makes concept maps an ideal vehicle for exchanging ideas for collaborative constructing knowledge. Several studies have reported that students who collaboratively generated concept maps achieved higher scores than those who constructed their concept maps individually (Okebukola, 1992b; Okebukola, 1989). p. 51

Inscriptions are different forms of external representations, and are central to the construction of knowledge in scientific practice (Lehrer et al., 2000; Roth & McGinn, 1998). p. 51

Having to negotiate one common expression, using argumentation, rather than each having their own "point of view". Like a Wikipedia article. p. 51

As each proposition can consist of only one link, students are required to negotiate which connection to revise or newly generate. Berland and Reiser (2009) found that trying to persuade a peer of your ideas encourages students to support their ideas with scientific evidence. p. 51

Concept maps can be used as assessment tools to elicit students’ connections between ideas (Edmondson, 2000; Ruiz-Primo, 2000; Stoddart, Abrams, Gasper, & Canaday, 2000; Mintzes et al., 2001). See Ruiz-Primo and Shavelson (1996) for a review of concept maps as assessment tools in science education. p. 52

Concept map assessments have been found to show varying correlation with conventional tests, depending on the type of conventional test, the concept map activity design, and the concept map scoring system (Stoddart et al., 2000) p. 52

More constrained forms of concept map assessment have been found to be highly correlated with multiple choice tests (Liu & Hinchey, 1993; Liu & Hinchey, 1996; Schau, Mattern, Weber, Minnick, & Witt, 1997; Rice, Ryan, & Samson, 1998). Course grades in a college biology course showed moderate correlation to concept mapping scores (Farrokh & Krause, 1996). Osmundson reported a moderate correlation between middle school essays and concept maps (Osmundson, Chung, Herl, & Klein, 1999). Since 2009, concept maps have been used in standardized large-scale assessments in the U.S. National Assessment of Educational Progress (NAEP) (Ruiz-Primo et al., 2009) to measure changes in conceptual understanding of science ideas. p. 52

A complete concept map activity consists of A) a concept map training phase, B) a concept map task, C) and a concept map analysis method. p. 54

O’Donnell et al. (2002) state that training is a key factor in making concept maps effective learning tools. An initial concept map training phase is important to familiarize learners with a) the concept mapping generation principles, and b) criteria for concept map evaluation (Novak & Gowin, 1984; Reader & Hammond, 1994; Shavelson, Lang, & Lewin, 1994). Ruiz-Primo (Ruiz-Primo, 2000) suggested that efficient concept map training activities could be designed. Concept map training activities can, for example, consist of studying a worked-out concept map, generating a small map on a familiar topic, or critiquing a flawed map. Kalyuga et al. (Kalyuga, Ayres, Chandler, & Sweller, 2003) suggest that worked-out concept maps have been found effective with inexperienced learners, while more experienced learners should learn to construct their own concept maps. p. 54

Link to Inst of Ed Singapore prof work on training students to use Knowledge Forum / football practice vs football matches p. 54

B) Concept mapping tasks are found in many different forms and provide different degrees of constraints. A concept map task consists of 1) a description of the task, 2) a concept map design, and 3) a concept map scoring system (Ruiz-Primo & Shavelson, 1996; Stoddart et al., 2000). The design of the concept mapping task is of importance as the task itself influences the outcome (Ruiz-Primo et al., 2001; Yin et al., 2005). The task and concept map constraints can range from less-constrained maps where students can choose their own ideas and labels to highly-constrained tasks where students select ideas from a given list to fill them into blanks in a provided skeletal network structure (Canas, 2006). Both extreme forms of concept mapping activities have advantages and disadvantages. p. 54

Table 5 suggests an overview of typical examples illustrating the broad diversity of different concept mapping tasks found in research and practice (also see (Anohina, Pozdnakovs, & Grundspenkis, 2007; Ruiz-Primo & Shavelson, 1996; Ruiz-Primo, Schultz, & Shavelson, 1997)). Concept mapping tasks can be distinguished by the degree of constraints, from few constraints (allowing students to create their own connections), to medium constraints (where student complete an existing map), to full constraints (where students study expert-made concept maps). The table distinguishes three different concept map task properties: Ideas (or concepts), labels, and links. p. 54

More degrees of freedom can provide more insight into students’ understanding, but make standardized comparison between students more difficult. Concept mapping tasks can start from an empty workspace, a workspace divided into horizontal hierarchy levels, or a workspace divided into specific vertical levels. In addition, a starter map can be provided to which students add additional ideas and connections. p. 55

Results indicate that periodic embedded concept mapping p. 56

can be more effective for science learning than summative concept mapping. Cheng (2001) and Kinchin (2005) noted that continuously revising embedded maps might cause students to revise propositions around the edges, but avoid revising the superstructure. p. 57

This dissertation research aims to develop and explore concept mapping forms that represent trade-offs between capturing students’ rich repertoire of ideas while allowing for efficient scoring and comparison across students. p. 57

For overviews of concept map scoring systems, see (Stoddart et al., 2000; Nicoll et al., 2001a; Vanides, Yin, Tomita, & Ruiz-Primo, 2005; Yin et al., 2005) p. 57

Literature indicates that concept map analysis is no trivial task and can use a wide variety of scoring methods. p. 57

Concept maps can be analyzed either qualitatively or quantitatively. p. 57

[See figure 8] provides an overview of different concept map analysis method. p. 57

The inclusion of concept maps as large-scale assessment tools, for example those used in the 2009 NAEP exam in science (Ruiz-Primo et al., 2009), requires economical as well as reliable and valid scoring methods. Several studies reported the validity and reliability of quantitatively evaluating concept maps as assessment tools (for example (Markham, Mintzes, & Jones, 1994; Ruiz-Primo, 2000; Ruiz-Primo & Shavelson, 1996; Ruiz-Primo et al., 2009; Ruiz- Primo et al., 1997; Ruiz-Primo et al., 2001; Stoddart et al., 2000; Yin et al., 2005)). p. 59

Concept maps contain several elements that can be quantitatively evaluated: Links, ideas (or concepts), hierarchy levels, and propositions. Links and ideas can be easily counted but their amount provides little insight into a student’s understanding. A higher number of links does not mean that the student understands the topic better as many links might be invalid or trivial (Austin & Shore, 1995a; Herl, 1999). p. 59

Novak (1984) suggested evaluating the number of hierarchy levels. The existence of hierarchies is linked to a higher level of expertise, but hierarchy levels are difficult to differentiate and many students create non-hierarchical, but still valid maps. p. 59

Propositions, the composite of two ideas, a link label, and an arrow, are the most promising elements of a concept map to be evaluated in order to learn about students’ understanding. It can be decided to evaluate all ideas equally, to weight certain propositions more than others (Rye & Rubba, 2002), or to analyze only certain core ideas (Ruiz-Primo et al., 2009). Yin (2005) showed that scoring each individual proposition on a four-point individual proposition scale, summed up to a ‘total accuracy score’, provided the best validity: 0 for scientifically wrong or irrelevant propositions, 1 for partially incorrect propositions, 2 for correct but scientifically ‘thin’ propositions, and 3 for scientifically correct and strong propositions. The ‘total accuracy score’ allows comparing the overall quality of students’ concept maps. The disadvantage of this method is its time consumption, and equal evaluation of links that show deeper understanding and trivial links. Yin compared the total accuracy score to a second concept map scoring method, the convergence score (Yin et al., 2005). Propositions of the students’ concept map are compared to an expert-generated benchmark map. The convergence score is the proportion of accurate propositions out of all possible propositions in the benchmark map. Study 1A will explore the generation of student and expert concept maps [See study 1A]. p. 59

Ruiz-Primo et al. (2009) suggest that scoring only essential links is more sensitive to measuring change because it focuses only on the key ideas of the concept map. p. 59

In addition to quantitative propositional analysis methods, the geometrical structure of concept maps can be analyzed. For example, 1) Network analysis focuses on the connectedness of selected ideas, or 2) Topological analysis describes the overall geometrical structure of the concept map. p. 59

1) Network analysis: The network analysis strategy uses the frequency of usage of selected ideas as indicators for a more integrated understanding. As students develop a more complex understanding, they might also identify certain ideas as more important and connect them more often [See study 3]. The network analysis method is based on social network analysis p. 59

(Wasserman & Faust, 1994) (Chapter 5). Network analysis method can be used to identify changes in “centrality” (outgoing connections) and “prestige” (incoming connections) of selected indicator ideas (for example “mutation” for genotype level; and “natural selection” for phenotype level). p. 60

2) Topological analysis: Kinchin (2000b; 2001) suggested a framework of four classes (simple, chain/linear, spoke/hub, net) that refer to the major structure of a concept map. This quick way to categorize concept maps can be used at the beginning of a lesson to pair students accordingly. According to Vygotsky’s ‘zone of proximal development’, it is beneficial to pair students of different ability levels (Vygotsky, 1978). Students who create a ‘network’ show a more coherent prior understanding than students who create a simple map or a chain. Yin (2005) suggested two additional classes (tree, circle) [See table 6]: p. 60

(0) Simple: Mostly isolated propositions (1) Linear/chain propositions, which are chained together; (2) Tree propositions, a linear chain that has branches attached; (3) Hub or spoke propositions, which emanate from a center idea; (4) Circular propositions, which are daisy-chained with the ends joined; (5) Network or net propositions, a complex set of interconnected propositions. p. 60

A ranking of these categories is only possible at the extreme ends, with simple and chain at one end and networks at the other. All others classes fall in between. p. 60

Limitations of Concept Maps p. 61

This dissertation research sees concepts maps not as exact representations of a person’s cognitive structure, but as a constraint and partial model thereof (Baumgartner, 2004; Trochim, 1989). p. 61

Concept maps constrain connections between two ideas to a single relationship, which require distinguishing and selecting between multiple possible relationships. p. 61

Concept maps do not directly reveal if students understand ideas themselves, but only indirect evidence through the connections they make to other ideas or by triangulation with other artifacts. p. 61

Concept maps often do not link to or contain supporting evidence. To better distinguish ideas, Tergan (2006) suggests using electronic concept mapping tools that allow the inclusion of supporting information in the map p. 61

Concept maps focus on declarative understanding, while other node-link diagram forms, for example flow charts and circle diagrams, focus more on procedural knowledge. p. 61

The design of WISE modules uses the scaffolded knowledge integration (SKI) framework (Linn et al., 2004) that translates the KI framework into four pedagogical meta- principles: Making science accessible, making thinking visible, helping students learn from others, and promote autonomy and lifelong learning (Linn & Hsi, 2000). p. 101

Formative assessment, like in NEXT-TELL p. 102

Making thinking visible serves the dual purpose of modeling scientific thinking and visualizing scientific principles through multiple representations, as well as providing insight into students’ thinking through embedded assessments. For example, students are prompted to make a prediction, make an observation in a scaffolded inquiry activity, and then revisit their ideas by generating an explanation for the observed phenomenon. p. 102

Traditionally, concept maps are generated using paper and pencil (Novak & Gowin, 1984), or moveable cards (Scheele & Groeben, 1988; Bonato, 1990). However, using computer software to create concept maps can facilitate learners in re-arranging, color-coding, adding, or deleting nodes and links with relative ease. Research suggests that concept mapping software can reduce many of the mechanical obstacles to editing complex maps (Novak, 1998; Novak, 2002; Canas, 2003). Sturm (2002) reported that learners usually prefer the higher flexibility of computer-generated concept mapping. Royer (2004) compared paper-pencil to computer-based (using “Inspiration”) [See appendix chapter 3: Table 65 for detailed comparison] concept map generation. Findings indicate that the computer-group generated significantly more complex maps than the paper-pencil-group. Computer-based concept maps can contain additional information (pictures, sounds, videos, texts, weblinks) and meta-information (source of information) (Tergan et al., 2006). Computer-generated concept maps can be stored, are easier to read than hand-written maps, and can support online collaboration. p. 102

To identify computer-based concept mapping tools suitable for this dissertation research, an evaluation survey of available tools was conducted. The survey included computer programs that were specifically designed for concept map generation. p. 102

The evaluation identified “Inspiration” as the best commercial concept mapping tool, and “Cmap” as the best freeware concept mapping tool. p. 103

Concept mapping can be a helpful metacognitive tool to visualize the interaction between new and prior ideas of the learner. The ability to construct a concept map illustrates two important properties of understanding: Representation and organization of ideas (Halford, 1993). The term representation is used differently in many contexts and therefore quite ambiguous. As a working definition for this study, a representation is a model (representing world), reflecting certain, but not all, aspects of the represented world (Palmer, 1978). A representation can be either a mental model or a materially embodied external representation (inscription) (Roth & McGinn, 1998). Internal and external representations are two indispensable parts of the representational system of any distributed cognitive task and interact with each other through cognitive processes. An inscription (e.g. diagram, text, maps, charts, tables, graphs) influences the existing mental model by providing new information that can be evaluated against the prior existing knowledge. Inscriptions are a medium that allows people to communicate. Through negotiation over an inscription, teacher and students can identify common ground and discrepancies. Inscriptions are not just peripheral aids to cognition, but provide a different form of representation (Zhang & Norman, 1994). They can provide memory aids or anchor and structure cognitive behavior. An external representation contains embedded rules that provide constraints by strictly determining what kind of information can be perceived, what processes can be activated, and what structures can be discovered from it. Zhang called this ‘representation determinism’. p. 106

A person’s internal representation affects the person’s perception of the world as well as the production, comparison, and critical evaluation of inscriptions. An inscription is perceived through our senses, but it does not interpret itself (Von Glasersfeld, 1987). Perceiving, from a constructivist viewpoint, is always an active process, rather than passive receiving. To give a perception meaning, it needs to be interpreted. Interpretations require both domain knowledge as well as knowledge about the socially constructed conventions that informed the creation of this inscription. p. 106

Novices’ insufficiency in both regards makes interpreting inscriptions very difficult for many students (Leinhardt, Zaslavsky, & Stein, 1990). Experts’ content knowledge allows them to distinguish salient surface features from structurally important features of a representation. They are (often) aware of the limitations of a representation as it shares only a limited number of features with the represented world. Experts can better decide if a certain inscription allows them to illustrate, communicate, and analyze a certain principle and create new inscriptions, if required. This ability has been named meta-representational competence (DiSessa, 2004; diSessa, Hammer, Sherin, & Kolpakowski, 1991). Experts, such as research scientists, show a high level of meta-representational competence when creating inscriptions to reflect and communicate with one another. Inscriptions are an important part of their work (Kozma, 2003). p. 106

The cognitive processes that connect the represented and representing world (directly or through intermediate steps) are complex and not yet well understood. Larking and Simon (1987) offered a computational model that focused primarily on internal representations: Diagrams are beneficial because they allow for faster indexing than text – but only to those who know the appropriate computational processes for taking advantage of them (experts). Diagrams allow to group information together, which reduces search time; automatically support a large number of perceptual inferences; use location to group information about a single element, avoiding the need to match symbolic labels (p. 98). This also applies to the concept maps used in this study [See chapter 2: Concept maps and Knowledge Integration]. Larkin and Simon later broadened their view and acknowledged the importance of external representations and the internal/external relationship. p. 107

Scaife and Rogers (1996) proposed an alternative framework which suggest three, somewhat overlapping, characteristics of this relationship: 1) “Computational offloading” refers to the extent to which an external representation reduces cognitive lead required to solve a problem. 2) “Re-representation” describes how the structural properties of an external representation can make problem solving easier or more difficult, for example familiarity with certain conventions (Zhang & Norman, 1994). 3) “Graphical constraining” refers to how graphical elements in a graphical representation constrain the kinds of inferences that can be made about the underlying represented world. As an inscription maps onto only a limited number of features in the represented world, it provides a constraint for the user. Expert and novice users have different awareness of these constraints. Constraints in inscriptions influence the reasoning about this inscription (Parnafes & diSessa, 2004). p. 107

Graphical constraining - constraints and salience? p. 107

Relating to their work, this current study will use two different modes of reasoning as the framework for analysis of the representations and creation process: ‘Constraint-based reasoning’ refers to the cognitive process of finding values for a set of variables that will satisfy a given set of constraints. When utilizing this kind of reasoning, learners focus primarily on the constraints, one at a time. They try to find a solution that satisfies all given constrains. The second mode is ‘model-based reasoning’, a holistic approach, were learners try to address all or most constraints at the same time. They create a global model of the whole scenario. p. 107

Concept maps as assessment tools are often analyzed as completed artifacts without considering the thought processes that lead to this stage. Study 1B investigates how biology experts and novices differ in their concept map construction using a talk-aloud protocol. This case study describes observations made from three biology researchers and three high school students completing a constrained concept mapping task using a given list of evolution ideas. Study 1B compares how domain experts and novices differ in their use of constraint-based and p. 107

model-based reasoning. Findings from study 1B aim to improve understanding of knowledge represented in concept maps. p. 108

Study 1A anticipates that domain experts create concept maps faster than novices because of their more integrated content knowledge; Experts’ concept maps are more hierarchical and grouped than novices’ because of their greater content knowledge; Experts’ concept maps contain more cross-links than novices’ do; Concept maps contain either a structural (e.g. organisms, cells, chromosomes) or a procedural/temporal pattern. p. 108

The training phase included a presentation of a sample concept map of familiar everyday field [See figure 11] and a step-by- step concept map protocol, based on the networking technique (McClure et al., 1999a). Using this technique, ideas (or concepts) are connected with labeled arrows to describe the relationship between them. The participants were instructed to 1) group related ideas, 2) link ideas with arrows, 3) label each link, 4) add cross-links, and 5) revise the whole map. p. 108

The think-aloud technique has been found to reveal thought processes in a variety of tasks (Ericsson & Simon, 1985), for example concept map generation (Ruiz-Primo, Shavelson, Li, & Schultz, 2001), multiple-choice test taking (Levine, 1998), performance assessment (Ayala, Yin, Shavelson, & Vanides, 2002), and problem solving (Baxter & Glaser, 1998). Ericsson suggests that verbalization is a direct encoding of heeded thoughts that reflects their structure (Ericsson & Simon, 1985). Verbalizing one’s inner dialogue does not need translation and does not cause additional processing; therefore talking aloud does not slow down task performance – as long as connections between ideas can be recalled from memory. When connections between ideas need to be newly generated, it leads to measurably slower verbalization. p. 109

Expert A began his concept map by dividing the ideas into two groups: cell division/meiosis/mitosis/clones and body cells/ sex cells/ sperm cells/ crossing over/ random fusion of gametes [See table 15]. He placed the most comprehensive idea “evolution” on top and grouped related terms, natural selection/ random segregation of chromosomes/ genetic variability, around it. In a second arrangement phase, he divided the terms into the groups meiosis and mitosis. Only after arranging all ideas, he began linking them. A said “I am thinking hierarchical, but the connectors are not going to be very hierarchical because sometimes an idea is the subject and sometimes an object”. And pointed at a horizontal chain which he just created. At the end of his systematic activity, which took only 15 minutes, he started, after prompting by the researcher, adding cross-links. This lead to the final concept map, which partially followed the ‘circle of life’-model p. 111