Design Process

The development team grounded its design in Learner-Centered and Participatory design theories utilizing a traditional methodology of observation, literature review, content research, user testing, prototyping and expert guidance. Follow the evolution of the Watson prototype through the process chart below or view the text version.

Defining the Learning Problem
Before choosing a topic for our master's project, the design team commenced several brainstorming sessions to determine our strengths as a team, our goals for a learning design product and our combined interest to fill a need in the learning design arena. See a couple of our agendas for these meetings. (agenda1) & (agenda2) It was also important for us to build something that could serve a practical use in the classroom where the teacher was still an integral piece of the learning puzzle. Finding that many tools using technology tended to focus on math and science, we chose to build something that focused on the humanities, specifically history as it represented our content interests and strengths. We quickly reviewed learning technology texts as well as the National History Standards to gain a better understanding of how we could combine the two. This fulfilled both the need for a tool focusing on subject matter other than math or science as well as the desire to build something that could be practically used in the classroom as history is commonly taught to varying degrees throughout the K-12 experience. In between brainstorming sessions, the design team researched different concepts for teaching history, learning that "there is so little research on how people understand the past (that); there is a lack of attention to historical processes and how people learn history using raw materials of historical understanding." (Seixas, 1996) We found that there is a wealth of content available online and through archives, yet there are few mechanisms to teach students what to do with that content when it is attained. See our research summaries of our preliminary history research and investigation of the National History Standards. By the end of this first stage of our process, we defined our learning problem.

Finding a focus
Armed with a learning problem, we were able to move forward with the project proposal. The team worked over email to write the proposal, dividing up responsibilities for different sections and then using the tracking feature in MSWord to edit each other's work. This proved a general practice we followed throughout our joint writing of all pieces of the product over the design process. Seeking advice on a solid design plan, the team met with several experts in different arenas to develop the specific points of the project. With regard to content, we met with advisor Prof. Walker and were directed to more research on teaching history to help us focus what we were doing with the tool; teaching a particular theme, how to research using primary documents or how to examine content now available to the student. Additionally, we met with Prof. Barron and discussed the concept of expert/novice historians. See our notes on that meeting and our review of Sam Wineburg's work in learning history. Concurrently, there was investigation into how students learn history and how technology could support it. Speaking with Prof. Greeno, we focused on concept mapping and helping students to organize information into "meaningful clusters". (Spoehrer, 1992) We also met with Prof. Goldman for input on finding a class to work with and focusing on a grade level. The final proposal was developed with several key points: we would build a tool that helped middle schoolers gain a better understanding of history through using the expert model of a historian. We would focus on the theme of independence, multiple perspectives of history and use technology to support a community of learners that included the students themselves as well as expert historians and virtual field trips. See our initial proposal. After our design review, as may be expected from such an ambitious plan, we were forced to scale down the project to a narrower focus and consequently developed key talking points and our framework that would be Watson. Through more research we learned of other tools for history, most notably the Sourcer's Apprentice, a project that seemed quite similar to ours. Through closer review, the team determined our idea was different in that it did not support students in the process of doing research as a whole. The team finalized its plan for building a tool that scaffolded students through the process of historical research.

Identify Audience
Interested in using Participatory Design to develop our tool, the team joined forces with an eighth grade US History class at a local middle school. We met with their teacher, Mr. Carothers to get a better understanding of what they were learning with regard to content and where their needs could be met through our curriculum and online tool. We confirmed their participation in our project over the next few months and set a schedule for visiting the school for early observations of the project-based classroom. To get some perspective on teaching social studies at the middle school level, the team attended a discussion with STEP students interested in using technology to teach their subject as well as solicited expert guidance from Alan Marcus, STEP lead TA and former history and social studies teacher. From these research sessions, we gained knowledge in the complexities of teaching often controversial content and the constraints of textbook information and lack of support for accessing and teaching primary document research. This research enabled us to determine our audience and how we would build a tool that best supported their learning of history.

Establishing Curricular Goals
The heart of our project development, this stage proved the most complex for determining our final product: a curriculum and interactive tool. In a long brainstorming session, the team looked to establish what we wanted to teach, how we wanted to teach it and what was needed to help the learners gain a better understanding of history and its many facets of construction. This brainstorm led us to concurrently do research on history research processes, national history standards and curriculum design as well as visit JLS Middle School for more observations. Meeting with Prof. Barron and Prof. Walker, the team revisited the expert/novice model of a historian's research process and discussed how to show the learners that history is built from storytelling and narrative. To fit the learner's classroom where students give constructive criticism of each other's work and collaborated on projects, we focused on designing a problem-based, project-based curriculum to support the online tool, utilizing the backward design process of Wiggins & McTighe and keeping in mind education curriculum design theories such as Progressivism and Rational Humanism. Additionally, we met with experts in curriculum design, Denise Pope and Alan Marcus as well as historian Kathi Kern to establish our instructional goals. To review our final plan, we met with Stephen Carothers, JLS teacher, to gain a clearer picture of how his students do historical research for their class projects. He noted that it was important to make history meaning for the lives of the kids. Features that emerged from this stage include the History as Mystery! curriculum that promotes collaboration and investigation, the investigative journal for student reflection, the expert research process map and the overall idea of a framework and environment for student work. Our pedagogy was established.

Watson Tool Defined
Completing the History as Mystery! curriculum, enabled us to focus on the Watson tool for the remainder of the project. Our next step was to conduct focus groups with the JLS students to ask them how they do research and what they wanted in an online tool to support their work. Through this informative session we learned that students wanted to be able to access the information online without having to sift through the extensive amount of bad data and advertisements found on the Web. Additionally, we worked with students in user-testing other online history environments such as Valley of the Shadows. The team brainstormed to refocus on our tool and its features. What would it do? How would it help the learners make sense of all that information that was now available to them? How do we include the teacher in a facilitative role? Continuing our review of history research, the team determined the core of the expert historian's process: sourcing, contextualization and corroboration of evidence. See an early sketch. Focusing on the learning theory of authentic practice (Bellamy) and learner control (Lepper & Malone), the design team began its first storyboarding. See our early storyboards. Our challenges of building an online environment that supported a linear process and developing a manipulable interface for the learner emerged from these early storyboarding sessions. This sent us back to more research of technology in the humanities, cognitive processes and collaborative environments. See our research of Zurbo, the Sourcer's Apprentice, Tapped In and Questia. With paper mockups and notes in hand, the team consulted with several experts for advice on next steps. See our notes from meetings with history teacher Hunter Gelbach, historian Kathi Kern and learning designer Chris Hoadley. Features that emerged from these meetings include: the research process map made visible to the student throughout the environment, manipulable environment (hand tool, highlighting tool, pull bar), selecting a topic and building a research question and thesis statement and onscreen feedback in the form of a checklist for student self-assessment. See sketches and storyboarding led to our first prototypes. (1) (2)

Developing the Prototype
At the final stage of our design process, the team sat down to develop the prototype. Following "Quick and dirty prototyping" (Borenstein), the team used paper mockups for using testing at JLS. See our protocol for user testing. From this first session, we learned students wanted a customized and friendly interface. Features that emerged from this stage were the history helper, customizable profile and search results with regards to zip code. Following reflective design process (Schon), we reviewed some literature of kids and usability and went through some design iterations for a second set of paper mockups to take back to more user-testing. See our user-testing video clips and summaries of sessions. These sessions provided us with authentic student responses and questions for US History topics. Taking our designs to advisor Prof. Walker, we discussed motivation for students to use the tool, making the process more natural for the learner and creating a more robust community for the student researchers. Features emerging from this session are the flexible navigation, (natural process) controversy of the day (student motivation) and the community (learner support). Again following reflective design, the team reworked some of the features to create a student work environment within the website depicted on our paper prototype. This new design of a notebook metaphor with an active window for students to work and archive their research as well as visit the web to collect data was a radical switch from our earlier, more linear designs. See the prototype iteration from general web page to student online environment. Meeting with experts Stephen Carothers and Mike Smith (technology policy) gave us corroboration for the environment and policies for utilizing content online in the classroom, respectively. A final review with Prof. Schwartz helped form our assessment for a piece of the project and meeting with Shelley Goldman guided us in adding a self-assessment feature within the design just before the student works on building her outline. Final prototyping consisted of working with finalizing our demo content and primary documents, (African-Americans in the Civil War) as well as user-testing the usability and clarity of the interface. See the transformation through iterations that became our final product: the History as Mystery! curriculum and the Watson online tool for doing historical research.

See the Watson prototype.
View the protocol for the Watson demo.