Notes taken by Kalee Gregory and Bhavin Shah at a meeting with Kristina Woosley and Bill Hill at Metadesign, July 22, 1999. This meeting was called to discuss the results of the user tests and Kalee and Bhavin’s tentative ideas for the redesign of VizAbility.
The meeting began with Kalee reading the transcript of Learner #1’s interaction with the VizAbility software. Following this, some general findings from the user tests were brought up. First, none of the learners tested had spent significant time listening to the introductions for the various segments, and none of them had been given the VizAbility text to guide their interaction with the software.
B: People often mistake VizAbility for a software product alone, but it’s software and book. When I demo it to people, I’m the book.
K: Yes, the choice of not listening to the introductions is really fatal. The introductions tell you why you’re doing it.
Many of the users were disappointed by the lack of assessment or feedback. Learner #2 wanted to know the "answer" for the Imagination/Guided Fantasy/Worlds section and Learner #1 exclaimed outright at the end of his session, "Where’s my point system?" Is this something we could work on for our redesign phase of this product?
K: When you do a drawing, people don’t say, "This is right." You might say, "This drawing shows a good sense of motion; this drawing shows an incomplete understanding of lighting," etc. but not "This is right, this is wrong." Feedback in the design world is very different from traditional assessment.
B: This isn’t a game. It’s a teaching tool, which is something very different.
K: With entertainment, you judge how good [the product] is. With education, you’re judging yourself, and that’s not something a software product can do.
B: On the other hand, the whole point of this is [fostering the ability to] make your work public and dialoguing about it and seeing what the collective comes up with. We might be able to bring the web in here.
K: Yes. We have to realize that putting things on the web changes the context of the dialogue. VizAbility is about how to communicate ideas so you can critique them. Abstract or concrete ideas. One of the things we’re trying to teach is that it really doesn’t matter how smart someone is; it’s how well you communicate ideas. Maybe you could prototype a web environment that people use to talk about what they did. How do you create a trusting web environment so you could put yourself out there to learn? Who gets more points? What technique would you use for assessment? That’s going to be your challenge.
One concern that came up in the feedback we got from our class was that people weren’t sure that this is the right approach to teaching visual thinking. What is the proof that this approach is the best one to use to accomplish your goal?
K: How do we have proof that this textbook teaches economics? How do we have proof that Decker’s class will teach you what he intends it to teach you?
B: I come to this from a design perspective. In my design classes, the structure was that you went to studios and you get feedback. As far as proof goes . . . a measure of success . . . I don’t know . . . you’re judged by your work, what you end up doing. It reminds me of the Apple ads: ‘Think different.’ Individually, it’s really hard to quantify some of this stuff. . . . This is about innovation. There isn’t a step-by-step process to become an innovator. The learning is all in the in-betweens.
One of our classmates wondered if it would be better to teach this using actual three-dimensional objects.
K: We’re trying to teach abstract thinking. On the Block-Builder, you can’t make those transformations in 3-space. We purposely omitted the steps you would take to get there: we only showed the first and the last configuration.
The user who wanted an "answer" to the Imagination/Guided Fantasy/Worlds exercise realized at the end of her session that that wasn’t the point. Not realizing what the exercises were about was common in our tests.
K: What might be really fun with your 3 users is to talk to them in a few weeks and see if there was anything they thought about in retrospect. Do any connections happen later? [This ambiguity] is the strength and weakness of the product.
B: Some of the responses that come back from the lectures I give are: "This would be really useful for pre-architecture students." I always saw this more as a fundamental skill that they would use all the time. They would be visual thinkers inside of whatever domain they’re in. Also, if you look out there, this world’s changing so fast that there are no answers. This product is teaching people about thinking, not about solving a particular problem.
K: Lead scientists have been shown to be more interested in post-docs who don’t know all the answers than know-it-alls. Though in an academic environment ‘know-it-all-ness’ is associated with prestige, the solutions that know-it-alls come up with are often too shallow to really get anywhere. We’re trying to teach people to think within a broader, more ambiguous paradigm, which I’m realizing we never made completely explicit.
I’m hearing that for our first redesign, some promising directions in which to go are 1. design of a web forum for discussion about visual thinking and 2. making the overall goals of the product and the thesis that it applies to everyone more explicit.
K: [In the class at Stanford] everyone walks in and thinks it’s a stupid thing to be doing at first. Engineers say, "this isn’t engineering!" But doing to projects in the Stanford class, be they as simple as analyzing a doorknob, builds your confidence and your faith that this is an important thing to be doing. . . . Your user tests seem to show a very distinct gender difference as far as attitudes toward the product go. One thing to keep in mind is that gender differences often show up because the tester is watching. That’s what some of the trick will be in designing your web forum. You need the right balance of anonymity.
B(musing about the webification ideas): Maybe we could co-print it with Hallmark or Crayola--associate it with some other product that sells sketchbooks or something.
Something else that came out of our user tests was that the concepts taught were important and the methodology seemed on target but the content wasn’t relevant.
K: Maybe one of the best things to do with VizAbility might be to have it tell you what to look at based on your background. It could show one exercise and then say to the business person, "this skill allows you to do ____," to the chemist, "this skill allows you to do ____," to the film major, "this skill allows you to do ____."
B: In the web forum, we could have people share how they use the skills in their real lives. The point is to do. None of your users actually did the story-making and few did the actual drawing. We need to make them draw it. You can’t learn to play the guitar by reading a book.
K: We’re also not trying to make people perfect at using the computer. We want people to draw. And we realize that it takes time to build the community, the trust, the culture of visualization. In the electronic world, you can’t tap into the relationships that are created.
B: Before we webify, you have to answer the question "What has the web given us that wasn’t available when we first did this four years ago? What is happening in education that needs the web? What would you be looking for as educators that could be web-specific?
K: Look at the book a little bit as the stand-in teacher. That book is the teacher component. Would that teacher component be better in the electronic world?