B: No. We had lots of people use it informally to test the interface. People didn’t have trouble with the instructions and pretty much everyone thought it was really neat. The problem was that people didn’t come away with a clear answer to the question "What did I just learn?"

K: In industry, user studies are very expensive. There was really no compelling reason to run those studies—based on people’s reactions and the input of experts we can be pretty sure that our educational objectives are being met, and that’s what you do formal user studies for. (Problems with the interface are determined by more informal feedback.) We may want to put this in a school for a year and see how it does and use that situation to do a more formal study.

B: We had that contention early on.

K: Assessment is tricky. Do you do a "gaming" sort of assessment, in which you put down how many right/wrong answers there are, and compute levels of mastery? We error correct in some places on VizAbility, but our point of view is that this is going to be in so many different environments that we didn’t want assessment. A professor in a college might want to use VizAbility to get students excited about a design project, or an art teacher might want to help students develop their artistic sides . . . maybe a list of possible ways a teacher could use the product to assess students would help.

B: Yes, you might want to look at the methods of assessment used in the Mechanical Engineering course at Stanford and see how useful they were and whether you think there’s any way they should be incorporated into the product.

K: We did. We decided not to allow that as we wanted students to focus on the process, not product. The activity is not so much about the stories you create, but the process of imagination that went into making them. The Magic Theater is designed to encourage your confidence with your imagination. We wanted to capture the fluidity of the experience of imagining.

K: Notions of constructivism are interwoven into these things. We do a lot with hands-on active learning. We were trying to teach a way of learning, and in so doing found that we incorporated many of the current cognitive models without calling them by their academic names.

B: Our primary teacher was experience. Scott and Gail had taught this. We all ended up arguing a lot about how the product should behave. The fights ended up producing a product that solved a lot of problems.

B: From a mechanical standpoint, it would be very easy. Our hesitation is that the producers are worried about how we would charge for that. Perhaps it would be possible to use the web for other things, like a teacher’s guide or creation of a web community of teachers/learners.

K: In the web environment you can form communities. Another possibility is to do something as for other products on the MetaDesign site—let people download a small portion of it for free and then give them the option to buy it on the web.

B: There was so much that went into this. I’ve never shown this to someone without eliciting a deep passion. The thing I would do differently is that I would not trust the publisher to handle the marketing. There’s nothing fundamental that we missed; if the need for visual thinking were recognized and the product given the proper press it should sell much better.

K: We over-estimated people’s ability to understand the concept. If I were to change something I would make it smaller—make that a first step toward a design curriculum. We spent a lot of time putting it all together into a coherent form, but I wonder if that was a mistake. We should have just given people pieces and the tools to put them together themselves without trying to capture every feature in there. We took on an awful lot of "brand-news" at once.

B: Ask us for the specific pieces you want to work on and we’ll see if we can dig it up. We’ll have to figure out about the copyrights so there are no guarantees, but if you have a specific request that is more likely to be do-able.

B: In thinking about the future of VizAbility, the first question to answer is "How can we get it out there more and then use that to fund any changes we would do?" Somebody has to fund it or we have to figure out a way to self-fund.

K: I guess in thinking about the future, we should look at general issues of how to redesign an interactive product. Is the topic correct? Is the interface working?

B: We need a fly-on-the-wall executive summary. If there were a few key things that VizAbility would show you how to do, people would have a better understanding of what it is.

K: It’s difficult to distinguish between people not looking at it and people not being interested in it.

B: When I demo it to 5^th graders, the students often understand it before their teacher. This is a real problem for us. The teachers are scared of it . . . they feel stupid because their students get it first.

K: These are the tools you need to enter the digital age.