Building Games for Improved Standardized Testing

My underlying desire is to improve the game experience for all players and to use these games to improve education (I'm designing educational games right now). I particularly like the multiplayer component of MMOs in the context of education, because so many of the things people want to learn to do involve interacting with other people and because anything we do has more meaning in a social context. So, I've been thinking about MMOs for education for awhile (particularly to teach languages). One property of using any kind of game for education is that assessment is built in -- the game knows if you're succeeding or not. However, I did not consider using these games to improve the sometimes-bane of every student's academic existence: standardized tests. These tests are, after all, basically the assessment without the learning.

Where is all of this coming from? My girlfriend is preparing to take the LSAT on Saturday, and has been studying for months. It's bringing back my own memories with the SAT and GRE. It has always struck me the degree to which a standardized test I consider arbitrary has influenced admissions committees' predictions of my future success or aptitude. I knew before college, before the SAT and GRE, that I wanted to design games "when I grew up." Now that I've been designing games for several years, and am much more aware of what the process involves, I find that my success and aptitude has very little to do with those standardized tests. Even my success in Computer Science, which was technically my major, had very little to do with these tests. A much better test would have been for me to actually design games (or, for computer science, write a simple program in simple a language with an unknown API/"toolbox").

I was reading some of Howard Gardner's work on multiple intelligences, and was struck by the following paragraph:

A shift away from standardized short answer "proxy" instruments to real-life demonstrations or virtual simulations. During a certain historical period, it may have been necessary to assess individuals by administering items that are themselves of little interest (e.g., repeating numbers backwards) but that are thought to correlate with skills or habits of importance. Nowadays, however, given the advent of computers and virtual technologies, it is possible to look directly at individuals' performances-to see how they can argue, debate, look at data, critique experiments, execute works of art, and so on. As much as possible, we should train students directly in these valued activities and we should assess how they carry out valued performances under realistic conditions. The need for ersatz instruments, whose relation to real world performance is often tenuous at best, should wane. (8)

For the most part, standardized testing has integrated computer technology by digitizing the same old tests. The big change in the case of the GRE is that the test is now adaptive on the computer -- it gives you harder questions worth more points if you correctly answer previous questions. However, now that we can construct virtual environments that simulate many of the tasks students are hoping to do someday, we can measure their aptitude at those tasks directly, instead of asking abstract, multiple-choice questions and then guessing at a presumably correlated aptitude.

It's not immediately obvious how a simulation would be constructed to test every field, but certain fields seem well suited: medicine, engineering, and aviation, for example. Even some softer skills like communication, leadership, resourcefulness, and creativity can be effectively simulated and scored in multiplayer virtual environments. Take communication, leadership, and ability to follow instructions -- all crucial factors in successfully organizing any team activity in MMOs. Admissions committees currently look to extracurricular activities, work experience, and personal statements for traces of these qualities in applicants, as they should. However, a test activity completed in a collaborative virtual environment could give even more insight and eliminate some of the guesswork. Players would need to repeat the test enough times to sufficiently reduce the variability introduced by the particular teammates or opponents the player had, so the test should probably be short.

The way the educational game space is proceeding right now, these virtual training/testing grounds will probably be applied first to teaching, perhaps in the context of a course, and only later to testing.

Another issue, of course, is that games are great learning environments for some students because some students enjoy games. But, the same has always been true of standardized tests -- they have always favored some students over others.

Gardner, H. (1998). A multiplicity of intelligences. Scientific American, 9, 19-23.