The BCI Development Process
While we have borrowed from the approaches used for development of the Force Concept Inventory (FCI) and the Astronomy Diagnostic Test (ADT), we have also contributed our own improvements to concept inventory development processes.

Our approach was as follows:  We collected student essays on a wide variety of content areas in biology.  Our goal was to design question that would provoke a thoughtful response of about 100 works.  Often, we were surprised and disappointed to discover that instead our questions, such as "What is diffusion and why does it occur/" produced stereotyped responses that did not address the question (read more here).  This lead us to designed questions that prodded students out of their comfort zones, often by putting them in novel or ambiguous situations.  

We collected and analyzed this data using our textual data tool developed for this project, Ed’s Tools.   Each essay was analyzed by multiple reviewers and coded using content analysis techniques (e.g., de Sola-Pool).

 

Content analysis looks at more than simple word frequency (although that can be important as well).   For our purposes, we followed the broader view of content analysis and looked for patterns of meaning.

As we collected students responses, we began to build multiple choice questions using student language and misconceptions as distracters.    These questions where then tested for validity through videotaped "talk-aloud" interviews in which students were encouraged to explain what they thought the question was asking, what they thought the choices meant, and their thinking on arriving at a final choice. 

The final results is an instrument, currently being tested for reliability, that provides information about what is in student's heads based on both "correct" and "incorrect" answers. 


bioliteracy.net © all rights reserved
last update 18-Jul-2007