Brain Explorers distills lessons from evaluating its neuroscience outreach programs: a research digest

Some of my favorite times in graduate school were sitting at an activities table for a neuroscience outreach event. I loved showing kids how to make neurons out of beads and seeing them light up when I showed them a real human brain. If someone asked me how it went, I would have answered with how a kid looked at a brain with awe as I explained the different lobes, or the way some lingered at the table to ask me questions. That would tell me that they enjoyed it, but what about the overall impact? How do I know they learned something? Did it deepen their appreciation of science in the long term? Few programs have the answer to these questions, mainly due to the difficulty of evaluating an outreach program. Brain Explorers, a neuroscience outreach program at Western Michigan University Homer Stryker M.D. School of Medicine, sought to answer these questions not just for their own program but as a way for other outreach programs to also answer these questions. 

In this research digest, I examine a 2024 paper titled “Evaluation of content knowledge and instructor impacts in a middle school outreach program: lessons from Brain Explorers” authored by Peter J. Vollbrecht, Carolyn E. A. Cooper, Joseph A. Magoline, Thomas M. Chan and Kirsten A. Porter-Stransky. The paper was published in Frontiers in Education. 

About Brain Explorers

Brain Explorers is a neuroscience outreach program based at Western Michigan University Homer Stryker M.D. School of Medicine geared toward middle school students in a small rural town. Medical students (1^st and 2^ndyear) design and execute lesson plans for classroom sizes of approximately 18 middle school students, 4 or 5 times a year. 

The medical students use active learning so the middle schoolers can connect ideas about how the nervous system works. However, in this program, the middle schoolers are not the only ones learning something new. The medical students are also learning how to design, execute, and evaluate their lesson plans and explain complex neuroscience concepts so that a middle schooler can understand them.

Dr. Vollbrecht and the team wanted to evaluate their own program to see if it was meeting one of their goals to increase understanding of neuroscience. 

Mission Objective 1: Understand the motivations and experiences of the medical students

This objective was to understand why medical students joined the program and how the program impacted them. Medical students were enrolled in the Active Citizenship course. The medical students would observe a Brain Explorers session, design their own curriculum, and then do a mock run in front of the faculty and peers. A part of the research was to host focus group sessions to understand their motivations and experiences. The researchers would then do a thematic analysis of the focus group sessions. 

Mission Objective 2: Assess the students’ understanding of the material

Brain Explorers would visit science classes 4 or 5 times a year. Over the course of two years, they issued the following types of assessments:

• Immediate post-event assessment: Review multiple-choice quiz via an online game called Kahoot!
• Immediate pre- and post-event assessment: A paper quiz before the lesson and after the event, consisting of short-answer questions and multiple-choice. 
• Spaced pre- and post-event assessment: Online test with short answer questions. Space pre is taken 1-2 days before the lesson. The Post is taken 1-2 weeks after the lesson and includes a question that would be covered during the lesson and one that was not. 

The team would then grade and analyze students’ scores.

Mission Outcomes

For the teachers, the medical student understood that the skills they learned from the course go beyond the classroom. As doctors, they need to communicate complex medical diagnoses to patients, and these communication skills take practice. Many wanted to give back because they remembered their experiences in science class. After being in the program, they felt more confident in communicating science concepts. 

For the middle schoolers, across all assessments, students retained information from the neuroscience lessons. For the first assessment, the students performed better than chance on the multiple-choice test (Figure 1). For the immediate pre- and post-event assessment, the students had higher test scores after the lesson compared to before the lesson. Notably, they remembered the lessons weeks later. The researchers showed that students retained lessons shorter after and long after across different testing formats.

Lessons Learned

Through all three methods of assessment, the researchers found that Brain Explorers helped middle schoolers understand the material. However, they emphasize that while they focus on knowledge gains, it is not the only important goal of an outreach program.  Another important point is that evaluation methods need to be matched to the format of the engagement. While structured quizzes may work well in classroom settings like Brain Explorers, they may be less effective for open-house-style informal outreach events. They concluded that, with thoughtful consideration, we need to continue to find ways to evaluate outreach goals and progress.

Reflections

One aspect of the paper that stood out is how the medical students prepared for Brain Explorers. The medical students put a lot of time and effort into perfecting their lesson plans, which showcase the importance of taking the time to learn and practice the necessary science communication skills. However, this would possibly be too time-intensive for other outreach programs. Dr. Vollbrecht and colleagues addressed this issue by incorporating it into the medical school curriculum, offering an example of how science communication training can be integrated within existing curricula rather than treated as a separate component —an approach that may make such training more sustainable for institutions.

I realize that the aim of the paper was on knowledge retention. However, it raises a broader question about whether similar qualitative analysis could be applied to the middle schoolers. Just as the medical students were becoming more confident in science communication, I wondered how the lessons affected the middle schoolers’ confidence, interest, or attitudes toward science. In addition, the program had multiple visits per year, raising questions about how this type of multi-visit engagement is positioned within the broader landscape of science engagement efforts, particularly regarding how outcomes are measured across programs with varying levels of engagement.  

Follow up with the first author

In responding to my questions, Dr. Vollbrecht, one of the founders of Brain Explorers and the paper’s first author, shared some insights about the program’s community impact, explaining that it serves as a bridge between the community and the medical school. “Brain Explorers has allowed our institution to highlight what we do for a lot of students and parents that might not otherwise pay a lot of attention to the medical school,” he told Civic Science Observer.  He added that the team has been able to “build connections with teachers in the community and better understand the educational needs of students in the area.” 

In responding to my question about evaluating middle school participants using qualitative approaches as they did for the medical school students, he noted that his team has actually done both qualitative and quantitative methods in evaluating the middle school student experience. He emphasized that their “primary goal is to improve attitudes towards science among the general population” and that they “hope to do this by engaging students early and continuing that engagement.” 

For Dr. Vollbrecht, “making science less ‘impressive’ and more accessible hopefully means that more students will see science, including medicine, as an accessible career path, not just one for ‘really smart people’.”

Looking ahead, Dr. Vollbrecht points to the program’s expansion beyond neuroscience, reflected in its new name, Brain Explorers & Beyond, which brings in additional areas of anatomy and physiology. He noted, “While neuroscience is my passion, it isn’t every student’s passion, so this is an exciting opportunity to get more participants excited about the content we present.” 

Julianna Goenaga

I am a neuroscientist with a bachelor’s degree in Biology and Society from Arizona State University and a Ph.D. in Neuroscience from the University of Minnesota. I am currently a postdoctoral researcher studying treatments for traumatic brain injury. I am also passionate about science communication and am a former writer for SciComm Bites.