The Evaluation Therapy Newsletter
Our monthly Evaluation Therapy Newsletter shares strategies, ideas, and lessons learned from our decades of evaluating learning in non-school spaces - museums, zoos, gardens, and after-school programs. Jessica is a learning researcher who is an educator at heart. She loves helping education teams really understand and build insights from data that they can use immediately – even those who are a bit wary of evaluation.
From Anecdote to Systematic: Ease on Down the Road
Hi Reader, I wrote the most popular LinkedIn post of my career a couple of weeks ago. It was a graph showing how many trick-or-treaters we get each year. (As a subscriber, you get first dibs on the 2025 tally: 640 kids in 2 hours.) Our neighbor first prepared us for Halloween on this block in anecdotal terms: “You need a lot of candy.” I discovered that anecdote was insufficient in the super store candy aisle. In 2016, we finally got systematic about our tracking, which made all future...
Formal & Informal Learning: Opposites Attract
Hi Reader, Back in June, I got to attend NASEM's Convocation on Informal Science Education. (Video of the whole event is at that link. If you have a lot of hours to burn, it was great.) Over two days, we dug into the state of informal science learning. Some looking back. Some tough-talk about the present. And some envisioning what we want to see in the next 15 years. One panelist said something that really stuck with me about the relationship between formal education (K-16 schooling) and...
Plan Slow, Implement Fast
Hi Reader, I'm sending this while still feeling a bit bummed that I missed the ASTC conference this year. My flight was booked, and then life stuff happened. (Thanks for continuing to be awesome, 2025!) It also got me thinking about a talk I gave at ASTC 2024. Karen Hammerness (of AMNH) and I gave a lightning talk unpacking how JSC, as external evaluators, collaborated with AMNH's internal research team on a Wave 1 evaluation of the (then) newly opened Gilder Center. There were scores of...
How do you make measurement meaningful? It's all about grain size.
Hi Reader, A few weeks ago, Rob and I were meeting with a new client. We were introduced around the office as “the new evaluators.” One young woman was casually sitting on a low filing cabinet, chatting with colleagues. At this introduction, she leapt to her feet, shook my hand, and said, “Whoops!” I was like, “Yeah, no, we’re not that kind of evaluator.” It was a stark reminder of the baggage this work can carry. People assume they're about to be judged against some impossible benchmark....
Maybe you DON'T need a logic model.
Hi Reader, I often wonder if every evaluator is, like me, a card-carrying fanatic of The Container Store. (Was I invited to the VIP opening of Pittsburgh’s very first Container Store? Yes. Was that because they knew I'd been driving to Ohio to get my fix for years? Also yes.) The reason I wonder that is because evaluators love organizing ideas. Putting things in boxes. Connecting those boxes with lines to show order within a system. To give it logic. For us, this process is second nature....
Observing Humans is Hard
Hi Reader, Back in my teacher training days at Bank Street, "Observation & Recording of Children" was a foundational course. On the first day, the instructor showed us a 2-minute video of a kid playing. She asked us to observe and take notes of the child's behavior. It was a humbling exercise. My class learned really fast how complicated it is to observe human behavior. You miss details. You infer intent or emotion (rather than describe behavior). You get distracted. Your hand cramps and...
1-2-3-4: Counting Visitors or Making Visitors Count?
Hi Reader, Based on conversations I've been having, it seems a lot of organizations updated strategic plans in the last few years. Now they’re trying to figure out how to track, measure, and otherwise document their progress toward those big-picture goals. More than once, I’ve reminded folks to start with meaningful, rather than immediately panic about measurable. This all made me think of a conference theme that has stuck in my memory for 19 years -- “Counting Visitors or Making Visitors...
Measuring Ch-ch-changes: When Pre/Post is not the answer
Hi Reader, You know that old saying about the weather in New England? Or Michigan? Or Pittsburgh? Or, well, every Northeast/Midwest city or region has a version of it. Don't like the weather? Wait 15 minutes. It'll change. It's been nothing but ch-ch-changes in the weather around here. 75 yesterday. 50 today. Maybe Mother Nature is on the collective Emotional Roller Coaster we are all riding these days. As an evaluator, the word change makes my ears perk up. Measuring change creates presents...
Teacher PD: The Art of Offering Everything & Nothing
Hi Reader, The hits just keep on coming, don't they? While we watch officials give a big middle finger to the Department of Education and IMLS,* it makes me think about how our orgs can step up to the plate for our K-16 learners. There are lots of ways you do this outside of schools. From field trips to afterschool and summer programs. Science cafes to mentoring and internships. But our out-of-school experiences rarely achieve the same intensity and consistency as classroom teachers. We're...
Meaningful evaluation with tiny humans
Hi Reader, If you're at all like me, you've been waking up each morning wondering, "What fresh hell will greet me in today's news?" I've been tracking updates from the National Council of Nonprofits and the policy statements from large universities. It's good to learn how the folks with lots of legal pros on staff are thinking. At this moment, it seems to be, "Keep calm and (cautiously) carry on." In these difficult days, I need things that bring me joy. And routinely, I can count on kids to...
Our monthly Evaluation Therapy Newsletter shares strategies, ideas, and lessons learned from our decades of evaluating learning in non-school spaces - museums, zoos, gardens, and after-school programs. Jessica is a learning researcher who is an educator at heart. She loves helping education teams really understand and build insights from data that they can use immediately – even those who are a bit wary of evaluation.