Our monthly Evaluation Therapy Newsletter shares strategies, ideas, and lessons learned from our decades of evaluating learning in non-school spaces - museums, zoos, gardens, and after-school programs. Jessica is a learning researcher who is an educator at heart. She loves helping education teams really understand and build insights from data that they can use immediately – even those who are a bit wary of evaluation.
When interest-driven doesn't mean interest-driven
Published about 7 hours ago • 8 min read
Hi Reader,
As we come out of a month of non-stop political analysis, a colleague asked me last week if I’d been drinking from the news firehose like she had. I had to confess I hadn’t. I've found I just can't handle the extreme highs and lows anymore, so I turned the volume way down.
That got me thinking about the weird ecosystem of news consumption. My colleague chose one approach, I chose another. And how a lot of what any of us hears is invisibly shaped by media tech that can now follow our individual interests: what we’ve clicked on, who we follow, what we pay attention to.
It doesn’t take much for exposure to ideas to become an echo chamber. Sometimes that's comforting. And sometimes it can be limiting.
This month’s issue looks at the same dynamic for learning: Why it isn't always empowering to make individual interest is the driver of experience.
And why I strongly believe that educators are the anti-algorithm.
Cheers,
Jessica
Keep feeling fascination.
One of my first writing assignments as a Museum Education student at Bank Street College (RIP, OG Museum Ed program) asked us to recall a time in our own schooling that reflected the constructivist principles of John Dewey.
Many of us (self included) focused on students forming the purpose of learning, interpreting that as times when we set the direction. When what to study was our decision. When we followed our interests.
My example was about a 4th grade project where I did a deep dive on endangered animals. Because, after a few family trips to the National Zoo, I had become fascinated. And if anyone gave me an opportunity, that was all I wanted to talk, read, or write about. It was one of those "islands of expertise" that kids develop.
But I still remember the course correction I got from our Bank Street instructors. By focusing only on learning driven by student interest, we'd taken the student's role to an extreme. If all of my schooling had followed that model, I'd have left 4th grade knowing everything about pandas, and nothing about anything else. And that wasn't what Dewey meant at all.
It's a lesson that rings in my ear whenever I hear programs talk about being interest-driven.
Why interest-driven doesn’t mean “interest-driven”
One of those moments was at a conference of afterschool and museum educators this year. Everyone's North Star was for their programs to be "interest-driven.” And as I listened, I heard that phrase being applied very literally – like my own story. The approach boiled down to, "We let the kids decide what, how, and why.”
And while I see its merits, I couldn't help but think about all the unspoken limitations.
The first is that following interests can lead to a narrowing of worldviews, experiences, and thinking. Those islands of expertise can be a double-edged sword – for any learner.
The second is that the simplicity of this approach neglects the fundamental truth that learners don't know as much about the world as you do.
Educators are not algorithms.
A recent article in the Journal of Science Communication looked at “incidental exposure” to science content. It concluded that incidental exposure on social media happens more for those who are already interested in science. Not unlike political news, existing interest-based gaps are reinforced through the science people encounter.
I was 0% surprised when I read that.
Social media algorithms are literally built to chase pre-existing interests. An algorithm sees you express an interest, and then magnifies it back at you. Click on one cat video, and you see more and more. If cat-video people also get sucked in by groundhog videos, maybe they entice you over there.
But no matter what, it's always spiraling to send you down a rabbit hole (or groundhog hole) of your interests. Because it doesn’t want you to learn. It only wants your time.
Why do I bring this up?
Because educators are the anti-algorithm. You are not here to run out the clock. You have actual aims. You want to introduce learners to new things. You want people to have aha moments. To test their limits. To broaden their interests. To build connections between disparate ideas. To question their assumptions.
So don't blindly chase interests. While incoming interests matter, they may not be the primary route to achieve your learning goals.
Why learners might not want “interest-driven”
Another fundamental conundrum with interest-driven is that interest is limited to the universe of what someone already knows. If existing interest is the starting point, you are stuck in that universe. The wide-open question, “What do you want to learn about?” may sound like freedom. But in reality, it can be a narrow monoculture of ideas.
My guess is that your aims are a lot broader than that. And, honestly, your audiences don't want to be trapped in the same old, same old either!
On Jonathan Alger’s Making the Museum podcast, I talked about why chasing interests is a pretty useless approach to the early stages of exhibition development. Because if you rely on visitors telling you what topics they want to see, we get stuck in the spiral of what they already know and expect.
Visitors back me up on this. When we’ve asked them to rate qualities of museum interactives, what things get top ratings?
Showing me something I wouldn't see otherwise.
In other words:
“Show me something cool that you know and I don’t!”
The real trick of our work is delivering on that promise. Figuring out how to take what you think is cool and spark interest and relevance for others.
Strategically Activating Interest
Don’t get me wrong; I’m not advocating for some kind of educational forced march. But education is also not a game of Follow-the-Leader, where the learner is in the front.
There's a balance you strike where interest is an asset you can leverage in different ways, places, and times. For productive learning, you are strategically using their interests within your framework as an educator or designer.
Here are a few ways interest can support your aims without driving them:
Interest as Hook; Skills by Design:For example, activities and programs that involve a table of familiar building toys – but carefully structure the interaction. Rather than interest-driven play, an educator-designed structure can get interest, creativity, and purpose working together to support learning goals.
Where'd I get that example? LEGO's Build the Change activity kit – and museums that have made it their own.
Defined Topic; Interest-Driven Investigation:This is where you've identified the content zone you want to explore, which may be totally unfamiliar. Once you're in that world, learners get agency to identify questions and where to go further. Maybe they explore an unfamiliar object to get into a new content area. That leads to questions about it. And collectively identifying where to dig deeper.
Introduce Options; Interest Sets Direction:Another variation is to seed people with options, ideas, and stories – topics they perhaps never knew existed – offering it as almost a menu of options. And then let them decide what interests them to pursue. For the educator, it not only honors their interests, but tells you a lot about how they are making connections about ideas based on what they choose.
Also (because I can't leave without talking evaluation): Options + Interest is a great front-end evaluation strategy! When you offer people a set of possible stories, narratives, or ideas you could explore, finding out what they'd choose and why is incredibly revealing. Often the answer isn't entirely in what interested them, but in why it interests them. That helps you think differently about interests in the future.
Real World Example:
This issue comes up a lot within the Teen Science Café Network. With programs built to be “for teens, by teens,” letting teens' interests lead seems intuitive. Right?
In reality, if the whole point of a Teen Science Café is to broaden teens' awareness of the possibilities and pathways in the giant world of STEM professions, relying on teens to get the ideas flowing is a roadblock.
It relies on them to know what’s out there. To know who is out there. In cutting-edge STEM fields. How, exactly, are they supposed to do that?
Each Adult Leader has lots of creative ways to get past this roadblock. But one shared tool is a Café Event Database, which crowd-sourced a menu of topics for teens nationwide.
After each event, a TSC member adds the title, topic, speaker, and STEM disciplines of their event. And all TSCN members have access to this database, which is a simple, searchable inventory of 538 (and growing) TSC speaker topics that have been hosted across the country.
Even though we created this database for participation counts (think of the funders!), its idea-generating power made it valuable to educators and teen leaders.
It has become a trove of ideas to be shared across the network. Any member (or their teens) can pull up the list and search or filter for inspiration. You can select the “computer science” tag and get all sorts of Café Event topics and speakers that relate to computer science.
Screenshot from the database. Full access is a benefit for TSCN members.
It’s meant to spark teens’ interests by seeing what other sites have covered. Not to replicate an idea, but to riff, remix, and take it in a direction relevant to their community.
Teens' interests are still in the lead, but they don't have to re-invent the wheel of interesting STEM topics every single year.
How much does audience or learner interest drive your approach to your work? Hit reply and share your strategies to balance interest and other priorities.
Newsletter Mix Tape
Rewinding to related past issues that you might have missed.
The Play vs. Content Tug of War: How educators can think about structuring and evaluating play without abandoning their content soul.
Let’s Get Curious About Curiosity: If you’ve ever wanted to measure curiosity in learners but felt stuck about how one can authentically examine such an abstract thing, this is for you.
Getting Beyond the "Glitter and Glue": There’s a Value Gap between how educators understand their value versus the role they feel they are asked to play. How do we get leaders to see beyond the “glitter and glue”?
P.S. Know someone who needs to hear this? That's why email comes with a Forward button!
P.P.S. Get this from a colleague? Sign up to get your monthly email "therapy." (No insurance required.)
Why the "Evaluation Therapy" Newsletter?
The moniker is light-hearted. But the origin is real. I have often seen moments when evaluation causes low-key anxiety and dread, even among evaluation enthusiasts. Maybe it feels like a black-box process sent to judge your work. Maybe it’s worry that the thing to be evaluated is complicated, not going to plan, or politically fraught. Maybe pressures abound for a "significant" study. Maybe evaluation gets tossed in your "other duties as assigned" with no support. And so much more.
Evaluation can be energizing! But the reality of the process, methods, and results means it can also feel messy, risky, or overwhelming.
I've found that straightforward conversation about the realities of evaluation and practical solutions can do wonders. Let's demystify the jargon, dial down the pressure, reveal (and get past) barriers, and ultimately create a spirit of learning (not judging) through data.
This newsletter is one resource for frank talk and learning together, one step at a time.
You are receiving this email because you signed up for our newsletters somewhere along the line. Changed your mind? No hard feelings. Unsubscribe anytime.
Wanna send us some snail mail? J. Sickler Consulting, 100 S. Commons, Suite 102, Pittsburgh, PA 15212
The Evaluation Therapy Newsletter
J. Sickler Consulting
Our monthly Evaluation Therapy Newsletter shares strategies, ideas, and lessons learned from our decades of evaluating learning in non-school spaces - museums, zoos, gardens, and after-school programs. Jessica is a learning researcher who is an educator at heart. She loves helping education teams really understand and build insights from data that they can use immediately – even those who are a bit wary of evaluation.