profile

The Evaluation Therapy Newsletter

1-2-3-4: Counting Visitors or Making Visitors Count?


Hi Reader,

Based on conversations I've been having, it seems a lot of organizations updated strategic plans in the last few years. Now they’re trying to figure out how to track, measure, and otherwise document their progress toward those big-picture goals.

More than once, I’ve reminded folks to start with meaningful, rather than immediately panic about measurable.

This all made me think of a conference theme that has stuck in my memory for 19 years -- “Counting Visitors or Making Visitors Count.” Catchy, right? This month, we'll dig into different ways you can think about counting participation that keeps what's meaningful at the center.

Also inside:

I didn't start with a lament about the state of things because, well, I'm feeling a little tired of eulogies. But if you want a positive story of how those past NSF Education dollars led to an awesome new universally-designed product for sale by the broader public, keep reading!

(PS - It involves an eight-year-old, a library, and an abacus!)

Hang in there,

Jessica

8-9-10-11, I'm just gonna keep on counting...

In 2006, the Visitor Studies Association conference theme was “Counting Visitors or Making Visitors Count.” To this day, it's a conference title I still remember. And it's not even one I helped plan!

It's a provocative statement.

Because, of course, we all care about "making visitors count." But there are many reasons why it is also critical to count visitors (or participants or students or whoever you work with). And that is never as easy as it sounds.

This comes up when trying to operationalize a strategic plan. Or frame an annual report. Or make a case for support.

Counting visitors or making visitors count? Trick question. It has to be both.

Being able to tell the story of how many, in a way that mirrors the story of what and why, gives you a complete story of impact.

Admittedly, it's not a simple solution. The key is to start by defining what is meaningful to your strategy. That can point toward how you might measure, count, and report that detail.

Let's try a few lenses to meaningfully count people and engagement.

Lens 1: Counting Your Reach

This is the knee-jerk reaction when we hear "counting visitors." And it is usually interpreted as the broadest measurement: headcount.

Counting Attendance. Tickets sold. Registrants. Bodies in the space. Clicks. Downloads. Likes. Whatever your mode of engagement. Everything counts as 1. Add it up.

This has merits. It's one clear number. And it's typically a Big Number. For example, it's the basis for the oft-cited statistic that more people visit museums than go to pro sporting events.

But, as sometimes happens with Big Numbers, it can feel a little meaningless. Because you've lumped all the jelly beans into a giant jar, it's like, "Cool, that's a lot of jelly beans. And?"

It will also start to fall apart if someone asks about the difference between visitation and unique visitors. Which leads to another way of thinking about reach.

Counting People. Individual human beings. Some of whom are regulars. Some who benefit just once.

Your Big Number may be more compelling when you can underline that it represents distinct, individual humans. This is actual reach. (This is why web metrics offer stats on "unique visitors," rather than just clicks.)

Also, your regulars offer a different way of counting. They may only be 1 human, but think about how frequently they come. That's a whole other count. With a little data behind it, it's not an anecdotal family that comes every week -- it's a Big Number of families for whom you are a part of their routine.

This can humanize the Big Numbers, each of which tell a slightly different story of reach.

Admittedly, any effort to count individuals is hard and not always achievable. But if this story feels compelling, it can be worth exploring options.

2: Counting Breadth

Here we acknowledge that not all engagements are created equal. Sometimes you need counting that celebrates strategic variety.

In this lens, you identify qualitatively distinct categories of how you engage audiences. And that becomes the basis for counting. The trick is finding categories that tell your story, because those are the puzzle pieces to create your final picture.

Maybe its different formats of engagement -- to showcase ways that learning is activated.

Maybe its types of audiences -- to showcase broad value across communities.

Maybe its platforms or venues -- to showcase reaching people where they are.

Maybe its via subject -- to showcase the value of interdisciplinary spaces.

A system of categorizing gives you the narrative thread for your counting story. By breaking down your Big Number into meaningful (and distinct) clusters of how you achieve your strategy, counting acknowledges the parts and the whole.

3: Counting Depth

Tell me if this sounds familiar: “Neat idea. But it costs $X,000 per participant. How do we justify that?”

When a strategy requires time investment to build relationships or learning, headcount is self-defeating. It's got a factory-style assumption about learning. (It's kind of the least publishable unit of program design.)

Alongside a solid outcome evaluation plan (I had to mention that!), you can think about participation depth. Are you building deep relationships, laying critical groundwork, empowering people? I'm telling you, there is evidence in the ways that participation is happening.

When they're in this boat, we've helped clients think about a ladder, hierarchy, or pyramid of engagement. (Shape is solely dictated by which geometric metaphor feels most apt.)

This reflects that deep engagement takes time. And that time has value. For the highest rungs, you can reframe how high degree of commitment manifests. This is where specifics diverge for every project. But it gives you ways to recharacterize counting that gets at the heart of what each "high investment" participant is putting in and why that's essential to get to bigger gains.

There's no right answer! But if you've been feeling like your efforts to "count visitors" is less than fulfilling, maybe it's time to try a different lens.


Real World Example:

We've worked with the Teen Science Cafe Network for years. As a national network of 80+ organizations leading independent Teen Science Cafe, the question of counting comes up.

Alongside outcomes, we gather information about participation in different ways. And all of these lenses come into play.

Headcount: This is a great starting point for member sites. Tracking how many teens are at each event. And it is important! Because that Big Number represents all the hours teens spent engaging with a scientist.

Repeat Attendance / Unique Teens: We encourage Adult Leaders to have some system to monitor repeat teens and new teens. It gives an estimate to report unique teens reached and the benefit of talking about repeaters. Because we know that teens coming back repeatedly has value.

Teen Leaders: As a "for teens, by teens" program, Teen Leaders are a critical component nurtured by organizations. This can be a distinct category for participation, alongside the events story.

Professional Network Engagement: As the Network has rebuilt (post-COVID), we've been working on ways to track and report engagement by TSCN organizations and Adult Leaders. It requires thinking about strategy and depth. What is meaningful (and reasonable) levels of engagement for professionals? And what makes sense over the journey in this work?

Which lens seems most meaningful for your organization to think more about? Reach? Breadth? Depth? Something else? Reply and let me know!

An NSF Ripple Effect Impact Story

If you're like me, perhaps you have occasionally struggled to explain to people outside of our Informal Education Bubble what is lost with the the defunding of organizations like IMLS, NSF, NOAA, NEH, and NEA.

In addition to all of the compelling research and evaluation results (which, let's face it, put people at dinner parties to sleep), frequently these investments in education have had ripple effects that go beyond anything we could have predicted at the outset.

Take our partners in the Build-a-Better Book Internship project at Mountain Lakes Public Library. A Cliff's Notes version of goes like this:

The BBB Internship is a multi-site research project. It took the core idea that Makerspaces could be a hub of innovation to create toys, books, and games accessible to blind and low-vision kids. And, hey, doesn't that sound like a great way to build engineering design and workplace competencies that will enable a future workforce? If you turn it into an internship working for real clients it does.

Mountain Lakes Public Library was a collaborator. Their Makerspace lead, Ian Matty, and his 2023 interns facilitate a pop-up Makerspace event in their area. (This is all while working on their specific client-centered projects.)

There, they meet a young girl (Juniper) who uses an abacus to learn math alongside her sighted classmates. She and her mom (Chi-Hoon) wrote a story about it. Juniper really wanted her story to be published for others to learn too.

The BBB Internship program stepped up. They spent the next year and a half collaborating with Juniper, her mom, and a publisher (Clovernook) to turn Juniper's story into a multi-modal book package, Juniper and the Red Swoosh, that can work for any family that wants to play with math:

  • Juniper's original story and illustrations
  • Large text, Braille text, and audio narration
  • 3D-Click™ Abacus, with haptic features making it easily used without sight*

*The 3D-Click™ "bump-click" rod system that enables the tactile abacus? Developed by a BBB Intern.

And if you want some things to count, try this fun fact, courtesy of Ian Matty:

"We partnered with Clovernook Center for the Blind and Low Vision to produce the book. Fun fact - Clovernook is over 100 years old and has brailled thousands of books and magazines that exist already. Juniper and the Red Swoosh is the first original book that they have ever published! It is even more special that it happened because of an eight-year-old child who is blind."

Want to learn more about the story, authors, and technology?

(You can even pre-order a copy.)

The ground-breaking creation of this book started from the seed of the ideas, know-how, and capabilities developed through our NSF Education research project (#2049109). Ian's team took what they'd learned and just kept innovating. Which is why these investments matter. When they're right, the innovation and new resources don't stop with the grant abstract. They keep improving lives.

That said: Any opinions, findings, and conclusions or recommendations expressed in this material are mine. They do not necessarily reflect the views of the National Science Foundation.

P.S. Got a question you'd like us to answer in an upcoming newsletter? Hit reply and tell me what's on your mind!

P.P.S. Get this email from a colleague? Sign up to get your very own copy every month.

Why the "Evaluation Therapy" Newsletter?

The moniker is light-hearted. But the origin is real. I have often seen moments when evaluation causes low-key anxiety and dread, even among evaluation enthusiasts. Maybe it feels like a black-box process sent to judge your work. Maybe it’s worry that the thing to be evaluated is complicated, not going to plan, or politically fraught. Maybe pressures abound for a "significant" study. Maybe evaluation gets tossed in your "other duties as assigned" with no support. And so much more.

Evaluation can be energizing! But the reality of the process, methods, and results means it can also feel messy, risky, or overwhelming.

I've found that straightforward conversation about the realities of evaluation and practical solutions can do wonders. Let's demystify the jargon, dial down the pressure, reveal (and get past) barriers, and ultimately create a spirit of learning (not judging) through data. This newsletter is one resource for frank talk and learning together, one step at a time.

Learn more about JSC and our team of evaluators. Or connect with us on LinkedIn:

Copyright © 2025 J. Sickler Consulting, All Rights Reserved

You are receiving this email because you signed up for our newsletters somewhere along the line. Changed your mind? No hard feelings. Unsubscribe anytime.

Wanna send us some snail mail? J. Sickler Consulting, 100 S. Commons, Suite 102, Pittsburgh, PA 15212

The Evaluation Therapy Newsletter

Our monthly Evaluation Therapy Newsletter shares strategies, ideas, and lessons learned from our decades of evaluating learning in non-school spaces - museums, zoos, gardens, and after-school programs. Jessica is a learning researcher who is an educator at heart. She loves helping education teams really understand and build insights from data that they can use immediately – even those who are a bit wary of evaluation.

Share this page