Hi Reader,

If you're at all like me, you've been waking up each morning wondering, "What fresh hell will greet me in today's news?"

I've been tracking updates from the National Council of Nonprofits and the policy statements from large universities. It's good to learn how the folks with lots of legal pros on staff are thinking.

At this moment, it seems to be, "Keep calm and (cautiously) carry on."

In these difficult days, I need things that bring me joy. And routinely, I can count on kids to bring some joy. So, this month we're going to answer a frequent question I've heard -- how the heck do you get to meaningful evaluation when you work with tiny humans?

Today we'll hit on some principles and tips, in case this is a nut you're trying to crack. If you want to chat some more, reply to this email. I'd love to talk!

Also included is:

• Downloadable Tip Sheet for anyone stuck in a "let's do a survey" rut
• How to update your email address if, for some reason, you need to do that right now.

Hang in there,

Jessica

I believe the children are our future.

In education, a common challenge I've heard is:

I work with kiddos in pre-K and early elementary. How can you do meaningful evaluation with tiny humans?

I'll be honest, Reader, this is the most fun challenge I get to face in my job.

If you know my backstory, I came to evaluation and research by way of teaching -- in museums and in K-5 classrooms. That has given me a lot of perspective of ways to find out what's really going on with those kiddos, whether I'm in a classroom, museum, or contrived research setting. Let's dig in!

​

First off, evaluation approaches have to reflect where kids are developmentally. Things to consider:

Literacy skills: Even when kids can read and write independently, collecting data that way privileges their early literacy skills over their thinking.

Metacognitive skills: Kids have abilities to think about their own thinking, but they're miles away from the convoluted questions often posted to adults.

Fewer mental shortcuts: (Or heuristics, if you wanna be all fancy about it.) Adult brains constantly infer from context to suss out the type of answer you want. Kids aren't reading between the lines like that.

Fewer mental shortcuts, part 2: On the other hand, they don't get bogged down by assuming what you're asking. They hear your question as you actually ask it.

Imagination: They are far more willing to get creative than adults are.

Eager: Once they confirm that you're "safe weird lady" and not "creepy weird lady," they are eager and adaptable to share what they know, think, or can do.

They really are tiny humans: They are making sense of the world exactly like you or I do. It's just from a perspective that we haven't experienced for a long time.

Let's dive into 8 tips to get you started showing how awesome your young learners are.

​

Tip 1: Use Your Words

You're probably going to have to talk to them. They're having thoughts. Feelings. Experiences. Talking is the best way to draw that out.

It also gives you flexibility to ask follow-ups and be sure you understand. This is important if your evaluation focus is something that's tricky to articulate. It takes time, but it's worth it.

And verbal data can take a lot of forms. I love the power of a 1:1 interview. But facilitating a group discussion or activity can be really great too. (The KWL Chart is a classic for a reason.)

Tip 2: Try Not to Lead

Kids learn early and often to agree with grown-ups, especially in school-adjacent settings. Whether it's true or not, "yes" is a safe course of action.

Evaluating? This is a big problem.

I have definitely interviewed a kid who answered "yes," only to continue talking and reveal that their "yes" was quite clearly a "no."

Focus on open-ended questions that do not lead to right answers. Use questions that can't be answered with one word. And, the hardest part, don't assume what they mean when they say something. Be curious and let them really tell you what's going on.

​

Tip 3: Say Yes

Taking a lesson from improv 101, always follow the kid's lead. Even when they take that weird turn you didn't expect. That is often where you get to some good stuff of how that little noggin' is ticking.

If you're interacting with kids, one of them will say something zany. Out of left field. Factually incorrect. Say yes.

Follow where they lead. Even if it feels bonkers to you. Remember: everything they say is the most interesting thing you've ever heard and you just want to understand more.

​

Tip 4: Watch & Listen

In some environments, you can learn a lot about young kids' learning and skills through observing them closely.

You'll have to get really specific about what you are looking for. And, to be fair, that's the hard part.

But if you can nail that, careful observations can reveal loads about kids' learning. It's powerfully authentic data about learning.

​

Tip 5: Gamify or Puzzlify

Kids aren't fools. They know a test when they see one. Looking for ways to turn data-gathering into a game can ease that barrier. In some of our data literacy research, we've found kids responded really well to a scenario-based interview that was trying to solve a puzzle. (See our Real World Example below for more.)

Our puzzle had no right answers and was admittedly challenging. But that kept students interested and focused. And that was the secret to finding out the extent of their skills.

Simpler game-like techniques can include sticker voting, using pre-fab games or toys, or emoji-based ratings (used by our AMNH colleagues) to get feedback or opinions in ways that feel more playful and less like a test.

​

Tip 6: Consider Drawing... with Caution

I make this suggestion with an asterisk. Yes, drawing is a natural and fun way for kids to express themselves and ideas. But anyone with tiny human art on their fridge knows... it may not speak for itself.

If you use drawings, I recommend pairing it with an interview. You need a way to find out what is meaningful to them about their drawing. (There are common literacy-building practices you can use in this way to have a win-win for evaluation and instruction.)

If you're thinking about drawings, maybe flip the script. Consider providing pictures and using those as a jumping-off point for interviews. Concept cartoons are a classic for a reason.

​

Tip 7: Let them be expert

Ask for a tour of something they created or visited or know a lot about. Suddenly they're the teacher, you're the student. Truths are revealed!

I saw Lorrie Beaumont present an awesome study that she did for the Adler Planetarium a bunch of years back. She evaluated a museum exhibit from children's point of view by strapping a camera and mic to them and asking for a tour.

PS - I would LOVE to replicate that study for any museum designing for kids. Call me.

​

Tip 8: It might NOT be the kid.

Sometimes, you dig into your evaluation questions and discover that the thing you really care about isn't something kids can tell you.

Turning to the significant adults in kids' lives is not a cop out! Parents, teachers, and other caregivers know a lot about the lives, experiences, thoughts, and growth of those kids.

And sometimes significant adults have a different take on things. From the perspective and long-view of adulthood, we see things in our significant kiddos that can be really important to an evaluation.

---

Real World Example:

In the Streams of Data project, Michelle and I interviewed elementary school kids to gauge their data reasoning skills

We wanted to figure out what data reasoning abilities kids of that age could actually bring to the party. Those insights would then let the educators create activities to get 4th graders working with real NOAA data.

Some key features of our approach to evaluating kids' data skills:

Puzzlify: We structured the interview around a task. It laid out some scenarios of imaginary kids doing a science project at home, asking our students to match up whose test conditions lined up with which data points.

Familiarity: We made the scenario, materials, and phenomenon things every kid had experienced and could relate to.

No numbers: We intentionally took numbers out of the puzzle. We didn't care about math skills. We cared about reasoning.

Say Yes Interview Guide: A kid forms a theory about how placing a glass under a bush could actually lead to more water? Tell. Me. More!

No right answers: The puzzle was so open-ended that there weren't right answers. It meant we couldn't lead them down a "right" path. And it dialed down the pressure when we told that to the kids.

Personally, I enjoyed telling them that a bunch of scientists had tried this puzzle and low-key freaked out saying, "BUT THERE'S NO RIGHT ANSWER?!?"

​

Check out this article if you want the full scoop on that study. (Paywall issues? Hit reply and let me know. We have author copies to share)

​

Do you have any fun stories of working with kids? Reply and brighten my day!

Tip Sheet Download

Kids aren't the only audience for whom surveys are, to use a technical term, the worst. They are useful, but can be a real bummer some settings.

If you feel stuck in a survey rut and want ideas of more ways to gather data (with any audience) that isn't a survey, our Tip Sheet download is for you.

Tip Sheet: 15 Ways to Collect Data without a Survey Put an end to QR codes and golf pencils!

P.S. Got a question you'd like us to answer in an upcoming newsletter? Hit reply and tell me what's on your mind!

P.P.S. Get this email from a colleague? Sign up to get your very own copy every month.

Why the "Evaluation Therapy" Newsletter?

The moniker is light-hearted. But the origin is real. I have often seen moments when evaluation causes low-key anxiety and dread, even among evaluation enthusiasts. Maybe it feels like a black-box process sent to judge your work. Maybe it’s worry that the thing to be evaluated is complicated, not going to plan, or politically fraught. Maybe pressures abound for a "significant" study. Maybe evaluation gets tossed in your "other duties as assigned" with no support. And so much more.

Evaluation can be energizing! But the reality of the process, methods, and results means it can also feel messy, risky, or overwhelming.

I've found that straightforward conversation about the realities of evaluation and practical solutions can do wonders. Let's demystify the jargon, dial down the pressure, reveal (and get past) barriers, and ultimately create a spirit of learning (not judging) through data. This newsletter is one resource for frank talk and learning together, one step at a time.

​Learn more about JSC and our team of evaluators. Or connect with us on LinkedIn: