profile

The Evaluation Therapy Newsletter

I kinda hate SMART goals.


Hi Reader,

In addition to candy canes and Mariah Carey on repeat, this is the time of year that brings up a lot of talk about goals. Did you meet your goals for this year? What are your goals for 2025?

Even Goodreads is all over me about reviewing and setting my annual reading goals!

All this talk about goals reminded me that I am a bit of a Grinch when it comes to so-called SMART goals. Since we are in December, I shall indulge in the traditional Festivus Airing of Grievances and dig into the troubles that come with trying too hard to be SMART.

Also included:

  • Quick Poll: What do you want to talk about next year?
  • A holiday laugh, courtesy of SNL and some highly flawed informal science educators.

Here's hoping you ring in 2025 with peace, joy, and whatever type of beverage keeps you warm in these cold days.

Cheers,

Jessica

I kinda hate SMART goals.

As someone who helps people measure if they achieved their goals, this may seem an odd position for me to take. But I’ve watched this acronym metaphorically beat creative people into submission.

SMART (typically) stands for:

  • Specific.
  • Measurable.
  • Achievable.
  • Realistic.
  • Time-Bound.

It's not all bad. Let's highlight the great parts of SMART.

I love that it starts with "Specific." I cannot stress enough how important specificity is. Whenever I lead outcome planning workshops, it's the first criterion on my list. Be as specific as you can be about what you envision achieving.

Hand-wavy goals get hand-wavy evaluations.

I also like ending with "Time-Bound." If you're setting a goal, it really can't be for "someday." (Face it, even a Bucket List goal is time-bound.)

Practically speaking, you need to set goals that are bound within the time you have available to evaluate.

I do have a minor semantic gripe with SMART. Come on, “Achievable” and “Realistic” are basically synonyms. There's a bit of nuance, but that Venn diagram is awfully close to a circle.

Extra words for acronym purposes get my BS detector fired up.

But my big gripe is with “Measurable.”

Here’s the thing: I’ve watched many informal educators dream up creative and inspiring experiences for audiences of all ages. But when told to write a SMART goal or outcome for that brilliant program, the air is sucked out of the room.

Pen goes to paper, and they are stuck. I hear:

“But that kind of stuff can't be the goal. It isn’t measurable.”

And then they write some basic statement that reflects some lame survey question they could imagine writing. It seems measurable, but doesn't at all reflect the magic of what they do.

Reader, that hurts my heart.

Look, measurable is great if the measurement is obvious. Take that Goodreads Challenge. A measurable goal to read more books in 2025 than I did in 2024 is perfectly clear and lovely.

But education initiatives are a wee bit more complicated than that. That’s why our clients will always hear:

“Forget about measurable. Tell me what you are striving for. Describe the outcomes or impact you envision. Draw a link of how the learning experience you create leads to that vision. If you can be specific about all of that that, leave the measurement part to us.

(Sidebar: I told you that I love the Specific in SMART.)

Look, figuring out how to measure complex things is hard! Evaluators are an entire profession of people who spend our days figuring out how to measure difficult-to-measure things.

For something as complex as education or community engagement, it's ridiculous to expect you to limit your vision based on whether you think it's measurable.

Let's get SMRT.

Wikipedia tells me that the "A" in SMART has been rebranded many times. (At this point, I'm 100% convinced the A is just acronym filler.)

Interestingly, no one has ever monkeyed with the "M." Let's fix that.

What if we changed the M to stand for Meaningful?

Instead of immediately worrying about how you are going to measure your goal, think about whether the goal actually means something to your program or institution. On the journey to creating change, what meaningful step will you take?

To be clear, a meaningful step does not mean a big step.

Meaningful just means something you care about. Something you'll be heartened to report that your work achieved, at the end of the day.

And, the heck with it, I'm just getting rid of that A. Today begins my campaign for simpler, albeit less pronounceable, acronyms.

I want museums, informal educators, and folks trying to change the world for good to dream that impossible dream! Don’t let the intimidation of measurement or evaluation hold you back. You create the inspiration and bring the educational know-how. We're here to help you prove it to the world.


Real World Example:

I kid you not, just yesterday, a client told me that they are facing a new mandate to construct "SMART" outcomes for a really complicated initiative.

I have no poker face at this point in the year. I audibly groaned.

And what part of the conventional SMART acronym has them freaked out? Yup, the measurable part.

So, we talked about their struggles with past outcomes and measures, which boiled down to:

Our partners have done so much good work! They see progress happening on the ground. But it's not reflected in the metrics or goals. We feel like we're clawing to just say we are holding steady.

So, where did we start? We set "measurable" aside, and we focused on meaningful.

We suspect the past metrics and goals are a little too big-picture -- aiming for a gold standard. But there's a lot of good (and gradations of progress) that can happen on a participant's journey to that gold standard. Their goals and metrics need to allow space to celebrate legit "wins" at earlier stages and better represent progress within the big-picture goals.

Progress over perfection. Yes, it's one of those cheesy, inspirational poster memes. But it's not wrong.

Fortunately, we have a trove of historic data to mine. We identified where we might find actual participants' stories about what things look like in the "progress zone." Anytime you can build goals from actual experiences and language of your target audience, you are in much better shape than taking a shot in the dark.

Eventually, they'll have to name some numbers and targets to satisfy the powers-that-be. But by starting with what is meaningful, we will identify some goals that qualify as SMART, but were actually built by being SMRT.

Anyone else with SMART Goal traumas? Impossible dreams they’d like to measure but aren't sure how? Reply and tell me.

Your Questions for 2025

We are planning newsletter topics for the new year! What are your burning questions about evaluation, research, or navigating the complicated landscape of progressive education in 2025?

Hit reply to tell me your questions, conundrums, worries, and struggles -- we'll address it in an upcoming newsletter!

If it's bugging you, I promise, you are not the only one.

A Holiday Laugh

Last year, a holiday-themed SNL skit floated across my radar. "The Science Room" is a parody of science education TV. (I assume it is specifically a parody of my childhood favorite, Mr. Wizard's World. Oh, the amazing things I learned from you, Mr. Wizard!)

In this episode, Steve Martin & Martin Short illustrate terrible informal educator questioning strategies.

You know, where the educator asks an "engagement" question. But they're actually fishing for a single, very specific answer. The answer that will enable them to say and explain their next scripted point.

They become baffled and flummoxed when a student answers with something off-the-wall -- or even perfectly reasonable, just not the answer they wanted. So they keep fishing and fishing until someone says the magic word. (And everyone in class learns that science = right answers only.)

It's painful to watch in real life. In parody, hilarious.

Do you want to watch Steve Martin & Martin Short be hilariously terrible informal science educators?

Warning: It is SNL. I can't promise it is 100% work appropriate.

P.S. Did you get this email from a colleague? Sign up to get your very own copy every month.

P.P.S. See you in 2025, everyone! Hang in there, it might be a bumpy year.

Why the "Evaluation Therapy" Newsletter?

The moniker is light-hearted. But the origin is real. I have often seen moments when evaluation causes low-key anxiety and dread, even among evaluation enthusiasts. Maybe it feels like a black-box process sent to judge your work. Maybe it’s worry that the thing to be evaluated is complicated, not going to plan, or politically fraught. Maybe pressures abound for a "significant" study. Maybe evaluation gets tossed in your "other duties as assigned" with no support. And so much more.

Evaluation can be energizing! But the reality of the process, methods, and results means it can also feel messy, risky, or overwhelming.

I've found that straightforward conversation about the realities of evaluation and practical solutions can do wonders. Let's demystify the jargon, dial down the pressure, reveal (and get past) barriers, and ultimately create a spirit of learning (not judging) through data. This newsletter is one resource for frank talk and learning together, one step at a time.

Learn more about JSC and our team of evaluators. Or connect with us on LinkedIn:

Copyright © 2024 J. Sickler Consulting, All Rights Reserved

You are receiving this email because you signed up for our newsletters somewhere along the line. Changed your mind? No hard feelings. Unsubscribe anytime.

Wanna send us some snail mail? J. Sickler Consulting, 100 S. Commons, Suite 102, Pittsburgh, PA 15212

The Evaluation Therapy Newsletter

Our monthly Evaluation Therapy Newsletter shares strategies, ideas, and lessons learned from our decades of evaluating learning in non-school spaces - museums, zoos, gardens, and after-school programs. Jessica is a learning researcher who is an educator at heart. She loves helping education teams really understand and build insights from data that they can use immediately – even those who are a bit wary of evaluation.

Share this page