“Can we even get that data?”
That question, or some version of it, is usually one of the first, if not the first, question I hear when planning or discussing a new evaluation project. People want to know if it’s possible to collect data on a particular outcome or from a particular group. There’s often an undertone of, “I bet we can’t,” in the question too. Read More
One of the first posts I put up here was one about advice for emerging evaluators, based on my own experiences of getting into the field. I wrote it about a year after moving to Vancouver and six months after I committed to building a consulting practice. Three years later, I’m pleased with how well that post holds up. I might word a few things differently now, but the gist would be the same. And I’m pleased to report that I took my own advice (for once) and it’s stood me well as a developing evaluator and consultant. So much so that the last year in particular has been one of transformation and levelling up! Read More
If you listened to our recent episode of Eval Cafe with Michael Quinn Patton on principles-focused evaluation, you’ll remember him sharing his new favourite example of principles in action. It’s from the introductory article of a recent special issue of The American Statistician, which is all about moving beyond the use of p < 0.05 as the threshold for determining statistical significance. The article offers an impassioned explanation of why abandoning the entire concept of statistical significance is necessary and also outlines the beginnings of an alternative practice for valuing and interpreting statistical findings. The reason it showed up in the podcast is because the authors ground this new framework in principles, or flexible advice that can guide decisions and give direction, but must be adapted and interpreted in context. In comparison, p < 0.05 is a rule—it is applied the same way regardless of any contextual factors. (Check out the podcast and also Michael’s book, Principles-Focused Evaluation, to learn more about the implications of principles for evaluative work.) Specifically, the principles that the authors offer are, “Accept uncertainty. Be thoughtful, open, and modest” (or “ATOM”, as a mnemonic), and the remainder of the issue (43 articles worth!) goes on to offer more depth around the issues of p < 0.05 and the discussion of alternatives.
For an academic publication about statistics, it is, frankly, stirring. Read More
Transformation is awkward.
I mean, we all know that change is hard, but it’s also awkward. Read More
I can’t recall for certain, but I think the first person I “had coffee” with in a professional capacity was my friend Brian Hoessler, a fellow evaluation consultant and now my co-host on our evaluation-themed podcast, Eval Cafe. And of course the podcast is all about, as we say in our intro, “informal chats on evaluation-related topics. The kind you might overhear a your favourite coffee shop, if your favourite coffee shop was frequented by evaluators.” I guess we really set the tone with that first conversation! Read More
This blog post has been jointly written by Carolyn Camman and Art Assoiants. We connected through the Canadian Evaluation Society’s Mentoring Initiative in 2018 and this post is our way of sharing some of the learnings from our journey together as mentor (Carolyn) and mentee (Art) over the last year.
To write this post, we did a sort of self/mutual interview. We chose some questions for ourselves, wrote up our responses separately, and then shared them back with each other. You can read the questions and our respective reflections below. Read More
Today I was asked to design a meeting outline for a hypothetical scenario in which a group of people needed to winnow down an ambitious list of topics for inclusion in a strategic plan. This was part of a training course I’ve been taking in facilitation (hence being hypothetical), but of course I’ve been in this scenario plenty of times in real life. Only so much time in the day, much money in the budget, staff in the organization, energy in the body, etc. Somewhere we have to decide how we’re going to focus resources that are not infinite.
I don’t object to prioritizing in principle, but there was something about this particular assignment that made me recoil and, ultimately, rebel. Read More
Have you ever had that experience of being really seen? Paid attention to in that deep way where the other person notices things about you that no one else ever seems to, maybe even sees things in you that you didn’t know were there, but now you see them too? Maybe with a therapist, or a romantic partner, or a really sensitive, observant friend or family member? There’s something tingly and terrifying about being seen that way, but also deeply satisfying and rewarding. The pay-off of that vulnerability is intimacy. Read More
Blogging is hard. I’m not sure why I find it such a struggle, though I know I’m not the only one who does. I marvel at the folks who seem to be able to write quickly, eloquently, and insightfully (I can usually manage 1-2 at a time but rarely all three). But I’m choosing to assume that blogging ability is a learnable skill so I keep practicing and looking for ways to make it easier on myself.
One thing I’m trying is to look for synergies between blogging and other creative work that I find easier, to see if I can borrow some of that momentum and inspiration. In this case, that’s the podcast I’ve been producing with my good friend and co-conspirator Brian Hoessler (of Strong Roots Consulting) for the last couple of years. Read More
There’s a really great device that Michael Quinn Patton suggests using in developmental evaluation called ‘sensitizing concepts’. He’s borrowed it from qualitative research methods as a way of providing guidance to inquiry in complexity. Here’s a definition he gives in his qualitative methods book that came out a few years ago:
“Sensitizing concepts are terms, phrases, labels, and constructs that invite inquiry into what they mean to people in the setting(s) being studied. ... Qualitative inquiry using sensitizing concepts leaves terms purposefully undefined to find out what they mean to people in a setting. Sensitizing concepts are windows into a group’s worldview.” Read More
I hurt my back last week.
This is not news. I’ve been hurting my back since 2013. Read More
Pronouns are small little words to cause such a fuss, am I right?
In my ideal world, my pronouns would require no footnotes, no extra explanation, or citations. But as we don’t live in a gender utopia, at least a little explanation is required, if for nothing more than to move the conversation along productively. Read More
In December I had the opportunity to moderate a breakfast club event organized by the local chapters for the Canadian Evaluation Society (CES-BCY) and the Institute of Public Administration Canada (IPAC Vancouver). The theme was next year’s CES 2017 Conference theme: Facing Forward: Innovation, Action and Reflection. Read More
Everyone knows the story.
It's a great story about perseverance and compassion, about the small but tangible impacts we can have even when faced with what seems like an insurmountable problem. In the difficult world of delivering social interventions, it is stories and ideas like these that get people out of bed and into work every day—the knowledge that whatever else is happening in the world, we can make a positive change for someone.
As an evaluator, I do count myself among those whose work is dedicated toward improving the lives of others. While evaluation may seem a few steps removed from the difficult and important work of direct service delivery, if you speak to evaluators you will find that most of us are not in it just because we love numbers and metrics so very much. Sure, lots of us enjoy a good spreadsheet, but we do what we do because we sincerely believe that our work is valuable and necessary and makes a difference to the organizations and people we support, and therefore to the organizations and people they support. 'Helping people who help people' is my professional motto for a reason. Read More
Officially blogging from my new office space! A big upgrade for me and a momentous step forward in my journey as a professional evaluator. It’s bitty and beautiful and all mine. And will hopefully soon have curtains!
Moving forward always makes me want to look backward as well, to see how far I’ve come and what I’ve learned along the way. (Must be the evaluator in me.) One of the things I’ve realized recently is how much farther back my journey goes than I thought, and how I was gravitating toward the path of program evaluation well before I even knew what it was or how to do it. Read More
It's nerve-wracking to be approached by a recent grad or student asking my advice on how to break into the evaluation field. I want to look around and see if they're really talking to someone else, someone who's been doing this forever who knows exactly what they're talking about, not someone in the midst of breaking into the field themselves.
Then I remind myself that no one ever knows exactly what they're talking about, especially in a field as dynamic and rapidly growing as evaluation. And while I may still be at the start of what I hope will be a long career, that also gives me an excellent perspective on things that in five or twenty years I may have forgotten about or may no longer be relevant. So none of this advice comes with a solid guarantee of success, but it does come with my absolute promise that you're not the only one using it. Read More
Last week I had a great opportunity to give a talk at the CES-BCY 2015 Evaluation Conference on the topic of engaging service delivery staff in evaluation, specifically in the context of non-profits. The talk was well attended and followed up with several interesting questions. More than one person also told me they wanted to attend the presentation but couldn't because of other talks scheduled at the same time.
So for people who weren't able to attend either the talk or the conference itself, I have uploaded the slides (link opens PDF file). Because the slides weren't meant to stand alone and may be a little cryptic without context, I'm also recapping my talk here on the blog. Read More