Evaluating Impact: Storytelling Versus Metrics, Which Matters?

Stories can capture the heart of a person like a number or statistic never can. Even mind-boggling statistics are just that – incapable of being fully processed and affecting us into action. Human experience on the other hand, touches us deeply and creates empathy that we are unable to ignore.

On Thursday, I listened to Jessica Jackley (co-founder of Kiva and founder of Profounder) and Jennifer Aaker (Stanford Business School Professor and author of The Dragonfly Effect) talk about the power of storytelling. It was at a Philanthropy and Civil Society event at Stanford University that was supposed to be on social media, but focused on something much more enlightening: the power of people's stories.

Many people are talking right now about the importance of metrics in measuring nonprofit impact. Metrics quantify activity and changes that are seen as a result of a program, service, or policy. While there is a place for metrics and quantifiable successes, I believe it can’t be the only way we assess change.

While some questions are amenable to a single numerical answer, many are nuanced and require skill in listening, capturing, synthesizing and communicating in creative qualitative ways. The latter begins through asking thoughtful questions and consequently by listening to peoples’ stories.

There are two reasons that we must consider the questions we ask about impact and how we gather information to answer those questions, whether through collecting numerical data or capturing stories.

The first reason is to know whether a program is having the intended effect on a population. For example, is a domestic violence program protecting and empowering women involved in abusive relationships? This could be accomplished by gathering numerical data – number of women served, number who leave abusive relationships, number who begin to work. It may also be collected by listening to women's stories and collecting nuanced information on changes in thought, attitudes, aspirations, and challenges that would be difficult, if not impossible to gather in a structured questionnaire.

The second reason is to learn about the program, make adjustments, and adapt to external changes in the environment (e.g. changes in funding; new government policies; heightened awareness of domestic violence among law enforcement; etc.). This second reason has to do with strategic learning – what can you learn about the program’s impact in order to improve, develop, and adapt in climate of constant, rapid change.

When your goal is learning a single metric or even a series of metrics will nearly always fall short. You may get a sense of impact, but it will never give you the richness of information that you need to take that next step to action.

Even more, as a donor or a funder, which would stir you to action: a number or a story of change? I think most of us would agree that well-told stories supported by metrics (or even stories alone) are the biggest motivators for our generosity.

I’ve been thinking about how I tell my story and capture the stories of those around me. How well do I communicate well the stories of my life and the experience of those around me?

I think I have a lot of learning to do in this area of storytelling. I am much better at reporting metrics, creating thoughtful, but rather dull, reports of survey data. It comes much more easily than painting vivid, expressive thought and activity to capture a person’s heart and mind.

So while I focus here on the importance of storytelling, as a skill and component to evaluating impact; I must be clear that I believe there is a role for metrics.

Metrics play an important role in setting the baseline for a community – to help set priorities among many different human needs. For an amazingly fun and creative approach to metrics check out County Sin Rankings. It takes the seven deadly sins and maps them onto community indicators such as crime rate, infectious disease rate, obesity rate, high school dropout, etc. It gives you sense of how poor or well your county lands in each of the 7 areas (Pride, Greed, Envy, Wrath, Sloth, Lust, Gluttony). You can search by county, zip code or city. It’s awesome!

Metrics also play an important role in benchmarking, but not as an end it itself. When you rank nonprofits, foundations, according to a series of metrics or indicators you start a conversation that can lead to meaningful change. No one wants to be at the bottom of a benchmarking list (I heard this from Adam when he saw how Lehigh, his Alma Mater, dropped in college rankings - sad). It gets leaders (and, I guess, alumni) talking about both the quality of the metrics and what change if any is needed to improve what’s happening in that organization or community.

How do you use storytelling in your work? I’d love to hear what tools or experiences you’ve had with excellent storytelling that has compelled you or others to action.

Happy Halloween readers! Safe trick-or-treating.