Measuring Nonprofit Performance – other approaches

In refining our vision for the Wolfhound Fund, we’ve come across different ideas that are relevant to our plans but don’t capture the entire idea. In this post, we’ll discuss some other approaches, but note that we will not go over what we’ve already discussed, namely Robin Hood.

The NYT recently published an article on GiveWell, a new organization that seeks to collect information about measurable results obtained by charities and uses that information to rank charities it has evaluated. They are incredibly open in their approach and vow to make the information, the evaluations, and their methodology available to the public. We think this is absolutely the right approach. The most disturbing thing they reveal is that many of the larger foundations they contacted would not disclose their rationales, decision-making process, or grant recipient information to the GiveWell parent foundation. We are shocked; we have been hoping to, in the course of developing our metrics in an open platform that we would be supported by foundations who have presumably spent a lot of time evaluating various programs. We have not yet contacted these organizations to “seed” the discussions with existing metrics, but it’s certainly on the list.

The WSJ recently published an article on due diligence on donations. They note the typical first step of evaluating along standard financial measures, using the excellent resources that make those results, and the Form 990s from which they are derived, available. Notably, they identify GiveWell as one of the new organizations helping evaluate charities in a different way. The article mentions several other groups that take a similar approach to evaluating measured results. One theme is that many organizations do not attempt to research and measure their results; this statement does not surprise us given our experience with organizations.

First, for-profit businesses seldom actually evaluate whether they are creating value, and the myriad financial publications seem almost unwilling to identify those companies that are regularly destroying value — not just those that are actually losing money or not producing cash, but those that are underperforming by not earning the cost of capital. This notion led us to believe philosophically that many nonprofits were not instituting similar rigor in their operations.

Second, our personal experience in forming nonprofits and advising founders indicates that often, the same fervent belief and desire to help that sparks the impulse to start an organization and bear those early burdens also negates the natural skeptic within those same founders; they start organizations because they believe. This mindset isn’t to be ridiculed; it’s to be understood. All organizations, whether for-profit, nonprofit, family, or recreational, start with some leap of faith, however small or large. Our response and reaction to this circumstance is to rely on a liberal arts education and quote Milton: “I can not praise a fugitive or cloistered virtue;” our second response: “The unaimed arrow never misses.” More melodramatic is Socrates: The unexamined life is not worth living.

WSJ also gives a sidebar mention and link to The Center for What Works. They provide a variety of benchmarking materials to nonprofits, and even provide basic education about benchmarking and how the process works. They give a great introduction for their audience, who typically doesn’t come from a big-firm corporate culture where benchmarking, the development of Key Performance Indicators, and a dashboard/balanced scorecard/3x5x15 report to track it all are more common. While WhatWorks has benchmarking information, it does not

Their initial work on developing a standard taxonomy (PDF) for various types of outcomes is exactly the type of thing we want to create. Building on this standardization effort by adding domain/sector-specific metrics is one way of visualizing the Wolfhound Fund’s approach. In addition, they’ve taken this general taxonomy and actually applied it to 14 specific program areas, revealed at this page. If anything, these magnificent efforts clearly show the way and exemplify the type of standardized tools we believe should be produced for, and more importantly by, nonprofits and the extended community of stakeholders, including donors and program recipients.

What separates the Wolfhound Fund from these organizations who have already started collecting research and making evaluations? Three things: first, we come from a strong background combining military leadership with traditional financial analysis, which has made us experts in the tools, language, and very idea of converting strategy into action through leadership, training, execution, and management; second, we believe that the organizations that are measuring their results are the best place to start the discussion — we hope to build on the thousands of internal discussions in dozens of organizations who have already addressed these questions rather than seek to substitute our judgment for theirs; third, we aim to collect this information from a broader cross-section of the charitable universe so that donors interested in a sector, such as homelessness, can evaluate charities with different strategies that are not directly comparable. That second order decision, of choosing a strategy to combat a problem, is not the primary inflection point the Wolfhound Fund seeks to influence but one that we hope to illuminate just the same.

Fundamentally, we believe that an open-source, continuous improvement model is the best way to create self-correcting, self-improving, and self-proving tools for creating positive change. This is a small part of the overall problem, and we in no way compare ourselves, sitting alongside board members and donors, as remotely as committed as those who actually run homeless shelters, who show up to hearings for protective orders, who perform surgeries in developing countries, who counsel and guide troubled youth. Our goal is to help the outsiders find those people who do these jobs better than everyone else and who have built programs to put themselves out of business.