Case Study: The Handsel Foundation

The Handsel Foundation is a small private foundation working to end the ill treatment and overpopulation of companion animals. Beginning in May 2006, HRC conducted three phases of research for the Handsel Foundation ("Handsel") related to grant-making effectiveness. In particular, our research focused on the efficacy of Handsel's recent grants, most of which were directed toward high-volume, high-quality spay/neuter programs. This information may be helpful to anyone interested in program assessment, particularly those working on companion animal overpopulation.

Measuring the Effectiveness of Spay/Neuter Programs

Background, Approach, and Challenges

Research Need: The Handsel Foundation wanted to understand the impact of its grantmaking program, both overall as a foundation and for its individual grants. Handsel also wanted to update its grantmaking program to encourage program evaluation among its grantees, while also keeping the application and reporting requirements reasonable for small organizations. The research conducted by HRC helped Handsel refine its grantmaking strategy and understand the immense challenges relating to program assessment.

Approach to Solution: Phase one of the Handsel-HRC research studied the landscape of spay/neuter program assessment from the perspective of both organizations and funders. HRC conducted interviews with grantmakers and experts in the field to determine how best to measure the impact of spay/neuter programs on animals lives. Phase two involved additional interviews, this time with a selection of Handsel's recent grantees. The goal of this phase was to understand the extent to which small groups already measure their impact and how difficult it would be for them to start doing so in the future. The results of phase two were shared with grantmakers through a PetSmart Charities webinar and a publicly available report (see below for details). Finally, phase three involved studying the efforts and impact of a single exemplary Handsel grantee, the Feral Cat Spay/Neuter Project, located in Seattle, WA.

Information and Tools: All three phases of the Handsel Foundation research involved collecting input from experts representing a variety of perspectives, including many who are currently "in the field." This first-hand input provided a realistic assessment of the state of program evaluation in companion animal work, particularly among shelters and spay/neuter advocates. HRC also relied on secondary information such as the articles by Peter Marsh, Esther Mechler, and others, which are available on the Best Friends website. Finally, some research pertaining to spay/neuter program evaluation is available from FIREPAW (http://www.firepaw.org/research/).

Challenges: The primary challenge to assessing the state of program evaluation is the difficulty of measuring something that doesn't exist. As discovered during the Handsel-HRC research, program assessment is a mostly anecdotal exercise for many companion animal groups. This is due mainly to the fact that accurately measuring program impact is very difficult, but it also stems from a lack of knowledge and/or motivation among some organizations and advocates. However, by conducting this research and sharing the results, the Handsel Foundation has taken one step to improve program evaluation and create a culture of accountability among companion animal advocates.

Solution and Results

While some of the Handsel Foundation research is for internal use only, most of the information has been made public to help other foundations and spay/neuter advocates with their own program evaluation efforts. Below is an executive summary covering the first two phases of the research, from the Handsel Report for Animal Grantmakers (October 2006). The full report is available online at http://www.handselfdn.org/initiative.htm.

1. The primary focus of this research was on companion animal program evaluation, with particular emphasis on measuring the impact of spay/neuter programs.
2. In general, it is clear that accurately assessing the performance and impact of companion animal projects and programs involves many challenges, and evaluation is therefore underutilized by all but a handful of organizations.
3. The overall challenges companion animal groups face include the unavailability of accurate and reliable data for the community, lack of time to collect and analyze data, and/or insufficient knowledge about measuring impact.
4. Spay/neuter programs are faced with unique measurement challenges; groups and foundations use a variety of statistical and anecdotal indicators of performance.
5. Regardless of the inability to accurately measure impact, there is widespread agreement among experts regarding a common set of S/N program best practices.
6. Within this framework of best practices, however, different foundations use significantly different processes to measure grantee accountability and impact.
7. From a grantee's perspective, despite concerns about having sufficient time or knowledge, they are willing to make an effort to meet grant requirements concerning program and impact evaluation.
8. Reactions from grantees to a few "targeted grant" ideas being considered by the Handsel Foundation were mixed, with greater interest in training and conference participation than program evaluation or optimization.

Outcomes and Next Steps

The Handsel Foundation's research clearly showed two findings. First, all companion animal advocates from grantmakers to group leaders to individual rescuers want to understand their impact on animals' lives. Second, despite strong interest in program evaluation, the inherent challenges make it too daunting for most groups to conduct accurately (and hence for most foundations to require such evaluation). From Handsel's perspective, grantee requirements should be commensurate with grant size, and in early 2007 the foundation revamped its strategy and updated its guidelines to reflect this belief.

The Handsel Foundation also continues to have a strong interest in program evaluation. As part of its revised grantmaking strategy, the foundation is making more concentrated investments in proven high-volume spay/neuter programs on the west coast of the U.S. Over time, the information provided by these grantees will help the foundation understand the broader impact of its grants and potentially the difference in effectiveness for different types of programs (e.g., stationary vs. mobile clinics). To the extent possible, findings from future research will be shared with the animal protection community through the Handsel Foundation website.

More Information

Client: The Handsel Foundation (www.handselfdn.org)

Author: Humane Research Council (HRC)

Date of Original Research Study: May-August 2006

Contact Person: Che Green, HRC; (206) 905-9887; cgreen@humaneresearch.org