The Surprising Truth Behind Beneficiary Feedback
Article written By Shuo Xie & Kris Sankaran
New research suggests that the feedback of nonprofit clients is less biased than many think.
In the business world, collecting user feedback has become a rule of thumb. It enables companies to improve products or services, and strengthen relationships with clients. The social sector also recognizes the importance of collecting and responding to client feedback, but critics cast doubt on its validity, citing “courtesy bias”—the tendency for people to give answers they think the asker wants to hear. We at the Stanford Statistics for Good Group (SSFGG) set out to understand whether courtesy bias truly exists among clients in the nonprofit sector and worked with the crowdsourced nonprofit review site GreatNonprofits.org to analyze user feedback via its nearly 200,000 nonprofit reviews. In particular, we wanted to learn whether low-income food bank and homeless shelter clients were more apt to give positive feedback because they are in need of the benefits provided by the nonprofits.
Reviewers on GreatNonprofits indicate which one of seven populations they most identify with: clients, donors, volunteers, board members, the general public, advisors, or professionals with expertise in the field they are reviewing. Using this data, we analyzed the differences between the ratings among nonprofit clients and the six other groups.
The illustration below shows how the ratings of different groups deviate from the overall rating of an organization. The more a curve sways from the 45-degree diagonal line, the more difference there is. For instance, on average, for the nonprofits that reviewers give five stars (out of five) 75 percent of the time, board members give five stars 83 percent of the time and the general public gives five stars 60 percent of the time. In the chart, each group for each nonprofit is represented as a dot in the background. The mass of dots on the upper-right corner indicates that all the groups rate most nonprofits highly.
So are client evaluations for nonprofits reliable? Our research indicates that they are, and that online reviews can help provide nonprofits with a true and honest picture of its performance. Here are our recommendations for organizations seeking this kind of feedback:
- Embrace third-party platforms or methods to help avoid courtesy bias.
- Invite clients, staff, donors, and others to review your services.
- Don’t be afraid of what you may find out; use the feedback to improve your services.
- When you receive feedback, respond promptly and publicly so that others know you’ve heard them.
A great example of an organization doing this is Mission in Citrus, a Florida-based nonprofit that offers homeless and shelter services and has successfully collected and leveraged reviews from clients. Executive Director James Sleighter notes, “When we ask clients after a few weeks, they gladly do the reviews. Clients who depart often leave reviews. No one is forced.” When time permits, Mission in Citrus also shares client reviews on its website, on Facebook, and in newsletters to attract others who need help. By actively sharing client reviews publicly, the organization communicates that client opinions matter.
Shuo Xie is a master’s student in statistics at Stanford University and an active member of Statistics for Good. In her spare time, she serves as a nonprofit consultant at iJoin Social Innovation Consulting.
Kris Sankaran is currently at Stanford University pursuing a Ph.D in statistics. He dreams about the potential for rigorous quantitative thinking and inference from data in the 21st century.