Forcing Nonprofits To Lie About Data

In Viewpoints by Laura Quinn17 Comments

 

Here’s a truth that’s rarely spoken: if a key funder asks a nonprofit for data it doesn’t really have, it’s standard practice for nonprofits to simply invent plausible data to fill the gaps.

 

On its face, this seems outrageous. As a sector, how can we ever hope to be able to effectively understand impact and compare interventions if half the results data we think we have is simply fabricated? Who are these liars bringing us down, and where’s the nearest jail cell we can slap them into?

 

It’s not so clear cut, however, and the road we’ve travelled to get here is considerably more subtle. Data invention is the inevitable result of tying money to data without ensuring that a nonprofit’s infrastructure is sufficient – and sufficiently funded – to collect that data. It’s expensive and time consuming to collect lots of data about lots of things – often as expensive as providing a simple service itself.

 

For example, let’s say I’m running a nonprofit that works to eliminate school bullying. We’ve got a public hearing on a school board issue coming up, and we’re down to the wire canvassing for it. We’ve been lucky enough to get funding to manage a big volunteer canvassing campaign at the same level as our last big volunteer campaign, but this time, the funder has put an additional focus on metrics. In response, we’ve added a data-collection element – in addition to talking to each person about the importance of the issue, volunteers are asked to collect their Zip code, gender, and apparent age and race.

 

We all know data is only as good as the collection methods, and in the heat of the campaign, our volunteer-reported data gathering got a little iffy. Some volunteers decided the issue was more important than the data and simply tallied how many people they talked to. Others did not turn in any data sheets at all, and it’s unclear if they were tracking anything.

 

Now we’re facing a funder report asking us for campaign results. On the surface, the campaign was successful – we won the school board vote – but we’ve got to come up with some way to complete the fields asking for detailed metrics.

 

When we sought funding, we said we would talk to 2,500 people. We’re unsure precisely how many people we actually talked to, but we’re confident it was at least that many. Let’s say we put 2,551 so the number is not suspiciously precise.

 

For the percent of people age 55 or older, our data – representative of maybe half the people we actually talked to – says it’s about 20 percent, but we know some volunteers who canvassed several retirement homes did not turn in demographic data. Let’s say 27 percent, then.

 

These estimates are probably close to accurate, and our best guesses are well-meaning, but we’re presenting them as data. From our perspective as a nonprofit, they’re close enough, and better than telling the foundation we didn’t successfully track the data we were committed to tracking. But if the foundation then takes that information and compares it to other nonprofits’ data – which might be equally iffy – then any conclusions drawn about which demographics to speak to in order to make change happen, or how many, are invalid.

It’s expensive and time consuming to collect lots of data about lots of things – often as expensive as providing a simple service itself.

This is not an obscure example. If we could collect accurate numbers on how many nonprofits have ever fabricated results data (an interesting challenge in and of itself), I would put money that’s it’s way north of 50 percent of all nonprofits. Fabricating at least some of the data they report to funders is the day-to-day reality for many nonprofits.

 

Whose fault is this? It’s tempting to place the blame squarely on nonprofits, but I would focus more on problems in the design and funding of programs. In our example, the foundation has funded a campaign at the same level as a previous one but added new data requirements without funding them. Right or wrong, the assumption is that the nonprofit will cover it, though it’s just as likely that neither the foundation nor the nonprofit itself recognized the additional cost required.

 

The truth at the core of this issue is this: It’s almost always easier and less expensive to carry out your mission without collecting data.

 

It might seem like data collection is hardly any extra work – why can’t volunteers just write the information down? – but in reality, data collection often requires additional training, hiring more-reliable staff members or volunteers, and spot-checking and overseeing collection, all of which cost more. Fundamentally, running an outreach campaign that reliably collects data is simply more expensive than running one that touches just as many people but doesn’t collect data.

 

So. How do we ensure nonprofits are able to collect data without feeling like they need to fabricate it?

 

First we need to acknowledge as a sector that collecting data is more expensive than not collecting data. We need to stop pretending that nonprofits can easily collect data and present metrics without additional money or any compromise in quality or quantity-served. Then we need to work through the tradeoffs implicit in that – do we prefer to serve more people with less data collected? More data with fewer people served? More data but lower quality service (a choice unlikely to be popular)?

It’s almost always easier and less expensive to carry out your mission without collecting data.

It doesn’t serve anyone to simply deny that tradeoffs exist. We can ask for more data and expect the nonprofit to serve the same amount of people at the same level of quality for the same amount of money, but that won’t make it possible.

 

Most frequently, nonprofits that care deeply about people and services will sacrifice the data instead, and simply fabricate it for funders. If we demand more data without more funding, it logically follows that what we’ll get is simply data that’s more made up.


Many thanks to Laura Quinn, Executive Director of Idealware, for this latest op-ed piece. Please let us know what you think in the comments below or on Twitter.

To stay up to date with the latest Markets For Good articles and news, sign up to our newsletter here

Comments

  1. Just desire to say your article is as amazing.
    The clarity in your post is simply spectacular and i can assume you’re an expert on this subject.
    Fine with your permission let me to grab your RSS feed to keep up to date with forthcoming post.
    Thanks a million and please carry on the rewarding work.
    0mniartist asmr

  2. It’s perfect time to make some plans for the longer term and it is time to be happy.
    I’ve read this publish and if I may I desire to suggest you some attention-grabbing things or advice.
    Perhaps you can write subsequent articles regarding
    this article. I want to read more issues about it! asmr 0mniartist

  3. Does your blog have a contact page? I’m having trouble locating it but, I’d like to
    shoot you an e-mail. I’ve got some ideas for your
    blog you might be interested in hearing. Either way, great blog and
    I look forward to seeing it develop over time. asmr 0mniartist

  4. For the reason that the admin of this web page is working, no question very quickly it will be famous, due to its quality contents.
    0mniartist asmr

  5. Excellent items from you, man. I have be mindful your stuff prior to and you’re just too great.
    I really like what you have obtained here, certainly like what
    you are stating and the best way by which you are saying it.
    You are making it enjoyable and you continue to care for to keep
    it smart. I can not wait to learn much more from you.
    That is actually a tremendous website.

  6. Wonderful site you have here but I was curious if you
    knew of any forums that cover the same topics
    talked about in this article? I’d really love
    to be a part of community where I can get responses from other knowledgeable people that share
    the same interest. If you have any recommendations, please let me
    know. Appreciate it!

  7. Hello! I know this is somewhat off topic but I was wondering which blog
    platform are you using for this site? I’m getting tired of WordPress because I’ve had
    problems with hackers and I’m looking at alternatives for another platform.

    I would be fantastic if you could point me in the direction of a good
    platform.

  8. Spot on with this write-up, I honestly feel this site needs much more attention. I’ll probably be back again to
    read through more, thanks for the information!

  9. I’m truly enjoying the design and layout of your website.

    It’s a very easy on the eyes which makes it much more enjoyable for me to come here and visit more often. Did you hire out a developer to create
    your theme? Superb work!

  10. It’s truly very complex in this active life to listen news on TV, so I only use world wide web for that reason, and get the latest news.

  11. Hello, Neat post. There is a problem together with your web site in web explorer, might test this?
    IE still is the market leader and a huge part of other people will leave out your great writing due to this problem.

  12. Howdy! I’m at work surfing around your blog from my new iphone 3gs!
    Just wanted to say I love reading your blog and look forward to all your posts!
    Carry on the fantastic work!

  13. Hey there, I think your website might be having browser
    compatibility issues. When I look at your blog site in Opera, it looks
    fine but when opening in Internet Explorer, it has some overlapping.
    I just wanted to give you a quick heads up!

    Other then that, amazing blog!

Leave a Comment