The Role of Civil Society in Ensuring AI Is Fair, Accountable, and Transparent

In Viewpoints by Rhodri Davies89 Comments

This piece was originally published by Charities Aid Foundation as “Sorting Algorithms: The Role of Civil Society in Ensuring AI Is Fair, Accountable and Transparent” on September 17, 2018. Some words have been changed to reflect American English usage.

I have recently found myself repeatedly making the case for the importance of involving civil society in the growing debate about the development of AI. (For instance, Where are charities in the great AI debate?, It’s more vital than ever that charities get to grips with AI, and Our voices must be heard in the Artificial Intelligence debate). Some might suggest I am in danger of being repetitive, although the more charitable among you might allow that I am merely being consistent. (For now, at least.)

Without revisiting that argument in full here, the key points are that AI has the potential to affect civil society three broad ways: Firstly, by offering new ways of delivering social and environmental missions; secondly, in terms of its wider impact on the operating environment for organizations, and finally, because deliberate misuses of the technology and unintended consequences of its honest application are going to create new problems for the people and communities that CSOs serve.

This means that civil society needs to get to grips with the issues now: both so that CSOs can harness AI’s potential for delivering impact in new and innovative ways, and so that they can play a part in the debate about shaping the development of the technology in order to minimize the potential negative consequences. Conversely, governments and the tech industry need to recognize that engaging with civil society when it comes to the AI debate is not merely a nice added extra, but an absolute necessity. If they are serious about the need for AI to be ethical (which the UK, in particular, has made a central pillar of its ambitions) then it is vital to involve CSOs, because they represent the very people and communities that are likely to be hit earliest and hardest by any negative consequences of the technology. It is important to recognize that in order for many CSOs to be able to bring their valuable perspective to the table, they may well require additional support. I would argue that it is incumbent on government and the tech industry to provide this support.

And just to make it clear that it isn’t just me saying this, here’s a good quote I came across recently in a new ebook published by Microsoft, making the case for the importance of engaging a wide range of voices in debates about where technology is going:

“As technology evolves so quickly, those of us who create AI, cloud and other innovations will know more than anyone else how these technologies work. But that doesn’t necessarily mean that we will know how best to address the role they should play in society. This requires that people in government, academia, business, civil society, and other interested stakeholders come together to help shape this future. And increasingly we need to do this not just in a single community or country, but on a global basis. Each of us has a responsibility to participate—and an important role to play.”

What Does This Mean in Practice?

Having re-mounted my hobby horse briefly, I want to move away from the case for engaging civil society in the debate about AI and instead explore another aspect of this wider issue: namely what practical role can CSOs play in the design, implementation, and oversight of specific AI systems? This is just as important, because even if CSOs do end up playing a vital role in framing the ethical debate about the overall development of AI, there will still be an ongoing need to put into practice whatever mechanisms are identified as necessary to ameliorate any negative unintended consequences. Furthermore, these mechanisms are unlikely to be perfect so there will still be a need to address problems where they do occur.

What, then, can CSOs do in practical terms? It is useful to break this down using the concepts of Fairness, Accountability, and Transparency, as have become key framing devices in the academic debate about the ethics of machine learning (ML). (There’s even a community of academics focused on ‘FATML’. It’s definitely worth checking out the resources on their website for anyone interested in these issues. Or, if you want a more easily digestible overview, I heartily recommend this Medium post by Fiontann O’Donnell from BBC News Labs). It is worth saying that Fairness, Accountability, and Transparency are not necessarily the only criteria one could use, of course, but they are a useful starting point.

Fairness

Fairness is a concept that sits at the heart of the work of many CSOs; so the fact that it is also one of the key concerns about the implementation of AI should immediately suggest that civil society has something relevant to bring to the table. But using a word like “fair” potentially begs an awful lot of questions such as “fair to whom?” And “fair in what way?”

Luckily, academics have begun to dig into some of these issues and to parse the concept of fairness when it comes to ML systems (such as in this great paper by Skirpan and Gorelick (2017)). What this leads to is not a uniform notion of fairness, but a series of context-relevant sub-questions that can often be assessed in more practical ways.

For example, in the first instance before we have even started building an ML system, we need to ask some fundamental questions such as:

  • Is it fair to apply ML in this context at all?
  • Do the risks evidently outweigh any potential gains?
  • Have the people and communities that this system will affect been given an opportunity to voice any concerns?
  • Are there demographic or cultural considerations that should give us cause for concern?
  • Does the system inherently require data that could compromise the privacy or rights of certain individuals or groups?

This is a point at which civil society clearly has a role to play. CSOs will be able to bring relevant insights to human rights and civil liberties issues, and knowledge of marginalized groups and communities. But perhaps even more importantly, unlike technologists, CSOs have no particular prior interest in building ML systems. This is crucial, because the question of whether it is appropriate to use ML at all must be on the table. The concern—if ethical and moral questions about issues like fairness are left solely to technologists—is that either tacitly or explicitly it will be taken as a given that the end result of considerations will involve the creation of an ML system; and that all that is in question is what kind of system. However, this immediately misses one of the fundamental dimensions of fairness.

Once all parties are agreed that there is a justifiable case for building an ML system in a particular instance, we can then move on to considerations about how that is done fairly in practice. For instance:

  • Can we involve the people and communities the system will affect in the design process to minimize any potential negative impacts?
  • Can we control for statistical bias in the training sets of data, so that the algorithms do not develop biases?
  • What mechanisms are there for ensuring that the operation of the system is actually understandable by the people it affects? (We will come back to this shortly when we consider transparency).
  • Do we need to involve external organizations in the oversight of the system on an ongoing basis, and how do we achieve this?

Finally, we need to give consideration to how we test our system for fairness on an ongoing basis; because even if we diligently follow the first two steps outlined above, the chances are that there will still be unforeseen consequences over time. For instance, the algorithmic processes might affect the target groups we have already identified, but in ways that we never guessed. Or the system may—through widespread adoption or interaction with other systems—come to affect other groups that we had not considered. That is why we need to keep assessing the impact of the ML systems we use, rather than seeing it as a one-shot deal.

Accountability

We have already touched on some of the questions we might ask about oversight or governance mechanisms that might be necessary to ensure fairness. But fairness is not the only consideration when it comes to AI. We also need to address the wider question of how we make algorithmic systems accountable when things go wrong, or when people take issue with the decisions they make or the outcomes they produce.

There are obviously major questions of law, regulation and governance here. They are ones that many governments, academics and others around the world are grappling with. Solutions proposed so far range from tough new laws governing ML systems to the creation of kitemarking systems for algorithms or new bodies that could defend user interests (like an AI-focused Which?).

There is a clear need for civil society to play a role here too. CSOs often represent the most marginalized people and communities within society, so they need to be in a position to speak out on their behalf when it comes to the impact of AI systems and to hold those responsible to account when necessary. Likewise the the application of ML systems in some contexts will have serious implications at a societal level for many of the kinds of issues that are core to the work of civil society such as human rights, civil liberties, equality, and poverty. Hence it is vital that CSOs are able to identify those responsible for any deliberate or unintended negative consequences and hold them to account.

Some of this may require new policy and legislative frameworks, but often these are likely to be extensions of existing international law and governmental agreements, so it is crucial that CSOs with relevant expertise in the current context adapt what they know and lend their weight to any advocacy efforts that may be required. There are some good examples of this already happening: Human Rights Watch, for instance, has joined a campaign to ban the development of autonomous weapons, while Amnesty International and Access Now recently published their “Toronto Declaration,” aimed at setting principles which ensure the right to non-discrimination and equality in ML systems.

Transparency

The other thing that is often cited as a key component of ethical AI is transparency. At its most basic level, this simply means that people should be able to determine that an AI system is being used. This might sound obvious, but evidence shows that many people are still unaware of the role that algorithmic processes play in determining the content they are presented with on platforms like Netflix, Spotify or YouTube; or that their Facebook and Twitter feed is likewise determined by highly complex machine learning algorithms that are constantly adapting and evolving based on their behavior. (One study analyzing awareness and understanding of the way in which Facebook’s news feed works (Eslami et al. (2015)), for instance, found that well over half of participants (62.5%) were not aware that their feed was curated by an algorithm, and that when they were informed they were not happy).

And not realizing that AI is involved in a given context may not be simply down to ignorance on the part of the user: there are worrying examples in which tech companies seem to suggest that their aim is to conceal this fact deliberately. Google, for example, showcased a new feature called “Duplex” in its voice assistant at a conference earlier this year, which can make simple phone calls (to reserve a table at a restaurant, say) on behalf of its owner. This immediately drew a storm of criticism for the fact that the system was designed to hide the fact that it was a machine talking (probably not helped by the rapturous, whooping reception the demonstration received from the roomful of developers either). Google subsequently did a swift retreat and announced that future versions of the feature would announce that an automated system is being used, but the potential future danger had been clearly highlighted by that point.

The challenge of knowing when an AI system is being used is also likely to get more widespread as our interactions with a broad range of technology come to rely on conversational interfaces (like Alexa or Siri), or virtual and augmented reality interfaces. These are likely to seem as though they are presenting us with an objective view of the available options, yet in reality they are underpinned by algorithms which filter our experience according to criteria that may well remain hidden. CSOs could play a valuable role simply by educating the communities they work with about the role that algorithms play in many of the platforms and interfaces they rely on.

But beyond simply being transparent about the fact that algorithms are being used, many also argue that there needs to be transparency about the algorithms themselves. This links back to the earlier goals of fairness and accountability. The theory goes that by making ML systems transparent, the people they actually affect will be able to see how they work and why certain decisions have been taken; and will therefore be able to identify instances where they feel they have been treated unfairly and seek recourse accordingly.

The problem is that in reality, transparency is likely to be far harder to achieve. In part this is because many applications of ML take place in contexts that are considered commercially sensitive, so explicit efforts are made to keep the underlying workings of the system opaque. It may be that this challenge can be overcome through new legislation or regulation requiring openness in contexts where there is a demonstrable public interest, but not everyone would agree that this is desirable; and in any case it certainly isn’t likely to happen without a significant amount of hard work in terms of lobbying and advocacy by civil society organizations and others.

But even if platforms and technology firms are willing to make the inner workings of their systems totally open, it is not clear that this would actually result in transparency in any sense in which we would ordinarily intend it. Presented with vast reams of technical data on a machine learning system, most of us would be none the wiser in terms of understanding how a particular decision had been arrived at. And this might not simply be a deficiency in our own understanding: it is possible that no amount of additional technical expertise would actually help. Machine learning systems are very good at identifying patterns in data, but they primarily do it through a vastly-iterated process of trial and error which doesn’t involve any deeper ‘understanding’ of why the pattern has emerged, and there may be no way of inferring any such understanding from the system. Furthermore, if we are talking about systems that incorporate unsupervised learning (see this previous blog for a more detailed discussion of what this means), the AI could arguably be seen as relying on modes of thinking that are entirely non-human and thus we could never understand them.

So should we just give up and go home? Well, no, not necessarily. Challenges such as these have led many to suggest that instead of straightforward transparency, what we should actually be demanding is something more akin to “explainability” or “interpretability” i.e. that those who are affected by algorithmic processes are able to get access to some meaningful account of how decisions have been taken, which enables them to take relevant action to challenge the decision or ensure that a similar one does not occur in future.

How would this work in practice, you might well ask? That is an area where a lot of interesting research is currently going on: for instance the development of algorithms that are able to “explain themselves”; or the use of counterfactual scenarios to give people a meaningful sense of how things might have been different had a given algorithmic decision not been taken. (For a useful overview of how humans might understand explanations of ML systems, see Narayanan et al. (2018)). It is not clear whether any one of these approaches is going to solve the problem: most likely it will be a combination of elements of all of them. But from the point of view of civil society organizations who see the need to engage in these issues and are evaluating what role they could play, the key point is that there are practical approaches being developed so they don’t need to start from scratch by any means.

Where Next?

It seems pretty clear to me that CSOs could have a valuable role to play in all of the various stages of ensuring fairness, accountability, and transparency when it comes to the use of AI. However, we cannot simply assume that this will happen naturally. Instead, is it likely to take an awful lot of careful thought and intermediation to ensure that they are able to engage in the design, implementation, and evaluation of ML systems. But if the ongoing pronouncements from governments and the tech industry about the desire to ensure that AI is “ethical” are genuine, then I would argue that engaging civil society is an absolute necessity, and that the effort or cost of enabling CSOs to play their part will be more than justified.


Rhodri Davies is Head of Policy at Charities Aid Foundation (CAF) and author of Public Good by Private Means, a book on the history of British philanthropy published in 2016. Davies leads Giving Thought, CAF’s think tank focusing on current and future issues affecting philanthropy and civil society.

Comments

  1. Admiring the persistence you put into your blog and detailed information you offer. It’s great to come across a blog every once in a while that isn’t the same old rehashed material. Excellent read! I’ve saved your site and I’m adding your RSS feeds to my Google account.

  2. This is really interesting, You are a very skilled blogger.
    I’ve joined your feed and look forward to seeking more of your wonderful post.
    Also, I have shared your web site in my social networks! asmr
    0mniartist

  3. If some one desires to be updated with most up-to-date technologies after that
    he must be pay a quick visit this website and be up to date daily.
    asmr 0mniartist

  4. I like the valuable info you provide in your articles.
    I’ll bookmark your blog and check again here regularly.
    I’m quite sure I will learn many new stuff right
    here! Good luck for the next! 0mniartist asmr

  5. When I originally commented I clicked the -Notify me when new comments are added- checkbox and now each time a comment is added I get four emails with the same comment. Is there any way you can remove me from that service? Thanks!

  6. I need to to thank you for this excellent read!!
    I definitely enjoyed every little bit of it. I’ve got you saved as a favorite to
    check out new things you post…

  7. you are in point of fact a just right webmaster.
    The web site loading pace is amazing. It sort of feels
    that you are doing any distinctive trick. Moreover, The contents are masterwork.

    you have performed a great task in this topic!

  8. May I simply just say what a comfort to find an individual who
    actually knows what they are discussing on the net.
    You certainly understand how to bring an issue to light and make it important.

    More people should read this and understand this side of your story.
    I can’t believe you aren’t more popular since you surely possess
    the gift.

  9. I think this is one of the most significant information for me.

    And i’m happy studying your article. But wanna commentary on few normal issues, The web
    site taste is great, the articles is really great : D.

    Just right task, cheers

  10. When I initially commented I appear to have clicked on the -Notify me when new comments are added- checkbox and from now on whenever a comment is added I get 4 emails with
    the exact same comment. Perhaps there is a means
    you are able to remove me from that service? Thank you!

  11. It’s actually a nice and helpful piece of information. I am satisfied that you just shared this useful information with us. Please keep us informed like this. Thank you for sharing.

  12. Hello there, You’ve done an excellent job. I will certainly digg it and personally suggest
    to my friends. I’m sure they’ll be benefited from this web site.

  13. What’s up everyone, it’s my first pay a visit at this
    website, and article is really fruitful in favor of me, keep up
    posting these articles.

  14. Hello there, You’ve done a fantastic job. I’ll definitely digg it
    and personally recommend to my friends. I am sure they will be benefited from this website.

  15. What i don’t realize is in fact how you’re now not really much more neatly-appreciated than you may be now. You’re so intelligent. You realize therefore significantly when it comes to this matter, made me in my view imagine it from numerous numerous angles. Its like men and women don’t seem to be involved unless it is something to do with Woman gaga! Your own stuffs outstanding. At all times care for it up!

  16. You could definitely see your skills in the work you write. The world hopes for even more passionate writers like you who aren’t afraid to say how they believe. Always go after your heart.

  17. scoliosis
    I really love your site.. Excellent colors & theme. Did you
    create this site yourself? Please reply back as I’m trying to
    create my own personal site and would love to know where you got this from or
    exactly what the theme is called. Thanks! scoliosis

  18. scoliosis
    I’ll right away take hold of your rss as I can’t to find your e-mail subscription hyperlink or newsletter service.
    Do you’ve any? Kindly allow me recognise so that
    I could subscribe. Thanks. scoliosis

  19. dating sites
    Hiya very cool web site!! Guy .. Excellent .. Wonderful .. I will bookmark your website
    and take the feeds additionally? I’m satisfied to find
    a lot of helpful info right here within the submit, we want develop more
    techniques on this regard, thanks for sharing.

    . . . . . dating sites

  20. scoliosis
    I feel that is among the so much important information for me.
    And i’m glad reading your article. However want to observation on few normal things, The site taste is ideal, the articles is truly
    excellent : D. Just right job, cheers scoliosis

  21. My programmer is trying to persuade me to move to .net from PHP.
    I have always disliked the idea because of the expenses.
    But he’s tryiong none the less. I’ve been using
    WordPress on a variety of websites for about a year and am
    nervous about switching to another platform. I have heard fantastic things about blogengine.net.
    Is there a way I can import all my wordpress content into it?
    Any help would be greatly appreciated!

  22. you’re actually a good webmaster. The site loading speed is amazing.
    It seems that you are doing any unique trick. Also, The
    contents are masterpiece. you’ve performed a excellent task in this matter!

  23. Aw, this was an exceptionally good post. Taking a
    few minutes and actual effort to make a really good article… but what can I say… I put things
    off a lot and don’t seem to get anything done.

  24. That is the right weblog for anyone who needs to find out about this topic. You realize a lot its virtually arduous to argue with you (not that I truly would need…HaHa). You positively put a brand new spin on a subject thats been written about for years. Great stuff, just great!

  25. I feel this is one of the so much vital info for me. And i am satisfied reading your article. However want to statement on few common things, The site style is ideal, the articles is truly excellent : D. Good task, cheers

  26. hello!,I like your writing so much! share we communicate more about your post on AOL? I need an expert on this area to solve my problem. Maybe that’s you! Looking forward to see you.

  27. Amazing! This blog looks exactly like my old one! It’s on a completely different topic but it has pretty
    much the same page layout and design. Wonderful choice of colors!

  28. Hello! This post could not be written any better!
    Reading through this post reminds me of my good old room mate!
    He always kept talking about this. I will forward this
    post to him. Fairly certain he will have a good read.
    Thanks for sharing!

  29. Hi, i think that i noticed you visited my web site thus i got here to go back the favor?.I’m trying to in finding issues to improve my
    web site!I suppose its adequate to make use of a few of your ideas!!

  30. Hello just wanted to give you a quick heads up and let you know a few of the pictures aren’t loading correctly. I’m not sure why but I think its a linking issue. I’ve tried it in two different internet browsers and both show the same outcome.

  31. Greetings! This is my first visit to your blog!
    We are a team of volunteers and starting a new initiative in a community in the same niche.
    Your blog provided us useful information to work on. You have done a wonderful job!

  32. Hey there just wanted to give you a quick heads up.
    The text in your article seem to be running off the screen in Safari.
    I’m not sure if this is a format issue or something to do with browser compatibility but I thought I’d post
    to let you know. The design and style look great though!
    Hope you get the problem solved soon. Kudos

  33. May I just say what a relief to uncover an individual who genuinely understands what they’re discussing on the internet.
    You certainly realize how to bring an issue to light and make it important.

    More people ought to read this and understand this side of your story.
    I can’t believe you’re not more popular because you certainly possess the gift.

  34. Thank you for any other informative website. The place else may I get that type of information written in such a perfect means? I have a mission that I am simply now running on, and I’ve been at the look out for such information.

  35. Excellent beat ! I wish to apprentice while you amend your website, how can i subscribe for a blog site? The account aided me a acceptable deal. I had been tiny bit acquainted of this your broadcast offered bright clear idea

  36. I was curious if you ever considered changing the layout of your site? Its very well written; I love what youve got to say. But maybe you could a little more in the way of content so people could connect with it better. Youve got an awful lot of text for only having one or 2 pictures. Maybe you could space it out better?

  37. Thank you, I’ve recently been looking for info approximately this topic for ages and yours is the greatest I’ve found out till now. However, what in regards to the conclusion? Are you certain in regards to the source?

  38. Nearly all of whatever you state is supprisingly appropriate and that makes me wonder the reason why I had not looked at this in this light before. This particular piece truly did switch the light on for me as far as this particular subject matter goes. However at this time there is actually one factor I am not necessarily too cozy with and whilst I attempt to reconcile that with the core theme of the point, let me see just what the rest of the visitors have to say.Well done.

  39. Greetings! Very helpful advice on this article! It is the little changes that make the biggest changes. Thanks a lot for sharing!

  40. Howdy very nice blog!! Guy .. Excellent .. Amazing .. I’ll bookmark your web site and take the feeds additionallyKI am glad to seek out so many helpful info right here within the put up, we’d like work out more techniques in this regard, thank you for sharing. . . . . .

  41. Nice blog right here! Additionally your site quite a bit up very fast! What web host are you the use of? Can I am getting your affiliate link on your host? I desire my web site loaded up as quickly as yours lol

  42. 935680 559984Spot on with this write-up, I need to say i believe this superb site needs significantly a lot more consideration. Ill probably be once once more to learn a fantastic deal a lot more, several thanks that details. 574155

  43. Hi there! I’m at work surfing around your blog from my new iphone! Just wanted to say I love reading through your blog and look forward to all your posts! Keep up the great work!

  44. Hello There. I found your blog using msn. This is a very neatly written article. I will make sure to bookmark it and come back to learn extra of your helpful info. Thank you for the post. I’ll definitely return.

  45. What a material of un-ambiguity and preserveness
    of precious knowledge on the topic of unpredicted emotions.

  46. Great work! This is the type of info that should be shared around the internet. Shame on Google for not positioning this post higher! Come on over and visit my website . Thanks =)

  47. In this awesome pattern of things you get an A+ just for effort and hard work. Exactly where you actually confused us was on your particulars. As it is said, the devil is in the details… And that could not be more correct in this article. Having said that, allow me say to you exactly what did give good results. The text is actually very engaging which is possibly the reason why I am making the effort in order to opine. I do not really make it a regular habit of doing that. Next, despite the fact that I can see a jumps in reasoning you make, I am not necessarily convinced of exactly how you seem to unite your details which produce your conclusion. For now I shall subscribe to your point however wish in the future you actually link your facts much better.

Leave a Comment