Big Data & Society· 2016Q1
How the machine ‘thinks’: Understanding opacity in machine learning algorithms
- 2,546citations
- Q1SCImago
- 2016year
Short summary
This article distinguishes three forms of opacity in machine learning algorithms used for classification and ranking: corporate/state secrecy, technical illiteracy, and inherent algorithmic opacity. It argues that identifying the specific type of opacity is crucial for selecting appropriate solutions to prevent harm.
AI-generated from the title and abstract; the full text is not read.
Key points
- Identifies three distinct types of opacity in machine learning: secrecy, illiteracy, and inherent algorithmic complexity.
- Examines opacity in socially consequential applications such as credit scoring, fraud detection, and search engines.
- Proposes that differentiating opacity types is essential for choosing effective prevention strategies.
- Utilizes computer science literature, industry practices, and code audits for analysis.
AI-generated from the title and abstract; the full text is not read.
Abstract
This article considers the issue of opacity as a problem for socially consequential mechanisms of classification and ranking, such as spam filters, credit card fraud detection, search engines, news trends, market segmentation and advertising, insurance or loan qualification, and credit scoring. These mechanisms of classification all frequently rely on computational algorithms, and in many cases on machine learning algorithms to do this work. In this article, I draw a distinction between three forms of opacity: (1) opacity as intentional corporate or state secrecy, (2) opacity as technical illiteracy, and (3) an opacity that arises from the characteristics of machine learning algorithms and the scale required to apply them usefully. The analysis in this article gets inside the algorithms themselves. I cite existing literatures in computer science, known industry practices (as they are publicly presented), and do some testing and manipulation of code as a form of lightweight code audit. I argue that recognizing the distinct forms of opacity that may be coming into play in a given application is a key to determining which of a variety of technical and non-technical solutions could help to prevent harm.
The authors' abstract, as published at the source. Big Data & Society, 2016 · DOI ↗
Continue with a free account
Ask the paper: 3 free questions a day about this paper; save it, get its citation, new summaries every day for your field. Takeaways are Premium.
Continue free on the webSign in with Google or Apple; no card needed. You come back to this paper.
On your phone:
Field: Safety Research
Safety ResearchSocial Sciences