The Wisdom of Crowds

When we talk about “outsourcing,” we mean asking a well-defined and select group of individuals, generally receiving a salary, to solve a task.¹ This activity is a cornerstone of the business world and has become extremely common in the last 20 years. “Offshoring” is a different process because, unlike outsourcing, in it, the functions of a company or business are relocated to a different country. Not surprising, offshoring can be a political hot potato, while outsourcing rarely raises eyebrows. In our profession, one can view the interpretation of images done at night elsewhere as outsourcing because the process of obtaining them and the equipment remains with us. We are simply buying a service to make our lives easier.

However, when India and South American countries begin building “medical cities” and our patients go there to obtain imaging studies, offshoring of radiology occurs. When both of these activities become a threat to our economic well-being, the opposite (“insourcing”) may occur—that is, we hire more radiologists or pay extra to some already working in our departments to stay overnight and interpret studies. Insourcing is a great way of controlling costs because it tends to use existing infrastructure or adds to it relatively cheaply. The most extreme way of insourcing is “backsourcing,” which happens when all activities that were once done outside (or offshore) are brought back in-house. In addition, of course, if one does backsourcing at a national level, then it becomes “onshoring.” One aspect that defines all these activities is the people doing them. Professionals who do work via outsourcing, offshoring, insourcing, and onshoring are groups of paid employees specifically hired to work at a task and many times are experts (or least, have some experience) at what they do.

Lately, a very different outsourcing model has become popular: crowdsourcing. In crowdsourcing, a nonspecific group of individuals, most unpaid, are given the chance to solve a task. If we can string a long line of computers to make work faster and more exact, the same idea can be applied to humans. In crowdsourcing, the “crowd” part refers to the users, and most are volunteers who are not paid for their services but receive some type of recognition. The entity that looks for the answers is the “crowdsourcer,” and when the issue is solved, it will own the solution.² It would be entirely possible to ask neuroradiologists to capture images from patients with aneurysms, perform computer-flow analysis via freeware, interpret the data obtained, and post their results on the Web site of the project. This would be an example of “community science” or crowdsourcing in which no one would be paid but all would contribute to the understanding of why aneurysms grow and rupture.

The first ever use of crowdsourcing is said to have been the Oxford English Dictionary (Oxford University Press, 1928), in which the entire British community contributed to index and define all words in the English language, a fascinating story told by Simon Winchester in The Professor and the Madman.³ However, crowdsourcing did not become truly efficient until the advent of Web 2.0. Because of its reach and instantaneous nature, Web crowdsourcing has become popular. The appeal of crowdsourcing is the concept that collective intelligence is better than individual, and this forms the basis of the “wisdom of the crowd” principle. Dr Sarah Fortune, an Associate Professor of Immunology and Infectious Diseases at the Harvard School of Public Health, used crowdsourcing to identify cells resistant to tuberculosis drugs.⁴ The only way she could have expedited her research was by hiring 100 graduate students or crowdsourcing it via the Web; she chose the latter. Using a specific computer program (Crowdflower; http://crowdflower.com), Dr Fortune was able to recruit 1000 on-line workers who looked at her cell images (costing her only a few pennies per image).⁵ She used Crowdflower, a crowdsourcing service that, as of this writing, had already done more than 775 million tasks (you can watch the number of tasks being performed in real-time at their Web site).

Crowdsourcing is bound to become an important way of doing research in the future (a search of the term “crowdsourcing in medicine” using Google Scholar yielded almost 2000 related items). When data produced by crowdsourcing are evaluated, most experts express high levels of confidence in the results.⁶ The pharmaceutical industry is looking into crowdsourcing as a means of encouraging alternative models of drug discovery.⁷ Using this model, drug companies are poised to save billions of dollars. Eli Lilly and Dupont have already used it to solve vexing research and development problems.⁸ Although these 2 companies offered money to those who solved their issues, the rise of crowdsourcing is based on the fact that everyday folks do it in their “spare cycles” (techie parlance for “free time”), and they represent an enormous new pool of cheap or free labor.⁹ Eli Lilly funded InnoCentive, a crowdsourcing platform now used by Boeing and Proctor and Gamble among others, with incredible results: Thirty percent of problems found on their Web site have been solved by no other than hobbyists! These collaborators are paid somewhere between US $10,000 and $25,000 when they solve a problem, which is cheaper than hiring a scientist to do it. As sales of many products decrease, research and development departments become too expensive to run and are no longer affordable, and crowdsourcing their functions becomes an attractive solution.

It seems to me that imaging research would be an ideal platform for crowdsourcing. Images can be cheaply and rapidly transmitted, exchanged, and analyzed, and data could be entered into an open Web site and modified as needed once it is there (Wikipedia works this way). In my experience, radiologists are always willing to give their opinions, contribute with their own cases, and have a good fund of knowledge of computers. In an interesting study, virtual colonoscopy studies were assessed with computer-aided detection and also made available to 228 “knowledge workers” via a crowdsourcing platform.¹⁰ When it came to identifying polyps, both systems produced similar results (about 85% each). In this study, the so-called “knowledge workers” (euphemism for laypeople) received only minimal training while the computer-aided detection was highly specialized. The corresponding author noted that radiologists may be too busy to participate in this type of experiment despite the fact that these tasks generally take less than 1 minute to complete. Money may have something to do with this. If one looks at the different pay schemes in several crowdsourcing platforms, it is obvious that one cannot make a living from them, and as an example, users of Mechanical Turk (http://aws.amazon.com/mturk/; the for-pay crowdsourcing Amazon platform that has over 400,000 registered workers) make less than the minimum wage.¹¹

As the financial situations of our governments continue to deteriorate, it seems logical to look for research funding elsewhere, perhaps in the general public. President Obama funded most of his first campaign with small contributions from a multitude of people. This is what is called “crowdfunding,” and the government is not too happy with it. Regulation of crowdfunding is a nightmare for the US Securities and Exchange Commission; and as of this writing, it had not been able to enact laws to define and regulate it.¹² In reality, if no more than $1 million is raised in this way, regulations are still pretty loose. Nevertheless, big brother is watching, and the US government has established 3 organizations related to crowdfunding (National Crowdfunding Association, Crowdfunding Professional Association, and CrowdFund Intermediary Regulatory Advocates).

Kickstarter is the largest crowdfunding Web platform and has raised over US $100 million for the funding of creative projects.¹³ Kickstarter posts a “daily project” and attempts to get donations for it from everyone. None of its projects involve the medical field, and these vary from workshops for science fiction and fantasy writers (106% funded) to making better marshmallows (1501% funded). If you click on a project. you can see its number of backers, the pledge goal, total collected money, and the days left to fund it. Crowdfunding seems to work well for disaster relief, support of artists, political campaigns, software development, and even scientific research. Unlike the traditional fundraising events that most universities and institutions hold, crowdfunding occurs outside the geographic boundaries. Most of the typical funding events are meant to bring the donor and recipient together with the idea that personal contact will entice donors to offer financial support. It seems that crowdfunding erases this aspect because the average distance between donors and recipient in crowdfunding platforms is, on average, 3000 miles.¹⁴ This may work for small-amount donors, however not for those who give us millions of dollars. Patients have also started using crowdfunding to pay their medical bills.¹⁵

There are 2 general crowdfunding platforms for science projects. The first one established was the Science Fund Challenge with a current rate for funding projects of 135% versus 20% for the federal government.¹⁶ This site is part of another larger one, RocketHub, that attempts to collect money for soft and hard sciences.¹⁷ Petridish (http://www.petridish.org) is perhaps one of the most serious enterprises that entices donors to “explore the world with renowned researchers,” and Healthtechhatch (https://www.healthtechhatch.com) was, as of the time of this writing, beta-testing its site dedicated to crowdfunding in medicine. It seems that the number of crowdfunding platforms for medicine projects is exploding. GiveForward (www.giveforward.com) labels itself as the “The #1 Medical Fundraising Site,” a claim that is probably true because they have raised over US $22 million since its recent inception. Other sites include www.medstartr.com and www.iamscientist.com. As expected, levels of suggested donations are higher on these than on RocketHub, and donors have the chance to give their money to respected institutions and researchers (on the homepage of Petridish, one can find projects from the Cold Spring Harbor Laboratory, State University of New York, California State University, and many more). There, one can also access the biographies and curricula of the investigators. Some are PhD candidates looking for funds to complete their dissertations.

Rather than just getting money for research, these crowdfunding sites have a more complex and perhaps positive societal influence: outreach—that is, they let the general public know what is going on with the sciences. Additionally, researchers must exactly detail on the Web what the money is used for, creating a greater transparency in the public eye than that seen with grants given by the government. Obviously, crowdfunding is not the solution for the ever-dwindling federal funding for research, but it is clearly an alternative on a smaller scale. Crowdfunding supports the concept of “open notebook science,” in which all data generated in experiments are made available on-line and, thus, are completely transparent. As the expense of delivering and advancing medicine continues to increase, participation by all involved, including patients, may be one answer to understanding the real costs and complexity of research. The Society for Participatory Medicine (and its journal) encourages this type of activity.¹⁸ Maybe it is time to allow the lay public and our patients to participate in our academic activities so that they can see what we do (because we do it so well). Our research has always been hampered by the small amount of patients whom we are able to reach for donations, and crowdsourcing projects may be an answer to this problem. Additionally, our professional societies could begin to explore the possibility of crowdfunding, or at least we should post some of our projects on Web sites dedicated to this activity.

• © 2013 by American Journal of Neuroradiology