Crowdsourcing Malaria Parasite Quantification: An Online Game for

Original Paper

Crowdsourcing Malaria Parasite Quantification: An Online Game for Analyzing Images of Infected Thick Blood Smears

Miguel Angel Luengo-Oroz^1,2, PhD; Asier Arranz³, MEng; John Frean^4,5, MBBCh, MMed

¹Biomedical Image Technologies group, DIE, ETSI Telecomunicaci?n, Universidad Polit?cnica de Madrid, CEI Moncloa UPM-UCM, Madrid, Spain
²Biomedical Research Networking Center in Bioengineering, Biomaterials and Nanomedicine, CIBER-BBN, Madrid, Spain
³Nebutek Soluciones SL, Vizcaya, Spain
⁴National Institute for Communicable Diseases, National Health Laboratory Service, Johannesburg, South Africa
⁵School of Pathology, Faculty of Health Sciences, University of the Witwatersrand, Johannesburg, South Africa

Corresponding Author:
Miguel Angel Luengo-Oroz, PhD

Biomedical Image Technologies group
DIE, ETSI Telecomunicaci?n
Universidad Polit?cnica de Madrid, CEI Moncloa UPM-UCM
ETSIT, Av. Complutense 30
Madrid, 28040
Spain
Phone: 34 913366827
Fax: 34 913367323
Email:

---

ABSTRACT

Background: There are 600,000 new malaria cases daily worldwide. The gold standard for estimating the parasite burden and the corresponding severity of the disease consists in manually counting the number of parasites in blood smears through a microscope, a process that can take more than 20 minutes of an expert microscopist?s time.
Objective: This research tests the feasibility of a crowdsourced approach to malaria image analysis. In particular, we investigated whether anonymous volunteers with no prior experience would be able to count malaria parasites in digitized images of thick blood smears by playing a Web-based game.
Methods: The experimental system consisted of a Web-based game where online volunteers were tasked with detecting parasites in digitized blood sample images coupled with a decision algorithm that combined the analyses from several players to produce an improved collective detection outcome. Data were collected through the MalariaSpot website. Random images of thick blood films containing Plasmodium falciparum at medium to low parasitemias, acquired by conventional optical microscopy, were presented to players. In the game, players had to find and tag as many parasites as possible in 1 minute. In the event that players found all the parasites present in the image, they were presented with a new image. In order to combine the choices of different players into a single crowd decision, we implemented an image processing pipeline and a quorum algorithm that judged a parasite tagged when a group of players agreed on its position.
Results: Over 1 month, anonymous players from 95 countries played more than 12,000 games and generated a database of more than 270,000 clicks on the test images. Results revealed that combining 22 games from nonexpert players achieved a parasite counting accuracy higher than 99%. This performance could be obtained also by combining 13 games from players trained for 1 minute. Exhaustive computations measured the parasite counting accuracy for all players as a function of the number of games considered and the experience of the players. In addition, we propose a mathematical equation that accurately models the collective parasite counting performance.
Conclusions: This research validates the online gaming approach for crowdsourced counting of malaria parasites in images of thick blood films. The findings support the conclusion that nonexperts are able to rapidly learn how to identify the typical features of malaria parasites in digitized thick blood samples and that combining the analyses of several users provides similar parasite counting accuracy rates as those of expert microscopists. This experiment illustrates the potential of the crowdsourced gaming approach for performing routine malaria parasite quantification, and more generally for solving biomedical image analysis problems, with future potential for telediagnosis related to global health challenges.

(J Med Internet Res 2012;14(6):e167)
doi:10.2196/jmir.2338

KEYWORDS

Crowdsourcing; Malaria; Image Analysis; Games for Health; Telepathology

Crowdsourcing methodologies leveraging the contributions of citizen scientists connected via the Internet have recently proved to be of great value to solve certain scientific challenges involving ?big data? analysis that cannot be entirely automated [1]. In the GalaxyZoo project, citizen scientists classified imagery of hundreds of thousands of galaxies drawn from the Sloan Digital Sky Survey and the Hubble Space Telescope archive [2]. Crowdsourced contributions can be achieved with different motivation strategies, such as micropayments or games. The ?serious games? concept refers to an intention not only to entertain users, but also to train or educate them [3]. The ?gamification? [4] of the crowdsourcing approach enables a higher motivation of the participants and, using the Internet as a vehicle, untaps an underexploited resource for scientific research [5,6]: it is estimated that 3 billion hours per week are spent playing computer and videogames worldwide [7]. For instance, Fold-It, an online game where players solve 3-dimensional puzzles by folding protein structures, has resulted in several breakthrough scientific discoveries [8-10]. Another recent growing trend is the use of crowdsourcing techniques for participatory health research studies in which individuals report in real time a variety of health conditions [11], providing a promising complement to traditional clinical trials. Considering crowdsourced image analysis, collective processing has been recently explored for earthquake damage assessment from remote sensing imagery [12]. However, this methodology has not yet been mainstreamed for biomedical image analysis.

In this context, analysis of microscopic images of malaria-infected blood samples is an appealing goal. Worldwide, there are more than 200 million malaria cases and approximately 800,000 deaths annually, mainly in children [13,14]. Careful optical microscopic examination of a well-stained blood film remains the gold standard for malaria diagnosis [15]. Confirmation of a negative diagnosis is ultimately dependent on the technician?s expertise and can take up to 20 minutes. In addition, as malaria prevalence decreases in one specific place over time, microscopy technician skills may now be needed in other regions. Fast, cheap, ubiquitous, and accurate diagnosis is a priority in the Agenda for Malaria Eradication [16]. Although automated processing methodologies have been used extensively for the analysis of digitized blood smears [17,18], currently there are no completely automated image processing systems that can achieve perfect parasite recognition [19-24]. The main problem in computer-aided malaria diagnosis is that algorithms are usually not very robust with respect to the variable appearance of the parasites and changing image acquisition conditions.

The goal of this research was to test the feasibility of a crowdcomputing approach for malaria parasite quantification in which nonexperts count parasites in digitized thick blood smears through an online game (crowdsourcing) and a decision algorithm combines the data generated by several players in order to achieve a collective detection with a higher accuracy rate than an individual analysis. This idea?gaming for distributed malaria image analysis?has been also explored in a recent study by Mavandadi et al [25], in parallel to and independently of this study. These researchers designed a video game and a processing pipeline to investigate whether nonexperts can assess if a single-cell image extracted from a digitized thin blood sample is infected with malaria or not. Although this study and the present research share a similar vision and goal, the research questions posed and solutions adopted differ substantially in terms of the data analyzed, the nature of the participants, the main task required of them, and the processing methodologies.

The proposed system in this study provides a new tool for parasite counting, but not malaria diagnosis, which is a more complex problem [26]. For this purpose, the microscopist protocol will need to be translated completely into a gaming protocol, including assessing the presence or absence of parasites, the parasite species, and growth stages and prognostic markers, such as schizonts or gametocytes, or pigment load. In the long run, crowdsourced remote telediagnosis from images acquired with optical microscopy and distributed worldwide through the Internet and possibly with systems that integrate the microscope into mobile phones [27,28], might have a potential impact for malaria-endemic countries because diagnosis availability and its cost could be optimized. However, in addition to the need for conventional laboratory processing and imaging equipment to prepare the material to a sufficiently high standard, this kind of analysis will require a communications infrastructure with enough bandwidth to distribute the images over the Internet and a critical number of online participants in order to ensure timely analysis of the images.

This work presents a proof-of-concept system that explores the feasibility of an online game-based, crowdsourced solution for malaria parasite quantitation in digitized images of thick blood smears.

We selected an image database of malaria-positive blood films that had been previously analyzed by experts to generate gold standards. These images were then incorporated into an online game. The player?s task was to click on the parasites. When a player found all the parasites present in 1 image (constituting a level) within a limited amount of time, the game continued by presenting a new image. Otherwise, the game was over. All the players? clicks were registered in a database. After 1 month, all the collected data was preprocessed in order to group all the clicks that players placed around the different objects in the image: parasites, white blood cells (leukocytes), and background noise. Finally, an algorithm that combined the different games to increase accuracy was developed and evaluated.

Ethics Statement

The malaria images used in this research were previously used to evaluate automated image analysis methods [20]. Original blood samples and resultant test images were collected and used with ethical approval from the Human Research Ethics Committee (Medical), University of the Witwatersrand, Johannesburg, South Africa (protocol number M051126). No new ethical review board approval was required since the digital images used in our work were not linked to any patient data or diagnosis and were digitally shared for microscopic training evaluation purposes. The data analyzed in this research were anonymously produced by online volunteers who agreed to play an Internet game. The participants were informed of the research purposes of the game on the game webpage.

Image Database

The image database was compiled from 28 Giemsa-stained thick films made from blood infected with malaria (Plasmodium falciparum) parasites, acquired using a 50? objective in a conventional laboratory optical microscope. Medium to low parasitemia images were selected for the game because of its design (1-minute games) and the fact that discrepancies between automatic counting methodologies and manual expert counting tend to be greater in low parasitemia cases. A gold standard mask image was generated for each of the 28 images to evaluate player performance.

Game Architecture

The objective of the MalariaSpot game was to tag as many parasites as possible in an image in 1 minute. The instructions?what is a parasite and what it is not?were briefly explained in the splash screen of the game website (Figure 1a). During the game, if the player found all the parasites in 1 image in the allowed time, a new image was presented (Figure 1b). Therefore, a player could analyze several images (levels) in a single game. In order to reinforce the game?s addictive nature, the players were given continuous feedback: each click was compared with the gold standard and an icon was placed immediately at the tag position to indicate a correct or incorrect selection. In addition, if the player misidentified an object and clicked in a wrong location (eg, on a leukocyte), the player was penalized by reducing the remaining time available to solve the level. Players were confronted with different, randomly selected test images. The difficulty of the levels increased as the time penalty for wrong tags grew with each level. As a motivation strategy, at the end of the game players were invited to register and provide their name, email address, and country in order to be included in the table of high scorers depicting the top daily, weekly, and monthly players.

Source: http://www.jmir.org/2012/6/e167/

doobie brothers jennie garth peter facinelli marques colston golden state warriors free agents nfl 2012 milwaukee bucks bear grylls