Eye in the Sky

Optical coherence tomography (OCT) scans can be used to diagnose and evaluate the effectiveness of various treatments and therapies for patients with Age-Related Macular Degeneration (AMD), the leading cause of severe visual impairment in the developed world. While there are treatments for some forms of wet AMD there is nothing to help the vast majority of patients suffering from the dry or degenerative form of AMD. One of the major obstacles in the development of an effective treatment is the inability to detect objective criteria that would either herald further progression of the disease or identify aspects that suggest improvement in patients undergoing treatment. While automated evaluation procedures exist, AMD OCT analysis often requires verification or modification by a trained OCT evaluator. With millions of OCT scans performed per year, demand for new evaluation approaches increases. As part of AMD, collagen and phospholipid vesicles known as drusen form between the retinal pigment epithelium (RPE) and Bruch’s membrane (BM). Below is a visual of an OCT retina scan and all of the layers that are present in each image

While there is no cure for AMD, some treatments have shown promise in slowing disease progression. Optical coherence tomography (OCT) is a valuable tool in the diagnosis and treatment evaluation of AMD.

Human computation has proven to be an effective way to crowdsource a variety of scientific problems, as well as leverage human pattern-recognition ability. Video games allow users to interact with the scientific data while also leveraging the elements game developers require to maintain engagement.

Eye in the Sky: Defender is a human computation game developed in partnership with BALANCED Media | Technology and deployed on HEWMEN platform to train players to evaluate AMD OCT images, by having players identify the RPE and BM as well as subretinal drusenoid deposits. Below is a trailer/tutorial of the human compuation video game. You can download and play the game by going to http://join.hewmen.com

In early prototype testing, results indicate individual players decreased their mean error score on a previously unknown image by 92.7% after 7 attempts. Averaging players’ input produced a decrease in mean error of 98.3% from a players initial image. These results suggest Human Computation games can train users to analyze AMD and OCT scans, encouraging expanded research.

The goal was to evaluate the effectiveness of each player the first time they analyzed a specific image. This allowed us to evaluate the learning rate of each player as they progressed to each new level. Below is a graph of the average error on each players first attempt to identify Bruch’s membrane boundry in the retina. An interactive verision of the image can be found here. the graph showed that players were able to quickly identify the boundary and after aproximately 7 levels (about 15-20 minutes of gameplay) they were identifying the boundary within the acceptable accuracy range.

A comparision between Level 1 and Level 7 can be seen below as well. The average error rate between the two images show that once players reached level 7, they results were within 1-2 pixels of the desired boundary.