Whoa! Is that invisible gorilla headed for the courtroom?
We’ve been not seeing it everywhere. Who knows how many invisible gorillas you’ve missed? And, it’s not just you. How about expert observers (like radiologists) not seeing those invisible gorillas who are taking up 42 times more space on their X-rays than a typical tumor? This is so not good. We probably need write no further. Because you know jurors are going to hate this too. But, we know you want to know more and to understand how this really could come to a courtroom near you. Just, hopefully, not when you are Defense. You probably find yourself wanting to skip ahead to see the ambiguous “gorilla” in the X-rays so instead of purposely taunting you, here it is now (taken from the article itself).
Not too ambiguous now, is it? Imagine explaining to a jury how your expert missed that as they carefully and fastidiously scoured the image looking for nodules indicative of lung cancer. It’s enough to cause a panic attack.
So. Let’s take a look at what the researchers did. Radiologists’ tools have become much more sophisticated. Instead of a single flat image, they now look at “reconstructed stacks of axial slices of the lung for nodules that appear as small light circles”. A typical stack of images from a chest CT scan, according to the authors, has between 100 and 500 “slices”, so radiologists are used to looking at potentially hundreds of images as they assess for the presence of the small light circles indicative of cancer.
The stack of research slices containing the gorilla had 239 slices and the gorilla was on 5 of them with fading and growing opacity–we’ve cropped Figure 1 in the article so you can actually see what the research participants saw. The gorilla was about the size of a book of matches. So the participants did not have just one chance to see the [relatively gigantic] gorilla. They had five chances.
Here is the baddest of the bad facts:
“In the scans used in this study, which were taken from the Lung Image Database Consortium, the average volume of the lung nodules was 153 mm. Thus, the gorilla was more than 48 times the size of the average nodule in the images.”
In the first experiment, 20 of 24 radiologists failed to report seeing a gorilla (that’s 83%!) as they “freely scrolled through the slices containing the gorilla an average of 4.3 times”. On the other hand, they did detect 55% of the lung nodules present.
So the researchers wondered if “naive observers” (like your jurors) would fare better with the guerilla-gorilla. All 25 naive observers missed the gorilla too, but (like the radiologists in Experiment 1), they all saw it when shown Figure 1 above. Naive observers, did less well than the radiologists in identifying the cancer nodules. They only detected about 12% of the lung nodules present, compared to the 55% identified by the radiologists. (The researchers say this is “reassuring” and we would vigorously concur–however, only 55%?!.)
Then, the researchers wanted to see if the gorilla was simply too difficult to see in the stack of slices from chest CT images. They selected another group of “naive observers” and had them look at the slices. (Apparently radiologists are too busy for this sort of thing, or they were so ticked off after the first experiment, they did not return the researcher’s calls.) The researchers put the slices into a “movie” for Experiment 3, and automatically advanced them, showing the slices at two different speeds (either 35 milliseconds per frame [fast] or 70 milliseconds per frame [slow]). Regardless of whether the slices were shown quickly or slowly, 88% of the research participants saw the gorilla.
The researchers say their results indicate that “under certain circumstances, experts can also miss the presence of a large, anomalous stimulus”. (They are referring to the gorilla. I guess they view a gorilla as anomalous in a lung tissue scan.) Later, and a bit more clearly, they also say, “Clearly, radiologists can miss an abnormality that is retrospectively visible when the abnormality is unexpected”. (Again, they are referring to the relatively huge gorilla one would never expect on a chest X-ray.) Finally, the researchers conclude, “It would be a mistake to regard these results as an indictment of radiologists. As a group, they are highly skilled practitioners of a very demanding class of visual search tasks.”
We can hear the jurors now.
“That radiologist only saw 55% of the cancer in that X-ray? Please. S/he is obviously incompetent.”
“I can see the gorilla. Why would an expert miss it if s/he was competent?”
“Obviously, this radiologist was just putting in time and not trying very hard to read the information accurately.”
“I’m confused. Do a lot of people have gorillas in their lungs?”
If this case comes across your desk as one you need to defend, it would be a good time for an extended vacation. Or early retirement. Missing really big gorillas in a lung scan could result in awards that are even bigger.
But ultimately, this particular scenario in its research form would only occur in a radiology residency prank, or an April Fool’s Day prank. The takeaway is quite real, though. Dancing gorillas, phantom gorillas, sleights of hand and latent biases all produce impressions based on what we expect, not what is in front of us. To liberally paraphrase Simon and Garfunkel,
“A man [sic] sees what he [sic] is prepared to see and disregards the rest.”