By translating images into the language spoken by
object-recognition systems, then translating them back, researchers hope to
explain the systems’ failures.
Object-recognition systems — software that tries to identify
objects in digital images — typically rely on machine learning. They comb
through databases of previously labeled images and look for combinations of
visual features that seem to correlate with particular objects. Then, when
presented with a new image, they try to determine whether it contains one of
the previously identified combinations of features.