Orange sheep, ML and sharpening data

Janelle Shane posted these two pictures, depicting some goats on a tree and some orange sheep on a green field. Some AIs mislabeled them as “a flock of birds” and “orange flowers in a field.

This problem, I argue, is a consequence of a low-frequency image recognition. In this post, I propose a solution to facilitate recognition.

Contextual information

The artificial intelligences working on the images above mislabeled them because they relied too much on contextual information (the tree setting, the green field and the orange color) to infer what was in an image.

Over relaying on contextual information is a symptom of seeing images after they are passed through a low-pass filter (in other words, of seeing blurred images). To demonstrate this point, please see the example below. The two images below are the blurred versions of the previous ones (blurring = passing images through a low-pass filter). As you can see, it’s easy to classify the images like the artificial intelligences did: the first blurred image looks like some birds on a tree, and the second one looks like some orange flowers in a field.

Seeing only low-frequency information causes the observer (human or machine) to over rely on contextual data (low-frequency data) and to neglect details (high-frequency data).

Blurring images where there are details at odd with the context will cause the observer to neglect the details and to jump to the wrong conclusions.

The solution is thus to sharpen the images. How can this be done?

Sharpening by reducing scope

There is a relationship between scope and sharpness (explained in the first part of this article, where magnifying glass = high-pass filter). Imagine looking at the image of the orange sheep in the green field without your glasses and seeing it blurry. To better see the sheep, you need to use a magnifying glass. However, if you use the magnifying glass, you will only be able to see a small area of the image (example: in the image below, the magnifying glass allows you to see the letters better, but you can only see a few at a time). To increase sharpness, scope has to be reduced.

In my latest paper (https://psyarxiv.com/xqdtb; section “Denser cortical column fields cause a High-Pass Filter Effect”) I explained how there is a relationship between sharpness, scope and number of neurons. Simplifying, sharpness / scope = number of neurons. If the number of neurons is kept constant, to increase sharpness, scope has to decrease. Applied to convnets, a common machine learning algorithm, it is possible that this means: “smaller subsets of the original image have to be fed to the convnet”.

For example, to allow for greater sharpness, a 100x100px rectangle containing the sheep should be fed to the convnet. This 100×100 rectangle would be big enough to see the sheep, but not big enough to see the tree. The absence of contextual details (the tree) would mean that the image has to be analyzed solely on non-contextual details (the sheep); this would be analogous to sharpening the algorithm.

Conclusions

As I showed in my book “The World Through a Magnifying Glass”, recognition mistakes happen because recognition algorithms effective for contextual fields are used with high-frequency information, or algorithms effective for detailed fields are used with low-frequency information. For example, to correctly recognize the tree image, one has to disregard low-frequency info (the context: the tree) and only focus on high-frequency one (the detail: the sheep). This problem can emerge in machine learning when algorithms developed with one set of data with a given context are used to evaluate another set of data with a different context.

Submitting images with reduced scope to the same algorithm could be a solution to avoid at least part of these problems.

References

“The World Through a Magnifying Glass” : Autism is at least partially caused by a high-pass filter effect.

Paper: denser fields cause sharpening perception.

Paper: high-pass filters cause contextual information to be disregarded (and this causes at least partially autism).