Poster Presentation

Poster E in Poster Session E: Thursday, August 6, 10:30 am – 12:15 pm, Kimmel Center, Shorin & Rosenthal Rooms

A deeper look into occlusion types and their impacts on object recognition models

Courtney M King¹, Elissa Aminoff¹, Daniel D. Leeds¹; ¹Fordham University

Presenter: Courtney M King

Modern computer vision offers a wide variety of models for recognition and object detection. Partial occlusions have been shown to have a negative impact on performance in both cases. Similarly, color can be used to create illusions in images and instances of camouflage, which can also have an impact on these two tasks. This work explores several types of occlusions that can make objects appear illusory due to their color with respect to the greater image context. Specifically, we study cases where the occluder blends in with an object or scene. These cases find commonality with optical illusions in human perception. In experiments, these artificial occluders are applied to 80 distinct objects as horizontal lines. Current experiments explore the impact of these types of occlusions on object recognition for several state-of-the-art computer vision models, ranging from simple convolutional neural networks (CNNs) to transformers to modern CNNs designed to compete with transformers. This work also offers a look inside these diverse models to visualize how the occlusion-based errors propagate through the networks to the final layer. Current results find that transformer-based models are far more robust to these types of occlusions when compared to standard CNN models, like ResNet, but that more recent CNNs like ConvNext, show similar robustness to transformers. A second more surprising observation is that these color-based occlusions show significantly higher performance when compared to black baseline occlusions in the CNN models, but similar performance in both the transformer and more recent CNN models. These results give more knowledge about the internal nature of neural networks as well as their robustness to specific types of partial occlusions which are not unseen in real-world scenes.

Topic Area: Computational Models of Vision & Visual Cortex