Image understanding

A Dive into Image Understanding

Jan. 20, 2024

At its core, image understanding is the process by which machines interpret visual content, extracting meaningful information just as a human would when looking at a picture. This involves not just recognizing objects within an image but also understanding the context, relationships between objects, even inferring the emotions or actions depicted in the visual content.

It has many potential real-world applications in fields like healthcare, retail, and automotive.

For instance, here are a couple of attempts of ChatGPT 4 to describe the scene in Figure 1:

  1. “The image depicts a group of men walking through a hallway with a tiled floor and white walls. In the foreground, a tall man in a dark suit is standing on a weighing scale, checking his weight, and he appears to be holding a clipboard or folder. He is smiling and looking at a shorter man standing next to him, who is laughing and pointing towards the scale, seemingly sharing a light-hearted moment. The shorter man is also dressed in a dark suit and is walking with an air of confidence and joy. Behind them, there are other men dressed in suits who are observing the interaction, some with smiles on their faces, suggesting an overall jovial atmosphere. The environment gives the impression of a professional setting, perhaps a workplace or a formal event, where a moment of humor is being shared among the individuals.”
  2. “This image captures a moment where a group of men are in a hallway with tiled floors and walls. One man is stepping on a scale to measure his weight. He is holding a clipboard, implying that he may be taking part in some form of a health check or tracking his weight for some reason. To his right, another man is looking on with a notably amused expression, while others in the background appear to be enjoying the scene as well, with smiles and laughter. The overall mood of the image seems lighthearted and congenial.”
Obama prank with weight scale

Despite describing the image accurately, it misses the main point: “Barack Obama is playing a prank, making the other man weight more by stepping on the scale.”