Panoptic segmentation is a very interesting concept in computer vision. It's a combination of semantic and instance segmentation. Let's explain both concepts in the simplest manner that I can.

Semantic segmentation algorithm aims to detect "things" such as grass, roads and skies. If it detects grass, then it highlights / classifies the whole grass within an image under one class. If it detects the skies, then it highlights / classifies the thing into one class within an image on a pixel basis. Nice. However, the problem lies if it detects "people", animals, or any "stuff" because then the algorithm will just highlight every people or any "stuff" under one class. That's where the problem lies.

Now the instance segmentation algorithm comes in. It classifies "stuff" on an individual basis per class. Now you can start dealing with them accurately like counting them, pointing the location, etc. Again, very nice. However, detecting "things" like grass, skies and seas becomes a problem because the algorithm instance segmentation is not design to this task.

Hence, the birth of Panoptic Segmentation! The algorithm deals with both "things" (grass, seas, roads, etc.) and "stuff" (people, animals, cars, etc). It is basically a reconciliation between instance and semantic tasks.

Here's the video that shows how panoptic segmentation works. I run panoptic segmentation via Detectron 2 / PyTorch on this video. Have fun!

(c) via Pexels | Videogrammer, Taryn Elliot, Tom Fisk

свернуть