Image segmentation is the task of labeling the pixels of objects ofinterest in an image.

In this tutorial, we will see how to segment objects from a background.We use the image from skimage.data.coins(). This image showsseveral coins outlined against a darker background. The segmentation ofthe coins cannot be done directly from the histogram of gray values,because the background shares enough gray levels with the coins that athresholding segmentation is not sufficient.

Simply thresholding the image leads either to missing significant partsof the coins, or to merging parts of the background with thecoins. This is due to the inhomogeneous lighting of the image.

A first idea is to take advantage of the local contrast, that is, touse the gradients rather than the gray values.

Let us first try to detect edges that enclose the coins. For edgedetection, we use the Canny detector of skimage.feature.canny()

As the background is very smooth, almost all edges are found at theboundary of the coins, or inside the coins.

Now that we have contours that delineate the outer boundary of the coins,we fill the inner part of the coins using thescipy.ndimage.binary_fill_holes() function, which uses mathematical morphologyto fill the holes.

Most coins are well segmented out of the background. Small objects fromthe background can be easily removed using the ndi.labelfunction to remove objects smaller than a small threshold.

However, the segmentation is not very satisfying, since one of the coinshas not been segmented correctly at all. The reason is that the contourthat we got from the Canny detector was not completely closed, thereforethe filling function did not fill the inner part of the coin.

Therefore, this segmentation method is not very robust: if we miss asingle pixel of the contour of the object, we will not be able to fillit. Of course, we could try to dilate the contours in order toclose them. However, it is preferable to try a more robust method.

Let us first determine markers of the coins and the background. Thesemarkers are pixels that we can label unambiguously as either object orbackground. Here, the markers are found at the two extreme parts of thehistogram of gray values:

We will use these markers in a watershed segmentation. The name watershedcomes from an analogy with hydrology. The watershed transform floodsan image of elevation starting from markers, in order to determine the catchmentbasins of these markers. Watershed lines separate these catchment basins,and correspond to the desired segmentation.

The choice of the elevation map is critical for good segmentation.Here, the amplitude of the gradient provides a good elevation map. Weuse the Sobel operator for computing the amplitude of the gradient:

From the 3-D surface plot shown below, we see that high barriers effectivelyseparate the coins from the background.

and here is the corresponding 2-D plot:

The next step is to find markers of the background and the coins based on theextreme parts of the histogram of gray values:

Let us now compute the watershed transform:

With this method, the result is satisfying for all coins. Even if themarkers for the background were not well distributed, the barriers in theelevation map were high enough for these markers to flood the entirebackground.

We remove a few small holes with mathematical morphology:

We can now label all the coins one by one using ndi.label: