Cartoonize effect

The last section of this chapter is dedicated to creating another effect, called cartoonize; the purpose of this effect is to create an image that looks like a cartoon. To do this, we divide the algorithm into two steps: edge detection and color filtering.

The cartoonCallback function defines this effect, which has the following code:

void cartoonCallback(int state, void* userData) 
{ 
    /** EDGES **/ 
    // Apply median filter to remove possible noise 
    Mat imgMedian; 
    medianBlur(img, imgMedian, 7); 
 
    // Detect edges with canny 
    Mat imgCanny; 
    Canny(imgMedian, imgCanny, 50, 150); 
     
    // Dilate the edges 
    Mat kernel= getStructuringElement(MORPH_RECT, Size(2,2)); 
    dilate(imgCanny, imgCanny, kernel); 
 
    // Scale edges values to 1 and invert values 
    imgCanny= imgCanny/255; 
    imgCanny= 1-imgCanny; 
     
    // Use float values to allow multiply between 0 and 1 
    Mat imgCannyf; 
    imgCanny.convertTo(imgCannyf, CV_32FC3); 
 
    // Blur the edgest to do smooth effect 
    blur(imgCannyf, imgCannyf, Size(5,5)); 
 
    /** COLOR **/ 
    // Apply bilateral filter to homogenizes color 
    Mat imgBF; 
    bilateralFilter(img, imgBF, 9, 150.0, 150.0); 
 
    // truncate colors 
    Mat result= imgBF/25; 
    result= result*25; 
 
    /** MERGES COLOR + EDGES **/ 
    // Create a 3 channles for edges 
    Mat imgCanny3c; 
    Mat cannyChannels[]={ imgCannyf, imgCannyf, imgCannyf}; 
    merge(cannyChannels, 3, imgCanny3c); 
 
    // Convert color result to float  
    Mat resultf; 
    result.convertTo(resultf, CV_32FC3); 
 
    // Multiply color and edges matrices 
    multiply(resultf, imgCanny3c, resultf); 
 
    // convert to 8 bits color 
    resultf.convertTo(result, CV_8UC3); 
 
    // Show image 
    imshow("Result", result); 
 
} 

The first step is to detect the most important edges of the image. We need to remove noise from the input image before detecting the edges. There are several ways to do it. We are going to use a median filter to remove all possible small noise, but we can use other methods, such as Gaussian blur. The OpenCV function is medianBlur, which accepts three parameters: input image, output image, and the kernel size (a kernel is a small matrix used to apply some mathematical operation, such as convolutional means, to an image):

Mat imgMedian; 
medianBlur(img, imgMedian, 7); 

After removing any possible noise, we detect the strong edges with the Canny filter:

// Detect edges with canny 
Mat imgCanny; 
Canny(imgMedian, imgCanny, 50, 150); 

The Canny filter accepts the following parameters:

• Input image
• Output image
• First threshold
• Second threshold
• Sobel size aperture
• Boolean value to indicate whether we need to use a more accurate image gradient magnitude

The smallest value between the first threshold and the second threshold is used for edge linking. The largest value is used to find initial segments of strong edges. The sobel size aperture is the kernel size for the sobel filter that will be used in the algorithm. After detecting edges, we are going to apply a small dilation to join broken edges:

// Dilate the edges 
Mat kernel= getStructuringElement(MORPH_RECT, Size(2,2)); 
dilate(imgCanny, imgCanny, kernel); 

Similar to what we did in the lomography effect, if we need to multiply our edges' result image with the color image, then we require the pixel values to be in the 0 and 1 range. For this, we will divide the canny result by 256 and invert the edges to black:

// Scale edges values to 1 and invert values 
imgCanny= imgCanny/255; 
imgCanny= 1-imgCanny; 

We will also transform the canny 8 unsigned bit pixel format to a float matrix:

// Use float values to allow multiply between 0 and 1 
Mat imgCannyf; 
imgCanny.convertTo(imgCannyf, CV_32FC3); 

To give a cool result, we can blur the edges, and to give smooth result lines, we can apply a blur filter:

// Blur the edgest to do smooth effect 
blur(imgCannyf, imgCannyf, Size(5,5)); 

The first step of the algorithm is finished, and now we are going to work with the color. To get a cartoon look, we are going to use the bilateral filter:

// Apply bilateral filter to homogenizes color 
Mat imgBF; 
bilateralFilter(img, imgBF, 9, 150.0, 150.0); 

The bilateral filter is a filter that reduces the noise of an image while keeping the edges. With appropriate parameters, which we will explore later, we can get a cartoonish effect.

The bilateral filter's parameters are as follows:

• Input image
• Output image
• Diameter of pixel neighborhood; if it's set to negative, it is computed from a sigma space value

• Sigma color value
• Sigma coordinate space

To create a stronger cartoonish effect, we truncate the possible color values to 10 by multiplying and dividing the pixels values:

// truncate colors 
Mat result= imgBF/25; 
result= result*25; 

Finally, we have to merge the color and edges results. Then, we have to create a three-channel image as follows:

// Create a 3 channles for edges 
Mat imgCanny3c; 
Mat cannyChannels[]={ imgCannyf, imgCannyf, imgCannyf}; 
merge(cannyChannels, 3, imgCanny3c); 

We can convert our color result image to a 32-bit float image and then multiply both images per element:

// Convert color result to float  
Mat resultf; 
result.convertTo(resultf, CV_32FC3); 
 
// Multiply color and edges matrices 
multiply(resultf, imgCanny3c, resultf); 

Finally, we only need to convert our image to 8 bits and then show the resulting image to the user:

// convert to 8 bits color 
resultf.convertTo(result, CV_8UC3); 
 
// Show image 
imshow("Result", result); 

In the next screenshot, we can see the input image (left image) and the result of applying the cartoonize effect (right image):