Cartoonize your images with Neural Networks

Cartoonized Images using Neural Networks and White Box Cartoon Representations:

Made By:

Xinrui Wang  Jinze Yu 

The Cartoonizer project allows users to generate cartoonish representations of their high quality images.

After consulting a lot of cartoon artists and observing their cartoon painting behavior, this research project has been proposed by Xinrui Wang and Jinze Yu to separately identify three white-box representations from images:

• Surface representation: It contains a smooth surface of cartoon images.
• Structure representation: It refers to the sparse color-blocks and flatten global content in the celluloid style workflow.
• Texture representation: It reflects high-frequency texture, contours, and details in cartoon images.

To achieve the cartoonish results on input images as shown below, GAN (Generative Adversarial Network) framework is used to learn the extracted representations and to cartoonize images.

Code available below to implement the cartoonization of images using this research project.

Some resulting outputs:

How it works:

As shown in the architecture image below, Images are decomposed into the surface representation, the structure representation, and the texture representations, and three independent modules are introduced to extract corresponding representations.

A GAN framework with a generator G and two discriminators Ds and Dt is proposed, where Ds aims to distinguish between surface representation extracted from model outputs and cartoons, and Dt is used to distinguish between texture representation extracted from outputs and cartoons.

Pre-trained VGG network is used to extract high-level features and to impose spatial constrain on global contents between extracted structure representations and outputs, and also between input photos and outputs. Weight for each component can be adjusted in the loss function, which allows users to control the output style and adapt the model to diverse use cases.

Proposed Image Cartoonizer System:

Demo:

This video shows how the neural network was used to make animation cartoonish filters on top of a video of Tokyo City.

Research information:

The Complete Research Paper is available here:
Cartoonize images using white box cartoonish representations

Code:

Want to try out this code? Implement it right away using this:
GitHub repo for Cartoonizing images

Get your AI project featured

Fill out this Google Form and get a chance to be featured in this growing AI community