Re:Draw - context aware translation as a controllable method for artistic production

Cardoso J. L., Banterle F., Cignoni P., Wimmer M.

Deep learning  J.5  image enhancement  anime editing  I.2.1  Computer Science - Graphics  Computer Science - Multimedia  Computer Science - Artificial Intelligence  I.2.6  Computer Science - Computer Vision and Pattern Recognition 

We introduce context-aware translation, a novel method that combines the benefits of inpainting and image-to-image translation, respecting simultane- ously the original input and contextual relevance – where existing methods fall short. By doing so, our method opens new avenues for the controllable use of AI within artistic creation, from animation to digital art. As an use case, we apply our method to redraw any hand-drawn animated character eyes based on any design specifications – eyes serve as a focal point that captures viewer attention and conveys a range of emotions; however, the labor-intensive na- ture of traditional animation often leads to compro- mises in the complexity and consistency of eye de- sign. Furthermore, we remove the need for produc- tion data for training and introduce a new charac- ter recognition method that surpasses existing work by not requiring fine-tuning to specific productions. This proposed use case could help maintain consis- tency throughout production and unlock bolder and more detailed design choices without the produc- tion cost drawbacks. A user study shows context- aware translation is preferred over existing work 95.16% of the time.

Publisher: International Joint Conferences on Artificial Intelligence (IJCAI)

Metrics