Interactively transforming chinese ink paintings into realistic images using a border enhance generative adversarial network

Traditional Chinese painting has a long history. When we appreciate such paintings today, although we can obtain an overview of the landscape and environment of that time, it can be difficult to feel like we are interacting with the paintings. Alongside the rapid rise of deep learning, much research has been conducted on style transfer—for example, transforming photographs into the style of Chinese painting, sketches, or cartoons—but no research has considered the transformation of Chinese paintings into realistic images or even enriching such paintings through user interaction. To address this research gap, we employed a generative adversarial network (GAN), which is a generative model, to create new images that resemble the training data through the process of confrontation. Additionally, compared with general image-to-image translation, converting Chinese ink paintings into realistic images requires additional input because ink paintings contain texture and border features of relatively low quality. We combined cycle-consistent GAN with pix2pix and added a label function to establish a border enhance GAN with the purpose of enhancing the detail of border images and producing more accurate realistic images. In this manner, traditional Chinese paintings can be invigorated. Finally, we compared the image generated using our model with other benchmarks. The results revealed that the image generated using our model exhibited greater similarity to the actual photograph than did the benchmark images. Therefore, our model mitigates a major problem encountered in previous works and renders more realistic results. These interactive images clearly and profoundly convey Chinese culture, offering the user a novel art experience. Moreover, when viewers can interact with the input image by selecting different geologic styles, they can derive a relatively profound immersive experience. Our study can serve as a reference in transforming images (such as watercolor and oil paintings) with blurry borders.