Scalable AI Pipeline for Specified Colour Transfer from Concept Sketches into Static Animation Frames

Every technological innovation brings new possibilities to the animation industry. This study explores the application of Artificial Intelligence (AI) in the 2D animation production process. In this study, we establish an AI Colour filling Pipeline, which includes a variety of steps and model components. For the core part of generative colour fill, we use the U-Net architecture of Pix2Pix as the foundation, combined with the Conditional Generative Adversarial Network (CGAN), to train a model that can generate colour filling results according to the specified colour, which we name 2DColorGAN. The training materials, sourced from Midjourney and Kaggle, are classified into character design drawings (CHD), character sketch drawings (CHS) and character sketch finished drawings (CHSF), with motion consistency, colour consistency and colouring accuracy as the evaluation indicators. The automatic colour filling and colouring process is divided into two stages: data collation and component colouring. The former helps users organize character images, while the latter relies on the segmentation model 2DColorSEG for colouring. The results show that 2DColorSEG has a high degree of stability and accuracy for general components but performs less effectively on unique components due to limited recognition capabilities. In contrast, 2DColorGAN has higher colour flexibility and can handle components that cannot be covered by colour filling, but it still faces colour stability and overflow issues, which could be mitigated by increasing data diversity. Overall, the combination of automated colour fill and generative colour fill can improve the overall colour filling effect and application flexibility.