Digital avatars are key to communication among the younger generation globally. On platforms like YouTube, Twitter, and similar social media sites, individuals from around the world forge friendships through these virtual personas, transcending barriers of race, gender, and geography to share a collective experience within a beautifully crafted virtual realm. However, the creation of these avatars is often prohibitively expensive, typically accessible only to professional VTubers who can afford to commission designers for these specialized services. In extreme cases, VTubers may end up allocating a significant portion of their income to IT and design teams to maintain their virtual personas. Addressing this challenge, RAIN introduces a revolutionary technology that democratizes the creation of digital avatars, making them accessible for widespread use in various social interactions without scene limitations. Whether drawn by hand, retrieved online, or generated by AI, RAIN enables users to create their own real-time, low-latency virtual avatars with just a basic consumer-grade graphics card and a single image. We are excited about how this technology will empower the younger generations and facilitate barrier-free communication, fostering connections across all divides.
Face morphing examples, expressions and head positions of real face are mapped into anime faces.
Test Example from UBC-Fashion Dataset, the model is only trained on 500 video clips from training dataset.
RAIN provides a possible way of rendering real-time animation with AI. We could expecting the future of combining AI together with CG in rendering games, live streams and virtual realities. In which we could take the great advantage of generalization from AI in rendering countless new scenes and objects, and also provide a more interactive way to participate in the synthesized world.