In miHoYo (http://www.mihayo.com/), we have great artists, and we are trying to use technologies to free them from the tedious works so they can focus on creation.
To do that, we are looking for talented and motivated research interns in the areas of computer graphics, deep learning, computer vision and human computer interaction to work on the following projects. We are also open to any other projects that could help improve the game/CG assets creation.
If you are interested in internships or external collaborations with us, please contact us via email@example.com (Jun Xing) with your resume. Any slide or meterial summarizing what you did is welcome.
1. cap2Cart: motion transfer from capture to cartoon
Due to the different body scales and motion styles, transferring the motion from a real human (e.g. via motion capture) to a cartoon character could be a challenging problem. The goal is transfer the high-level captured motion, while maintaining the style of the target character. Deep neural network has shown to be powerful in high-level feature extraction, merging, and transfer.
2. pix2Anim: style transfer from image to animation
Image stylization is a popular topic in deep learning and lots of amazing results have been achieved. The goal is to transfer the desired style to an animation sequence. In particular, the artists could specify the style of one or a few frames via manual drawing, and the system automatically propagates the style to the rest frames while maintaining temporal coherence.
3. smartWalk: smart walking animation synthesis
In games, walking is one of the most important animation asset. However, the walking animations are usually predefined and limited in expression. The goal is to make the walking stylized and adaptive to/aware of the environment, e.g. different poses under different emotions.
4. smartAnim: smart editing of animation sequence
Keyframing the animation sequence is a tedious task that requires experienced artists. Usually, the artists tune the shape and pose of the objects (e.g. cloth, hair, body) by rigging the bones, or even more detailed editing on the vertices, edges and faces, frame-by-frame. The goal is to propagate the editing from local to global (e.g. edit one finger, and the whole hand/arm/body is automatically adjusted), and from one frame to the whole animation sequence.
5. hairAnimBrush: interactive hair animation
The hair animation should be stylized, physically-plausible, and easy to control via simple gestures. Physical simulation can produce realistic animations, but can be hard to control and stylize. The goal is to design a sketch-based system to make animation authoring easy, while exhibiting the time-varying dynamics of simulation.
6. terrainBrush: interactive terrain creation in game world
Designing the terrain of a large area is a tedious task, which includes the numerous layout of trees, grasses and stones, and the terrain altitude and texture adjustment. The goal is to design a terrain brush that could allow the artists to easily create the desired terrain at different scales, with the detailed terrain geometry/texture and object layout automatically generated while being aware of the environment.
7. fakeVoice: synthesize the audio of expert actor by anyone
Dubbing the character voice using human actors is an expensive and time-consuming task, and the same actor may not be always available. The goal is to synthesize the voice of an expert actor A, driven by the audio and emotion of another person B, given a large paired audio dataset of A and B.