July 2015 [CG]
"The Boy and the Beast" was a 2015 summer movie shown on 458 screens nationwide. The Boy set a record number of moviegoers for a Mamoru Hosoda-directed film, greatly surpassing the 3.4 million for Wolf Children, which was released in 2012.
"The Boy and the Beast" is Hosoda’s first original feature film in 3 years since "The Girl Who Leapt Through Time" and "Summer Wars". The relationship between a lonely boy, Kyuta, and a beast, Kumatetsu, is portrayed with the backdrop of a world in which the Beast Kingdom exists alongside the human realm. Unique characters and a fantastical story told in the Hosoda-esque view of the world captured the hearts of the audience, young and old, male and female alike.
The staff that had worked on the previous films, trusted not only by Digital Frontiers (hereinafter DF) but also Hosoda himself, reunited after "The Girl Who Leapt Through Time" and "Summer Wars" to oversee compositing and CG production for all cuts of the film. With the greater number of cuts and the higher quality of CG as compared to the previous works, “integration of 3DCG mobs within 2D framework” was a consummate technique that could have only come from DF. There were a total of 1550 cuts in "The Boy and the Beast", of which 480 involved the 3DCG mob.
We spoke to Ryo Horibe, who directed CG production, as well as unit leaders, whose names appear in the credits, about behind-the-scene stories of some memorable scenes.
(Hereinafter, CGI is referred to as CG)
(Interviewer: Kao Yamashita, ZAKKA MEKKA)
Rendering of the downtown Shibuya and Jutengai - production of mobs that transcended background characters
Production work is divided at DF. At the start of a project, a project team is composed of staff members assigned from each team. Senior members of the characters, animation, rigging, and CG development units share their techniques used to achieve seamless integration of 2D and 3D. The making of crowds (= mobs), which were the showstopper in this project, was the product of the whole unit's blood, sweat, and tears.
Created characters, facial features, and props in CG mobs and was responsible for modeling the character silhouettes in the opening sequences and the whale. Mr. Eom joins the interview.
In charge of animation of the opening sequences, CG mobs, and the whale, as well as base motion in the crowd simulation tool. Mr. Kamekawa and Mr. Imatsuji join the interview.
In charge of setting up CG characters, as well as cloth simulation of drapery, beaded curtains, and Tanabata Festival decorations. Mr. Tabuchi, Mr. Tanaka, and Mr. Kanda join the interview.
[TD Unit (Technical Direction), Crowd Unit]
Responsible for constructing plug-ins and the rendering system, as well as shader development. A technical director, Mr. Ishida, and a crowd lead, Mr. Gonokami, join the interview.
(A mob in the text refers to a CG mob)
“Shibuya Scramble Crossing, which also appeared in "The Girl Who Leapt", was drawn by hand back then. According to Horibe, they knew that they would use CG to create the crossing for "The Boy".
Overwhelming CG mob at Shibuya Scramble Crossing
The number of assets in the mob was enormous; there were three models, one each for far-distance, medium-distance, and close-distance. When considering humans and beasts collectively, we created about 600 bodies. We manually (rather than automatically) combined each part. In this film, there are two different eras - one in which the main character is 9 years old and today. As such, we had to be mindful of different styles of fashion. We researched a variety of items, including accessories and cell phones. Facial expressions of the mobs were created under Horibe’s guidance. The animation team took care of positioning the mob characters for the most part. Basically, they were picked up from layouts and contents and Horibe checked them at the end.
We (CG production) had freedom with the Shibuya mobs, but we were asked for a color specification once. There is the color setting for cels, so we had our submitted work reviewed and received color samples for that.
Creating a color palette for the mobs in advance was labor intensive. The palette for color specification from the animation side was an image, so we pulled out colors from it and used them as data for the palette. When we looked at the mob afterwards and noticed some parts with repeated colors, we corrected them on the composites.
Two types of crowd simulation systems were used for the mob characters
For this film, we used two tools - Massive and Miarmy on Maya. We used Massive primarily for walking crowds and Miarmy for scenes in which the crowd is seated inside the stadium, for example, and there are only upper-body movements. There is less foot slippage with Massive, which is to say it is easier to create a sense of grounded-ness. Miarmy, on the other hand, does not require detailed settings such as the brain setting and incorporation of motions, which saves time and labor. It is advantageous in that it cuts down on settings, so Miarmy was used for scenes in which people are seated to eliminate as many processes as possible. Because it is Maya based, basically everything can be done using one piece of software without having the need to convert anything.
Is the ground contact inevitably slightly less firm with Miarmy?
Yes, you could say that if you compared it to Massive. But if you work at it, this tool still gives you a high-quality result. Although there is certainly much work involved with Massive, it gives you a decent result with a just little tweaking, so there is no unnecessary trial and error.
Some work overlapped by transferring from Massive to Maya. The number of bodies in mobs kept increasing as we tweaked parts that were unfitting due to things like bone structure. We also pared down the outfits because they are simpler in drawings than the actual photographic images.
Figuring out the right timing to ask the director to check the progress was also tricky. The timing is scary [laughs]. It is hard for the director to give feedback if you submit too early before it has taken shape and you end up with a lot of edits. But if it is too close to being finished, the editing process may become more complicated.
These motion data are gathered from actual performance using motion capture and a list is created for each crowd simulation and for general-purpose motions to be manipulated by hand. Even if the base motion is predetermined, the number of motions skyrockets when variations, such as a bag or a phone being carried, are added. Including the acted motions that are generated, for example stepping forward on the right foot, there were more than 200 motions for Massive alone. Creating this list alone undoubtedly required significant man hours.
Motions are components in simulation, so checking them in detail was also time consuming. Although they are real motions recorded using motion capture, we had to avoid them being too realistic and standing out in a mob. So the motions had to be without distinctiveness. We paid attention to creating refined data.
Development of tools for creating mobs
The TD unit developed new tools for this film, including plugins and a shader for mob creation. Based on the view that this project could not be achieved using the existing mob generator tools alone, they decided to go ahead and develop tools dedicated to creating mobs.
Even in a mob, there are individual parts. So we developed a tool that reads out parts in combination, rather than simply creating individual bodies. The scope of work using this tool was up to the point where the characters are animated in Maya; that is, until they were rendered.
Every detail from the body and down to hair is broken up as an object and these are combined to create the body of a character. It starts with creating the initial parts and individual parts are read into the tool to get a certain look for each character. Then, the rigging team manipulates the rig for each part and color codes them. Finally this tool is set up to allow a character to be created at the touch of a button and can then be animated by the animation team. One cut is thus complete. The next step is rendering. Sometimes this may go back to a mob.
I developed the tool mostly by myself and it took me about 3 months. It took me longer than expected because various parts sections in parallel. This tool was involved in all the processes from modeling to rendering, so I spent a lot of time thinking about how best to connect them. A mistake in one step would affect the rest of the steps down the line like a line of dominos. This was last autumn. A cut of the Shibuya crowd was going to be used for the movie trailer, so I worked hard to make that happen. At that point, the tool was in fact not yet finished.
The meeting started early last year (2014) and the production began full swing in July. Shibuya Scramble Crossing appears in the very first scene of the storyboard, so we had to deal with it from the get-go or there would be no movie [laughs]. The first one to be tested was the Shibuya Scramble Crossing cut.
Drawing enhancing artistry
Lines made using CG shaders look homogeneous, so I developed the tool to be capable of generating shading with unevenness, as in cel animation. I made several patterns for cel outlines. For the foreground characters, inhomogeneous shading with a thicker line, like ones drawn for celluloid. For outlining the background characters, the line of a fine-point pen is generated. There are dozens of shaders, but I created 3 or 4 customs shaders for this film.
We made objects specifically for casting shadows in order to create shadows in the Hosoda style. In his drawings, there are generally just sparsely drawn round, simple shadows. So in CG, we placed simple objects separate from characters to generate shadows. These shadow objects are incorporated in all the characters in a mob, and when a shadow is thrown over, it is designed to cast shadows over these objects. If you add regular shadows, it would look too computer generated.
Where is red Waldo?
Tsukamoto (the TD unit) was asked when creating the shader to set it up in a way to show any parts that could not be properly rendered due to some errors in red. Any parts that turned up red indicated that they had some sort of error in the shader (parts), so we checked one by one to pinpoint the error. … The movie trailer was a particularly insane amount of work. When the Shibuya Scramble Crossing scene used in the trailer was rendered, we found many instances of this red error. There were many overlapping mobs in this particular scene and we could not tell with one look at it where the red spots were. So I called it “Where is Waldo?” [laughs]. We had to visually find where the errors were and manually determine the ID number for each of them and in which part the error occurred. First, the cut is rendered and if an error was found, we would guess what the error was and fix it, then the cut is rendered again to see if the error was still there; this was a mind-boggling task. While also debugging the tool, I was barely able to meet the deadline. In the movie trailer, perhaps because it was where you could see everything, even the minute details turned red. But after working on this scene, nearly all the errors were debugged, so we could proceed without errors thereafter.