Progress Report - May 28, 2021 (Patreon)

Hello, everyone!

This past week has been uncharacteristically productive, in the best way possible! All week I've been able to put out daily renders of the project, with noticeable improvements on each. I've put together a small 30-second clip of the project, back-to-back with each of these dailies, so you all can see and hear just how quickly progress has moved this past week.

You can watch the video by clicking here. I have put the Mass Effect Galaxy Map music underneath the video, to discourage people from trying to rip Oolay's and Bordeaux's wonderful voice-work from the video. Plus, I mean, it's thematically relevant, right? Note that the video may take a hot minute to buffer before it'll play - not sure why that is, but I've been noticing it a lot lately with these dailies I've been sharing internally.

As you all may or may not know, over the past few years, I have been slowly but surely building up a collection of tools and techniques to assist me in and speed up the process of animating these videos.

The two most significant processes involved here are the Gentle Auto-Lipsync workflow, and the Kinect motion-capture. And this past week, for the first time, I was able to extensively exploit the both of them.

And in doing so, I managed to do what would have taken me just shy of 2 weeks to do the way I've done it in years past... in only three days.

As you can see by this week's video post, I managed to get the lip-sync done, start to finish, on the 24th; all of Miranda's motion-capture animation done on the 25th; and all of Femshep's motion-capture on the 26th.

Now, this motion capture is not 100% finished. It needs a lot of polish, and I will probably give it a one-over to smooth out all of the micro-stutter. I am using a single cheap $40 Xbox Kinect in a small, poorly-lit room, and so it's prone to tracking errors. I've smoothed out the worst of them, but even then, mixing a 30Hz framerate down to a 24Hz project introduces some artifacts outside of the tracking errors.

But even if I need to spend, say, two days polishing all of the mocap, that still saves me at least a week's worth of work, and it yields far more interesting and complex animation (such as characters stepping forward and back, or shifting their weight - animations I typically don't do by hand, simply because of the enormous time-sink involved).

And then the automatic lip-sync was an even greater success than the motion-capture. It's not perfect, but in total I had to hand-animate / correct less than 20 words total. Out of a screenplay with almost exactly 300 words lip-synced, having to hand-animate less than 7% of them is pretty darn good, all things considered. The lip-sync as of now is certainly final - I couldn't do it better myself if I tried.

The lion's share of the work is done now, with the next week being focused on smaller, independent tasks:

- I still need to light the scene. I normally light far earlier, but I am rushing this project through and wanted to get the big tasks out of the way first, to fast-track my sound editor being able to work as fast as possible.

- I need to detail-animate and lip-sync the lewd audio. There's no getting around this one, I have to do it by hand. The automatic lip-sync completely falls apart with squeals and moans, and motion capture is just not in the cards for sex scenes. It's not even worth trying either. Luckily, in general, lewd audio is a lot easier to lip-sync than narrative animation, and I can get a long way with some jitter combined with some well-timed head motions for sex scenes - they're largely static, all things considered.

- I need to animate the fingers. The motion-capture setup I use doesn't even attempt to capture finger motion. It's just far too nuanced for both the hardware and the software. So I have to animate them myself. Luckily, fingers generally don't move too much - putting the hands in an idle "trumpet hand" pose, and having the fingers explode and curl as characters emote, should get the lion's share of the work done.

- I need to polish the motion-capture animation. The bits shown in this preview here aren't too bad, and outside of cleaning up some jitter, I honestly think they're fine (sans the fingers not being animated, of course). But there are some other sequences that are really bad. Like, "will probably take a full day by themselves to fix" bad. But they'd take even longer to have done by hand, so it's still a win in my book in terms of time savings.

- I need to choreograph the camera, and clean up the visual editing / split-screen. The split screen is a bit sloppy in places, and it'll be far more pronounced when the final 4k video is released. And animating the camera is going to be particularly tricky, because I don't want characters falling off-screen. Should be an interesting puzzle to solve.

- I need to polish up clipping. This will be a joint effort of fixing the outfits, and fixing the animations. Not much more to be said. It's just something that needs to be done.

- I need to render! This isn't really an independent variable, because it is dependent on literally everything else, but when everything above is done, this is all that will remain. It takes me about 45 minutes to render this project out right now, in 720p, with no lighting, no ambient-occlusion, and minimum quality settings. It's going to probably take at least a full 24 hours to render everything in final-quality 2160p / 4k footage. And that'll be with the assistance of my dedicated rendering machine.

All in all, I am estimating 1 day each for lights, fingers, choreography and clipping. I am estimating 2 days to clean up the mocap, and 3 days to lip-sync and detail the lewd sequences. All together, that sums up to 9 days of work I am projecting.

My plans for getting this project released by the end of the month are obviously not going to happen. Technically speaking I could push what I have right now, with some lighting quickly thrown together, and honestly it'd probably be good enough for a great many people. But I like to hold myself to a higher standard than that.

All things considered, the fact that I have about 6 minutes total of animation (in a 4-minute video, due to the split-screen shenanigans), coming together from conception to release in the span of about 5 weeks (the project's first draft was written on May 9, and I am projecting another 2 weeks before release), I think is extremely impressive.

Once this project is out the door, I am returning my attention to Rachaos. I am feeling extremely chuffed with this detour - not only am I getting something out that I am genuinely excited by due to its experimental nature, but the exploration (and success!) of technologies I have spent years developing all culminating on this project gives me great inspiration for using the same techniques on Rachaos to shave weeks off its already massive production time.

That's all I have for you this week! Until next time, everyone!