Want to know how you can make on-the-fly tweaks to animation assets without their source files, or mock up the concept art for your next project with a few clicks? Unity Technical Artist Victor Kam shows you how to unlock these workflows easily and quickly with ArtEngine.
Do you work with prebuilt animation assets from public libraries and get frustrated when you can’t easily make adjustments to the material on a character or prop because you don’t have a source file? Have you ever had a “vibe” for a new animation project in mind but struggled to recreate it quickly and effectively for a mood board or previs mockup?
If these scenarios feel all too familiar, Unity ArtEngine may be able to help. With ArtEngine, these workflows not only become easy, but incredibly quick. Built on the philosophy of the example-based workflow, the tool takes data and human intent as input, and uses AI to quickly generate output that fits your target outcome, greatly accelerating your art creation processes, be it material creation, visual exploration, or other artistic work.
At Unity’s first Media and Entertainment Digital Developer Day this year, during a 20-minute demo, I showed real-time content creators in the film and animation space how ArtEngine unlocks new workflows and eliminates manual tasks.
Watch the demo below, or read on for a brief summary.
Speedy material extraction with Mask Paint
As an example, let’s take the character Shanks from the Realtime Rascals animation project, available for free download on the Unity Asset Store. The Shanks model is fully unwrapped and textured, which is great, but what if you wanted to create a version of the material for future use? Using traditional tools, this can be a time-consuming and manual process. With ArtEngine, you can do it in minutes.
After importing the texture into ArtEngine, add a Mask Paint node. Using the brush, loosely paint in areas that correspond to the areas that you want to keep. Connect this Mask Paint node to an Invert node, and then connect the Invert node to the Ignore input of a new Seam Removal node. This will recompute into a fully tileable material using the areas you defined and bring all the channels from the original source.
The modified Shanks frog material is looking quite slick and is ready to be applied to another model (perhaps on their best friend, Sydney).
Quickly tiling tricky patterns with Mutation Structure
What if we wanted to create a new shirt for Shanks?
If you have experience working with fabric materials, you are probably aware of the headache that can ensue from trying to extract a specific pattern from a source image and make it tileable. This proves even more challenging when your image is not square, which is common when working with physical samples. With exceptional AI pattern recognition, plus a bit of human input, ArtEngine’s Mutation Structure node can analyze the image, detect the pattern and create a tileable version to the specified dimensions, all within a few blinks of an eye.
Your source can be as simple as a photo you snapped on your phone of your favorite shirt, or a texture off the internet (such as from a public library). Here, we’ve chosen a brown plaid image scanned by a Unity artist. Unfortunately, our material is not seamless, nor a size that is friendly to Unity (textures should be power of 2). With the Mutation Structure node, we can quickly address both problems inside ArtEngine.
First, connect your material to a Mutation Structure node. You can either choose “Auto Definition” or manually define the pattern by using the square widget. Manually defining the area and choosing Neural Match results in greater accuracy. Then, execute with your specified resolution and boom: your material is perfectly seamless and ready to apply to another model.
Keep in mind that Mutation Structure requires your source image to have around three or four identifiable repeatable units to work. To read more about this node, head over to the Mutation Structure page on the ArtEngine Knowledge Base.
For fun, let’s test out both materials on Sydney.
I can’t say this is Sydney’s best look, but hey – we now have two tileable materials that we can use for future projects, and it only took us a few minutes.
We’ve had some fun with the characters, but let’s not forget about the environment assets that will really bring your story to life.
Get the creative juices flowing with Style Transfer
Visual exploration forms a critical part of any animation project, whether you are crafting a mood board to get the green light, or creating the more refined lookdev or previs during preproduction. The ability to quickly experiment and iterate with the look and feel of your artwork is key to developing a successful end product that is delivered on time and within budget.
If you have the luxury of several concept artists on your team, this may not be a concern, but if you’re seeking this kind of rapid, creative prototyping in the absence of a full art department, this can be a challenge. The solution? ArtEngine’s Style Transfer node.
Using Style Transfer is as simple as importing your original texture and style image, and connecting them to their respective inputs in a Style Transfer node. Style images can be anything from your own drawings or paintings, to inspiration found online.
Adjust any settings to your preference. After executing, you will have a completely new image that you can add to your mood board or export directly to your Unity project, as shown in the video demo and also illustrated below.
Note: This feature is still in beta testing, but will be available in the coming months.
If you want to experience how AI can speed up your animation pipeline and creative workflows, we invite you to give ArtEngine a try. Until May 17, ArtEngine is available for $19/mo (vs regular price of $95/mo). If you have any feedback or questions, we welcome you to drop us a line at [email protected] or engage on our Discord channel.