Introduction#
In 2024, TCL, launched an AI-driven content creation studio. Starting from scratch I had the chance to build an AI/VFX studio pipeline from the ground up; rethinking conventional approaches and emphasizing a minimal and nimble setup that would be easy to maintain and improve upon during production. This article explores the key technologies and strategies I used to create a future-ready VFX studio and pipeline.
OCIO / OIIO#
One of the challenges was ensuring colour accuracy when working with AI-generated VFX elements, on-set photography, and a variety of output formats. By adopting ACES and OCIO the team could align all assets in the ACEScg colour space for VFX work. One of the pain points was using OCIO's YAML based config file - a motivation to create OCIO.cc a web based GUI OCIO config tool.
To run transforms in the pipeline I built OpenImageIO and created Python bindings and types.
RV#
For dailies and artist review I wrote a simple Python script that walked our shot directory structure and integrated with Notion’s API to load shots into RV. It recursively scanned the shot directory to grab shot codes, then hits Notion’s API pulling cut order and metadata; skipping shots marked "omit." Artists also had the option to load shots within context or full sequences directly from Nuke.
Notion#
I setup Notion as an alternative to Shotgrid or FTrack, significantly cheaper and covered all the use cases for a small studio.
This created a single source of truth that everyone could access.
Notion's relational database tracked everything and didn't require complex migrations. As the comp supervisor I could see which shots were taking longer than expected, and what the bottlenecks were.
This analytics approach extended beyond just tracking.. we used the data to predict project timelines, and continuously improve our processes. The charting and visualization features made it a lot easier to get a big picture on how things were going.
PyInstaller#
I used PyInstaller to deploy Python-based tools in an environment where nested virtualization was unavailable (AWS G5.4xlarge instances). PyInstaller was great for packaging code along with an interpreter as a standalone executable. For updates, I used symlinking allowing a continuous deployment of executables without disrupting artists, simply by retargeting the symlink to the latest version and rolling back as necessary. This approach minimized dependencies, simplified maintenance, and enabled rapid iteration, keeping the pipeline nimble during production.
Nuke#
One the big speed ups came from working in Nuke under a multi shot workflow. The old "Don't repeat yourself" adage applied here where traditionally a lot of time and effort is wasted synchronizing work between Nuke sessions or artists. Why not work on similar shots as one bulk effort. I made a custom Nuke write node that could infer context from its upstream read node or be set manually.
Other Tools#
Shot Launcher#
Some shots being one-offs I also made a shot launcher that would put the shot context into the Nuke session and populate bookmarks in the artists read and write dialogues. Taking advantage of QSettings to store a list of recently opened shots and defaulting to the last shot, so artist could one click open Nuke in the right context. From experience working as an artist 90% percent of the time it will be the last or recent shot that needs to be worked on.
EXR to Quicktime#
The media and entertainment industry stands upon the work of ffmpeg. As with any Qt tool I separated the UI from the business logic and made sure to never block the main thread by using either QThread or QRunnable. For this I used multi threading and optimized it that that it would only take a few 15-20 seconds max to run locally leaning hard on the speed of numpy and ffmpeg and not asking Python to do any of the heavy lifting.
Submit To Dailies#
Part of a CI/CD effort borrowing from tech where artist shot publishing would backup source files and generate ProRes and h.264 files for preview and editorial use. Notifications sent to subscribers on an opt in/out basis on Notion (no sweeping email alias spam). Zero config UI.
Comfy UI Launcher#
Making it easy for artist to launch ComfyUI - centralized repo for models and configurations as well as EXR image saver nodes.
Hardware#
For hardware I setup cloud workstations and artists could connect to a personal cloud machine with access to a shared network drive, and with the help of TCL's in house IT team we setup Parsec as a cheaper and superior alternative to Teradici. For some ComfyUI work, accessing H100 GPU's with per second billing on Modal was great. Parsecs tablet support and 4:4:4 colour mode worked great and with a remote team around the world our US-West instances had minimal latency.
Open Source Knowledge Base#
The entire team maintained all our technical documentation, workflows, and best practices in open source knowledge base. This transparency not only helped onboard new team members quickly but also cultivated a genuinely inclusive workplace.
Tech Stack#
Overall these were the tools/technologies/frameworks used
- Beeble
- Comfy
- SDXL + AnimateDiff
- Runway
- Notion
- Nuke
- Maya + Arnold
- Python
- OCIO, OIIO
- Luma Gaussian Splats
- Nuke
- Comfy UI
- OpenRV
- FFMPEG
- PySide
- AWS
- PyInstaller
- OIIO
- Parsec
Directory Structure
1/show
2└── /nsp
3 ├── /art
4 ├── /assets
5 ├── /editorial
6 └── /shot
7 └── /nsp_010
8 └── /nsp_010_010
9 ├── /camera
10 │ ├── /layout
11 │ │ ├── nsp_010_010_layout_v001.abc
12 │ │ └── nsp_010_010_layout_v001.fbx
13 │ └── /track
14 │ ├── nsp_010_010_mp01_track_v001.abc
15 │ └── nsp_010_010_mp01_track_v001.fbx
16 ├── /elements
17 │ ├── /ai
18 │ │ ├── /bg
19 │ │ ├── /rediffusion
20 │ │ └── /other
21 │ └── /cg
22 │ ├── /bg
23 │ ├── /depth
24 │ └── /other
25 ├── /export
26 │ ├── nsp_010_010.mov
27 │ ├── nsp_010_010_v001.mov
28 │ └── nsp_010_010_v002.mov
29 ├── /plate
30 │ └── /nsp_010_010_mp01_v001
31 │ └── nsp_010_010_mp01_v001.####.exr
32 ├── /reference
33 │ └── ?
34 ├── /render
35 │ ├── /nsp_010_010_comp_v001
36 │ │ └── nsp_010_010_comp_v001.####.exr
37 │ ├── /nsp_010_010_slapcomp_v001
38 │ │ └── nsp_010_010_slapcomp_v001.####.exr
39 │ └── /nsp_010_010_precomp_v001
40 │ └── nsp_010_010_precomp_v001.####.exr
41 └── /scene
42 ├── /blender
43 ├── /mocha
44 └── /nuke