CorridorKey: Physically Accurate Unmixing for Green/Blue Screen Keying

CorridorKey: Physically Accurate Green Screen Unmixing for Clean, Realistic Composites

Image: CorridorKey demo

The challenge with green or blue screen work is not simply removing a background color. Edges bleed, motion blur softens silhouettes, and semi-transparent fringes around hair and glassy surfaces often get mangled by traditional keyers. CorridorKey tackles this unmixing problem at the pixel level. Instead of producing a binary mask or fragile edge mattes, CorridorKey’s neural network predicts, for every single pixel, the true foreground color and a clean, linear alpha channel. It reconstructs the foreground as if the green screen never existed, preserving translucency, hair detail, motion blur, and other nuanced elements that make a composite feel natural in post.

What this post explores

• Why CorridorKey exists: solving the unmixing problem that traditional keyers struggle with.
• How CorridorKey works: two inputs, a sophisticated neural network, and outputs designed for modern VFX pipelines.
• Features and capabilities: resolution independence, 16/32-bit EXR outputs, auto-cleanup, and screen-color handling.
• Practical getting-started guidance: installation, dependencies, and how to run CorridorKey on different platforms.
• Backend options, options for Apple Silicon, and how to leverage optional AlphaHint generators.
• Advanced usage and troubleshooting notes, including Docker, ROcm, and MLX considerations.
• Licensing, community extensions, acknowledgments, and how to contribute.

What CorridorKey is solving

In traditional keying, edges that have blended with the green or blue screen produce mixed-color pixels that are difficult to disentangle. Even modern AI roto tools can yield masks that are harshly binary, destroying the subtle semi-transparent pixels essential for believable composites. CorridorKey changes the game by estimating, on a per-pixel basis, both the straight foreground color and a linear alpha that represents exactly how opaque that pixel should appear in the final image. The result is a clean separation that captures fine details like hair strands, highlights, and translucency that would otherwise vanish or become brittle in a standard matte.

Key idea in practice

• Input 1: The Original RGB frame captured against a green or blue screen, in a gamma space compatible with the pipeline (sRGB or linear).
• Input 2: A coarse Alpha Hint that roughly isolates the subject. This hint does not need to be precise; it can be a rough chroma key or an AI roto mask.
• Output: A foreground color pass in linear space and a high-fidelity alpha pass, enabling realistic compositing when paired with the original background or a new shot.

How CorridorKey works: the workflow in practice

• The Alpha Hint is not a perfect mask—it’s a coarse guide that helps the model reconstruct details hidden by the screen. The better the hint, the more precise the result, especially around tricky edges like motion blur and out-of-focus areas.
• CorridorKey supports two screen color checkpoints: green and blue. By default, the system auto-detects the dominant screen color from the first frame, but you can force a specific color with --screen-color green or --screen-color blue to fix the checkpoint.
• The green or blue screen pathway is chosen based on the detected background, but CorridorKey also performs a despill pass that targets the actual screen color being keyed, reducing color spill that would otherwise contaminate the foreground.

Two broad approaches to AlphaHints

• GVM (Generative Video Matting): Completely automatic and drop-in for most scenes, particularly effective with people. It might be challenged by certain inanimate subjects.
• VideoMaMa: Requires a rough VideoMamaMaskHint that you provide (hand-drawn or AI-generated) to guide the key. Its results can be highly controllable, especially when you want to steer the key with an explicit mask hint.
• CorridorKey’s design integrates these two as optional modules within the clip management system, so you can choose the approach that best fits your shot.

A note on “unmixing” and how CorridorKey does it differently

• CorridorKey doesn’t simply guess opaque versus transparent; it actively reconstructs the color of the foreground element as if the green screen wasn’t there, and generates a clean, linear alpha channel to go with it.
• This reduces ghosting, preserves fine detail in hair and translucent materials, and avoids the common “core” versus “edge” trap that many keyers fall into.
• The result is a more natural edge that remains stable under motion, with less need for manual rotoscoping.

Features that stand out

• Physically Accurate Unmixing: Foreground color and linear alpha are extracted with precision, preserving delicate edges, motion blur, and translucency.
• Green or Blue Screen support: Dedicated checkpoints for green and blue plates; auto color detection with the option to force a specific screen color when needed.
• Resolution independence: The engine scales inference to handle 4K plates while leveraging a robust 2048x2048 backbone for high fidelity.
• VFX standard outputs: Native support for 16-bit and 32-bit linear EXR files, ensuring compatibility with Nuke, Fusion, Resolve, and other core compositing tools.
• Auto-cleanup: A morphological cleanup system to prune tracking markers or small background specks that slip through detection.

Hardware requirements and practical notes

• CorridorKey was designed and tested on a Linux workstation with an NVIDIA RTX Pro 6000 (96 GB VRAM). The project is actively optimized for consumer GPUs, and early builds have run on systems with as little as 6–8 GB VRAM, including some M1+ Mac configurations with unified memory.
• Windows users (NVIDIA): CUDA 12.8 or newer is required for native GPU acceleration; older drivers may force CPU fallback.
• AMD GPUs (ROCm): ROCm support exists on Linux for RX 7000 and 9000 series GPUs, with Windows ROCm support being experimental. See the AMD ROCm setup notes for details.
• Optional modules: GVM, VideoMaMa, and BiRefNet are optional; the project supports AlphaHint generation via integrated modules or your own method. The quality of the AlphaHint strongly influences the final result.
• VRAM considerations: 2048x2048 inference can require around 10 GB VRAM on NVIDIA and closer to 18 GB on some AMD configurations due to ROCm allocations. If you’re on a 16 GB card and run into memory issues, GTT (graphics translation tables) memory can help by sharing system RAM as overflow for VRAM, at the cost of slower performance.

Getting started: installation and setup

• The project uses uv to manage Python and dependencies. uv handles Python installation, virtual environments, and package installation in one step, so you don’t need to install Python separately.
• Windows (automated):
• Clone or download the repository.
• Run InstallCorridorKeyWindows.bat to install uv, set up the Python environment, install dependencies, and download the CorridorKey model.
• Optional: InstallGVMWindows.bat and InstallVideoMaMaWindows.bat to fetch heavy optional weights.
• Linux / macOS (automated):
• Clone or download the repository.
• In a terminal, run bash, then drag and drop InstallCorridorKeyLinux_Mac.sh into the terminal to execute.
• Optional: Drag and drop InstallGVMLinuxMac.sh and InstallVideoMaMaLinuxMac.sh for the heavy alpha-hint generators.
• Linux / Mac (manual):
• Install uv (curl -LsSf https://astral.sh/uv/install.sh | sh).
• Install dependencies with uv sync (CPU/MPS by default); add --extra cuda for CUDA acceleration or --extra mlx for Apple Silicon MLX.
• Download models:
  • CorridorKey v1.0 (~300MB): auto-downloaded on first run or from HuggingFace if missing.
  • CorridorKeyBlue 1.0 (~300MB): dedicated blue-screen checkpoint, downloaded on demand.
  • Optional: GVM, VideoMaMa weights, and the Stable Video Diffusion base model for VideoMaMa workflows.
• Docker (Linux + NVIDIA GPU): You can run CorridorKey in Docker. Build the image, then run inference with the appropriate GPU access. The container expects your clips and outputs to be mounted, and you’ll place model weights in the expected folders inside the container.

Two inputs CorridorKey requires for each frame

1) The Original RGB image: The green/blue screen footage in sRGB (or linear) gamma. 2) A Coarse Alpha Hint: A rough mask isolating the subject. It does not need to be precise; a rough chroma key or AI roto suffices.

The model pair uses these inputs to produce:

• A foreground color pass (linear, for color accuracy in compositing).
• A clean alpha channel (linear), representing the true opacity of the foreground.

Project structure and outputs

• Matte: The raw linear alpha channel (EXR).
• FG: The raw foreground color (linear/straight, typically interpreted in sRGB for viewing in composites).
• Processed: An RGBA image where the foreground is premultiplied by the linear alpha (EXR). This is convenient for quick previews in Premiere/Resolve.
• Comp: A simple preview image with the key over a checkerboard background (PNG).

Backends and performance: Torch vs MLX

• CorridorKey supports two inference backends:
• Torch (default on Linux/Windows): CUDA, MPS, or CPU. This is the general-purpose backend.
• MLX (Apple Silicon): Native Metal acceleration with no Torch overhead; typically faster on M1+ machines.
• Backend selection order: CLI flag overrides environment variables, which override auto-detection. The recommended approach is to use MLX on Apple Silicon when available, otherwise rely on Torch on other platforms.
• MLX setup (Apple Silicon):
• Install MLX with uv sync --extra mlx.
• Obtain MLX weights (safetensors). You can either download pre-converted weights or convert from a .pth checkpoint using the provided scripts. The final weight file must be CorridorKeyModule/checkpoints/corridorkey_mlx.safetensors.
• Run with CORRIDORKEY_BACKEND=mlx to enable MLX inference.
• Additional notes:
• First-run on a new AMD ROCm GPU can trigger kernel autotuning (10–20 minutes). This caches for future runs.
• 16GB GPUs on Linux may require GTT to avoid VRAM OOM. GTT uses system RAM as overflow, which is slower but enables larger models on smaller GPUs. On Windows, this overflow is handled transparently.
• WSL2 support for ROCm-based workflows is more limited; WSL2 has a hard VRAM limit and cannot efficiently use GTT. For 16GB cards, Windows native or Linux native with a GPU that has more VRAM is advisable.

Advanced usage: AMD ROCm and Windows native

CorridorKey includes detailed guidance for AMD ROCm setups (Linux, Windows native, and WSL2). On ROCm, CorridorKey will:

• Enable experimental ROCm features for better performance on RDNA3/4 GPUs.
• Use rocBLAS and optimized kernels to accelerate attention and diffusion-like components.
• Auto-detect ROCm with environment variables and system paths for a smooth setup.
• Address first-run autotuning by caching results to minimize repeated tuning in subsequent runs.
• Warn about VRAM constraints on 16GB GPUs and suggest GTT as a workaround.

Troubleshooting and advanced tips

• If an auto-picked Torch backend unexpectedly selects MLX or vice versa, explicitly set CORRIDORKEYBACKEND=mlx or CORRIDORKEYBACKEND=torch to force a backend.
• Apple Silicon MPS troubleshooting:
• Enable verbose logging to see which device was selected and check MPS-related errors.
• If certain PyTorch operations are not yet supported on MPS, you can enable CPU fallback for those ops with PYTORCHENABLEMPS_FALLBACK=1.
• If MPS fallback is too slow, consider using native MLX (if available) to avoid PyTorch’s MPS layer entirely.
• For AMD ROCm users: ensure you’re on ROCm 7.2+ and follow the Linux native steps. Windows ROCm support is experimental and may not be fully functional for all configurations.
• If you encounter missing MLX weights, use the weight download/conversion steps described in the MLX setup to place corridorkey_mlx.safetensors in CorridorKeyModule/checkpoints.

Training, datasets, and future plans

• The author is open to sharing training programs and datasets if there is sustained interest and community support.
• Because training and fine-tuning involve large hardware requirements, progress depends on demand and community participation.
• In the meantime, CorridorKey’s architecture is designed to accommodate improvements and new AlphaHint generators as optional modules.

Licensing, permissions, and community

• CorridorKey is released with a license that blends permissive and non-commercial components. You can use CorridorKey in your projects, including commercial ones, but you may not repackage CorridorKey as a paid API service.
• If you release improvements or forks, you should keep the Corridor Key name and license intact.
• The project acknowledges and credits two major open-source efforts that contribute AlphaHint generation:
• Generative Video Matting (GVM) by AIM at Zhejiang University (BSD-2-Clause license).
• VideoMaMa by CVLAB KAIST (CC BY-NC 4.0), with certain model checkpoints under the Stability AI Community license. Users must respect those licenses when using VideoMaMa outputs and weights.
• Community extensions:
• CorridorKeyOpenVINO for Intel hardware (OpenVINO backend).
• The project invites enthusiasts to join the Corridor Creates Discord to share ideas, forks, and patches.

Where to go from here

• If you’re ready to experiment, start with the Unity of inputs: a green screen frame and a coarse AlphaHint. Try with auto-detected green screen first, then force green or blue to see how the blue-screen checkpoint improves results on blue-screen footage.
• For a smoother workflow, consider enabling the helper modules (GVM or VideoMaMa) to optimize the AlphaHint generation and refine results on challenging subjects.
• Use the Docker path if you want an isolated environment without local dependency management or if you’re provisioning a render farm. Ensure you mount your clips and the CorridorKeyModule/checkpoints folder into the container.
• Share your results and findings on the Corridor Creates Discord server so the community can iterate quickly, propose improvements, and push CorridorKey toward even better unmixing performance.

Closing thoughts

CorridorKey represents a focused attempt to rethink chroma-keying—not as a brittle binary mask task, but as a physics-informed unmixing process that respects color, illumination, and translucency. By predicting per-pixel foreground color and a linear alpha channel, CorridorKey allows VFX artists to preserve hair detail, motion blur, and delicate edges, enabling more seamless composites with less manual rotoscoping. It’s a brand-new release with a growing ecosystem, inviting collaboration, experimentation, and community-driven improvements.

If you’re curious to explore more or contribute, the CorridorCreates Discord and the project pages linked within the repository are the best starting points. The author welcomes patches, feature ideas, and real-world feedback as CorridorKey evolves toward an even more robust, accurate, and flexible tool for modern visual effects workflows.