How I Upscaled 4,000 Video Frames with RunPod and ComfyUI

I spent a weekend building an AI video upscaling pipeline from scratch. Not by sending files to a polished enhancement API, but by renting GPUs, running ComfyUI myself, and distributing the work across multiple machines.

The final pipeline processed roughly 4,000 frames across four GPU workers in about two to three hours. The total RunPod bill was $21.20.

The upscaling was only half the experiment. The more interesting problem was turning thousands of independent images into a workload that could be distributed, monitored, restarted, and kept within a sensible budget.

The Problem I Wanted to Solve

I had several videos with image quality I wanted to improve. The simplest option would have been a hosted enhancement service, but that would have meant giving up control over the model, workflow, processing limits, and potentially the content I could submit.

I wanted to understand the full pipeline instead:

• Extract frames from the source videos
• Process them with a reproducible ComfyUI workflow
• Run the workload on rented GPUs
• Add more workers when one machine was too slow
• Keep the inputs and outputs after shutting the GPUs down
• Know exactly what the experiment cost

My first requirement was deliberately specific: extract one frame every two seconds, upscale every extracted frame, and finish in hours rather than days.

That definition mattered. “Improve these videos” is vague. A numbered directory of independent images is a batch-processing problem, and batch-processing problems are much easier to reason about.

Turning Video into a Batch Workload

I used FFmpeg to sample the source videos:

ffmpeg -i input.mp4 -vf "fps=1/2" frames/frame_%06d.png

This produced one PNG every two seconds with zero-padded, sequential filenames.

The naming convention was not cosmetic. Stable filenames gave every frame a unique identity, preserved the original order, and made it possible to check whether a frame had already been processed. That last property becomes important as soon as workers can fail or jobs need to be restarted.

After extraction, I had roughly 4,000 independent inputs. No frame depended on another frame, so the dataset could be divided safely between workers without coordination during inference.

This is the ideal shape for horizontal scaling.

Why I Used RunPod and ComfyUI

I used RunPod for GPU infrastructure and ComfyUI for the image-processing workflow.

RunPod gave me temporary GPU machines without committing to permanent infrastructure. More importantly, I could attach persistent shared storage to the workers. The expensive compute could disappear when the job finished while the models, source frames, workflow, and results remained available.

ComfyUI gave me a visual way to build and validate the upscaling graph before automating it. Once the workflow produced the result I wanted, the same graph could be submitted repeatedly for every input frame.

That separation worked well:

• ComfyUI defined how one frame should be processed
• The batch scripts defined which frame each worker should process
• RunPod provided where the workflow should run
• Shared storage provided the common state between workers

I did not need a complex distributed framework. The problem was already divisible.

The Architecture

The final setup looked like this:

Source videos
      |
      v
FFmpeg frame extraction
(one frame every two seconds)
      |
      v
RunPod persistent storage
  ├── input frames
  ├── ComfyUI workflow
  ├── model files
  └── upscaled output
      |
      +------------+------------+------------+
      |            |            |            |
      v            v            v            v
 GPU Worker 1  GPU Worker 2  GPU Worker 3  GPU Worker 4
   ComfyUI       ComfyUI       ComfyUI       ComfyUI
      |            |            |            |
      +------------+------------+------------+
                         |
                         v
                 Upscaled frame set

Each pod mounted the same persistent volume and ran the same ComfyUI workflow. The only meaningful difference between workers was the subset of frames assigned to each one.

Shared storage acted as the simplest possible coordination layer. Workers could read from one input directory and write to one output directory without copying the complete dataset to every machine.

There are limits to this design. Shared storage can become a bottleneck when many workers read large files simultaneously, and multiple workers must never claim the same input accidentally. At this scale, four workers and deterministic partitions kept both problems manageable.

Why Four GPU Workers Beat One Faster Script

My initial throughput suggested that a single pod could take a couple of days to finish the whole dataset.

I could have spent the weekend profiling one worker, tuning every setting, and trying to save a few seconds per image. Instead, I split the input set into four partitions and ran them in parallel.

The entire job finished in roughly two to three hours.

This worked because the frames were independent. Adding workers increased aggregate throughput without changing the ComfyUI graph or the individual inference process.

The useful optimization was not hidden inside the model. It was in the architecture.

Horizontal scaling is not always the answer. It would have helped much less if every task depended on the previous one, if model startup dominated the runtime, or if shared storage could not feed the GPUs quickly enough. But for thousands of independent images, parallel workers were the obvious lever.

Making the Pipeline Restartable

Starting four machines is easy. Making sure a partial failure does not waste the entire run is the part that matters.

I treated the output filename as the completion record for each input. Before submitting a frame to ComfyUI, a worker could check whether the corresponding output already existed. If it did, the frame was skipped.

That made the processing effectively idempotent at the file level:

• Restarting a worker did not require starting its entire partition again
• Completed frames remained on persistent storage
• A failed pod did not destroy the results produced by the other pods
• Duplicate GPU work could be avoided

For a weekend experiment, this was enough. A production system would use a real job queue, explicit task states, retry limits, and structured error reporting. But the underlying idea would be the same: every unit of work needs an identity and a visible completion state.

The Results

The pipeline processed approximately 4,000 frames. Clean source material with visible texture produced the strongest results. Heavily compressed or blurry frames improved less.

That is the reality of AI upscaling: a model can reconstruct plausible detail and improve perceived sharpness, but it cannot recover information that was never captured. More generated detail does not automatically mean more accurate detail.

I evaluated the output using identical crops rather than relying on zoomed-out comparisons. At full-frame size, almost every sharpening operation looks convincing. Matching crops make halos, invented textures, damaged faces, and inconsistent fine detail much easier to spot.

The results were good enough to validate the workflow, but I would still test different models and settings before applying one configuration to every kind of source video.

What It Cost

According to the RunPod billing summary, the experiment cost:

Item	Cost
GPU Cloud usage	$21.17
Serverless	$0.03
Total	$21.20

At roughly 4,000 frames, that works out to about $0.0053 per processed frame, or just over half a cent. I treat that as the cost of this experiment, not a universal benchmark: GPU type, model, resolution, storage, and utilization can change the number significantly.

Almost the entire bill came from GPU compute. That means the largest cost improvements would come from keeping the GPUs busy and shutting them down immediately when their partitions finish.

A production version could reduce waste further by:

• Benchmarking several GPU types by cost per completed frame
• Building a container image instead of installing dependencies at startup
• Downloading model files once to persistent storage
• Automatically terminating idle workers
• Skipping outputs that already exist
• Recording failed frames for targeted retries
• Scaling the worker count based on queue depth

The cheapest GPU per hour is not necessarily the cheapest GPU for the job. What matters is the cost per valid output.

How AI Helped Me Build It

I used AI heavily throughout the project. It helped me research the available options, generate setup and automation scripts, troubleshoot dependency problems, and iterate on the batch-processing logic.

That did not make the engineering decisions disappear.

I still had to define the workload, choose the architecture, validate the ComfyUI output, decide how to partition the frames, watch the infrastructure cost, and determine when the result was good enough.

This is the part of AI-assisted engineering that interests me most. The advantage is not that an AI can type shell commands. The advantage is that I can move from an unfamiliar infrastructure problem to a working experiment much faster while keeping responsibility for the system.

AI accelerated execution. It did not own the outcome.

What I Would Change for Production

The weekend version was intentionally small. If I were turning it into a service, I would change several parts of the design.

Use a proper job queue. Static partitions are fine for a known dataset, but a queue balances uneven processing times and lets replacement workers pick up unfinished frames.

Package the environment. A versioned container image would make every worker start with the same ComfyUI version, custom nodes, model configuration, and system dependencies.

Add observability. I would track queue depth, frames per minute, GPU utilization, failures, retry counts, and cost per output instead of monitoring folders manually.

Separate validation from processing. Outputs should be checked for missing files, invalid dimensions, corrupt images, and naming gaps before the job is considered complete.

Treat credentials and network access seriously. Temporary GPU machines are still infrastructure. SSH keys, exposed ports, secrets, and stored source material need the same care as any other production environment.

Choose models based on the input. A single workflow is convenient, but animation, faces, compressed footage, and clean digital video do not necessarily benefit from the same upscaler or settings.

The prototype proved the processing model. Production engineering would be about making it repeatable, observable, and safe.

What I Learned

Clear Requirements Make AI More Useful

“Upscale a video” leaves too many decisions undefined.

“Extract one frame every two seconds, process every frame with a fixed ComfyUI workflow, distribute the inputs across four workers, preserve ordering, and skip completed outputs” is something an AI assistant can help implement and verify.

The quality of the automation still depends on the quality of the specification.

Parallelism Can Be More Valuable Than Premature Optimization

I did not need to make one worker perfect. I needed to recognize that the workload could be divided.

Four ordinary workers finishing in a few hours were more useful than one carefully tuned worker running for days.

Persistent Storage Changes the Failure Model

Keeping inputs and outputs outside the lifecycle of a GPU pod made the system much easier to restart. Compute became disposable; the job state did not.

That is a small architectural decision with a large operational impact.

Self-Hosting Buys Control, Not Convenience

Running the model myself gave me control over the workflow, data, model choice, and scaling strategy. In exchange, I became responsible for installation, reliability, security, monitoring, cleanup, and cost.

That trade-off is worth it when control matters. It is unnecessary when a hosted API already satisfies the requirements.

The Bottom Line

I started the weekend with several videos and no finished implementation. A few hours later, I had a distributed GPU pipeline that had processed around 4,000 frames for about $21.

The result was not a giant platform or a complicated distributed system. It was a focused batch pipeline built around the shape of the problem: independent inputs, disposable compute, shared persistent state, and enough parallelism to finish quickly.

The same pattern applies beyond video upscaling. OCR, document conversion, media processing, synthetic data generation, and other GPU-heavy batch jobs can all benefit from the same approach.

Knowing how to call an AI model is useful. Knowing how to turn it into a reliable, cost-aware workflow is where the real engineering starts.

Working on an AI or GPU-processing idea that needs to move from experiment to working system? I help teams design practical AI workflows, validate them quickly, and build the infrastructure needed to run them reliably.

Contact me at [email protected] to discuss your project.