Turning Bad Seller Photos Into a Clean Catalog, Automatically

On any marketplace, your catalog is only as good as the worst seller photo on it. Sellers upload what they have: busy backgrounds, harsh lighting, the product half out of frame, a watermark from wherever they sourced the image. Clean, consistent imagery measurably lifts trust and conversion, because a uniform grid of crisp product shots simply sells more than a jumble, but hand-editing thousands of uploads does not scale and outsourcing it is slow, expensive, and inconsistent. So you build the cleanup into the pipeline itself, with three stages, each a different kind of model doing the one thing it is best at.

Segment

The first job is to separate the product from its background. Segment Anything (SAM2) is the right tool because it is promptable and class-agnostic: you do not train a model per product category, which matters enormously when sellers list everything from sneakers to furniture to handmade jewellery. You get a clean mask and a cutout you can composite onto anything. Getting the mask right is the foundation, because every later stage inherits its mistakes, so this is where you spend your quality budget. A botched edge here becomes a product that looks like it was cut out with scissors three steps later, so it is worth an extra pass to get clean boundaries on hair, transparency, and thin structures.

Understand

The second job is to know what you are looking at. CLIP embeddings give you zero-shot classification and tagging: you match an image against a vocabulary of categories and attributes without a labelled training set, which auto-organizes listings and powers search. A vision-language model like LLaVA goes further, describing the image and answering questions about it, including quality questions like "is the product clearly visible, is it blurry, is there a watermark, is there a person in the shot?" That is the stage that lets you flag low-quality or plainly wrong uploads for a human instead of silently shipping garbage into the catalog. It is also where you catch the listing whose photo does not match its title, which is its own quiet category of marketplace problem.

Generate

The third job is to make it consistent. A generative model, diffusion or a DALL-E style system, composites the clean cutout onto a uniform, on-brand background and produces standardized variants: a square crop, consistent padding, a neutral or branded backdrop. The output is a catalog that looks like one company shot every product in the same studio, from inputs that were anything but. Underneath, this is a Python (FastAPI) service fed by a crawler that ingests the catalog, all running containerized with GPU access in the cloud. The models are the easy part to name and the hard part to operate.

Why three models and not one

It is reasonable to ask why this is three separate models rather than one big one that takes a messy photo and returns a clean one, and the answer is control and debuggability. A single end-to-end model is a black box: when it produces a bad result, you cannot tell whether it misunderstood the product, botched the cutout, or invented the background, and you cannot fix one stage without retraining the whole thing. Splitting the pipeline into segment, understand, and generate gives each stage a single responsibility, an inspectable output you can check and gate on, and the freedom to swap or upgrade one model without disturbing the others. It also lets you spend effort where it pays: segmentation quality matters more than anything, so you invest there, while the generation step can be the cheapest thing that produces a consistent background. Composability beats a monolith here for the same reason it does in any other system, because you can reason about, test, and improve the parts independently.

Cost is the architecture

These models are slow and GPU-hungry, and GPU time is the dominant cost of the whole system, so the engineering is mostly about not wasting it. You batch aggressively, because a GPU processing many images at once is far cheaper per image than one at a time. You cache by content hash so you never reprocess the same image twice, which matters more than you would expect because sellers re-upload identical photos constantly. And you run cheap pre-filters before anything touches a GPU at all: is this even a product photo, is it a near-duplicate of one you have already done, is it large enough to be worth processing? Each cheap "no" you can answer before the expensive stage is money saved at scale. The architecture, in other words, is shaped less by the models and more by the economics of running them.

Keep a human in the loop

You also set confidence thresholds and route low-confidence cases to a human rather than trusting the pipeline blindly. That branch is what makes the automation safe to leave running unattended, because the failure mode of a fully-automatic pipeline is that it fails silently and at scale.

The review queue is not a dumping ground, it is a feedback signal. The categories of input that land there most often tell you where to improve the pipeline next, and the human decisions become labelled data you can use to tune your thresholds. A good loop gets quieter over time as the automatic path learns to handle more.

The honest parts

Two more things you learn the hard way. First, you have to evaluate the pipeline, not just run it. You spot-check a sample of outputs continuously, because a model update or a new category of input can quietly degrade quality, and "looked fine on the three images I tried" is not a quality bar for a catalog of millions. Pick a few metrics you can actually track, segmentation failures caught, human-review rate, conversion on cleaned versus original images, and watch them. Second, you resist the temptation to make photos misleadingly perfect. Generative tools will happily invent product detail that is not there, a texture, a logo, a colour the real item does not have, and crossing that line is both a trust problem and, depending on the category, a legal one. The goal is a faithful, clean product shot, not a fictional one. The through-line is what makes the whole thing work: segment, then understand, then generate, turns an unscalable manual chore into an automated quality layer, as long as a human stays in the loop exactly where the model is unsure.