A supermarket or hypermarket migrating online rarely starts from zero — it starts from an existing POS or ERP export of 15,000 to 30,000 SKUs, most of it messy: inconsistent category names, missing images, barcodes that do not quite match between systems. The failure mode is not usually the import itself timing out. It is launching with a catalogue full of quiet errors nobody checked for before go-live.
Clean the file before it ever touches the platform
Every hour spent cleaning the source file before import saves several hours of after-the-fact fixes in the live catalogue. The three fields worth the most attention are barcode (deduplicated and consistently formatted), category (mapped to a single canonical tree rather than whatever the old POS called things), and price (checked for the classic spreadsheet errors — a misplaced decimal turning AED 45 into AED 4,500 is the single most common launch-week support ticket for a large migration).
Import in batches, not one 20,000-row file
Even with a background job and a live progress bar built specifically for large catalogues, splitting a 20,000-item migration into batches of 2,000 to 5,000 makes errors far easier to isolate — if batch six fails validation, you are debugging 3,000 rows, not 20,000. It also means the store can start listing categories and testing the storefront experience with real data well before the entire catalogue is loaded, rather than waiting for one enormous file to succeed or fail as a single unit.
Barcode matching is what makes re-imports safe
The first import creates the catalogue; every import after that should update it, not duplicate it. Matching subsequent files by barcode — for price updates, image refreshes, translation additions or external-ID syncs — is what lets a shop re-run a weekly price file from head office without recreating every product from scratch and without a duplicate-removal clean-up becoming a recurring monthly chore.
Test the picker and search experience, not just the admin list
A catalogue that looks correct in the back-office product list can still be broken for the two people who actually rely on it operationally: the picker scanning barcodes on the floor, and the shopper searching the storefront. Before launch, physically scan a random sample of barcodes against the picker app, and run a dozen real-world search queries — including common misspellings and Arabic-dialect terms — against the live storefront search. Both catch a different class of import error than a spreadsheet review ever will.
None of this is exotic engineering. It is the same discipline any large data migration needs — clean the source, batch the load, match rather than duplicate, and test from the user’s side rather than the admin screen — applied specifically to the two systems that actually notice when a catalogue is wrong: the picker on the floor, and the customer searching for what they came to buy.
Keep the master catalogue as the source of truth
Once the initial migration is live, resist the temptation to let branch staff edit products directly in the live catalogue one at a time. A master-item catalogue that head office maintains and pushes down — for a franchise or multi-branch operation especially — keeps 20,000 items consistent across every branch, rather than slowly drifting into 20,000 items with five slightly different versions of the truth depending on which branch touched them last.
Plan the launch-day checklist before batch one runs
Write the go-live checklist before the first batch, not after the last one — it should cover a spot-check of tax category assignment across a sample of products, a review of which categories carry stock control versus which are hidden-when-out-of-stock, and confirmation that new-arrival and sort-position fields imported correctly rather than defaulting to whatever the spreadsheet happened to leave blank. Catching a wrong tax category on 200 products during a scheduled check is a five-minute fix; catching it from a customer complaint about an incorrect invoice three weeks after launch is a much longer one.