Two delivery records contain the same item and quantity. One may be a repeated import, or both may describe real deliveries. Similar wording is not enough to decide which row can disappear.
Define identity before asking for matches
Choose fields that identify a record in this particular list. An order identifier plus a line number might work if that combination is unique in the source system. A product name alone usually answers a different question: what was ordered, not which event this was.
Ask the assistant for proposed duplicate groups, keeping every original row and identifier. Separate exact matches from approximate matches and leave uncertain cases for review. NIST describes the risk of confident but incorrect generative output; the checking routine here is a practical application, not a tested guarantee for a particular model.
Prepare examples with expected decisions before evaluating the answer:
- The same order and line imported twice: a candidate duplicate to verify.
- Different order identifiers with the same item and quantity: keep both events.
- A spelling variation with no shared identifier: investigate rather than merge automatically.
Preserve what a merge would remove
Compare all fields within each proposed group. Different notes, dates or quantities need an explicit decision, even when the identifying fields match. A conflicting field can signal a correction, not a disposable copy.
Work in a separate output file. Record which original rows map to each retained record and why you accepted the merge. Compare row counts and relevant totals before and after, accounting for each expected change. If you cannot explain a difference, keep the source intact and review the group. A shorter list is useful only when it still represents the original events correctly.