“Most users struggled with setup” sounds clear until you ask how many users supplied the comments. One person may have written several times. Before asking AI to summarize feedback, choose whether the question concerns messages, people, or mentions of a theme. Those totals answer different questions.
Build a count you can trace
Here is a fictional set of four comments from three participants:
- C01, P01: setup.
- C02, P01: setup.
- C03, P02: setup and speed.
- C04, P03: speed.
Setup appears in three comments but comes from two participants. Speed appears in two comments, also from two participants. If the question is how many respondents mentioned each theme, both answers are 2/3. Adding those fractions gives more than the whole group because P02 belongs to both categories. That overlap is information, not a rounding error to repair.
Keep a stable participant code and a separate comment ID. If a participant cannot be identified reliably, mark that uncertainty instead of merging similar names or writing styles. Use only material suitable for the chosen AI service; participant codes do not by themselves remove all identifying details from free text.
Ask for evidence before the headline
Request a theme assignment for each comment, the supporting IDs, and separate comment and participant totals. Then check the assignments against the original wording. “Setup” is an editorial classification in this example, not a fact established merely by placing it in an AI table.
NIST's Generative AI Profile describes the risk of confidently wrong generated content. The worked example here is our own illustration, not a NIST counting procedure. A fluent summary still needs arithmetic and source checks.
Write the conclusion with its scope: “Two of the three respondents in this set mentioned setup.” Do not turn three respondents into all customers, or treat people who did not answer as satisfied. Keep the denominator beside the result so the next reader can understand what the number actually measures.