Sentiment Sampling Audit (QA)

Sentiment Sampling Audit (QA)

1. 01

   Pull the raw mention export behind the client-facing sentiment chart, not the dashboard summary

   Most platforms aggregate before display, so the summary hides the classification decisions. Export 200 to 500 mentions from the reporting window and work from the raw rows.

2. 02

   Hand-label a 100-mention random sample and record your own positive, negative, and neutral calls

   Use two reviewers on the same sample when the retainer supports it. Disagreements between reviewers are the most useful signal about where the model will also disagree.

3. 03

   Compute agreement between your labels and the platform's labels, then set a floor for what ships

   Below roughly 80 percent agreement, the sentiment split is not defensible in a client deck. Report the disagreement rate instead of the chart, or reclassify by hand.

4. 04

   Isolate the failure modes: sarcasm, emoji-only posts, non-English mentions, and quoted news headlines

   A headline that mentions the brand negatively is not the same as a negative customer post. YouScan and Brandwatch both expose enough raw text to separate these; if the tool does not, flag it as a coverage limit.

5. 05

   Check whether the query is catching the brand name in unrelated contexts

   Common-word brand names pull in noise that reads as negative sentiment. Add exclusion terms to the query and re-run the sample before the next report.

6. 06

   Document the sampling window, sample size, and known blind spots in the report appendix

   State the coverage gap in plain language: which channels, which languages, and which time zones are underrepresented. Clients accept limits they can see.

7. 07

   Route any sentiment claim that drives a client decision through a second reviewer before delivery

   Crisis framing, competitive share-of-voice claims, and campaign performance verdicts all qualify. One reviewer signs off on the number, one signs off on the interpretation.