Key Takeaways
- NAD acknowledges that AI‑based testing can be useful for substantiating advertising claims, but it requires solid validation that the AI results reflect real‑world consumer experience.
- Claims such as “better than the rest in noisy conditions” and “up to a 6.5 dB advantage (70 % improvement)” were deemed unsubstantiated because Starkey did not show that Whisper’s speech‑to‑text measurements matched what hearing‑aid wearers actually perceive.
- Competitive comparisons must be based on current market data; using outdated product models weakens the legitimacy of the claim.
- Advertisers should validate any AI testing method against human‑subject outcomes, ensure the data are up‑to‑date, and be ready to adapt claims as the competitive landscape evolves.
Background of the NAD Dispute
In a recent National Advertising Division (NAD) case, hearing‑aid manufacturer Sonova challenged a series of performance claims made by rival Starkey about its Omega AI hearing aid. The decision touched on numerous advertising law issues—including the definition of an “ad” and the substantiation of preference claims—but the central question for this analysis was whether AI‑generated test data can legally support superiority claims. As NAD framed the matter, “the use of AI for claims substantiation is an emerging area with many potential benefits, which should be accommodated, but there must be evidence that the testing has been properly validated for the claims being supported, especially when those claims assert real‑world consumer benefits.” This statement set the stage for a detailed examination of Starkey’s methodology and the evidentiary burden it faced.
Starkey’s Performance Claims for Omega AI
Starkey advertised that the Omega AI hearing aid was “better than the rest in noisy conditions” and delivered “up to a 6.5 dB advantage (70 % improvement)” in speech intelligibility compared to all major brands. These statements were positioned as objective, quantifiable benefits that would help consumers understand how the device outperforms competitors in everyday listening environments. The NAD panel noted that a reasonable consumer would interpret “speech intelligibility” and “noisy conditions” as promises about real‑world performance, not merely laboratory metrics. Consequently, the claims required robust evidence linking the test results to actual user experience.
The AI‑Based Testing Methodology
To substantiate its assertions, Starkey placed various hearing aids on a dummy head, exposed them to recorded noisy soundscapes, and captured the devices’ audio output. The recordings were then processed by Whisper, an AI‑driven speech‑to‑text tool, which calculated the percentage of words correctly recognized. Starkey argued that this approach yielded a consistent, objective measurement free from the variability inherent in human listening tests. Moreover, the company contended that AI allowed for a far larger number of measurement repetitions than human subjects could tolerate, thereby improving the statistical reliability of the results and offering better predictions of real‑world performance.
Starkey’s Defense of AI Testing
In its submissions, Starkey emphasized three advantages of AI testing: consistency, objectivity, and scalability. The company asserted that “AI testing is consistent, objective, and removes the variation that comes with human test subjects.” It also highlighted that the ability to run numerous trials quickly produced a more robust data set, which it claimed translated into superior predictive power for how hearing aids would behave in actual noisy environments. Starkey’s own internal paper, however, cautioned that “more work was needed to establish whether AI‑based measurements are valid,” acknowledging a gap between the experimental setup and real‑world user perception.
NAD’s Position on Validation Requirements
The NAD panel directly addressed the evidentiary shortfall, stating: “AI testing for claims substantiation is an emerging area with many potential benefits, which should be accommodated, but there must be evidence that the testing has been properly validated for the claims being supported, especially when those claims assert real‑world consumer benefits.” This quotation underscored NAD’s willingness to accept novel methodologies, provided they are rigorously linked to the consumer experience the advertiser promises. The panel made clear that merely demonstrating technical precision in a lab setting is insufficient if the connection to everyday hearing‑aid use remains unproven.
NAD’s Determination of Insufficient Validation
Applying its validation standard, NAD found that Starkey’s record lacked any demonstration that Whisper’s word‑recognition scores corresponded to what actual hearing‑aid wearers experience in noisy settings. The panel noted that “nothing in the record showed that Whisper’s results match what actual hearing aid wearers experience.” Although Starkey’s own research admitted the need for further validation, the company attempted to shore up the claim by adding the term “predictive” to its language. NAD concluded that this qualifier did not cure the deficiency; the claim remained unsubstantiated because the underlying AI test had not been shown to predict real‑world speech intelligibility for consumers.
Concerns About Current Competitive Data
Beyond the validation issue, NAD raised a second problem: the competitiveness of the comparison relied on outdated product models. By the time the challenge was filed, four of the five competing brands Starkey had tested had already released updated flagship models. NAD stressed that “competitive claims must be based on current data,” and that using superseded devices distorted the market comparison and could mislead consumers about the relative performance of the Omega AI. The panel therefore recommended discontinuing the challenged claims until Starkey could substantiate them with testing against the current generation of rival products.
NAD’s Remedy and Guidance for Future Claims
While NAD advised Starkey to stop using the specific “better than the rest in noisy conditions” and “up to a 6.5 dB advantage” statements, it left the door open for responsibly crafted AI‑supported claims. The decision emphasized that “Starkey may still use AI to support ‘carefully tailored claims,’” provided the advertiser can demonstrate that the AI method has been validated for the specific benefit being asserted and that the competitive baseline reflects the latest offerings. This nuanced outcome signals NAD’s willingness to embrace innovation, but only when backed by rigorous evidence linking the novel test to consumer‑relevant outcomes.
Practical Implications for Advertisers
The takeaway for marketers, especially those in technology‑driven sectors, is twofold. First, before relying on any AI‑based testing to substantiate a performance claim, advertisers must conduct validation studies that show the AI’s output correlates with real‑world consumer experiences—such as human listening trials or field studies. Second, any comparative advertising must employ data drawn from the current marketplace; using legacy models risks violating the substantiation requirement and invites challenges from competitors or self‑regulatory bodies like NAD. As the NAD decision illustrates, a claim that adds a predictive qualifier without proper validation will likely be deemed insufficient. Advertisers should therefore treat AI as a tool that augments, rather than replaces, traditional substantiation methods, and be prepared to adjust claims as both technology and the competitive landscape evolve.
Looking Ahead: AI in Advertising Substantiation
The NAD ruling captures a pivotal moment in advertising law: the recognition that artificial intelligence offers promising efficiencies for claim support, yet it also raises novel evidentiary challenges. As AI tools become more sophisticated and pervasive across industries, regulators will likely continue to refine the validation threshold, insisting that any algorithmic metric be demonstrably tied to consumer perception. Companies that invest early in robust validation frameworks—pairing AI analytics with human‑subject testing or real‑world performance data—will be best positioned to leverage AI’s advantages while staying within the bounds of truthful advertising. In short, the future of substantiation lies not in choosing between AI and traditional methods, but in integrating them intelligently to deliver claims that are both innovative and provably true.
https://www.kelleydrye.com/viewpoints/blogs/ad-law-access/nad-addresses-whether-advertisers-can-use-ai-to-substantiate-claims