Excerpt below from https://www.nngroup.com/articles/ai-magic-8-ball/
Should Designers Consider Magic-8-Ball Thinking When Building AI Features?
Intentionally or unintentionally exposing users to AI hallucinations introduces the possibility that they feel disappointed and confused. In the worst situations, those feelings could metastasize into lower engagement or abandonment of the entire product. While AI hallucination rates are still high enough to be an ongoing issue, professionals designing for AI integrations can add features to help users check or trace where outputs are coming from.
When implementing a genAI feature into a product, conduct user research to identify the likelihood that users will engage with magic-8-ball thinking in the context of your implementation. If your product’s implementation of AI includes generating text-based content, consider adding features to prevent magic-8-ball thinking.
Examples of such features include:
- Annotating genAI outputs with references to information sources used in generating the response (e.g., in the form of in-line citations, information cards at the top or bottom of the output, or See sources button)
- Despite reprehensible behavior regarding intellectual property theft, Perplexity has a good demonstration of annotating responses with sources in its core product.
- Dovetail, a qualitative research platform, summarizes transcripts from video recordings using genAI; to improve users’ ability to check its output, Dovetail provides links to timestamped sections of each recording inside the summary.
- Allowing users to conduct a traditional web or literature search inside the product, within the context of the genAI response
- Google’s Gemini chat product includes a Double check response button, which evaluates the genAI’s output and provides dropdown options to expand separate Google searches for different components of Gemini’s output.
