Verification
AI can help you find and organize sources, but you own the accuracy. Here's how to verify AI-assisted work before publishing.
The Fluency Problem
AI generates confident-sounding text. This is called fluency — the text reads smoothly, uses proper grammar, and sounds authoritative. But fluency is not accuracy.
The danger: Because AI output "sounds right," we're tempted to trust it without checking. This is the "fluency illusion" — mistaking smooth prose for verified truth.
Your course has discussed this concept: "grasping a coherent mechanism" that doesn't actually exist. AI can construct a mechanism that feels coherent but isn't grounded in reality.
What Can Go Wrong
Hallucinated Sources
AI can invent sources that don't exist — fake studies, non-existent articles, fabricated quotes. These citations look real but lead nowhere.
Red flag: A source that's suspiciously perfect for your argument, with a title that exactly matches what you need.
Misrepresented Statistics
AI might cite a real source but misstate what it says. "40% of jobs will be eliminated" might actually be "40% of jobs will be affected in some way."
Red flag: Statistics with no context (affected how? over what timeframe?).
Quote Drift
AI might paraphrase a quote so much that it no longer reflects what the person actually said, or combine multiple quotes into one.
Red flag: Quotes that sound too perfect or too convenient.
Outdated Information
AI's training data has a cutoff date. It might cite old statistics, reference defunct organizations, or miss recent developments.
Red flag: Claims about "current" or "recent" events with no specific dates.
The Verification Workflow
Go through your draft and mark every statement that could be verified or falsified:
- Statistics and percentages
- Quotes attributed to people
- Claims about what studies or reports say
- Historical facts or events
- Descriptions of how things work
If you're unsure whether something is a factual claim, mark it anyway.
For each factual claim, ask: "Where did this come from?"
- If you added it yourself: You should have a source
- If AI added it: You need to verify it exists
- If it's common knowledge: Consider whether your reader would agree it's common knowledge
For each source AI mentioned, search for it:
- Google the title exactly as written
- Search for the author + organization
- Check the organization's website directly
If you can't find a source after a reasonable search, assume it doesn't exist.
Once you find the source, read it yourself. Verify:
- Does it say what AI claims it says?
- Is the statistic accurate? In context?
- Are quotes exact or paraphrased?
- Is the interpretation fair?
AI often gets the gist right but the details wrong. The details matter.
When you find an error:
- Correctable: Fix the claim to match the actual source
- Source doesn't exist: Remove the claim or find a real source
- Source says something different: Update your argument or find a source that supports your actual point
Don't keep a claim just because it sounds good. Accuracy beats eloquence.
NotebookLM vs. Gems: Verification Differences
NotebookLM: Grounded but Limited
NotebookLM only answers based on sources you provide. This is safer — but with caveats:
- Good: Answers cite your sources, so you can check them
- Good: Won't invent sources it doesn't have
- Caution: Still might misinterpret what sources say
- Caution: Discover Sources feature finds real sources, but you still need to evaluate their credibility
Verification with NotebookLM: When it cites a source, click the citation number to see the passage. Check that the passage actually supports the claim.
Gems: Powerful but Ungrounded
Gems generate from training data unless you give them sources. This means:
- Risk: Can invent sources, statistics, and quotes
- Risk: May blend real information with fabrications
- Mitigation: Use [CITATION NEEDED] placeholders in your Gem instructions
- Mitigation: Always run verification step before publishing
Verification with Gems: Treat every factual claim as unverified until you check it yourself. The Gem doesn't know what's true — it knows what sounds true.
Specific Things to Verify
Statistics
| Claim Type | What to Check |
|---|---|
| "40% of jobs are exposed to AI" | Find the original report. What does "exposed" mean? Is it 40% exactly? |
| "Studies show that..." | Which studies? By whom? Published where? When? |
| "Research indicates..." | This is often a red flag for AI vagueness. Get specific. |
Quotes
| Claim Type | What to Check |
|---|---|
| Direct quotes from people | Search for the exact quote. Is it attributed correctly? |
| Quotes from peer posts | Check the actual post. Did they say exactly that? |
| Paraphrased positions | Does the paraphrase fairly represent what they said? |
Claims About AI and Technology
| Claim Type | What to Check |
|---|---|
| "AI can/cannot do X" | This changes rapidly. Check recent sources. |
| "Companies are doing X" | Find a specific example. Name the company. |
| "The consensus is..." | Whose consensus? Among whom? Is there actually consensus? |
The "Could I Defend This?" Test
For every claim in your post, ask yourself:
If someone challenged this claim in class, could I point to a source and explain why it's true?
If the answer is no, either:
- Find a source that supports it
- Qualify it as your opinion ("I believe..." / "It seems likely...")
- Remove it
When AI Verification Helps
You can use AI to help with verification — but you still need to do the final check:
- Ask NotebookLM: "Does [source] actually say [claim]? Quote the passage." — Then read the passage yourself.
- Ask a Gem: "What claims in this draft need verification?" — Then verify each one manually.
- Search with AI: Use Gemini to help find sources — then check that the sources exist and say what AI claims.
The verification loop: AI can help you identify what needs checking. But the final "yes, this is accurate" judgment is always yours.