From the Google edition of October 31, 2026
Searchable midterm study: 1 in 12 AI chatbot answers on local political issues contradicted official data; Gemini false in 6.2%, Google AI Mode 2.9%
Searchable Limited (AI search analytics company) · State and local government / elections and civic information · United States (Atlanta, Charlotte, Houston, Las Vegas, Portland, Maine and their states)
- Publisher
- Searchable
- Original publication
- 2026-10-07 (updated)
- Source retrieved
- 2026-11-01
What happened
Searchable asked ChatGPT, Gemini, Google AI Mode and Perplexity 106 questions about local political issues in five U.S. metro areas and graded 7,449 answers against official government data. Overall 8.2% (610) of answers were false. Perplexity was false 22.6% of the time, Gemini 6.2%, ChatGPT 4.4% and Google AI Mode 2.9%. Gemini was most often wrong on cost of living and housing (13.0%) and schools (8.7%). Gemini read an average of 2 pages from 2 websites per answer, the fewest of the four, and drew under a third (32.2%) of its pages from government websites, the lowest share, with 15.1% from company websites. Why now: the page was updated Oct. 7, before the last google run, but the Nov. 3, 2026 midterms are three days after this edition, making its findings on Google tools newly relevant to election offices and civic-information teams.
Why it matters
Relevant to state and local election offices, communications teams and agencies that publish official data residents may access through Google AI products: it quantifies how often Gemini and Google AI Mode misstate local official figures and how little Gemini relies on government pages. It evaluates consumer chatbots, not Google Public Sector enterprise deployments grounded on agency data.
Evidence and measured results
Structured test: 106 questions (59 single-answer factual, 47 built from fact-checked false claims), multiple prompt forms, each asked five times in fresh sessions via anonymized logged-in accounts (not APIs); an answer counted false only when two graders agreed, with correct answers confirmed by two researchers against official sources (Census, BLS, FBI, EIA, CDC, NAEP). 43,079 pages read were classified by type. Not peer reviewed; model versions and test dates are not stated on the page.
Limitations and uncertainty
Produced by a company that sells AI-visibility products, a commercial interest in showing chatbot errors; five metros only; model versions, dates of testing and confidence intervals not disclosed; consumer interfaces only. Published before the previous google run; included for election timing. Interpretation in role guidance is ours.
Put this evidence to work
Lighthouse Advisory interpretation, grounded in this source. Enriched 2026-11-01; this does not change the original publication date. Each role follows the same checklist so takeaways compare across sources.
Sales
Role takeaway
- Customer problem
- Election offices and agencies need residents to receive accurate local facts, but AI chatbots, including Gemini, sometimes contradict official data.
- Stakeholders
- Election officials
- Agency communications and public information officers
- State and local CIOs
- Open data program managers
- Discovery questions
- Which official figures do residents most often ask about?
- Is any AI assistant planned for resident or voter questions, and how will it be grounded?
- How does the agency currently monitor how AI tools describe its data?
- Who signs off on AI-generated public answers?
- Value hypothesis
- A grounded, citation-required assistant with an accuracy test suite could reduce misstatements relative to consumer chatbots; this must be demonstrated on the agency's own data.
- Potential engagement
- A bounded accuracy-evaluation exercise replicating Searchable's dual-graded method on an agency's published figures before any resident-facing launch.
- Unsupported claims
- Accuracy of Gemini for government services or agency-grounded deployments
- That Google AI Mode is reliable for election information
- Error rates for model versions not tested
- Results beyond the five metros studied
Pre-sales engineering
Role takeaway
- Fit
- Applies to any agency considering Gemini or Vertex AI for resident-facing information retrieval.
- Architecture and integration
- Ground on authoritative agency datasets with retrieval restricted to official sources, mandatory citations, and logging of answers for review.
- Prerequisites
- Machine-readable official datasets with update dates
- Question bank including known false claims
- Two-reviewer grading process
- Constraints
- Study covers consumer interfaces only
- Model versions and test dates undisclosed
- Five metro areas only
- Security
- Integrity controls: source allow-lists, citation verification and answer logging to detect and correct misinformation.
- Proof of value
- Replicate the question-bank test on current Gemini and Vertex AI configurations
- Measure false-answer rate on frequently updated figures
- Compare grounded versus open-web retrieval error rates
Delivery
Role takeaway
- Work
- Build and run a recurring accuracy test suite for any AI assistant answering resident questions about local data.
- Dependencies
- Curated official data
- Reviewer time
- Vendor configuration access
- Ownership
- Elections or communications office, with agency IT and open data teams.
- Skills and adoption
- Staff need training in evaluation design and grading consistency; the study's two-grader method is a practical template.
- Governance checkpoints
- Pre-launch accuracy test
- Pre-election re-test
- Post-incident review of any false answer
- Acceptance criteria
- Proposed: false-answer rate below an agency-set threshold on the agency question bank
- Proposed: 100% of answers cite an official source
- Proposed: grounding data refreshed within a defined period after each official update
- Risks
- Misinformation near elections
- Over-reliance on vendor accuracy claims
- Stale data on frequently updated figures
Implementation considerations
Lighthouse Advisory interpretation across the operating dimensions a public-sector buyer must settle before this evidence becomes a design. Each note answers the question under its heading for this specific source.
Architecture and integration
What must connect, and where does the AI sit in the workflow?
Shows that retrieval breadth and source mix differ sharply between Gemini (few pages) and Google AI Mode; for agency deployments, grounding on authoritative datasets rather than open web retrieval is the relevant design control the study implicitly supports.
Governance
Who approves, reviews and stays accountable for outcomes?
Supports agency practices such as publishing machine-readable official figures, monitoring how AI tools describe agency data, and labeling AI-assisted public answers. It does not test enterprise configurations with agency-controlled grounding.
Security and privacy
What data, permissions and controls need testing?
No privacy findings; the risk is integrity of public information, including potential misinformation about local conditions near an election.
Accessibility and workforce
Who is affected, and what skills or accommodations follow?
Residents relying on AI summaries, including those with limited time or language barriers, are exposed to these errors; agencies should keep authoritative plain-language sources easy to find.
Procurement
What should contracts, pricing and exit terms secure?
Agencies can require vendors to support agency-run accuracy testing and source restrictions as contract terms.
Operating model
Which teams own the service once it runs?
Monitoring how AI tools represent agency data becomes an ongoing communications function, especially around elections.
Publication history
- 2026-10-31Google · Issue 034 resources
Stable resource ID: searchable-ai-local-issues-misinformation-midterms-gemini-6-2pct-false-20261007