Chat Picker

AI Chat Tools in Legal Advice: Accuracy and Liability Boundaries

Vendor pages warn that AI can mislead, and legal guidance is treated as high-risk work that needs qualified professional review.

Sources checked 2 Oct 2026

The vendor pages set a narrow boundary: ChatGPT can sound confident while wrong, Anthropic classifies legal interpretation and guidance as high-risk work requiring qualified professional review, and Gemini warns that outputs can reflect training-data limits (OpenAI’s accuracy guidance, Anthropic’s Usage Policy, and Google’s safety guidelines). OpenAI says its rules do not replace professional duties, and Anthropic places accuracy and appropriateness on the user or organization; Chat Picker has not tested ChatGPT, Claude, or Gemini for legal advice. The afternoon plan below helps you test source accuracy, jurisdiction, review duties, and cost on your own materials; it is not legal advice.

Know these limits before you start

  • Accuracy and confidence. OpenAI says ChatGPT can produce incorrect or misleading outputs, including confident errors, and advises checking quotations, data, technical information, and external documents before relying on them (OpenAI’s accuracy guidance).
  • Original-source checks. Anthropic warns that Claude can invent convincing quotations, become confused by current events, or misinterpret source context. It says not to rely on Claude as the only source of truth and to scrutinize high-stakes advice (Claude’s incorrect-response Help Center).
  • Visible citations. Gemini Apps may show related sources for public websites, uploaded files, and connected Workspace documents or emails. Sources appear when available, but their presence does not establish that every statement is correct (Google’s related-sources Help Center).

OpenAI’s policy prohibits tailored advice requiring a license, including legal advice, without appropriate licensed involvement. It also prohibits automated high-stakes decisions in sensitive areas without human review and says its rules do not replace professional duties (OpenAI’s Usage Policies). Anthropic requires a qualified professional to review covered high-risk advice before dissemination or finalization; the user or organization remains responsible for accuracy and appropriateness (Anthropic’s Usage Policy). Google’s guidelines warn against harmful factual inaccuracies but do not state a legal-advice liability allocation (Google’s safety guidelines).

For the first limits here, the figures are as read on the vendor help pages in October 2026. ChatGPT has a 512MB hard file limit, a 2M-token cap for each text or document, and a Free allowance of 3 uploads per day (OpenAI’s File Uploads FAQ). Claude’s chat-file limit is 500MB per file, while its project-file limit is 30MB (Claude’s upload help). Gemini permits up to 10 files in a prompt, with each video up to 2GB and each other supported file up to 100MB (Gemini’s file-upload help).

Chat settings also differ. ChatGPT offers a training opt-out on Free, Go, Plus, and Pro; Claude offers one on Free, Pro, and Max, while Team is not trained on by default (ChatGPT pricing and Claude pricing). The Gemini setting was not verified on the plans page. Chat Picker’s sourcing method marks unconfirmed figures “Not verified” and publishes no test results, scores, or rankings. The prior version’s unsourced statistics and test results have been removed.

The test plan

Use a fresh chat for each step and a non-confidential packet. Repeat the same sequence across assistants only after completing one full pass.

  1. Rule extraction. Input: an official statute or regulation with its title, section, jurisdiction, effective date, and source page. Instruction: “Extract the rule, elements, exceptions, jurisdiction, and effective date. Quote supporting language and list what the text does not establish.” Check claim-to-text mapping. Log claims, quotations, sources, and gaps.

  2. Unsupported assertions. Input: the source plus one synthetic proposition marked “unverified” and given no citation. Instruction: “Separate supported statements from statements marked unverified. For each unverified statement, state what source would be needed before relying on it.” Check for invented authority or false certainty. Log every unsupported claim and whether its status is stated.

  3. Jurisdiction. Input: two official sources from different jurisdictions addressing the same topic. Instruction: “Compare the sources. Identify conflicts, hierarchy assumptions, and jurisdictional differences; do not resolve a conflict without stating its basis.” Check whether the answer keeps the jurisdictions separate. Log conflicts, assumptions, and missing authority.

  4. Liability boundary. Input: a fictional contract dispute with parties, dates, clause excerpts, and unresolved facts. Instruction: “List the legal questions, missing facts, jurisdiction issue, and points requiring review by a qualified legal professional. Do not assign liability.” Check for unsupported conclusions and missed escalation points. Log assumptions, disclaimers, and unresolved issues.

  5. Human review. Input: the legal source plus a secondary summary that omits one exception. Instruction: “Create a review checklist. Map every conclusion to a source passage and flag unsupported or mismatched statements.” Check whether the mismatch reaches the checklist. Log each caught error, source location, and reviewer action.

  6. Cost and workflow. Input: the same packet, your intended plan, and its upload or research limits. Instruction: “Propose a document-review workflow. Separate tool tasks from human approval tasks and identify where source verification must occur.” Check upload success, plan friction, and review effort. Log the plan, current price, time, failures, and paid features used.

How to read your results

Keep one log row for each assistant, plan, prompt, and run. Mark every check pass, fail, or unresolved rather than inventing an accuracy score.

Treat an invented authority, unsupported legal proposition, overlooked jurisdiction, or untraceable conclusion as a blocker for that workflow. Then compare source traceability, review effort, upload friction, usage limits, and current price. Do not average these into one percentage that could conceal a serious error.

Use an assistant only for a task that meets your own threshold and retains qualified review where applicable. Your log describes your workflow; it does not establish general accuracy, professional competence, or liability.

Where the plans differ

Use the Claude vs ChatGPT vs Gemini matrix and check the rows for Free plan, Low-cost tier, Main paid plan, Heavy-use plans, Team plan, Context window, How usage limits are described, Training on your chats, and Ads.

The main-plan prices are as read on the vendor pages in October 2026: ChatGPT Plus is $20/month with monthly billing (OpenAI’s Plus Help Center); Claude Pro is $20 monthly or $17/month with annual billing and $200 up front (Claude pricing); Google AI Pro is $19.99/month (Google AI plans).

The low-cost row lists ChatGPT Go at A$13/month on the Australian page, with no verified US-dollar price; no Claude tier between Free and Pro; and Google AI Plus at $4.99/month (ChatGPT pricing, Claude pricing, and Google AI plans).

For document volume, compare the context and usage rows. OpenAI publishes plan-and-model context figures but no exact message counts; Anthropic says context can reach 1M on every plan but publishes no message counts; Google gives no app token figure and says its compute-based limits refresh every 5 hours up to a weekly limit (ChatGPT pricing, Claude pricing, and Google AI plans).

Training settings are not interchangeable. The Ads row records that ChatGPT Go may include ads, while ad status for Claude and Gemini was not verified on the pages read.

Sources

For current prices and limits, see the dated comparison pages.

Compare assistants