Chat Picker

AI Social Survey Design: Generating Questionnaires and Analyzing Samples

A practical afternoon test plan for drafting questionnaires, generating synthetic samples, checking survey flow, analyzing pilot data, and auditing bias without mistaking vendor features for proof of quality.

Sources checked 2 Oct 2026

Vendor pages establish that OpenAI’s data-analysis page says ChatGPT can analyze uploaded files and create tables or charts, Anthropic’s upload page lists document and spreadsheet support, and Google says Gemini Apps can analyze spreadsheets and other files; they do not establish that any assistant will design or interpret a social survey correctly. Chat Picker has not tested ChatGPT, Claude, or Gemini for this use, so use the afternoon plan below to log your own results.

Know these limits before you start

As read on October 1, 2026, the linked upload pages show different ceilings. OpenAI’s File Uploads FAQ sets a 512 MB hard limit for each ChatGPT conversation file, caps CSV files and spreadsheets at about 50 MB, and limits Free users to 3 file uploads per day. Claude’s upload page sets 500 MB per chat file and 30 MB per project file; XLSX requires code execution and file creation. Google’s Gemini file page lists up to 10 supported files per prompt, subject to availability, and 100 MB for each non-video file. Recheck them before a larger run.

Before using real records, inspect account settings. OpenAI’s pricing page lists a training opt-out for Free, Go, Plus, and Pro. Claude’s pricing page lists one for Free, Pro, and Max and says Team is not trained on by default. The Google AI plans page does not state a training setting, so the page read does not settle that point. Do not treat a training control as consent to collect or disclose data.

Outputs still need checking. OpenAI says ChatGPT may be incorrect and sound confident; Anthropic says Claude can produce convincing but ungrounded statements and should not be the only source of truth; Google says Gemini outputs are probabilistic and may reflect limited viewpoints or overgeneralizations. OpenAI’s usage policy bars unauthorized aggregation or profiling of private information and automated high-stakes decisions without human review; Anthropic’s usage policy calls for relevant human expertise in elevated-risk uses. Keep this test synthetic or properly authorized.

The test plan

Run each step in a fresh chat, with a single assistant at a time. Keep the input, prompt, plan, account settings, and every follow-up correction in one log so you can distinguish the first answer from later repair.

  1. Questionnaire drafting. Input: A brief naming the audience, decisions, topics, and any draft questions. Prompt: “Design a questionnaire for Portland residents registering for evening classes at a community center. Include an introduction, consent wording, closed and open-ended questions, and a closing message. Do not collect names or addresses. Return the questionnaire and a brief rationale.” Look for: Double-barreled or leading items, suitable response options, and clear instructions. Record: Omissions, edits, and ignored constraints.

  2. Synthetic sample generation. Input: A blank CSV schema and a codebook of allowed categories. Prompt: “Create a clearly labeled synthetic CSV that follows this schema and codebook. State that every row is fictional. Do not use real names, infer protected characteristics, or present the file as evidence about a population. Return a data dictionary and every assumption.” Look for: Schema drift, invented categories, and unsupported demographic patterns. Record: Corrections, assumptions, and labeling gaps.

  3. Question order and flow. Input: The unchanged draft plus notes on skip logic and sections. Prompt: “Review this questionnaire for order effects, confusing transitions, missing routing instructions, and respondent fatigue. Preserve each item’s meaning. Return a proposed order, the reason for each move, and replacement transition text.” Look for: Logical progression and usable routing. Record: Every proposed move and whether it preserves meaning.

  4. Pilot data analysis. Input: The synthetic CSV, codebook, and a specific analysis question. Prompt: “Analyze the uploaded synthetic pilot file. Report missing values, category frequencies, group comparisons, and uncertainty. Do not infer causation. List every transformation, show the formula for each calculated value, and separate observations from interpretation.” Look for: Reproducible denominators, consistent categories, and unsupported claims. Record: Which calculations and charts you can reproduce, plus every claim to verify.

  5. Ethics and bias audit. Input: The questionnaire, codebook, synthetic sample, and analysis log. Prompt: “Audit this survey for leading wording, missing response options, order effects, sampling limitations, subgroup undercoverage, and claims unsupported by the supplied data. Quote each affected item or field, explain the concern, propose a narrower revision, and label anything you cannot check.” Look for: Unsupported judgments and omitted perspectives. Record: Unresolved concerns, revisions, and your final disposition.

How to read your results

Read the log row by row, not by prose. Before comparing outputs, set acceptance checks: schema compliance, no unsupported population claims, reproducible calculations, respected data boundaries, and visible corrections. Mark each workflow “meets checks,” “needs review,” or “not suitable”; do not average these labels into a score or ranking.

A fabricated category, unreproducible calculation, unsupported causal claim, or real identifier is a failed check regardless of writing quality. Make your selection with required supervision, plan limits, and data settings in view. If two workflows pass, compare correction effort and cost; if none passes, change the workflow or tool. Synthetic data tests the process, not whether its pattern represents a real population.

Where the plans differ

Chat Picker’s three-way comparison matrix separates Free plan, Low-cost tier, Main paid plan, Heavy-use plans, Team plan, Context window in the app, How usage limits are described, Training on your chats, and Ads. These are plan and capacity entries, not survey-quality ratings.

As read on October 1, 2026, OpenAI lists Plus at $20 a month, Anthropic lists Pro at $20 monthly or $17 monthly with annual billing and $200 upfront, and Google lists AI Pro at $19.99 a month. For the low-cost row, OpenAI’s Australian page shows Go at A$13 a month, but no U.S.-dollar price was verified, Anthropic lists no tier between Free and Pro, and Google lists AI Plus at $4.99 a month.

Read on October 1, 2026, the usage rows also use different descriptions. ChatGPT’s pricing page gives no exact message counts; Claude’s page says Pro has “more usage” than Free and Max offers 5x or 20x Pro usage; Google’s page describes compute-based limits that refresh every five hours, up to a weekly limit. Reconfirm the vendor page before subscribing.

Sources

For current prices and limits, see the dated comparison pages.

Compare assistants