Chat Picker

2026 AI Customization Compared: Fine-Tuning and Preference Learning

Current vendor documentation separates model fine-tuning from saved instructions and memory, with different availability and privacy controls across the three assistants.

Sources checked 3 Oct 2026

2026 customization is not one feature: OpenAI's supervised fine-tuning guide says its platform is winding down and closed to new users, while Google's model-tuning page says no available model supports fine-tuning in Gemini API or AI Studio. Model tuning and in-product personalization are separate surfaces. As read on October 1, 2026, these pages document different availability and privacy choices, not a common tuning feature. Chat Picker has not run quality, speed, accuracy, reliability or benchmark tests of any assistant, so this page gives you an afternoon test plan, not a quality ranking.

Know these limits before you start

OpenAI's SFT guide, read October 1, 2026, says the winding-down platform is closed to new users. Existing users can create jobs for coming months, while fine-tuned models remain available for inference until their base models are deprecated. The guide makes SFT the default method, requires at least 10 training examples in JSONL, and recommends evaluation with holdout data. OpenAI's model optimization guide also says outputs are non-deterministic and calls for representative evaluations of prompted and fine-tuned behavior.

Google's model-tuning page, read October 1, 2026, says fine-tuning is available through Gemini Enterprise Agent Platform, but not for an available model in Gemini API or AI Studio. Google states that it has no immediate plans to add API or AI Studio support. This is separate from Gems: Google says a Gem has a name and instructions, can include files under Knowledge, and is subject to Google's Terms of Service and Prohibited Use Policy. Personal-account Gems are scheduled to transition automatically to skills starting November 2026.

The cited pages do not present “preference learning” as one shared feature. OpenAI says Memory can retain relevant preferences, while Temporary Chats do not create or update memories. Claude memory can be disabled for one chat; incognito chats are excluded from past-chat search and, according to Anthropic's privacy note, are not used to improve Claude even when model improvement is enabled. Google says personalization may use past chats, connected app content, response preferences and instructions, but requires a personal Google Account and is not available to everyone. The cited Claude documentation does not specify a custom-instruction control, so do not treat memory as one.

For open-weight models, Hugging Face's PEFT documentation says PEFT adapts pretrained models by training a small number of extra parameters. Ollama's FAQ says local operation does not see users' prompts or data, while its cloud service processes prompts and responses without storing, logging or training on their content. These are conditions to record during your test, not evidence about output quality.

The test plan

Prepare synthetic order tickets, each with a fictional policy, customer request and status. Use this seed: Northstar Bookstore is fictional; its policy says an address cannot be changed after dispatch; the customer says they moved; the order status is “Shipped.” Make variants by changing one policy or status fact at a time.

  1. Baseline and instructions. In a fresh session, use this prompt: “Return one JSON object with category, allowed_change, policy_quote, next_step, and needs_review. Use only the supplied policy; if it cannot decide the case, set needs_review to true.” If a system or custom-instruction control is exposed, place the same text there and log that placement. Look for required keys, unsupported claims and exact policy grounding. Record the product, plan, settings, raw output and deviations.

  2. Saved instructions and Gems. In Gemini, create a Gem named “Northstar reviewer,” paste the Step 1 instruction and add the policy file under Knowledge. In ChatGPT or Claude, use a custom or system-instruction control only if the current interface exposes one; otherwise record “not exposed.” Do not relabel memory as a custom instruction. Run every ticket with the saved text unchanged. Look for adherence across cases and whether the attached file is used. Record the saved text, attached source, settings and output.

  3. Memory and preferences. Start a regular session and say: “For Northstar tickets, put allowed_change before policy_quote and use US spelling.” Ask the sample question, then open a new session with a different ticket. Repeat with Temporary Chat or Incognito where offered. Look for preference carryover, overreach and reset behavior. Record the preference's scope, available controls and any memory entry you can inspect or delete.

  4. Fine-tuning path. Split the synthetic records into training and holdout sets before changing a model. If you already have OpenAI fine-tuning access, prepare at least 10 JSONL lines and follow the linked SFT schema. If you use Gemini API or AI Studio, record that no tuning control is documented there. For an open-weight route, use a PEFT setup and apply the Step 1 output contract to every holdout record. Look at holdout behavior and job status. Record the access route, dataset version, settings and exact charge shown; do not estimate a cost.

  5. Privacy and deletion. Put the fictional marker ORCHID-VIOLET in one record, then submit the Step 1 packet in regular and nonpersistent modes. Inspect history, memory, export and deletion controls without using real personal data. Look for marker persistence, clear warnings and model-improvement settings. Record each visible control and cleanup result. The Google AI plans page does not state a plan-by-plan model-training toggle, so leave that cell unresolved until you verify it.

  6. Evidence check. Build an answer key from the synthetic policy and ask: “Identify the sentence that decides this case, explain why, and flag any missing fact.” Record unsupported statements, missing citations and whether each source came from your file or the product. OpenAI warns that ChatGPT can sound confident when wrong; Anthropic says not to rely on Claude as the only source of truth; and Google documents source panels that can include uploaded or connected Workspace content. Verify the case before accepting it.

How to read your results

Keep one log row per run: customization surface, product and plan, case identifier, input version, settings, raw answer, observed deviation, evidence, memory behavior, privacy setting and cleanup status.

Define your must-pass checks before testing. A missing required field, unsupported policy claim or unacceptable data setting matters more than preferred tone. Change one layer at a time: baseline prompt, saved instruction, attached file, memory or model adaptation. If several change together, your log cannot show which mattered.

Turn the log into a decision by selecting the least complex setup that meets your own requirements. Do not import an external leaderboard score: it cannot represent your cases, files, plan or settings. Mark a run inconclusive when the interface changes, required context is missing or a privacy control remains unclear.

Where the plans differ

Use the three-way comparison matrix rows for Free plan, low-cost tier, main paid plan, heavy-use plans, Team plan, context window, usage limits, training on chats and ads. The matrix records vendor-published information rather than your workload. Under Chat Picker's sourcing method, an unverified cell is left blank; blank does not mean the feature is unavailable.

On the same October 1, 2026 reading, the main paid prices are ChatGPT Plus at $20 per month, Claude Pro at $20 monthly or $17 annually with $200 upfront, and Google AI Pro at $19.99 per month. Confirm the vendor page before subscribing.

Treat a Gem, saved instruction or memory as a product configuration layer. Treat API fine-tuning or PEFT as a model-adaptation workflow. The matrix can verify published plan differences; your log must determine which layer fits the cases you actually need.

Sources

For current prices and limits, see the dated comparison pages.

Compare assistants