Chat Picker

How to Train Your Own AI Chat Model: Open-Source Options and Fine-Tuning Basics

An afternoon test plan uses PEFT, which updates a small number of extra model parameters, to adapt an open-weight chat model and compare self-hosted deployment on your own hardware.

Sources checked 3 Oct 2026

The documented route is to adapt an existing open-weight base with Hugging Face’s PEFT documentation or a project-supplied fine-tuning framework, then serve it through a self-hosted runtime; this is not from-scratch pretraining. Project and vendor pages document license, access, memory, data-handling, and interface terms where stated, but not which setup will train fastest or answer best on your machine. Chat Picker has not tested ChatGPT, Claude, or Gemini for this use, as stated on its method page, so make the comparison from your own log.

Know these limits before you start

These are the projects we read, not the whole field, and they are not ranked: Meta’s Llama models, Mistral models, ChatGLM3, PEFT, Ollama, Open WebUI, LM Studio, and llama.cpp.

  • Adaptation: Hugging Face’s PEFT documentation says PEFT updates a small number of extra parameters instead of all parameters and integrates with Transformers, Diffusers, and Accelerate. The page read does not provide a LoRA, QLoRA, or DoRA hardware comparison, so verify the exact repository and configuration.

  • Weights and licenses: Meta’s Llama README says its weights are licensed for researchers and commercial entities and require license acceptance and an approved request. ChatGLM3’s repository says its weights are open for academic research, with free commercial use after questionnaire registration. Mistral’s model docs call Mistral Large 3 open-weight; the page read gives no license or hardware requirement.

  • Memory: As read on October 1, 2026, Llama’s README says full-bf16 Llama 4 inference needs at least four GPUs; for Llama-4-Scout-17B-16E-Instruct, it lists two 80GB GPUs for FP8 or one 80GB GPU for Int4. These are inference requirements, not training guarantees. ChatGLM3’s repository says default FP16 loading uses about 13GB VRAM, four-bit causes some performance loss, and CPU inference is slower and needs about 32GB memory.

  • Local data: Ollama’s FAQ says local operation does not see prompts or data. Its cloud-hosted models process prompts and responses but do not store, log, or train on their content, and disabling cloud features removes cloud models and web search. Record which path you use.

  • Answer checks: OpenAI’s accuracy note says ChatGPT can be incorrect or misleading; Anthropic’s help article says Claude can hallucinate and should not be the only source of truth. Google’s Gemini Apps help page documents when the Sources button appears; the page does not state an accuracy guarantee. Apply the same verification to every baseline.

The test plan

  1. Set the base. Prepare a card for each candidate: parameter count, license, access route, precision, documented memory, intended task, and serving path. Use this rule: “Proceed only if the stated license covers my use and the stated memory requirement fits my machine; otherwise mark blocked.” Record the exact model identifier and every reason a candidate was excluded.

  2. Prepare the data. Create fixed train, validation, and held-out sets. Each item needs context, question, reference answer, and an unsupported claim to avoid. Use this complete item: Context: “ERROR 42 means the inlet sensor is open. The manual says to inspect the inlet filter and rerun the diagnostic.” Question: “What should the operator do first?” Reference: “Inspect the inlet filter, then rerun the diagnostic.” Disallowed claim: “The board must be replaced.” Give annotators this instruction: “Use only the context; if the answer is absent, say so.” Record split counts, exclusions, duplicates, and source terms.

  3. Run PEFT. Keep the base, data split, precision, and seed fixed while changing one PEFT setting at a time. For a support adapter, use: “Answer from the supplied context, preserve units, and state when the context does not contain the answer.” Inspect training curves, sample errors, and memory use. Record the adaptation method, trainable parameter count, sequence length, learning rate, batch settings, hardware, and configuration files.

  4. Complete one run. Load the documented base, capture an unadapted baseline, train, save the PEFT output, reload it, and run one smoke test. Use ChatGLM3’s supplied fine-tuning framework or the current integration instructions for PEFT; do not transplant commands from another model. Record software versions, install failures, start and finish times, peak memory, checkpoint location, and the exact command or notebook cell used.

  5. Evaluate. Send the held-out prompts and manual item to the unadapted base and tuned model under the same settings. Have reviewers independently mark pass, fail, or needs review against the references, and record disagreements. Check factual fit, unsupported claims, instruction following, refusal when context is insufficient, response time, and peak memory. Save each output and decision; do not import a benchmark score.

  6. Test deployment. Choose one documented path: Ollama exposes a REST API; Open WebUI supports Ollama and OpenAI-compatible APIs; LM Studio can run offline after model files are obtained and offers a REST API; llama.cpp documents a server, web UI, quantization, and hybrid CPU/GPU inference. Use a path only if its documentation accepts your output. Record installation steps, model loading, context setting, successful chats, resource use, and failure text.

How to read your results

Fix your decision rule before viewing outputs: acceptable license, documented fit, correct held-out behavior, reproducible run, and a deployment path you can operate. Mark each gate pass, fail, or needs review; do not turn this private log into a public ranking.

If the base and tuned model both miss a reference, inspect the data and context before changing training settings. If only the tuned run fails, compare its configuration and samples. If training fits but deployment fails, keep the result unresolved and check the output format and runtime requirements. Keep time and memory observations tied to the exact machine and software.

Where the plans differ

Use the Claude vs ChatGPT vs Gemini matrix as a hosted-plan reference, not a training benchmark. Its relevant rows are free plan, low-cost tier, main paid plan, heavy-use plans, team plan, context window in the app, how usage limits are described, training on your chats, and ads. Every figure there carries a vendor source and read date; confirm before subscribing.

The matrix’s US-dollar main-paid figures, read on October 1, 2026, are $20 per month for ChatGPT Plus, $20 monthly or $17 per month with annual billing and $200 up front for Claude Pro, and $19.99 per month for Google AI Pro.

The training-on-chats row lists an opt-out for ChatGPT on Free, Go, Plus, and Pro; an opt-out for Claude on Free, Pro, and Max, with Team not trained on by default; and no verified setting for Gemini. These rows do not report training throughput, trainable parameters, adaptation formats, or VRAM, so they cannot settle the self-hosted training choice.

Sources

For current prices and limits, see the dated comparison pages.

Compare assistants