5 Best Sources for Chatbot Training Data in 2026

The fastest path to a working conversational model starts with three moves: pull from a proven repository, convert everything into the messages JSONL schema, and run it through a filtering pass before you split it. For chatbot training data specifically, that means checking Hugging Face first, Kaggle and GitHub second, and your own product logs only once the public options run dry.

Here’s the starting lineup that actually works. WildChat-4.8M gives you nearly millions of filtered, non-toxic human-to-ChatGPT conversations for general instruction tuning. GasaiAI’s Chat-Agent-1M adds roughly 919,000 conversations formatted in ChatML with heavy tool-use coverage. ThinkNet’s HQ-Chat-1K offers a small, high-quality example set when you need something small and clean to prototype with. Bitext’s intent-labeled dataset on Kaggle covers numerous customer-service intents across many utterances. Kaggle’s Simple Dialogs set rounds things out for anyone who just wants a lightweight sandbox.

Every one of those needs to land in the same shape before a training run touches it: JSONL, one JSON object per line, using a messages array of role and content pairs, with the assistant’s turn always last. If you’re fine-tuning a completion-only model instead of a chat model, you’ll use prompt-completion pairs rather than messages, and if your pipeline runs through Anthropic-style tooling, you’ll see the same idea reflected in ChatML format.

• Pick a source that matches your scale. Large corpora for breadth, curated sets for speed.
• Convert to messages JSONL immediately. Don’t train on mixed formats.
• Filter before you split. PII removal, deduplication, and toxicity checks come before your 80/10/10 split, not after.

Pro Tip: Don’t mix formats across a single training run. A dataset half in prompt-completion and half in messages format will confuse your loss calculation before you even get to model quality.

---

TL;DR:

• Use large public datasets like WildChat-4.8M for broad instruction tuning, but always convert data into the messages JSONL format before training.
• Filter datasets for PII, toxicity, and duplicates prior to splitting into training, validation, and test sets to prevent data leakage and model issues.
• Keep tool-invoking examples around 20% of your dataset to teach the model when to call tools and avoid reflexive tool use.
• Start with a learning rate between 1e-5 and 3e-5, using masked assistant-only loss to prevent overfitting and improve fine-tuning stability.
• Supplement public datasets with domain-specific logs or synthetic data generated via taxonomy-guided methods to better match your use case.

---

Table of Contents

• Where Do You Find Chatbot Training Data?
• What File Format Does Chatbot Training Data Need?
• How Do You Clean Chatbot Training Data Before Fine-Tuning?
• How Should You Label and Annotate Conversational Data?
• Can You Generate Synthetic Data to Scale Training?
• How Do You Format Data for Tool-Calling Chatbots?
• What Split and Hyperparameters Should You Start With?
• How Do You Evaluate a Chatbot Before Launch?
• Your First Fine-Tune: A Step-by-Step Checklist
• Key Takeaways
• Primary Sources and Docs to Bookmark
• What Developers Get Wrong About Chatbot Training Data
• Sources
• FAQ

Where Do You Find Chatbot Training Data?

The right repository depends on what shape of data you actually need, not just how much of it exists. Developers chasing scale look to Hugging Face. Those needing something narrow and fast look to Kaggle or GitHub. And teams building for a specific vertical often end up collecting their own logs because no public dataset covers their domain well enough.

Hugging Face is where the largest, most reusable chatbot datasets live. WildChat-4.8M is the standout: 3.2 million real conversations between people and ChatGPT, already filtered to remove toxic content, which makes it a strong base layer for instruction tuning or pretraining augmentation. Chat-Agent-1M sits alongside it with a different specialty. If you’re building an agent that needs to search, calculate, or query an API mid-conversation, that’s the dataset that teaches the pattern.

Kaggle and GitHub serve a different purpose. They’re where you go for curated, purpose-built sets rather than raw scale. ThinkNet’s HQ-Chat-1K is a good example: roughly 1,000 high-quality conversation pairs with a friendly tone across varied topics, small enough to inspect by hand before you commit compute to it. Bitext’s training dataset for chatbots and virtual assistants, hosted on Kaggle, takes the opposite approach: it’s built around structured intents rather than open conversation, covering more than 20,000 utterances across 27 intent categories with linguistic flags for things like politeness, typos, and colloquial phrasing. Kaggle’s Simple Dialogs dataset is the smallest and most approachable of the group, a solid pick if you’re testing a training pipeline for the first time and don’t want to wrestle with a multi-gigabyte download.

Domain-specific vendors and your own logs come into play once the public sets stop covering your use case. A dataset trained on general customer chat won’t know your return policy, your product SKUs, or the specific way your users phrase complaints. That’s when it makes sense to mine your own support transcripts or CRM history, assuming you’ve already got consent and a legal basis to reuse that data for training. Before you touch any public dataset, check its license page. Some allow commercial fine-tuning outright, some require attribution, and a few restrict redistribution of derivative models entirely.

Source Type	Example Dataset	Best Use Case
Large general corpus	WildChat-4.8M	Broad instruction tuning, pretraining augmentation
Agentic/tool-use corpus	Chat-Agent-1M	Training tool-calling and multi-turn agent behavior
Small curated set	HQ-Chat-1K	Fast prototyping, manual quality review
Intent-labeled set	Bitext customer support dataset	Domain-specific NLU, slot and intent training
Lightweight sandbox	Kaggle Simple Dialogs	First-time pipeline testing

• Match dataset scale to your compute budget, not just your ambition.
• Check the license before you download anything you plan to fine-tune on commercially.
• Blend a large general corpus with a small domain-specific set rather than relying on either alone.

If you’re building a chatbot for a specific business function like lead qualification and routing, a public general-purpose corpus will only get you partway there. You’ll still need labeled examples that reflect your actual sales conversations.

What File Format Does Chatbot Training Data Need?

Every modern chat model expects the same basic shape: JSONL, one training example per line, structured as a messages array. Each message carries a role (system, user, or assistant) and a content field, and the assistant’s response has to be the final entry in the array. NVIDIA’s NeMo documentation lays out this schema precisely, and it’s become close to a de facto standard across fine-tuning pipelines.

A single line looks like this in practice:

{"messages": [{"role": "system", "content": "You are a helpful support agent."}, {"role": "user", "content": "How do I reset my password?"}, {"role": "assistant", "content": "Go to Settings, then Security, then click Reset Password."}]}

If you’re fine-tuning a completion-only model rather than a chat-tuned one, you’ll skip the messages array and use a flat prompt-completion pair instead, a simpler two-field structure with prompt and completion keys. That format still shows up in older pipelines and some open-source completion models, but for anything built around a conversational interface, messages format is what you want.

A few technical details trip people up more than they should:

• UTF-8 encoding is non-negotiable. Malformed characters silently break tokenization in ways that are hard to debug later.
• Each JSON object must sit on a single line. No pretty-printing, no multi-line objects. That’s what makes JSONL streamable.
• Watch your context window. Most fine-tuning setups work best when individual examples stay within 2,000 to 8,000 tokens depending on the base model. A single training example that blows past your model’s context limit gets truncated or dropped, and either outcome quietly wastes a chunk of your dataset.
• Tool calls need their own fields. If a conversation includes a function call, the ChatML-adjacent convention embeds tool_call and tool_result as structured content within the assistant’s turn rather than as free text.

Format	Structure	Best For
Messages (JSONL)	Array of role/content pairs	Chat-tuned models, multi-turn conversations
Prompt-completion	Flat prompt/completion pair	Completion-only or legacy models
ChatML with tool fields	Messages plus tool_call/tool_result	Agentic, tool-invoking assistants

How Do You Clean Chatbot Training Data Before Fine-Tuning?

Raw conversational data is messy by default, and the cleaning pass is where most quality problems actually get solved, not during training. Skipping it doesn’t just risk a worse model. It risks leaking personal information or reinforcing toxic patterns straight into your weights.

1. Detect and remove personally identifiable information. Automated tools like Microsoft Presidio catch names, emails, phone numbers, and addresses using regex and named-entity recognition, but a manual spot check on a sample of flagged and unflagged examples catches what the automated pass misses. Log every removal decision so you can audit the process later if a compliance question comes up.
2. Filter or label toxic content. WildChat’s public release already strips its non-toxic subset before distribution, which is exactly why it’s usable off the shelf without a heavy cleanup pass. If you’re working with raw scraped or logged conversations instead, you’ll need to run that filtering yourself, and you’ll need to decide whether to discard toxic examples entirely or keep a small labeled subset for training a refusal behavior.
3. Deduplicate aggressively. Near-duplicate conversations inflate your effective dataset size without adding real diversity, and they can cause a model to overfit on repeated phrasing.
4. Normalize whitespace and encoding. Inconsistent line breaks, smart quotes, and mixed encodings cause silent tokenization errors that only surface once training is already underway.
5. Drop empty-turn and truncated examples. A conversation that cuts off mid-response teaches the model to produce incomplete answers, which is the opposite of what you want.

A practical toolchain runs in this order: load the raw data, tokenize it to check length distribution, filter for toxicity and quality, anonymize PII, deduplicate, and export to JSONL. Each stage should log how many examples it removed, so you know whether your final dataset is a few percent smaller than your raw pull or half the size.

Pro Tip: Run your PII scanner twice, once before deduplication and once after. Duplicate removal sometimes changes which examples survive, and a PII leak in a rare, non-duplicated conversation is easy to miss if you only scan once.

How Should You Label and Annotate Conversational Data?

Two labeling philosophies dominate chatbot training, and picking the wrong one for your use case wastes annotation budget; learn more about how to humanize AI text with instructions to enhance your annotation approach. Intent and slot labeling tags each user utterance with a category (like “check_order_status”) and pulls out the specific values inside it (an order number, a date). That approach fits task-oriented bots that route requests or trigger backend actions. Instruction-response labeling, by contrast, treats the whole exchange as a single example of desired behavior, without breaking it into discrete categories. That’s the format general-purpose assistants need.

Most production chatbots end up needing both, layered on top of each other rather than choosing one exclusively.

• Build a hierarchical taxonomy of intents before you start annotating, not after. It surfaces coverage gaps early, when you can still go collect more examples for a thin category.
• Use stratified sampling to make sure your dataset actually contains slang, indirect phrasing, overly polite requests, and typo-riddled input, not just clean textbook sentences. Bitext’s dataset builds this in directly with linguistic flags marking these registers, and stratified sampling across these registers matters specifically because models trained only on canonical phrasing tend to break the moment a real user deviates from it.
• Run inter-annotator agreement checks on a subset of your labels. If two annotators disagree more than occasionally on the same examples, your label definitions probably need tightening before you scale up annotation.
• Hold back a small validation set exclusively for label quality review, separate from your training/validation/test split for the model itself.

If you’re designing a bot around specific business goals, structuring your dataset around those same intents from the outset saves a re-annotation pass later, once you discover the taxonomy you started with doesn’t match how customers actually talk.

Can You Generate Synthetic Data to Scale Training?

Yes, and it’s become one of the more reliable ways to expand an instruction-tuning dataset without paying for a small army of human annotators. The approach that’s gotten the most traction is taxonomy-guided synthetic generation, laid out in the LAB paper (Large-scale Alignment for chatBots). The method starts with a small set of seed examples, uses a larger teacher model to generate variations across a defined taxonomy of skills and topics, then filters the output for quality before it ever reaches your training set.

The part of LAB’s approach that matters most for practitioners isn’t the generation step. It’s the multi-phase tuning that follows it.

Rather than fine-tuning on everything at once, LAB’s pipeline separates knowledge tuning from skill tuning into distinct phases, using replay buffers that reintroduce earlier training examples during later phases. That structure is specifically designed to prevent catastrophic forgetting, where a model gains a new skill but quietly loses ground on something it already knew how to do.

Generating synthetic data well requires the same discipline as sourcing real data:

• Rate teacher-model outputs before including them. A simple 3-point quality scale (good, borderline, discard) catches a surprising share of low-quality generations before they pollute your training set.
• Check diversity, not just volume. A generation pipeline that produces 50,000 examples clustered around three phrasing patterns hasn’t actually expanded your coverage.
• Monitor the output distribution against your taxonomy. If your seed prompts skew toward one topic area, your synthetic data will skew the same way, and you’ll end up amplifying an existing gap instead of closing it.
• Spot-check with human reviewers periodically. Automated filtering catches obvious problems, but a human reading a random sample every few thousand generations catches subtler drift that scoring rubrics miss.

How Do You Format Data for Tool-Calling Chatbots?

Training a model to call functions, hit an API, or query a database mid-conversation requires structure that plain conversational data doesn’t have. You need dedicated fields for the tool call itself and for the result it returns, both embedded inside the assistant’s turn rather than floating as separate messages.

Chat-Agent-1M is a useful reference point here because its authors built roughly 29% of the dataset around tool-use trajectories, which turns out to be a reasonable target ratio. Train on too few tool-call examples and the model rarely reaches for a tool even when it should. Train on too many and it starts invoking tools reflexively, even for questions it could answer directly.

Three practical rules govern this kind of formatting:

• Define your tool metadata once, shared across every example, so the model learns a consistent schema for how tools are described and invoked.
• Include realistic tool results, not placeholder text. A model trained on fake or generic tool outputs learns to ignore what a real tool actually returns.
• During training, mask the system and user tokens, or at minimum downweight them in the loss calculation, so the model’s gradient updates focus on the assistant’s tool-calling and response behavior rather than on memorizing prompt phrasing.

Training Focus	Approximate Share	Purpose
Direct-answer examples	approximately 80%	Teach the model when NOT to call a tool
Tool-call examples	approximately 20%	Teach selective, appropriate tool invocation

That masking practice extends beyond tool-calling specifically. Practitioner guidance on instruction tuning shows that masking prompt tokens during loss computation reduces the odds of a model producing incoherent output, because it stops the model from being penalized or rewarded for predicting tokens it never actually needs to generate.

What Split and Hyperparameters Should You Start With?

An 80/10/10 split between training, validation, and test data remains the standard starting point, and it’s not an arbitrary convention. Fine-tuning guidance built around this ratio gives you enough training volume to actually learn patterns, while reserving enough held-out data in validation and test to catch overfitting before it reaches production. Stratify that split by intent or domain category rather than splitting randomly, or you risk a validation set that’s accidentally missing an entire category your model needs to handle.

For learning rate, that same guidance points to a starting range of 1e-5 to 3e-5, paired with small batch sizes, typically 8 to 32 examples per batch. Push the learning rate much higher than that on a pretrained base model and you risk destabilizing weights that already encode useful general knowledge. Set it too low and training crawls without meaningful improvement.

• Run a few epochs on small, curated datasets like HQ-Chat-1K, where the risk is overfitting on a limited set of examples.
• Scale epoch count down for very large corpora like WildChat-4.8M, where a single pass already exposes the model to enormous variety.
• Mask assistant-only tokens during loss computation, following the same principle covered in the tool-calling section above, whether or not your dataset includes tool use.
• Consider LoRA or adapter-based fine-tuning if you’re compute-constrained. It trains a small set of additional parameters rather than updating the full model, and it gets you most of the quality gain at a fraction of the GPU cost.

None of these numbers are fixed law. They’re a stable starting point you adjust once you see how your specific dataset and base model respond during the first training run.

How Do You Evaluate a Chatbot Before Launch?

Evaluation needs both automated signals and human judgment, because neither one catches everything on its own. A model can post excellent perplexity numbers and still produce answers that are confidently wrong.

1. Track perplexity trends across training checkpoints. A steadily declining perplexity curve suggests the model is learning the patterns in your data, though a low perplexity by itself doesn’t guarantee good real-world responses.
2. Measure intent accuracy against your labeled validation set, if you’re working with an intent-driven bot. This is where the stratified sampling from your annotation phase pays off. A model that performs well on canonical phrasing but poorly on the slang and errorful examples in your validation set has a coverage gap you need to fix before launch.
3. Run automated toxicity and PII scans on model outputs, not just on your training data. A model can generate problematic content even when none of its training examples contained it directly.
4. Put a human reviewer through structured scenario testing. Have them rate a sample of real conversations for helpfulness and factual accuracy, and include a handful of adversarial prompts specifically designed to probe for unsafe or off-brand responses.
5. Build a feedback loop for logging unknown or misclassified intents once the bot is live, feeding those examples back into your next training round.

Pro Tip: Keep a running log of every adversarial prompt that broke a previous version of your model. Re-testing that same list against every new fine-tune catches regressions that a fresh evaluation set might miss entirely.

Your First Fine-Tune: A Step-by-Step Checklist

Getting from zero to a validated first fine-tune is realistically a multi-day project, not a multi-week one, if you follow the steps in order and don’t skip the filtering stage.

1. Confirm the license on any public dataset you plan to use before you download or train on it. Some datasets restrict commercial use or model redistribution.
2. Sample and manually inspect 100 to 200 examples from your chosen dataset before committing to it at scale. This catches formatting quirks and quality issues early.
3. Convert the dataset into JSONL messages format, with role/content pairs and the assistant’s turn last in every example.
4. Run PII detection and toxicity filtering, then deduplicate and normalize whitespace and encoding across the full set.
5. Annotate or synthetically generate any missing instruction pairs your taxonomy review flagged as underrepresented.
6. Split the cleaned dataset 80/10/10, stratified by intent or domain where applicable.
7. Run a small-scale fine-tune using masked assistant-only loss, starting at a learning rate between 1e-5 and 3e-5.
8. Validate with both automated checks and a human review pass before considering the model ready for a wider test.

Once your pipeline works end to end on a small dataset, scaling it up to a larger corpus like WildChat-4.8M is mostly a matter of compute budget rather than new process. If you’d rather have a team handle the dataset curation and deployment work directly, Depechecode’s AI chatbot starter plan builds that pipeline for you rather than leaving it as a side project between other priorities.

Key Takeaways

Good chatbot training data requires the right source, the correct JSONL messages schema, disciplined PII and toxicity filtering, and an 80/10/10 split with a 1e-5 to 3e-5 learning rate to train reliably.

Point	Details
Choose sources by shape, not size	Use WildChat-4.8M for breadth, Chat-Agent-1M for tool-use, HQ-Chat-1K or Bitext for curated intent data.
Standardize on messages JSONL	Convert every source to the same role/content schema before merging datasets.
Filter before splitting	Run PII removal, deduplication, and toxicity checks before creating your 80/10/10 split.
Start hyperparameters conservatively	Use a 1e-5 to 3e-5 learning rate with masked assistant-only loss on your first run.
Balance tool-call examples	Keep tool-invoking examples to roughly 20% of the dataset to avoid reflexive tool use.

Primary Sources and Docs to Bookmark

• allenai/WildChat-4.8M on Hugging Face for a large, pre-filtered general conversation corpus.
• GasaiAI/Chat-Agent-1M on Hugging Face for ChatML-formatted tool-use trajectories.
• ThinkNet/HQ-Chat-1K on Hugging Face for a small curated set to prototype with.
• Bitext training dataset for chatbots on Kaggle for intent-labeled customer support data.
• NVIDIA NeMo format documentation for exact schema requirements.
• The LAB paper on arXiv for taxonomy-guided synthetic data generation and multi-phase tuning.

What Developers Get Wrong About Chatbot Training Data

Most guides on this topic treat dataset selection as the hard part and formatting as an afterthought. That’s backward. The datasets covered here, WildChat-4.8M, Chat-Agent-1M, HQ-Chat-1K, Bitext, are all easy to find with a five-minute search. What actually derails a fine-tuning project is a mismatched schema, an unmasked loss function, or a learning rate picked without any reference point.

The conventional advice to “just grab a big dataset and fine-tune” undersells how much the multi-phase, taxonomy-guided approach from LAB has changed what’s realistic for smaller teams. You no longer need to choose between a massive general corpus and expensive human annotation. Synthetic generation, filtered properly, closes that gap.

If there’s one thing worth prioritizing over everything else in this guide, it’s the masking discipline covered in the tool-calling and hyperparameter sections. Teams get the dataset right and still ship a model that hallucinates or rambles, because they trained on every token instead of just the assistant’s output. Fix that one habit before you touch anything else.

— Donovan

Sources

• allenai/WildChat-4.8M · Datasets at Hugging Face
• Format Training Dataset | NVIDIA NeMo Platform
• GasaiAI/Chat-Agent-1M · Datasets at Hugging Face

FAQ

Can I Train a Chatbot With My Own Data?

Yes. Your own support logs or product conversations, once cleaned of PII and converted to JSONL messages format, often outperform generic public datasets for domain-specific tasks because they reflect exactly how your users talk.

Where Can I Get AI Training Data?

Hugging Face hosts the largest conversational corpora, including WildChat-4.8M and Chat-Agent-1M, while Kaggle and GitHub offer smaller, curated sets like Bitext’s intent-labeled dataset and Simple Dialogs.

What Are Some Examples of AI Training Data?

Common examples include WildChat-4.8M’s 3.2 million filtered ChatGPT conversations, Chat-Agent-1M’s tool-use trajectories, and Bitext’s 20,000-plus intent-labeled customer support utterances.

Can AI Generate Its Own Training Data?

Yes, through taxonomy-guided synthetic generation, where a teacher model produces new examples across a defined skill taxonomy and those outputs get filtered for quality before entering the training set, as described in the LAB methodology.

What Format Should Chatbot Training Data Be In?

JSONL using a messages schema, with role and content fields and the assistant’s response as the final message, is the standard format for chat-tuned models; completion-only models use flat prompt-completion pairs instead.

Recommended

• The Future Of Customer Interaction – AI Chat Bots • Depeche Code
• AI Chat Bots – Enhancing Efficiency In Customer Support • Depeche Code
• 10 Best Practices For Designing An Effective AI Chat Bot • Depeche Code
• The AI Chat Bot Growth • Depeche Code