TL;DR: Validate a small MiniMind dataset before installing a training stack or renting hardware. The standard-library script below catches common text and conversation formatting mistakes on a CPU. It is a deliberately narrow preflight for plain text, not proof that tokenization, supervision, or a full training run will work.

What you’ll be able to do after this guide:

• Distinguish pretraining text records from supervised chat records.
• Reject broken JSONL and incomplete plain-text conversations with line numbers.
• Keep a reproducible dataset snapshot for the next training check.

Honest take: Start with the local preflight and a tiny sample you can read; launching a long training job is a poor way to discover that your exporter used the wrong field name.

MiniMind is an open-source project for learning how language-model training works. Its repository code uses the Apache License 2.0. That makes the code available for study and modification under that license; it does not establish the permissions for every dataset you might feed it. Check the source and terms of your data separately.

This guide focuses on the step before model training. You need Python 3 and a local text file. Our validator uses no GPU, no model download, no API key, and no paid service. We tested the validator with synthetic fixtures, not MiniMind’s training loop, and make no claims about training time, model quality, or hardware capacity.

First choose which dataset you are preparing

We inspected MiniMind at commit 233d00fa068ec2b8c0d4f07b703934d6391a5a73 on November 5, 2026. Its dataset implementation distinguishes pretraining records with a text field from SFT records with conversations. The latter contain message roles and content; upstream also has handling for richer reasoning and tool data.

Our script intentionally accepts a smaller format. It requires nonempty strings instead of silently converting arbitrary values to text. For chat, it allows an optional initial system message, then alternating user and assistant messages, ending with an assistant answer. It rejects extra fields so a tool conversation cannot accidentally pass as a plain-text example.

These are our preflight rules, not a claim that MiniMind universally forbids other schemas. If your data includes tool calls, reasoning fields, or metadata, adapt a separate validator to the pinned loader and template instead of deleting meaningful information to satisfy this one.

Check	Pretraining mode	Plain-text SFT mode
Top-level field	text only	conversations only
Required content	Nonempty string	Nonempty role/content strings
Conversation order	Not applicable	Optional system, then user/assistant pairs
What passing proves	Record meets this text policy	Record meets this restricted chat policy
What remains untested	Tokenization and training	Template, loss labels, tokenization, and training

Use this distinction when debugging an exporter. A beautifully formatted chat transcript inside a text string can pass a text validator, but that does not turn it into the supervised conversation structure you intended. Pick the mode based on your training stage, not on whichever command happens to return success.

Step 1: create a small, inspectable JSONL file

JSON Lines stores one valid JSON value per line using UTF-8 without a byte-order mark. A blank line is not a record. Ordinary JSON can span many lines; JSONL records cannot be pretty-printed across physical lines in that way.

For the pretraining preflight, save the following as tiny-text.jsonl, with each object on one physical line:

{"text":"A small dataset should be easy to inspect."}
{"text":"This second example has a different sentence."}

For the supervised chat preflight, save this single line as tiny-chat.jsonl:

{"conversations":[{"role":"user","content":"What is two plus two?"},{"role":"assistant","content":"Four."}]}

The examples are synthetic formatting fixtures, not a useful training corpus. Their purpose is to show the shape of a record without introducing a download, a license question, or a large preprocessing job. Keep the originals when testing a conversion pipeline, and write converted output to a different path.

An escaped newline inside a JSON string is fine: "first line\nsecond line" remains a single JSONL record when written correctly. A literal line break inside an unfinished quoted string is different and will fail parsing. Use a JSON serializer when generating records instead of manually concatenating quotes and commas.

Step 2: run the local validator

Save this original helper as check_minimind_jsonl.py. It reads records sequentially, reports the first failure, and does not modify the dataset. A single unusually large record can still consume substantial memory; this is a small preflight helper, not a hardened ingestion service.

import argparse
import json
from pathlib import Path

def nonempty(value):
    return isinstance(value, str) and bool(value.strip())

def reject_constant(value):
    raise ValueError("nonstandard JSON constant: " + value)

def unique_object(pairs):
    result = {}
    for key, value in pairs:
        if key in result:
            raise ValueError("duplicate key: " + key)
        result[key] = value
    return result

def check(row, mode):
    if not isinstance(row, dict):
        raise ValueError("record must be an object")
    field = "text" if mode == "pretrain" else "conversations"
    if set(row) != {field}:
        raise ValueError("expected only the field: " + field)
    if mode == "pretrain":
        if not nonempty(row["text"]):
            raise ValueError("text must be a nonempty string")
        return
    messages = row["conversations"]
    if not isinstance(messages, list) or not messages:
        raise ValueError("conversations must be a nonempty list")
    expected = "user"
    answers = 0
    for index, message in enumerate(messages):
        if not isinstance(message, dict) or set(message) != {"role", "content"}:
            raise ValueError("each message needs only role and content")
        if not nonempty(message["content"]):
            raise ValueError("message content must be a nonempty string")
        role = message["role"]
        if index == 0 and role == "system":
            continue
        if role != expected:
            raise ValueError("expected role: " + expected)
        if role == "assistant":
            answers += 1
        expected = "assistant" if role == "user" else "user"
    if answers == 0 or expected != "user":
        raise ValueError("conversation must end with an assistant answer")

parser = argparse.ArgumentParser()
parser.add_argument("mode", choices=["pretrain", "sft"])
parser.add_argument("path", type=Path)
args = parser.parse_args()
count = 0
line_number = 0
try:
    with args.path.open(encoding="utf-8") as stream:
        for line_number, line in enumerate(stream, 1):
            if not line.strip():
                raise ValueError("blank line")
            row = json.loads(line, parse_constant=reject_constant,
                             object_pairs_hook=unique_object)
            check(row, args.mode)
            count += 1
    if count == 0:
        raise ValueError("empty file")
except (OSError, ValueError) as error:
    raise SystemExit(f"FAIL {args.path}:{line_number}: {error}")
print(f"PASS {args.mode}: {count} records; structure only")

Run the two checks with your installed Python interpreter:

python check_minimind_jsonl.py pretrain tiny-text.jsonl
python check_minimind_jsonl.py sft tiny-chat.jsonl

Expected output for these exact fixtures is:

PASS pretrain: 2 records; structure only
PASS sft: 1 records; structure only

The helper returns a nonzero exit status on failure, which lets you place it before a later command in your own pipeline. Do not launch training merely because a console printed some output: check the exit status. The wording “structure only” is part of the success message because that limitation should travel with copied logs.

Python’s JSON documentation describes decoding and the hooks used here. We use those hooks to reject duplicate keys and nonstandard constants rather than accepting a record whose apparent content differs between parsers. An encoding error also fails the preflight. The displayed line counter is useful for record errors; a decoding failure can occur while the reader fills its buffer, so do not treat that counter as the exact location of an invalid byte.

Step 3: deliberately test failures before trusting the exporter

We executed this helper locally with Python 3.14 on November 5, 2026. The positive fixtures above passed with two text records and one chat record. We also tested an optional system message followed by two complete exchanges. The negative cases below were rejected with nonzero status.

Fixture change	Why this preflight rejects it	Useful repair
Use messages instead of conversations	Wrong top-level field for this mode	Fix the export mapping
Store a number as text	Text policy requires a string	Inspect the original column type
End after the user message	No completed assistant answer	Recover the answer or remove the incomplete sample
Repeat a user role	Alternation rule is violated	Reconstruct turns from the source
Add a tool-call field	Outside this helper’s plain-text scope	Use a tool-aware validation path
Insert a blank line	Not a JSONL record	Fix the writer’s record separators
Repeat a JSON object key	Ambiguous exported object	Fix the serializer input
Truncate a closing bracket	Invalid JSON	Regenerate the damaged record

We also checked empty files, empty content, a UTF-8 byte-order mark, and a nonstandard numeric constant. These are checks of our validator’s behavior. We did not import PyTorch, instantiate MiniMind’s dataset classes, or launch an optimizer step during this exercise.

When a large export fails, keep the reported line and inspect nearby records. A missing quote can be a one-row problem; an unexpected top-level key can reveal that the whole file came from the wrong conversion path. Fix the upstream transformation when possible. Manually repairing one output row while leaving the exporter broken creates the same failure on your next run.

Step 4: preserve the data you actually checked

Give the accepted export a stable filename and record its checksum, record count, source, and conversion command. Keep that note next to the experiment configuration. If you edit or regenerate the file, rerun validation and replace the recorded checksum.

A count alone cannot identify a dataset. Two exports can both contain one thousand records while changing every assistant answer. A checksum identifies bytes, but it cannot tell you whether the text is accurate, whether private information should be removed, or whether you have permission to use it. Those require separate content and provenance checks.

For a first experiment, manually read a small sample of both prompts and answers. Look for empty boilerplate, accidental HTML, duplicated examples, truncated responses, and answers that contradict the question. A syntactically valid corpus can still teach unwanted behavior. Make these checks before increasing the training budget, not after interpreting a disappointing result.

What should happen after the preflight passes?

The next milestone is a loader and tokenizer smoke test in the actual pinned MiniMind environment. Inspect a rendered conversation, the tokenized length, and which tokens contribute to the loss. Confirm that truncation does not remove the answer you meant to supervise. This guide does not supply an untested end-to-end training command.

The pinned pretraining entry point and SFT entry point expose separate data-path and sequence-length arguments. Read those files for the revision you actually use rather than treating a class constructor’s default as your training command’s setting.

If your goal is understanding a complete small-model training project, compare the workflow with our nanochat guide. If you already have a base model and want to adapt it, our Unsloth fine-tuning guide covers a different starting point. Dataset preparation is shared work, but pretraining a model and adapting existing weights are different experiments.

Honest take: keep the first success small and specific

The local validator is the right starting point for a plain-text MiniMind export because it provides a fast, inspectable failure boundary without installing the training stack. Its strict schema is also its main limitation: richer conversations need a different checker, and even a perfect format can contain poor supervision.

Treat each milestone separately: valid records, correct loader output, correct tokenization and labels, then a short training experiment. Record which one you actually completed. That makes the next failure easier to locate and prevents a simple formatting check from becoming an unsupported promise about model training.

Sources and verification

• MiniMind repository: project and source entry point.
• Pinned dataset implementation: record fields and chat processing.
• Pinned Apache 2.0 license: repository code license.
• Pinned pretraining script and SFT script: training entry points.
• JSON Lines specification and Python JSON module: file format and parsing behavior.
• Local synthetic fixtures tested the published helper only; no training benchmark or GPU measurement was performed.

Last updated November 5, 2026. Source revision pinned and standard-library preflight exercised locally.

What self-hosting actually costs

Real cost breakdowns for self-hosted AI: hardware floors, power, maintenance hours, and the honest comparison against paying for it. No spam, unsubscribe anytime.

Email address Subscribe