Key insights

• Fine-tuning no longer requires a data scientist. InstructLab uses plain YAML files with question-and-answer pairs. Anyone who knows their subject can contribute training data.
• The data problem is solved with a trick: a local model generates hundreds of training examples from just a few you write yourself. You do not need a large dataset to start.
• Everything runs on your laptop with no third parties involved. That makes fine-tuning practical for industries where privacy matters, like insurance and law.

In Brief

Most AI models are trained to handle general questions, but they struggle when you need deep knowledge about your specific field. Cedric Clyburn, a Senior Developer Advocate at Red Hat, demonstrates how InstructLab (an open source project from Red Hat and IBM) lets you fine-tune a large language model (LLM) directly from your laptop. No developer background required.

The process has three steps: write a handful of training examples, use a local model to generate more examples automatically, then run a single command to bake that knowledge into the model. The result is a specialized model that understands your domain, runs fully offline, and never shares your data with a third party.

Step 1: Write your training data

InstructLab organizes training data in a taxonomy: a folder structure divided into skills (things the model should be able to do) and knowledge (facts it should know).

Each entry is a plain YAML file (a lightweight text format for structured data) with questions and answers. No code. No machine learning experience needed. If you know your subject, you can write the training data.

To show how this works in practice, Cedric uses a concrete example: the 2024 Oscars. The model being used is Merlinite-7B (a model with 7 billion parameters, a number that reflects its size, based on the open source Mistral model), and it has a gap in its knowledge. It was trained before the 2024 ceremony.

Step 2: Expose the gap, then fill it

Ask Merlinite-7B which film had the most Oscar nominations in 2024, and it answers "The Irishman." That film came out in 2019. The correct answer is Oppenheimer. This is exactly the kind of knowledge gap fine-tuning is meant to fix.

Writing enough training data by hand to fix this would take a very long time. InstructLab solves this by running a teacher model locally — a second LLM that reads your few examples and generates many more automatically. This is called synthetic data (data created by an AI rather than by humans).

The ilab data generate command kicks off this process. In the demo, it generates three new training examples in seconds, each asking something different about the Oscar nominations. A built-in filtration step removes low-quality examples before training begins, because not all data is good data.

Step 3: Train the model

With the data ready, one command starts the fine-tuning: ilab model train.

This uses a technique called LoRA (Low-Rank Adaptation), which updates only a small subset of the model's parameters instead of retraining the whole thing from scratch. That is why this works on a consumer laptop rather than requiring a data center. The trade-off is time: depending on your hardware, training can take a few hours.

The result

After training, the fine-tuned model is served locally in a quantized (compressed) format so it runs efficiently on a regular laptop. Ask the same question again ("which film had the most Oscar nominations in 2024?") and the answer is now Oppenheimer, with full details about the categories.

Real-world use cases

The Oscar example is a toy problem to make the concept easy to follow, but the same approach works for serious applications.

Imagine an insurance company fine-tuning a model on years of past claims and internal best practices. Field agents could get precise answers about policies without digging through documentation. Or a law firm could train a model on past contracts, letting it review and summarize new agreements much faster.

Both of these scenarios involve sensitive data that cannot be sent to a cloud provider. With InstructLab, the entire process stays local. "You've effectively taken an open source LLM, locally trained on specific data without using a third party."

Community contributions

InstructLab is also designed for collaboration. The taxonomy structure means anyone can contribute new skills and knowledge back to the broader project. Rather than each company building a private model in isolation, teams can share improvements and collaborate on domain-specific models the same way developers share code on GitHub.

The project represents a bet that the future of AI is open: models that communities build and improve together, not just models that large companies release and control.

---

Glossary

---

Sources and resources