What is InstructLab?

InstructLab simplifies the process of customizing large language models (LLMs) with private data. 

LLMs serve as the foundation for generative AI (gen AI), like chatbots and coding assistants. These LLMs can be proprietary (such as OpenAI’s GPT models and Anthropic’s Claude models) or offer varying degrees of openness around pretraining data and usage restrictions (such as Meta’s Llama models, Mistral AI’s Mistral models, and IBM’s Granite models).

AI practitioners often need to adapt a pretrained LLM to suit a particular business purpose. But there are limits to the ways you can modify an LLM:

• Fine-tuning an LLM to understand a specific area of knowledge or skills typically involves expensive, resource-intensive training.
• There’s no way to incorporate improvements back to the LLM, and thus no way for models to continuously improve from user contributions.
• LLM refinements have typically required large amounts of human-generated data, which can be time-consuming and expensive to get.

InstructLab follows an approach that punches through those limitations. It can enhance an LLM using far less human-generated information and far fewer computing resources than are typically used to retrain a model.

InstructLab is named after and based on IBM Research’s work on Large-scale Alignment for chatBots, abbreviated as LAB. The LAB method is described in a 2024 research paper by members of the MIT-IBM Watson AI Lab and IBM Research.

InstructLab is not model-specific. It can provide supplemental skills and knowledge fine-tuning to an LLM of your choice. This “tree of skills and knowledge” improves continuously from contributions and can be applied to support regular builds of an enhanced LLM. 

The InstructLab project prioritizes fast iteration and intends to retrain models on a regular basis. Organizations can also use the InstructLab model alignment tools to train their own private LLMs with their own proprietary skills and knowledge.

How to apply AI at the enterprise 

The LAB method consists of 3 components:

• Taxonomy-driven data curation. Taxonomy is a set of diverse training data curated by humans as examples of new knowledge and skills for the model.
• Large-scale synthetic data generation. The model is then used to generate new examples based on the seed training data. Recognizing that synthetic data can vary in quality, the LAB method adds an automated step to refine the example answers, making sure they’re grounded and safe.
• Iterative, large-scale alignment tuning. Finally, the model is retrained based on the set of synthetic data. The LAB method includes 2 tuning phases: knowledge tuning, followed by skill tuning.

The dataset grows through contributions of skills and knowledge, with each addition improving the quality of the model being tuned. 

Let’s compare InstructLab to the other ways of customizing an LLM to serve your domain-specific use cases.

Pretraining

Pretraining represents the deepest level of model customization, and allows organizations to curate what training data shapes the models’ foundational understanding. 

During pretraining, an LLM is trained to predict the next token using trillions of tokens of unlabeled data. This gets really expensive, sometimes requiring thousands of GPUs and months of time. Pretraining a highly capable LLM is only possible for organizations with significant resources.

Alignment tuning

After pretraining, LLMs undergo alignment tuning to make the model’s answers as accurate and useful as possible. The 1st step in alignment tuning is typically instruction tuning, in which a model is trained directly on specific tasks of interest. 

Next is preference tuning, which can include reinforcement learning from human feedback (RLHF). In this step, humans test the model and rate its output, noting if the model’s answers are preferred or unpreferred. An RLHF process may include multiple rounds of feedback and refinement to optimize a model.

Researchers have found that the amount of feedback at this alignment tuning stage can be much smaller than the initial set of training data―tens of thousands of human annotations, compared to the trillions of tokens of data required for pretraining―and still unlock latent capabilities of the model.

InstructLab

The LAB method emerged from the idea that it should be possible to realize the benefits of model alignment from an even smaller set of human-generated data. An AI model can use a handful of human examples to generate a large amount of synthetic data―then refine that list for quality―and use that high-quality synthetic data set for further tuning and training. In contrast to instruction tuning, which typically needs thousands of examples of human feedback, LAB can make a model significantly better using relatively few examples provided by humans.

How is InstructLab different from retrieval-augmented generation (RAG)?

The short answer is InstructLab and retrieval-augmented generation (RAG) solve different problems.

RAG is a cost-efficient method for supplementing an LLM with domain-specific knowledge that wasn’t part of its pretraining. RAG makes it possible for a chatbot to accurately answer questions related to a specific field or business without retraining the model. 

With RAG, knowledge documents are stored in a vector database, then retrieved in chunks and sent to the model as part of user queries. This is helpful for anyone who wants to add proprietary data to an LLM without giving up control of their information, or who needs an LLM to access timely information. 

This is in contrast to the InstructLab method, which sources end-user contributions to support regular builds of an enhanced version of an LLM. InstructLab helps add knowledge and unlock new skills of an LLM.

It’s possible to "supercharge" a RAG process by using the RAG technique on an InstructLab-tuned model.

Learn more about RAG

Red Hat AI InstructLab on IBM Cloud allows for contributions to an LLM without the need to own and operate hardware infrastructure.

On its own, Red Hat InstructLab is an open source project that simplifies LLM development, enabling a cost-effective approach to align models with less data and fewer resources.

IBM Cloud is an enterprise cloud platform designed for even the most regulated industries, delivering a highly resilient, performant, secure, and compliant cloud.

Together, Red Hat AI InstructLab on IBM Cloud offers a scalable, cost-effective solution to simplify, scale, and secure the alignment of your LLMs to your organization’s unique use-cases.

Learn more about Red Hat InstructLab on IBM Cloud