Instruction Tuning

(generated with leonardo.ai)

Most LLM has few specialized tasks that they could do like summarize, classification, entity extraction, etc. We could train pre-trained LLM with lots of other tasks that would include domain specific like sentiment analysis, question answering, code generation, domain specific tasks etc. All these tasks could be trained individually to include dataset with lots of examples which is Fine-tuning. Similarly, instruction tuning is a specialized technique to perform specific tasks based on instructions dataset to train LLM.

Problem to solve

This works on the basis to turn a vanilla next word predictor model to task specific answer generator for prompts.

Bridging the gap of training multiple tasks with Instruction tuning — standard fine-tuning an LLM using pairs of instructions and their corresponding outputs

While standard fine tuning (done with zero-shot, few-shot prompting and supervised fine tuning) focuses on specific task, instruction tuning can train a model simultaneously on many tasks

Difference between Instruction tuning and Supervised fine tuning?

• Supervised fine tuning — depends on input examples and their corresponding outputs

• Instruction tuning — augments input-output examples with instructions

Instruction turning methods differ on how the instruction turning data is generated. There are mostly two category on instruction dataset

1. To the existing NLP tasks, instruction are added to those specific tasks
2. new dataset is generated based on the model trained from data in #1

Popular instructions dataset

Collection: https://github.com/yaodongC/awesome-instruction-dataset

• Benchmarking Cross-Task Generalization
  https://instructions.apps.allenai.org/

Fig. Benchmarking Cross-Task Generalization

• Super-Natural Instructions: Generalization via Declarative Instructions on 1600+ NLP Tasks https://aclanthology.org/2022.emnlp-main.340/

Fig. Super-Natural Instructions

• Self-instruct : Aligning Language Models with Self-Generated Instructions
  https://aclanthology.org/2023.acl-long.754/

Fig. Self-instruct

• Multitask Prompted Training Enables Zero-Shot Task Generalization
  https://openreview.net/forum?id=9Vrb9D0WI4

Fig. Multitask Prompted Training Enables Zero-Shot Task Generalization

Note: Soon will share colab notebook for instruction tuning.