Meta's Llama2 and it's impact

META recently released collection of LLMs -- Llama 2 -- ranging from 7B to 70B parameters - with viable commercial as well as research license. These models are also available via SageMaker Jumpstart on the very first day of the launch!!! 🎵 My thoughts on what does it mean for open source vs. closed source models and how RLHF will have bigger impact on the future of LLMs.

🎉 Per their benchmark, Llama 2-chat outperforms open source models (Mosaic's MPT and Falcon) which are top performing models in OSS community
🎈 In some benchmarks (MMLU), Llama 2 is almost close to GPT-3.5 which is a closed source system
===
<Ritesh’s take> This is heading in the right direction where OSS models are competing with each other to become better -- at the same time OSS models are getting closer to the performance of a close source models.. Hopefully in few more iteration we should see them performing better than closed source models..let's see how far that future is :)
===
🎇 META describes lot many details of how the supervised fine-tuning (SFT) as well as reinforcement learning with human feedback (RLHF) was done for their models.
<Ritesh’s take> The amount of details shared by META is going to open multiple doors for fine-tuning via RLHF mechanism for multiple models. We may see new shops popping up which provide RLHF based tuning for customizing open or even closed source models - if model provider allows unfreezing of weights.

Certainly there is a science behind model performance improvement but majority of the performance gain we are going to achieve is using human feedback and/or annotated data.
(originally posted by author on LinkedIn)

#foundationmodels #generativeai #meta