Meta Llama 3's new AI model is here

Already at the beginning of April 2024, rumors appeared that Meta will soon release a new AI model. On your own blog Meta recently announced the latest generative AI model in the Llama series: the Llama 3. The Llama 3 family currently includes two models, the Llama 3 8B and the Llama 3 70B, with more models in the near future. In July 2023, Meta launched the previous version, Llama 2a large language model now used in many contexts – from Meta's own services to Opera AI to the Groq AI-powered answering machine.

Performance advantage over previous models and competition

The new models have eight billion and 70 billion parameters respectively, providing significantly increased performance over previous models of the Llama 2 series. According to Meta, the models were trained on two custom-built GPU clusters with a total of 24,000 GPUs, which are among The most powerful generative AI models currently available.

When comparing performance with other generative AI models, Meta reports the results of Llama 3 models on AI benchmarks such as MMLU (measurement of knowledge), ARC (measurement of skill acquisition), and DROP (test of model reasoning across passages of text). ). For example, the Llama 3 8B outperforms other leading models in nine different parameters. With Llama 3, users can benefit from improved control over forms, a lower probability of non-response and higher accuracy for questions in different subject areas.

Training methods and data

The size of the training data set is particularly noteworthy. Llama 3 was trained on a set of 15 trillion symbols, which is approximately 750 billion words. This is seven times larger than the dataset used in Llama 2, according to Meta. Meta is still relatively tight-lipped about the exact details of the training data, and notes that both publicly available data and synthetic data, i.e. data generated by artificial intelligence, were used to train the new models. The more comprehensive variant was trained using data up to December 2023. The smaller version with data up to March 2023.

Many companies in the field of generative AI view their training data as a key competitive advantage and are therefore reluctant to reveal details. This reluctance is also due to concerns that detailed information about training data could lead to legal disputes over intellectual property. Large AI companies, including OpenAI, are increasingly being targeted by investigations for using copyrighted content without proper permissions. The commercial use of data for AI training purposes has proven to be a lucrative business area. A recent example is Reddit, which entered into an agreement with Google prior to its IPO under which Reddit user content would be shared with Google for AI training purposes in exchange for $60 million per year.

Meta continues to rely on open models

Write on your blog deadThat the company continues to follow the open approach and that Llama 3 is available as open source for users and developers:

In support of our long-term, open approach, we are putting Llama 3 into the hands of the community. We want to unleash the next wave of AI innovation across the stack – from apps to developer tools to assessments, inference improvements, and more. We can't wait to see what you build and look forward to your feedback.

Llama 3 models are now available for download and support Meta's AI on Facebook, Instagram, WhatsApp, Messenger, and the web. It will also soon be hosted on a variety of cloud platforms, including AWS, Google Cloud, and Hugging Face. The AI ​​model can also be used as the basis for various services, such as Perplexity's autoresponder.