The first release is the two models in the Llama 3 series with parameters 8B and 70B respectively.
A 400+ B model with the maximum parameter is still under development and is expected to be released in a few months.
The largest model we are currently developing is the 400B + parameter Llama 3 model, and while still under development, we are very excited about the possibilities in the future.
The model architecture is a basic Transformer architecture that contains only decoders.
Llama 3 ‘s training is based on more than 15 trillion Tokens, all of which come from public data sources. This dataset is seven times the size of Llama 2 and contains four times the amount of code. Considering future multi-lingual applications, more than 5% of our pre-training data is high-quality non-English data covering more than 30 languages. However, the performance of this part of the data may not match that of English.
Meta has launched a new ChatGPT similar http://Meta.ai website,
Experience website:https://meta.ai
Llama 3 8B and 70B are already available for official download:https://llama.meta.com/llama3/
You can also experience it on HuggingChat: https://huggingface.co/chat/
Official blog:https://ai.meta.com/blog/meta-llama-3/
Video: