Mistral 7B

Mistral 7B is introduced as a 7–billion-parameter language model engineered for superior performance and efficiency. It outperforms the best open 13B model (Llama 2) across all evaluated benchmarks, and the best released 34B model (Llama 1) in reasoning, mathematics, and code generation. The model leverages grouped-query attention (GQA) for faster inference, coupled with sliding window attention (SWA) to effectively handle sequences of arbitrary length with a reduced inference cost. Additionally, a model fine-tuned to follow instructions, Mistral 7B– Instruct, surpasses Llama 2 13B– chat model both on human and automated benchmarks. These models are released under the Apache 2.0 license.