# Falcon 2: An 11B parameter pretrained language model and VLM, trained on over 5000B tokens and 11 languages

DevFeed: [Falcon 2: An 11B parameter pretrained language model and VLM, trained on over 5000B tokens and 11 languages](<https://devfeed.tech/articles/falcon-2-an-11b-parameter-pretrained-language-model-and-vlm-trained-on-over-5000b-tokens-and-11-languages-7189.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/falcon2-11b>)

Author: Quentin Malartic; Nilabhra Roy Chowdhury; Ruxandra Cojocaru; Mughaira; Giulia Campesan; Sanath Narayan; Ankit Singh; Clémentine Fourrier; Nathan Habib

Published: 2024-05-24T00:00:00Z

Content type: article

Language: en

Sources: [Hugging Face - Blog](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [vlm](<https://devfeed.tech/topics/vlm.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [Open Source](<https://devfeed.tech/topics/open-source.md>), [model-deployment](<https://devfeed.tech/topics/model-deployment.md>), [dataset](<https://devfeed.tech/topics/dataset.md>), [model architecture](<https://devfeed.tech/topics/model-architecture.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>)

Tags: [3d](<https://devfeed.tech/tags/3d.md>), [architecture](<https://devfeed.tech/tags/architecture.md>), [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [community](<https://devfeed.tech/tags/community.md>), [data](<https://devfeed.tech/tags/data.md>), [ecosystem](<https://devfeed.tech/tags/ecosystem.md>), [flash-attention-2](<https://devfeed.tech/tags/flash-attention-2.md>), [inference](<https://devfeed.tech/tags/inference.md>), [llm](<https://devfeed.tech/tags/llm.md>), [model](<https://devfeed.tech/tags/model.md>), [multimodal](<https://devfeed.tech/tags/multimodal.md>), [nlp](<https://devfeed.tech/tags/nlp.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [research](<https://devfeed.tech/tags/research.md>), [vision](<https://devfeed.tech/tags/vision.md>), [vlm](<https://devfeed.tech/tags/vlm.md>)

## AI overview

The article announces Falcon 2, a family of smaller open-source models from TII that includes an 11B pretrained language model and an 11B vision-language model. Falcon 2 targets improved performance, multimodal support, cheaper inference, and broader downstream use. The VLM supports text-based conversations about visual content, while the models primarily support English and have capabilities in ten additional languages.

## Source excerpt

We're on a journey to advance and democratize artificial intelligence through open source and open science.