# sentence-transformers

Python library for computing, training, and fine-tuning embedding and reranker models.

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## Training and Finetuning Multi-Vector Embedding Models with Sentence Transformers

DevFeed: [Training and Finetuning Multi-Vector Embedding Models with Sentence Transformers](<https://devfeed.tech/articles/training-and-finetuning-multi-vector-embedding-models-with-sentence-transformers-7526.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/train-multi-vector-encoder>)

Author: Tom Aarsen

Published: 2026-08-26T00:00:00Z

Content type: tutorial

Language: en

Sources: [Hugging Face - Blog](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [sentence-transformers](<https://devfeed.tech/topics/sentence-transformers.md>), [AI search](<https://devfeed.tech/topics/ai-search.md>), [AI, ML & Data Engineering](<https://devfeed.tech/topics/ai-ml-data-engineering.md>)

Tags: [community](<https://devfeed.tech/tags/community.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [embedding](<https://devfeed.tech/tags/embedding.md>), [embeddings](<https://devfeed.tech/tags/embeddings.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [guide](<https://devfeed.tech/tags/guide.md>), [nlp](<https://devfeed.tech/tags/nlp.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [retrieval](<https://devfeed.tech/tags/retrieval.md>), [sentence-transformers](<https://devfeed.tech/tags/sentence-transformers.md>), [training](<https://devfeed.tech/tags/training.md>)

### AI overview

A tutorial on finetuning multi-vector embedding models with Sentence Transformers. It explains late-interaction token-level retrieval, training components, and domain-specific retrieval improvements.

### Source excerpt

We're on a journey to advance and democratize artificial intelligence through open source and open science.

## Multi-Vector (Late Interaction) Embedding Models with Sentence Transformers

DevFeed: [Multi-Vector (Late Interaction) Embedding Models with Sentence Transformers](<https://devfeed.tech/articles/multi-vector-late-interaction-embedding-models-with-sentence-transformers-7360.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/multi-vector-encoder>)

Author: Tom Aarsen; Antoine Chaffin; Raphael Sourty

Published: 2026-08-18T00:00:00Z

Content type: article

Language: en

Sources: [Hugging Face - Blog](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [sentence-transformers](<https://devfeed.tech/topics/sentence-transformers.md>), [Embeddings](<https://devfeed.tech/topics/embeddings.md>), [Open Source](<https://devfeed.tech/topics/open-source.md>)

Tags: [community](<https://devfeed.tech/tags/community.md>), [embedding](<https://devfeed.tech/tags/embedding.md>), [embeddings](<https://devfeed.tech/tags/embeddings.md>), [guide](<https://devfeed.tech/tags/guide.md>), [multimodal](<https://devfeed.tech/tags/multimodal.md>), [nlp](<https://devfeed.tech/tags/nlp.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [retrieval](<https://devfeed.tech/tags/retrieval.md>), [search](<https://devfeed.tech/tags/search.md>), [sentence-transformers](<https://devfeed.tech/tags/sentence-transformers.md>), [transformers](<https://devfeed.tech/tags/transformers.md>), [vector](<https://devfeed.tech/tags/vector.md>)

### AI overview

This article explains multi-vector, or late-interaction, embedding models with Sentence Transformers. It covers token-level representations, MaxSim scoring, retrieval over text and page images, integration with search systems, and index-size tradeoffs.

### Source excerpt

We're on a journey to advance and democratize artificial intelligence through open source and open science.

## Introducing the Ettin Reranker Family

DevFeed: [Introducing the Ettin Reranker Family](<https://devfeed.tech/articles/introducing-the-ettin-reranker-family-7186.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/ettin-reranker>)

Author: Tom Aarsen

Published: 2026-05-19T00:00:00Z

Content type: article

Language: en

Sources: [Hugging Face - Blog](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [sentence-transformers](<https://devfeed.tech/topics/sentence-transformers.md>), [Transformer](<https://devfeed.tech/topics/transformer.md>), [long-context](<https://devfeed.tech/topics/long-context.md>), [Agent Skill](<https://devfeed.tech/topics/agent-skill.md>)

Tags: [ai-coding](<https://devfeed.tech/tags/ai-coding.md>), [community](<https://devfeed.tech/tags/community.md>), [long-context](<https://devfeed.tech/tags/long-context.md>), [nlp](<https://devfeed.tech/tags/nlp.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [ranking](<https://devfeed.tech/tags/ranking.md>), [research](<https://devfeed.tech/tags/research.md>), [retrieval](<https://devfeed.tech/tags/retrieval.md>), [sentence-transformers](<https://devfeed.tech/tags/sentence-transformers.md>)

### AI overview

The article introduces six Sentence Transformers CrossEncoder rerankers built on Ettin ModernBERT encoders. It explains their distillation-based training, retrieval-then-rerank usage, quality and cost trade-offs, and support for up to 8K tokens of context.

### Source excerpt

We're on a journey to advance and democratize artificial intelligence through open source and open science.

## Training and Finetuning Multimodal Embedding & Reranker Models with Sentence Transformers

DevFeed: [Training and Finetuning Multimodal Embedding & Reranker Models with Sentence Transformers](<https://devfeed.tech/articles/training-and-finetuning-multimodal-embedding-reranker-models-with-sentence-transformers-7527.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/train-multimodal-sentence-transformers>)

Author: Tom Aarsen

Published: 2026-04-16T00:00:00Z

Content type: tutorial

Language: en

Sources: [Hugging Face - Blog](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [multimodal](<https://devfeed.tech/topics/multimodal.md>), [sentence-transformers](<https://devfeed.tech/topics/sentence-transformers.md>), [datasets](<https://devfeed.tech/topics/datasets.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [Open Source](<https://devfeed.tech/topics/open-source.md>)

Tags: [community](<https://devfeed.tech/tags/community.md>), [data](<https://devfeed.tech/tags/data.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [embedding](<https://devfeed.tech/tags/embedding.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [models](<https://devfeed.tech/tags/models.md>), [multimodal](<https://devfeed.tech/tags/multimodal.md>), [nlp](<https://devfeed.tech/tags/nlp.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [optimization](<https://devfeed.tech/tags/optimization.md>), [performance](<https://devfeed.tech/tags/performance.md>), [retrieval](<https://devfeed.tech/tags/retrieval.md>), [sentence-transformers](<https://devfeed.tech/tags/sentence-transformers.md>), [train](<https://devfeed.tech/tags/train.md>), [training](<https://devfeed.tech/tags/training.md>), [transformers](<https://devfeed.tech/tags/transformers.md>), [vision](<https://devfeed.tech/tags/vision.md>)

### AI overview

A practical guide to fine-tuning multimodal Sentence Transformer embedding and reranker models for visual document retrieval. It explains the training pipeline and shows how domain-specific fine-tuning improved retrieval performance from 0.888 to 0.947 in the example evaluation.

### Source excerpt

We're on a journey to advance and democratize artificial intelligence through open source and open science.

## Multimodal Embedding & Reranker Models with Sentence Transformers

DevFeed: [Multimodal Embedding & Reranker Models with Sentence Transformers](<https://devfeed.tech/articles/multimodal-embedding-reranker-models-with-sentence-transformers-7361.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/multimodal-sentence-transformers>)

Author: Tom Aarsen

Published: 2026-04-09T00:00:00Z

Content type: article

Language: en

Sources: [Hugging Face - Blog](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [sentence-transformers](<https://devfeed.tech/topics/sentence-transformers.md>), [Embeddings](<https://devfeed.tech/topics/embeddings.md>), [multimodal](<https://devfeed.tech/topics/multimodal.md>), [Retrieval Augmented Generation (RAG)](<https://devfeed.tech/topics/retrieval-augmented-generation-rag.md>), [vlm](<https://devfeed.tech/topics/vlm.md>), [GPU](<https://devfeed.tech/topics/gpu.md>), [Cloud](<https://devfeed.tech/topics/cloud.md>)

Tags: [cloud](<https://devfeed.tech/tags/cloud.md>), [community](<https://devfeed.tech/tags/community.md>), [embedding](<https://devfeed.tech/tags/embedding.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [multimodal](<https://devfeed.tech/tags/multimodal.md>), [nlp](<https://devfeed.tech/tags/nlp.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [rag](<https://devfeed.tech/tags/rag.md>), [retrieval](<https://devfeed.tech/tags/retrieval.md>), [search](<https://devfeed.tech/tags/search.md>), [sentence-transformers](<https://devfeed.tech/tags/sentence-transformers.md>), [transformers](<https://devfeed.tech/tags/transformers.md>), [vision](<https://devfeed.tech/tags/vision.md>)

### AI overview

This article explains how multimodal embedding and reranker models in Sentence Transformers map text, images, audio, and video into shared spaces or score cross-modal pairs. It covers visual document retrieval, cross-modal search, multimodal RAG, hardware requirements, model loading, and similarity computation.

### Source excerpt

We're on a journey to advance and democratize artificial intelligence through open source and open science.

## Sentence Transformers is joining Hugging Face!

DevFeed: [Sentence Transformers is joining Hugging Face!](<https://devfeed.tech/articles/sentence-transformers-is-joining-hugging-face-7472.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/sentence-transformers-joins-hf>)

Author: Tom Aarsen

Published: 2025-10-22T00:00:00Z

Content type: news

Language: en

Sources: [Hugging Face - Blog](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [sentence-transformers](<https://devfeed.tech/topics/sentence-transformers.md>), [hugging face](<https://devfeed.tech/topics/hugging-face.md>), [Embeddings](<https://devfeed.tech/topics/embeddings.md>), [Natural language processing](<https://devfeed.tech/topics/nlp.md>), [Open Source](<https://devfeed.tech/topics/open-source.md>)

Tags: [announcement](<https://devfeed.tech/tags/announcement.md>), [collaboration](<https://devfeed.tech/tags/collaboration.md>), [community](<https://devfeed.tech/tags/community.md>), [contributors](<https://devfeed.tech/tags/contributors.md>), [embeddings](<https://devfeed.tech/tags/embeddings.md>), [hugging-face](<https://devfeed.tech/tags/hugging-face.md>), [natural-language-processing](<https://devfeed.tech/tags/natural-language-processing.md>), [nlp](<https://devfeed.tech/tags/nlp.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [sentence-transformers](<https://devfeed.tech/tags/sentence-transformers.md>)

### AI overview

Sentence Transformers, also known as SentenceBERT or SBERT, is joining Hugging Face. The popular open-source library generates embeddings that capture semantic meaning and supports natural language processing tasks such as semantic search, similarity analysis, clustering, and paraphrase mining. It will remain community-driven under the Apache 2.0 license, with contributions welcomed from researchers, developers, and enthusiasts.

### Source excerpt

We're on a journey to advance and democratize artificial intelligence through open source and open science.

## Welcome EmbeddingGemma, Google's new efficient embedding model

DevFeed: [Welcome EmbeddingGemma, Google's new efficient embedding model](<https://devfeed.tech/articles/welcome-embeddinggemma-google-s-new-efficient-embedding-model-7180.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/embeddinggemma>)

Author: Tom Aarsen; Joshua; Alvaro Bartolome; Aritra Roy Gosthipaty; Pedro Cuenca; Sergio Paniego

Published: 2025-09-04T00:00:00Z

Content type: article

Language: en

Sources: [Hugging Face - Blog](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [Embeddings](<https://devfeed.tech/topics/embeddings.md>), [sentence-transformers](<https://devfeed.tech/topics/sentence-transformers.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [Google](<https://devfeed.tech/topics/google.md>), [Retrieval-Augmented Generation](<https://devfeed.tech/topics/retrieval-augmented-generation.md>), [Open Source](<https://devfeed.tech/topics/open-source.md>), [transformers.js](<https://devfeed.tech/topics/transformers-js.md>), [Inference](<https://devfeed.tech/topics/inference.md>), [onnx](<https://devfeed.tech/topics/onnx.md>), [dataset](<https://devfeed.tech/topics/dataset.md>), [LangChain](<https://devfeed.tech/topics/langchain.md>), [llamaindex](<https://devfeed.tech/topics/llamaindex.md>)

Tags: [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [community](<https://devfeed.tech/tags/community.md>), [embedding](<https://devfeed.tech/tags/embedding.md>), [google](<https://devfeed.tech/tags/google.md>), [guide](<https://devfeed.tech/tags/guide.md>), [langchain](<https://devfeed.tech/tags/langchain.md>), [llamaindex](<https://devfeed.tech/tags/llamaindex.md>), [nlp](<https://devfeed.tech/tags/nlp.md>), [on-device](<https://devfeed.tech/tags/on-device.md>), [onnx](<https://devfeed.tech/tags/onnx.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [rag](<https://devfeed.tech/tags/rag.md>), [sentence-transformers](<https://devfeed.tech/tags/sentence-transformers.md>), [transformers-js](<https://devfeed.tech/tags/transformers-js.md>)

### AI overview

Google introduces EmbeddingGemma, a compact multilingual embedding model designed for fast, efficient on-device use. The article covers its architecture, training, multilingual capabilities, benchmark performance, framework integrations, and domain fine-tuning for retrieval applications.

### Source excerpt

We're on a journey to advance and democratize artificial intelligence through open source and open science.

## Training and Finetuning Sparse Embedding Models with Sentence Transformers

DevFeed: [Training and Finetuning Sparse Embedding Models with Sentence Transformers](<https://devfeed.tech/articles/training-and-finetuning-sparse-embedding-models-with-sentence-transformers-7530.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/train-sparse-encoder>)

Author: Tom Aarsen; Arthur BRESNU

Published: 2025-07-01T00:00:00Z

Content type: tutorial

Language: en

Sources: [Hugging Face - Blog](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [sentence-transformers](<https://devfeed.tech/topics/sentence-transformers.md>), [Embeddings](<https://devfeed.tech/topics/embeddings.md>), [hugging face](<https://devfeed.tech/topics/hugging-face.md>), [Benchmark](<https://devfeed.tech/topics/benchmark.md>)

Tags: [benchmark](<https://devfeed.tech/tags/benchmark.md>), [bm25](<https://devfeed.tech/tags/bm25.md>), [classification](<https://devfeed.tech/tags/classification.md>), [community](<https://devfeed.tech/tags/community.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [embedding](<https://devfeed.tech/tags/embedding.md>), [examples](<https://devfeed.tech/tags/examples.md>), [guide](<https://devfeed.tech/tags/guide.md>), [hugging-face](<https://devfeed.tech/tags/hugging-face.md>), [models](<https://devfeed.tech/tags/models.md>), [nlp](<https://devfeed.tech/tags/nlp.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [search](<https://devfeed.tech/tags/search.md>), [sentence-transformers](<https://devfeed.tech/tags/sentence-transformers.md>), [training](<https://devfeed.tech/tags/training.md>), [transformers](<https://devfeed.tech/tags/transformers.md>), [vector](<https://devfeed.tech/tags/vector.md>)

### AI overview

This tutorial explains how to fine-tune sparse embedding models with Sentence Transformers. It covers the required components, pretrained sparse encoders available through the Hugging Face Hub, the distinction between dense and sparse embeddings, interpretability through vocabulary tokens, and neural query/document expansion compared with BM25.

### Source excerpt

We're on a journey to advance and democratize artificial intelligence through open source and open science.

## Training and Finetuning Reranker Models with Sentence Transformers

DevFeed: [Training and Finetuning Reranker Models with Sentence Transformers](<https://devfeed.tech/articles/training-and-finetuning-reranker-models-with-sentence-transformers-7528.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/train-reranker>)

Author: Tom Aarsen

Published: 2025-03-26T00:00:00Z

Content type: tutorial

Language: en

Sources: [Hugging Face - Blog](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [sentence-transformers](<https://devfeed.tech/topics/sentence-transformers.md>), [datasets](<https://devfeed.tech/topics/datasets.md>)

Tags: [community](<https://devfeed.tech/tags/community.md>), [embedding](<https://devfeed.tech/tags/embedding.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [guide](<https://devfeed.tech/tags/guide.md>), [models](<https://devfeed.tech/tags/models.md>), [nlp](<https://devfeed.tech/tags/nlp.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [retrieval](<https://devfeed.tech/tags/retrieval.md>), [search](<https://devfeed.tech/tags/search.md>), [sentence-transformers](<https://devfeed.tech/tags/sentence-transformers.md>), [training](<https://devfeed.tech/tags/training.md>), [transformers](<https://devfeed.tech/tags/transformers.md>)

### AI overview

This tutorial explains how to fine-tune reranker models with Sentence Transformers. It covers datasets, loss functions, training arguments, evaluators, and the trainer class, and describes how Cross Encoder rerankers compare with embedding models in a two-stage retrieve-and-rerank search system.

### Source excerpt

We're on a journey to advance and democratize artificial intelligence through open source and open science.

## Train 400x faster Static Embedding Models with Sentence Transformers

DevFeed: [Train 400x faster Static Embedding Models with Sentence Transformers](<https://devfeed.tech/articles/train-400x-faster-static-embedding-models-with-sentence-transformers-7491.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/static-embeddings>)

Author: Tom Aarsen

Published: 2025-01-15T00:00:00Z

Content type: article

Language: en

Sources: [Hugging Face - Blog](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [sentence-transformers](<https://devfeed.tech/topics/sentence-transformers.md>), [Embeddings](<https://devfeed.tech/topics/embeddings.md>), [Natural language processing](<https://devfeed.tech/topics/nlp.md>), [On-device AI](<https://devfeed.tech/topics/on-device-ai.md>), [Inference](<https://devfeed.tech/topics/inference.md>), [datasets](<https://devfeed.tech/topics/datasets.md>), [Open Source](<https://devfeed.tech/topics/open-source.md>), [recommendation systems](<https://devfeed.tech/topics/recommendation-systems.md>)

Tags: [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [community](<https://devfeed.tech/tags/community.md>), [dataset](<https://devfeed.tech/tags/dataset.md>), [edge-computing](<https://devfeed.tech/tags/edge-computing.md>), [embedded](<https://devfeed.tech/tags/embedded.md>), [embedding](<https://devfeed.tech/tags/embedding.md>), [embeddings](<https://devfeed.tech/tags/embeddings.md>), [guide](<https://devfeed.tech/tags/guide.md>), [inference](<https://devfeed.tech/tags/inference.md>), [low-power](<https://devfeed.tech/tags/low-power.md>), [natural-language-processing](<https://devfeed.tech/tags/natural-language-processing.md>), [nlp](<https://devfeed.tech/tags/nlp.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [recommendation-systems](<https://devfeed.tech/tags/recommendation-systems.md>), [retrieval](<https://devfeed.tech/tags/retrieval.md>), [sentence-transformers](<https://devfeed.tech/tags/sentence-transformers.md>)

### AI overview

This Hugging Face blog post presents a method for training static embedding models that run 100x to 400x faster on CPU while retaining most of the quality of state-of-the-art models. It introduces released models for English retrieval and multilingual similarity, along with their training strategy, scripts, evaluation reports, and datasets. The approach supports on-device, in-browser, edge, low-power, and embedded use cases.

### Source excerpt

We're on a journey to advance and democratize artificial intelligence through open source and open science.

## Introducing the Hugging Face Embedding Container for Amazon SageMaker

DevFeed: [Introducing the Hugging Face Embedding Container for Amazon SageMaker](<https://devfeed.tech/articles/introducing-the-hugging-face-embedding-container-for-amazon-sagemaker-7464.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/sagemaker-huggingface-embedding>)

Author: Philipp Schmid; Jeff Boudier

Published: 2024-06-07T00:00:00Z

Content type: tutorial

Language: en

Sources: [Hugging Face - Blog](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [model-deployment](<https://devfeed.tech/topics/model-deployment.md>), [Amazon SageMaker](<https://devfeed.tech/topics/amazon-sagemaker.md>), [Amazon Bedrock](<https://devfeed.tech/topics/amazon-bedrock.md>), [Inference](<https://devfeed.tech/topics/inference.md>), [Embeddings](<https://devfeed.tech/topics/embeddings.md>), [Transformers](<https://devfeed.tech/topics/transformers.md>), [sentence-transformers](<https://devfeed.tech/topics/sentence-transformers.md>), [AWS IAM](<https://devfeed.tech/topics/aws-iam.md>)

Tags: [amazon-sagemaker](<https://devfeed.tech/tags/amazon-sagemaker.md>), [aws](<https://devfeed.tech/tags/aws.md>), [batching](<https://devfeed.tech/tags/batching.md>), [cloud](<https://devfeed.tech/tags/cloud.md>), [container](<https://devfeed.tech/tags/container.md>), [embedding](<https://devfeed.tech/tags/embedding.md>), [embeddings](<https://devfeed.tech/tags/embeddings.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [guide](<https://devfeed.tech/tags/guide.md>), [hugging-face](<https://devfeed.tech/tags/hugging-face.md>), [inference](<https://devfeed.tech/tags/inference.md>), [partnership](<https://devfeed.tech/tags/partnership.md>), [prometheus](<https://devfeed.tech/tags/prometheus.md>), [sdk](<https://devfeed.tech/tags/sdk.md>), [sentence-transformers](<https://devfeed.tech/tags/sentence-transformers.md>), [telemetry](<https://devfeed.tech/tags/telemetry.md>), [tracing](<https://devfeed.tech/tags/tracing.md>)

### AI overview

This tutorial explains how to deploy open embedding models to Amazon SageMaker using the Hugging Face Embedding Container. It uses Text Embeddings Inference for efficient, production-ready serving and covers container selection, CPU and GPU variants, batching, and observability features.

### Source excerpt

We're on a journey to advance and democratize artificial intelligence through open source and open science.

## Training and Finetuning Embedding Models with Sentence Transformers

DevFeed: [Training and Finetuning Embedding Models with Sentence Transformers](<https://devfeed.tech/articles/training-and-finetuning-embedding-models-with-sentence-transformers-7529.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/train-sentence-transformers>)

Author: Tom Aarsen

Published: 2024-05-28T00:00:00Z

Content type: tutorial

Language: en

Sources: [Hugging Face - Blog](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [sentence-transformers](<https://devfeed.tech/topics/sentence-transformers.md>), [datasets](<https://devfeed.tech/topics/datasets.md>), [Embeddings](<https://devfeed.tech/topics/embeddings.md>), [hugging face](<https://devfeed.tech/topics/hugging-face.md>), [AI Development](<https://devfeed.tech/topics/ai-development.md>)

Tags: [community](<https://devfeed.tech/tags/community.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [embedding](<https://devfeed.tech/tags/embedding.md>), [guide](<https://devfeed.tech/tags/guide.md>), [hugging-face](<https://devfeed.tech/tags/hugging-face.md>), [json](<https://devfeed.tech/tags/json.md>), [nlp](<https://devfeed.tech/tags/nlp.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [parquet](<https://devfeed.tech/tags/parquet.md>), [sentence-transformers](<https://devfeed.tech/tags/sentence-transformers.md>), [training](<https://devfeed.tech/tags/training.md>)

### AI overview

This tutorial explains how to fine-tune Sentence Transformers for task-specific similarity, covering datasets, loss functions, training arguments, evaluators, and the Trainer. It also describes loading training data from the Hugging Face Datasets Hub or local CSV, JSON, Parquet, Arrow, and SQL sources.

### Source excerpt

We're on a journey to advance and democratize artificial intelligence through open source and open science.

## Hugging Face x LangChain : A new partner package

DevFeed: [Hugging Face x LangChain : A new partner package](<https://devfeed.tech/articles/hugging-face-x-langchain-a-new-partner-package-7306.md>)

Original publisher: [Read original article](<https://huggingface.co/blog/langchain>)

Author: Joffrey THOMAS; Cyril KONDRATENKO; Erick Friis

Published: 2024-05-14T00:00:00Z

Content type: article

Language: en

Sources: [Hugging Face - Blog](<https://devfeed.tech/sources/hugging-face-blog.md>)

Topics: [hugging face](<https://devfeed.tech/topics/hugging-face.md>), [LangChain](<https://devfeed.tech/topics/langchain.md>), [Retrieval Augmented Generation (RAG)](<https://devfeed.tech/topics/retrieval-augmented-generation-rag.md>), [Open Source](<https://devfeed.tech/topics/open-source.md>), [sentence-transformers](<https://devfeed.tech/topics/sentence-transformers.md>), [LLMs](<https://devfeed.tech/topics/llms.md>), [tgi](<https://devfeed.tech/topics/tgi.md>)

Tags: [api](<https://devfeed.tech/tags/api.md>), [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [collaboration](<https://devfeed.tech/tags/collaboration.md>), [community](<https://devfeed.tech/tags/community.md>), [hugging-face](<https://devfeed.tech/tags/hugging-face.md>), [langchain](<https://devfeed.tech/tags/langchain.md>), [llm](<https://devfeed.tech/tags/llm.md>), [llms](<https://devfeed.tech/tags/llms.md>), [nlp](<https://devfeed.tech/tags/nlp.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [rag](<https://devfeed.tech/tags/rag.md>), [sentence-transformers](<https://devfeed.tech/tags/sentence-transformers.md>), [tgi](<https://devfeed.tech/tags/tgi.md>)

### AI overview

This article introduces Hugging Face's partner package for LangChain. It explains how the package exposes Hugging Face pipelines, models, serverless inference endpoints, TGI deployments, chat templates, and embedding models, including integrations with RAG and agent use cases.

### Source excerpt

We're on a journey to advance and democratize artificial intelligence through open source and open science.

## Semantic Search - Word Embeddings with OpenAI

DevFeed: [Semantic Search - Word Embeddings with OpenAI](<https://devfeed.tech/articles/semantic-search-word-embeddings-with-openai-24996.md>)

Original publisher: [Read original article](<https://codeahoy.com/2023/03/28/semantic-search-intro/>)

Author: umer

Published: 2023-03-28T00:00:00Z

Content type: tutorial

Language: en

Sources: [Code Ahoy - Articles](<https://devfeed.tech/sources/code-ahoy-articles.md>)

Topics: [AI search](<https://devfeed.tech/topics/ai-search.md>), [Natural language processing](<https://devfeed.tech/topics/nlp.md>), [Databases](<https://devfeed.tech/topics/databases.md>), [sentence-transformers](<https://devfeed.tech/topics/sentence-transformers.md>), [Computer science](<https://devfeed.tech/topics/computer-science.md>)

Tags: [databases](<https://devfeed.tech/tags/databases.md>), [embeddings](<https://devfeed.tech/tags/embeddings.md>), [nlp](<https://devfeed.tech/tags/nlp.md>), [openai](<https://devfeed.tech/tags/openai.md>), [search](<https://devfeed.tech/tags/search.md>), [techniques](<https://devfeed.tech/tags/techniques.md>), [vector](<https://devfeed.tech/tags/vector.md>), [word-embeddings](<https://devfeed.tech/tags/word-embeddings.md>)

### AI overview

This introductory tutorial contrasts semantic search with lexical search, explaining how semantic search uses query context and intent to improve result relevance. It also introduces NLP, embeddings, and vector databases as implementation components.

### Source excerpt

According to Wikipedia, Semantic Search denotes search with meaning, as distinguished from lexical search where the search engine looks for literal matches of the query words or variants of them, without understanding the overall meaning of the query. For example a user is searching for the term "jaguar." A traditional keyword-based search engine might return results about the car manufacturer, the animal, or even the Jacksonville Jaguars football team. However, semantic search would analyze the context and intent behind the user's query, such as whether they are interested in cars or wildlife, and then prioritize results accordingly. In this blog post, we will explore the underlying principles of semantic search, discuss its advantages over other types of search, and examine real-world applications that are transforming the way we access and consume information. Lexical Search Engines Lexical (Traditional) search engines have served us well using keyword-based search methods, looking for matching exact words or phrases in users' queries with those in documents/database. For example, if we search for the term "computer science intro" in a lexical / traditional search engine, it will return results that match one or more of my search terms. As you can imagine, the keyword matching approach often falls short when it comes to understanding what the user actually meant, often producing less accurate results. Semantic Search Enter semantic search -- a context-aware search technology that aims to improve search results by focusing on understanding the meaning and context behind queries. When a user inputs the query "computer science intro" in a semantic search engine, it would first attempt to understand the intent behind the query. In this case, the user is likely looking for introductory resources related to computer science. Based on this understanding, the search engine would prioritize search results such as introductory computer science courses or textbooks or other