Update dependency sentence-transformers to v6 - #2170
renovate[bot] wants to merge 1 commit into
Conversation
|
Warning Your comment is too long (maximum is 65536 characters), so the coverage report was not added. See the job log for how to reduce it.
|
1f3af67 to
d271de6
Compare
17aa0bd to
a9f10af
Compare
617f2a6 to
7ebe8c7
Compare
003fbb2 to
c33d363
Compare
🤖 Autofix self-reviewVerdict: fix is sound, all invariants hold. The Root causeThe failing check was
assert abs((created - latest).total_seconds()) < 1
This is not the dependency update
Run duration varies 20–24 min across runs of identical code, i.e. runner noise, not a regression FixBoth tests now pin the two aggregate timestamps to the version rows they are derived from, assert created == version_created_at[1]
assert latest == version_created_at[2]Two files, one hunk each — Invariant 2: the check got stronger, not weakerNet +3 / −1 assertions, and the replacement is strictly more discriminating. Mutation-tested: — a 10µs error that the old Verification
Unit tests run against SQLite via a dependency override, and the CI job declares no Invariants
Caveat for reviewersThis makes two flaky assertions deterministic; it does not make the suite immune to runner 🤖 Generated with Claude Code |
|
Understand this PR’s impact Explore downstream dependencies and potential security impact with Blast Radius. Important Review skippedReview was skipped as selected files did not have any reviewable changes. ⛔ Files ignored due to path filters (1)
⚙️ Run configuration
⛔ Files ignored due to path filters (1)
You can disable this status message by setting the Use the checkbox below for a quick retry:
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Advanced Run ID: ⛔ Files ignored due to path filters (1)
📒 Files selected for processing (1)
Included review availability: Your plan provides up to 10 included reviews per hour; 4 remain after this review. 📝 WalkthroughWalkthroughThe pull request updates the pinned ChangesDependency Update
Priority: ⬇️ Low Estimated code review effort: 1 (Trivial) | ~2 minutes Change: Other 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
This PR contains the following updates:
==5.7.0→==6.1.0Release Notes
huggingface/sentence-transformers (sentence-transformers)
v6.1.0: - Documentation and input handling improvementsCompare Source
This is a small release with refreshed inference benchmarks, clearer documentation, and a few improvements to multimodal input handling.
Install this version with
Changes
For existing multimodal dict inputs, message content now follows each input's key order, which can change embeddings or scores. Video-frame URLs without recognizable image extensions need an explicit frame wrapper to remain one video. See #4037 for details.
All Changes
docs] Document that optional imports in a custom module become load-time requirements by @abtonmoy in #3922Full Changelog: huggingface/sentence-transformers@v6.0.1...v6.1.0
v6.0.1: - Restore the PyLate prefix on prompted checkpoints, 80 documented multi-vector modelsCompare Source
This patch release fixes a multi-vector loading bug: PyLate checkpoints that carry both a
[Q]/[D]prefix and a text prompt lost the prefix, so they were encoded without a marker they were trained with. It also grows the documented multi-vector model tables from 51 checkpoints to 80.Install this version with
Compose the PyLate prefix with the saved prompt (#3968)
PyLate supports a prefix as well as prompt text, and Sentence Transformers assumed that these were mutually exclusive. However, the following three checkpoints carry both a prefix and a prompt, and were trained with the prefix prepended to the prompt text:
lightonai/ColBERT-Zerolightonai/ColBERT-Zero-unsupervisedlightonai/ColBERT-Zero-supervisedPyLate trained these checkpoints on
[CLS] [Q] search_query: ...and[CLS] [D] search_document: ..., but Sentence Transformers dropped the prefix and encoded them as[CLS] search_query: ...and[CLS] search_document: .... After the fix, the performance of theColBERT-Zeromodel improved from 0.6569 NDCG@10 to 0.6824 on the NanoBEIR benchmark. To my knowledge, only these 3 checkpoints used both a prefix and a prompt, so this is the only case where the bug would have affected you.Forward
taskto routed modules inRouter.preprocess(#3967)Router.forwardhas always passedtaskdown to the routed module, butRouter.preprocessdid not, so anything a module does with the task at preprocessing time did nothing behind aRouter:query_lengthanddocument_lengthcaps,query_expansion, and the chat-templatetaskkeyword. No released checkpoint combines aRouterwith those settings, so this is a latent bug rather than one you are likely to have hit. It would have affected anyone building such a model themselves, with no error to indicate it.Documentation
revisionandtrust_remote_codenotes refreshed as upstream pull requests merged (#3972).transformersv4.41.0+, where v6.0 requires PyTorch 2.2+ andtransformersv5.0+. This is the text rendered as the PyPI project description.All Changes
taskto routed modules inRouter.preprocessby @tomaarsen in #3967Full Changelog: huggingface/sentence-transformers@v6.0.0...v6.0.1
v6.0.0: - MultiVectorEncoder for ColBERT & late interaction models, transformers v5, float32 scoring, faster training & encodingCompare Source
This major release introduces Multi-Vector Embedding models, also known as late interaction or ColBERT-style models, as a fourth model type alongside
SentenceTransformer,CrossEncoder, andSparseEncoder. Going forward, you'll be able to use Sentence Transformers for training, inferencing, and interpreting Multi-Vector Embedding models.It also modernizes the dependency floors to
transformersv5, fixes a class of silent scoring bugs caused by half precision, and speeds up both training and encoding.Install this version with
MultiVectorEncoder: ColBERT-style late interaction models (#3794)
Sentence Transformers v6.0 introduces
MultiVectorEncoder, for ColBERT-style late interaction retrieval. Where a regular embedding model compresses a whole text into one vector, a multi-vector model keeps one vector per token and scores query against document with the MaxSim operator. That preserves token-level matching information that a single vector has to average away, which usually means stronger retrieval at the cost of a bigger index. It is also the state of the art for visual document retrieval, where a text query is matched against page images directly, with no OCR step in between.Any PyLate checkpoint and any Stanford-NLP ColBERT checkpoint loads straight into it, and colpali-engine models for visual document retrieval work too, through the same familiar API you already use for dense, sparse, and reranker models.
Mars wins, as it should, though notice how close the four scores are. That is normal for MaxSim: the scores often look similar, but the ranking is still exact. The blogpost explores this in more detail.
Note what you get back: a list of 2D tensors on the model device, one per input, each of shape
(num_tokens, embedding_dim). Unlike dense embeddings, you cannot stack these into one rectangular tensor, because every input has its own token count. Passconvert_to_numpy=Truefor a list of numpy arrays instead, which is what you want once a corpus outgrows device memory.Multi-vector models are also asymmetric: queries and documents go through different prefixes, different length caps, and different scoring masks. Unlike many dense models, where the two are interchangeable,
encode_queryandencode_documentare required to get correct embeddings.The MaxSim operator
Scoring uses MaxSim: for each query token, take its highest similarity against any document token, then sum those maxima across the query.
You can read the operator as a soft alignment: every query token points at the one document token that best explains it, and the score is how well the document explains the query overall. The alignment does not have to be lexical, since the token embeddings are contextualized. But when an exact match does matter to you (a product code, a surname, a function name), MaxSim has a token sitting right there to match it, where a single-vector model had to fold it into an average.
Because MaxSim sums over query tokens, its magnitude scales with the query token count, so scores are not comparable across models with different query recipes. If you want scores on a bounded scale, use
similarity_fn_name="meanmaxsim", which divides by the query token count and gives you an average cosine similarity in[-1, 1].Scoring builds a 4-dimensional intermediate of every query token against every document token, which is the largest tensor in the operation. Every scoring function takes a
chunk_elementsbudget that bounds it, defaulting to 100 million elements (roughly 400 MB in float32), so lower it if you run out of memory. Scores and gradients are bit-identical whatever you set it to.maxsimandmaxsim_pairwisealso take adevice, which scores one chunk at a time on that device and moves each result straight back, letting you score a corpus larger than your VRAM on the GPU. Both are reachable throughsimilarity, which forwards any extra keyword arguments to the scoring function:When training, pass the budget to the loss instead, with
similarity_fct=partial(colbert_scores, chunk_elements=1_000_000). It chunks the document axis, so it composes with the loss-levelscore_mini_batch_size, which chunks the query axis.Are they any good?
lightonai/LateOnandlightonai/DenseOnwere trained by LightOn on the same data with the same ModernBERT backbone and the same 149M parameters, differing only in whether they keep one vector per token or pool down to one per document. Running both over all 13 NanoBEIR datasets isolates what that choice buys:Late interaction wins on 9 of the 13 datasets and on the mean, by roughly one NDCG point. The four it loses (ArguAna, FiQA2018, SCIDOCS, and SciFact) are the shape of the tradeoff you should expect: a real gain in retrieval quality at the same model size, paid for in index footprint, rather than a universal win on every dataset. The same pair scores 57.22 against 56.20 on the full 15-dataset BEIR, a comparable gap, so the margin is not an artifact of the small benchmark.
That footprint is the real cost. One vector per token instead of one vector per document is a lot more vectors, only partly offset by the smaller dimension. Encoding 4,874 Natural Questions passages with
lightonai/LateOnproduced 608,414 token vectors, an average of 124.8 per passage:all-MiniLM-L6-v2gte-modernbert-baseLateOnThat is about 42x the storage of the MiniLM index. Token Pooling cuts the vector count before any of that, real late interaction indexes compress heavily (the same vectors take 88 MB as a fast-plaid PLAID index), and using a multi-vector model as a reranker over a dense first stage avoids building an index at all.
Every checkpoint format loads
Multi-vector checkpoints have been published in several formats over the years.
MultiVectorEncoderreads all of them, so loading looks the same whatever the model started life as:The recipe knobs that differ per checkpoint (marker prefixes for queries and documents, length caps, whether queries are padded out with
[MASK]tokens, and which tokens are skipped when scoring documents) all live in the module configs, soprint(model)shows you exactly what you loaded:Following the design principle of the rest of the library, this behavior lives in swappable modules rather than in the model class: a
Transformerproducing contextualized token embeddings, a token-levelDenseprojecting each of them down, aMultiVectorMaskdeciding which tokens count during scoring, and a token-levelNormalize.Supported models
These are the checkpoints we test against directly, ranked by retrieval quality. The
sentence-transformerstag on the Hub is the list that stays current, and for text retrieval in particular, any PyLate or Stanford-NLP ColBERT checkpoint loads whether or not it carries the tag yet. Where arevisionis listed, pass it until the pull request on that repository is merged.Text retrieval (29 models). NanoBEIR is the mean NDCG@10 over the 13 NanoBEIR datasets, a fast proxy for English text retrieval quality. A
-means the model was not evaluated on it, which is the case for the non-English models.trust_remote_code=Truetrust_remote_code=TrueVisual document retrieval (22 models). These embed page images as documents and text as queries. NanoViDoRe is the equivalent proxy over the ViDoRe benchmark subsamples.
trust_remote_code=Truetrust_remote_code=Truetrust_remote_code=Truetrust_remote_code=TrueNote that NanoBEIR and NanoViDoRe are small benchmarks, so their scores are not a substitute for evaluating on your own data, which is always the right way to pick a model.
Visual, audio, and video document retrieval
Late interaction is the state of the art for visual document retrieval: matching a text query against page images, with charts, tables, and layout intact, and no OCR step. This is what the ColPali family of models does, and those checkpoints run through the same API. Image documents are passed as URLs, local paths, or PIL images:
The code is unchanged from the text case. Underneath, the processor handles the visual prompt and the image patches, and MaxSim scores query text tokens against document image patches. Page images are not the only non-text modality either: text, images, audio, and video are all accepted, and a checkpoint supports whichever of those its processor does, which
model.modalitiesreports.Because MaxSim is a sum of per-query-token maxima, a ranking decomposes exactly: every point of a document's score belongs to one query token and one document token. The new
sentence_transformers.multi_vector_encoder.interpretabilitymodule overlays that decomposition onto the page as the standard ColPali heatmap, either aggregated over the query or one map per query token.Token pooling
If the index footprint worries you, the most effective knob is to store fewer token vectors.
HierarchicalTokenPoolingimplements the token pooling technique from Clavié, Chaffin, and Adams: it clusters each document's token vectors with Ward linkage on cosine similarity and replaces each cluster with its mean, keeping roughly1 / pool_factorof the tokens.By default, pooling applies to documents only, since queries are short and are the side you cannot afford to distort. On the Natural Questions corpus above, the reduction tracks
pool_factorclosely:pool_factorThe original experiments measured the retrieval cost of this on BEIR and found very little of it: 100.6% of the unpooled performance on average at
pool_factor=2, and 99.0% atpool_factor=3. How much it costs on your data is corpus-specific, so measure it with an evaluator before you settle on a factor.Update Stats
Introducing
MultiVectorEncoderhas been one of the largest updates to Sentence Transformers, introducing all of the following:MultiVectorMask,BaseTokenPooling,HierarchicalTokenPooling, andLambdaTokenPoolingmaxsim,maxsim_pairwise,mean_maxsim,mean_maxsim_pairwise) plus 6 named ColBERT scorers and 5 XTR scoring entry pointstorch.compileResources
🚨 transformers v5, torch 2.2, and new dependency floors (#3794)
Sentence Transformers v6.0 requires
transformersv5. The v4.x compatibility branches have been removed, which is what allows the new modality handling, chat template support, and unpadding paths to be relied upon rather than feature-detected. The floors that moved:transformers>=4.41.0,<6.0.0>=5.0.0,<6.0.0huggingface-hub>=0.23.0>=1.3.0,<2.0.0torch>=1.11.0>=2.2numpy>=1.20.0>=1.24.0scikit-learn>=0.22.0>=1.1.0typing_extensions>=4.5.0>=4.10.0datasets(train)>=2.0.0>=2.16.0accelerate(train)>=0.20.3>=1.3.0optimum-intel[openvino]>=2.0.0requires-pythonis unchanged at>=3.10. Note that multi-GPU training with streaming (IterableDataset) datasets needsaccelerate>=1.13.0in practice.🚨 Higher-precision scoring (#3892, #3893, #3924, #3926)
Half precision ties too many scores together to rank with. Three separate places where that mattered are now computed in float32.
Reranker scores are the big one.
CrossEncoder.predict(andrank) now upcast the logits to float32 before applying the activation function. A sigmoid in bfloat16 saturates and collapses the top candidates onto a handful of tied values, which randomizes their order. Measured oncross-encoder/ettin-reranker-32m-v1in bfloat16 over three NanoBEIR datasets with 100 candidates per query:NanoMSMARCO NDCG@10 alone goes from 0.0965 to 0.7093. If you run a half precision reranker with the default sigmoid activation, its ranking was essentially randomized before this release. Models using
activation_fn=nn.Identity()(raw logits) were unaffected, as bf16 logits keep enough relative spacing.Similarity scores from
model.similarity/similarity_pairwiseand thecos_simfamily are now computed in float32 for float16 and bfloat16 embeddings. With 10,000 realistic cosine scores (mean 0.7, standard deviation 0.05), float32 keeps 9,983 distinct values where float16 keeps 593 and bfloat16 keeps just 93. bfloat16 can represent only 129 distinct values in the whole of[0.5, 1.0).MaxSim sums over query tokens, reaching magnitudes where the bfloat16 grid is 0.125 wide, so
maxsimandmaxsim_pairwiseaccumulate the per-token maxima in float32 and always return float32 scores. The 4-dimensional scoring intermediate stays in the input dtype, so this does not change peak memory.Note that
encode()output dtypes are unchanged. Only the scoring step is upcast. ForCrossEncoder.predict, the returned dtype changes only withconvert_to_tensor=Trueorconvert_to_numpy=False, as the default numpy output was already float32.Separately, the multi-vector bf16 benchmarks were re-measured under this float32 accumulation (#3924). Most of the previously reported bf16 quality drop came from the scoring accumulation rather than from the embeddings: plain bf16 now sits at 99.0% of fp32 retrieval quality (was 95.0%), and bf16 with FlashAttention-2 is indistinguishable from fp32 at 99.96% (was 97.9%).
🚨 Other breaking changes (#3794, #3927, #3935)
similarityandsimilarity_pairwiseare methods, not properties. Calls likemodel.similarity(embeddings1, embeddings2)work unchanged, but assigning a custom function tomodel.similarityis no longer supported: it now silently shadows the method where it previously raised anAttributeError. Setmodel.similarity_fn_name = "dot"instead, which updates both. Note also thatmodel.similarity.__name__is now"similarity"rather than the resolved function name, which affected lossget_config_dict()output and generated model cards. The newsentence_transformers.util.similarity_fct_name()resolves it properly and the losses use it.model.encode([{"role": "user", ...}, {"role": "assistant", ...}])produces one embedding, where v5.x read it as a batch of two inputs. Wrap each conversation in its own list to encode a batch:model.encode([[msg1], [msg2]]). This applies toSentenceTransformer,SparseEncoder, andMultiVectorEncoder.CrossEncoderis unaffected.trust_remote_code=True(#3935). Loading a model whosemodules.jsonreferences a class outsidesentence_transformersexecutes third-party code, and a local directory no longer implies trust. This closes the bypass reported in #3801 and completes the deprecation cycle announced in v5.6 and v5.7. Unmet, it raises aValueErrornaming the class and pointing at the repository or local path to inspect. Trainer checkpoint reloading (load_best_model_at_end,resume_from_checkpoint) keeps working for programmatically built models without the flag.quantize_embeddingsreturns a list of per-input matrices when given a list of 2D arrays, where it previously stacked them into one 3D array. Update callers that indexed the stacked array. An empty list now returns[]instead of raising, and a(0, dim)matrix returns a correctly shaped empty result.encode(pool=..., precision="int8")now quantizes once after merging the worker results, so the calibration ranges match single-process encoding. Quantized indexes built with v5.x multi-process encoding are not bit-compatible and should be regenerated. Peak memory is higher, because the full float32 matrix is materialized before quantization.CrossEncoder.rankreturns Python floats (#3927) as its"score"values, where it previously returnednumpy.float32scalars or 0-dimensional tensors. The results are directly JSON serializable, matchingsemantic_search.convert_to_numpyandconvert_to_tensoronrankare now deprecated no-ops: callpredictdirectly if you want an array or a tensor. Beyond the cleaner output, this avoids a device synchronization per comparison when sorting, which took 212ms for 1000 CUDA scalars against 0.089ms for Python floats.Normalizemoved tosentence_transformers.base.modules. Existing models load fine and silently, but a model saved by v6.0 with aNormalizemodule cannot be loaded by Sentence Transformers older than v6.0.SentenceTransformercheckpoint as aCrossEncoder(or any other such conversion) no longer picks up the source'sprompts,default_prompt_name,similarity_fn_name,truncate_dim, oractivation_fn, as those describe a model you are not loading. A reranker's default prompt being prepended to everyencodecall was the motivating case. Explicit keyword arguments still win. These conversions are now also logged at warning level, so they are visible at default verbosity.SimilarityFunction.possible_values()now includes"maxsim"and"meanmaxsim". Setting an unsupportedsimilarity_fn_nameonSentenceTransformerorSparseEncoderraises immediately rather than failing later, and a newSUPPORTED_SIMILARITY_FN_NAMESclass attribute documents what each model type accepts.Faster training and encoding (#3938, #3794)
Multi-column losses now run one forward pass over merged columns (#3938). A training batch arrives as one feature dict per column (anchor, positive, negative_1, and so on), and the classic pattern runs the model once per column. The
SentenceTransformerandSparseEncoderlosses now pad and concatenate the like-width candidate columns into a single batch, keeping the anchor on its own forward pass since a 12-token query padded into 256-token documents costs more than it saves:Loss trajectories match, up to dropout sampling. Losses fall back to per-column forward passes whenever the columns cannot be merged safely, for example with differing feature keys, disagreeing prompts or router tasks, or flattened Flash Attention inputs. The cached losses keep using GradCache, and
AdaptiveLayerLossopts out.Backend benchmarks were re-measured for all four model types, with new Flash Attention columns and rewritten recommendations. For
SentenceTransformer, float16 with Flash Attention and unpadding is now the fastest GPU configuration at 3.87x over float32, and ONNX on GPU is no longer recommended for short texts as float16 now beats it. ForCrossEncoder, Flash Attention is explicitly not recommended, as unpadding does not apply to classification heads. ForSparseEncoder, plain float16 remains the recommendation even though FA2 unpadding is now supported. See Speeding up Inference for the flowcharts.Models can declare their dependency versions (#3934)
Model authors can now record which package versions their checkpoint needs, and loading verifies them up front instead of failing in a confusing way later. Add a
requirementsmapping toconfig_sentence_transformers.json, using PEP 440 specifiers:{ "model_type": "SentenceTransformer", "requirements": { "transformers": ">=5.15", "peft": { "specifier": ">=0.18,<0.20", "reason": "Older versions ignore the key_mapping, which silently randomizes the adapter weights." } } }Loading that model in an environment that does not satisfy it raises an
ImportErrorlisting every unmet requirement at once, with the optionalreasonincluded and a ready-to-run install command: