v5.12.1

Core

• [fix] Fixed crash during error handling in IndexUpdate when PK is missing
• [fix] Fixed composite values validation in ALLSET operator
• [fix] Fixed composite indexes error handling in equal_position
• [fix] Fixed negative radius validation in DWithin condition for geo index

Vector indexes

• [fix] Fixed disk ANN cache for composite primary keys. Previously it could lead to crash on startup
• [fix] Fixed KNN search with radius for quantized HNSW index

Go connector

• [fix] Fixed type tags handling for empty slices

v5.12.0

Core

• [fea] Added now()-function support into WHERE-clause. Now it may be used both in UPDATE SET and WHERE clauses
• [fea] Added flat_array_len()-function into UPDATE SET. Now it may be used both in UPDATE SET and WHERE clauses
• [fea] Added checksum field into #memstats-namespaces as better alternative for datahash
• [fea] Changed grouping logic for equal_position. New syntax/logic has better match with standard json-paths and also supports nested arrays in explicit way
• [fix] Fixed possible memory leak during composite-indexes substitution inside WHERE-clauses (in cases, when int->string convertion was performed before composite substitution)
• [fix] Fixed SQL parsing for queries with combination of or inner join(...) and left join(...)
• [fix] Fixed storage data migration, when Primary key index was changed
• [fix] Fixed 2D points convertion on WHERE-clause (previously it could led to crashes on assertion)
• [fix] Added explicit check for rtree Primary keys. Geo-indexes can not be PK anymore
• [fix] Fixed forced sort errors handling for KNN-queries, when query has LIMIT and OFFSET
• [fix] Fixed UUID->string conversions for nested arrays on UUID-index deletion

Fulltext

• [fea] Added optional terms boost, that allows to set rank multiplier for specific terms

Vector indexes

• [fea] Added 8 bit scalar quantization for HNSW-index. Read more...
• [fea] Added more effective vectorized implementations for L2, IP and cosine metrics.

Replication

• [fea] Added checksum check instead of datahash - checksum implementation has lower collisions rate and higher impact from each document's field
• [fix] Fixed some rare case, when temporary namespace could remain alive after replication error

Reindexer server

• [fea] Changed FilterDef in Query DSL: some of the fields were marked as deprecated and left_expression/right_expression were as more unified alternatives for better functions support and future filtering expressions development

Go connector

• [fea] Added unified WhereExpressions method for better functions support and future filtering expressions development
• [fix] Fixed deserialization crash for queries, where inner join stays before equal_position in brackets

Face

• [fea] Added Explain visualization for queries with MERGE
• [fea] Added Boost for specific fulltext terms into fulltext config tab
• [fix] Fixed up/down buttons for custom field on pagination section
• [fix] Fixed the issue related to the page opening in a new window from left bar
• [fix] Fixed the issue related to the page opening in a new window from Namespace tabs

v5.11.1

Fulltext

• [fix] Fixed possible heap-use-after in composite fulltext indexes, created over non-indexed fields
• [fix] Fixed composite fulltext index cache invalidation after UPDATE-queries
• [fix] Fixed deleted docs handling, when selection results exceeding merge_limit and there are multiple build steps in incremental index

Vector indexes

• [fix] Fixed possible buffer overflow in transactions logic in case of multithreading insertion into HNSW

v5.11.0

Core

• [fea] Optimized indexes memory layout for namespaces with large amount of items. Index IdSet-structures now produce noticeably less overhead
• [fea] Added support for JOIN on composite-indexes (i.e. queries like SELECT * FROM ns1 INNER JOIN (SELECT * ns2) ON ns1.composite = ns2.composite)
• [fea] Added support for serial()/now() precepts with non-indexed fields
• [fea] Added more optimal preselect for JOIN-queries in cases, when right namespace is small and right query does not have filtering conditions with IdSets
• [fea] Added new EXPLAIN format for SELECT-queries with MERGE. Now it contains aggregated timing information and separate explains for each query
• [fix] Fixed serial()/now() precepts with indexed fields, when target jsonpath is missing in document
• [fix] Fixed UPDATE-queries for indexed fields, when target jsonpath is missing in document
• [fix] Fixed indexing of empty arrays after UPDATE-queries: previously those arrays won't be selected by IS NULL condition
• [fix] Fixed memory leak in composite-indexes after particular item update via UPDATE query
• [fix] Fixed UPDATE DROP for composite-index parts, when jsonpath of subindex has nested field
• [fix] Fixed UPDATE-query interaction with null-fields
• [fix] Fixed handling for duplicate sparse-indexes in DISTINCT with multiple fields
• [fix] Fixed storage data migration after Primary key index update

Fulltext

• [fea] Changed indexing structure for typos handling. New structure has noticeably less memory consumation
• [fea] Added support for ORDER BY ft_composite created over non-indexed fields
• [fix] Fixed few incorrect interactions between UPDATE-queries and text composite index with null/missing fields

Vector indexes

• [fix] Fixed situation, when some row IDs in KNN results with range search could be incorrect (due to missing internal/external index mapping)

Reindexer server

• [fix] Fixed QPS in Prometheus-metrics for SELECT-queries (after 5.9.0 it was always equal to UPDATE-queries QPS)
• [fix] Fixed columns list content in HTTP query results response (now it will contain full list of existing columns)

Face

• [fea] Removed autocomplete from index fields for create/edit index forms
• [fea] Added caching of added float vector data config
• [fea] Deleted is_appendable field from index config

v5.10.0

Core

• [fea] Added filtering by field length
• [fea] Optimized selection plan for tree sparse-indexes with is null/is not null conditions
• [fea] Support multifield sort by tree sparse-indexes
• [fea] Improved index detection logic for target fields in update-queries in cases, when jsonpath does not equal to index name
• [fix] Fixed arrays concatenation for sparse-indexes
• [fix] Fixed assertion throw for non-existing fields in forced sort
• [fix] Fixed multiple issues with collate numeric index option: null-values handling and space characters handling
• [fix] Fixed original strings content preservation for collate ascii and collate utf-8 (previously those strings could be normalized)
• [fix] Disabled invalid config with multiple jsonpaths for geo indexes
• [fix] Fixed update drop for heterogenious arrays with sparse-indexes
• [fix] Fixed array fields rollback for unsuccessful update-queries in some corner cases

Fulltext

• [fea] Added terms concatenation. Enabled by default. Check EnableTermsConcat flag and ConcatProc value
• [fix] Fixed crash on null-values with enable_preselect_before_ft: true index option

Vector indexes

• [fea] Added embed_input_traffic and output_traffic prometheus metrics for auto-embedding
• [fea] Added skip_embedding() precept for vector fields. Check embedding configuration for details
• [fix] Fixed auto-embedding statistics in #perfstats after vector index update

Replication

• [fea] Added queued_namespace_syncs field into #replicationstats. It shows current WAL/force-sync queue size for each node
• [fea] Improved namespaces sync ordering. Now replicator tries to achieve better vectors data sharing
• [fea] Extended admissible_replication_tokens functionality: now those tokens may be used on leader to protect it from role switch by other node (useful in scenarios, when follower has to become new leader)

Go connector

• [fix] Fixed panic in case of inner join with closed namespace

C++ connector

• [fea] Added support for array-fields setting via Item::operator[]

Reindexer tool

• [fea] Improved interaction with between DB dump restoration and auto-embedding: auto-embedding will be skipped for existing data

Face

• [fea] Added enable_terms_concat and concat_proc setting into text-index page
• [fix] Fixed item template on New item page for interfering nested keys

v5.9.0

Core

• [fea] Added direct support for nested arrays storing/indexing in JSON, CJSON and MsgPack (i.e. JSONs like this { "id": 7, "arr": [ 1, "string", [ 1, 2, 3], { "field": 10 }] } now may be stored into database)
• [fea] Allowed to sort null-values in hash-indexes (including null's inside arrays). Nulls-order is now consistent for different indexes/fields: null is considering less than any other value. This changes behavior for some queries with sparse tree indexes: previously nulls-order was inconsistent and had depent on the selection plan and index/field type
• [fea] Added TagsMatcher's info into #memstats
• [fea] Added fields check according to current StrictMode for joined fields inside ON-clause
• [fea] Added Distinct-support for composite-indexes
• [fix] Fixed assertion in ordered queries with Distinct over fulltext-indexes
• [fix] Fixed background index optimization in cases, when target index contains null-values
• [fix] Fixed CJSON-corruption after UPDATE-queries with non-existing array indexes

Fulltext

• [fea] Improved merging logic, when MergeLimit is exceeded. Search engine will try to find documents with maximum corresponding terms. This may be slower, but provides better quality. You may set environment variable REINDEXER_NO_2PHASE_FT_MERGE=1 to disable 2-phase merging and fallback to the old merge logic
• [fea] Supported select functions for array values in composite indexes
• [fix] Allowed to index null-fields in fulltext composite indexes

Vector indexes

• [fea] Added performance metrics for auto-embedding logic. Check indexes performance stats in #perfstats namespace for details (make sure, that perfstats are enabled in #config)
• [fix] Fixed segmentation fault in KNN-queries with radius, when target index is empty
• [fix] Disabled vector indexes update/create operations, when namespace does not have PK-index (it could led to disk storage corruption)

Go connector

• [fea] Added support for nested arrays into CJSON-coding/decoding
• [fix] Fixed Transactions with Update/Delete-queries. Now such transactions will return actual count of items, affected by the queries

Reindexer server

• [fea] Added Prometheus-metrics for auto-embedding logic
• [fix] Fixed screening in api/v1/db/:db/namespaces/:ns/meta* endpoints

Reindexer tool

• [fix] Fixed screening in \meta-calls

Face

• [fea] Added total for Memory Statistics of namespace
• [fix] Fixed issue appeared on Performance Statistics refresh
• [fix] Fixed Embedder's URL validation to allow local domains

v5.8.0

Core

• [fea] Added new EqualPosition syntax to perform grouping conditions over object arrays
• [fea] Added MERGE support for hybrid select results
• [fea] Optimized comparator for multifield Distinct (for conditions like Distinct(field1,field2,...))
• [fea] Added Distinct support for fulltext indexed (works the same way as Distinct for regular indexes)
• [fea] Optimized selection plan for empty query results
• [fix] Fixed error handling in composite-index update/delete operations
• [fix] Fixed DSN masking in #config-namespace
• [fix] Fixed token's positions in SQL parsing error descriptions
• [fix] Fixed data race in namespaces renaming

Fulltext

• [fea] Added extra strict validation for non-existing fields/indexes in fulltext dsl

Vector indexes

• [fea] Added vector's data sharing between multiple query results to reduce peak memory footprint for results, containing vectors

Replication

• [fix] Fixed possible "split mind" in RAFT-cluster

Sharding

• [fea] Added forced RAFT-leader elections request after proxying errors during #replicationstats request for more stable errors handling
• [fix] Disabled sharding by vector indexes

Reindexer tool

• [fix] Fixed interactive mode termination after error

Face

• [fea] Added new vectors keeper size field to the Statistics page

v5.7.0

Core

• [fea] Added support for sorting with array fields (i.e. ORDER BY array_field)
• [fea] Json-paths ordering for composite indexes was made consistent and now depends on initial json-paths ordering in indexes definition array
• [fea] Improved error messages in cases, when user tries to create new PK-index over the field with duplicated values
• [fea] Optimized dynamic memory allocations count in JOIN-queries
• [fix] Fixed crash during null-values handling in equal_position
• [fix] Fixed quotes handling in sort expressions

Fulltext

• [fea] Sufficiently optimized ranks merging loop for queries with large relevant results count (up to 25% performance boost according to our CPP-benchmarks)
• [fix] Fixed composite fulltext indexes update when target index has individual fields configs
• [fix] Fixed crash when indexing arrays with enable_numbers_search
• [fix] Fixed fast-path index update

Reindexer server

• [fea] Added support for transaction in Protobuf and MsgPack format (in /api/v1/db/:db/namespaces/:ns/transactions endpoint)
• [fix] Fixed crash on incorrect JSON for equal_positions and join_query fields in Query DSL parser
• [fix] Fixed response for GRPC EnumNamespaces with onlyNames-option

CXX API

• [fea] Added few more safety checks for client::Reindexer
• [fix] Fixed handling of nested json-paths in reindexer::Item::operator[] (i.e. cases like item["obj.field"] = 10)
• [ref] Method Select(std::string_view sql) was renamed to ExecSQL(std::string_view sql)
• [ref] Removed deprecated temporary flag from namespace's #memstat

Deploy

• [upd] Updated base docker image from alpine:3.21 to alpine:3.22

v5.6.0

Core

• [fea] Added subqueries and or inner join support for UPDATE and DELETE queries
• [fea] Added more informative error message in case of unsuccessful index creation
• [fea] Improved anti-join handling (excessive braces do not required anymore)
• [fea] Added more strict validation for incorrect conditions with LEFT JOINS
• [fea] Improved protobuf/msgpack content validation
• [fea] Added more strict validation for UPDATE-queries targeting non-array fields
• [fix] Fixed SQL/DSL(JSON) parsing of NOT-operator inside JOIN's ON-clause
• [fix] Fixed case-insensitive namespaces names in DSL(JSON) queries
• [fix] Fixed automatic indexes substitution for array-indexes with multiple jsonpaths and sparse-indexes
• [fix] Fixed compatibility in empty arrays JOINs
• [fix] Fixed incorrect LIMIT handling in queries with combination of array-field DISTINCT/multi-DISTINCT and forced sort
• [fix] Fixed matched field value in explain results for conditions with NOT operators

Vector indexes

• [fea] Added automatic fallback in hybrid query, when embedder is not available. This query will be executed as pure fulltext query without KNN-part
• [fix] Fixed incorrect handling of the deleted vectors by KNN-conditions with radius
• [fix] Changed embedders validation logic to avoid indexes creation error on startup

Replication

• [fea] Added proxying for UPDATE and DELETE queries with subqueries and inner joins

Reindexer tool

• [fea] Added storage convertion tool

Deploy

• [upd] Added deployment for debian:13 (trixie)
• [upd] Removed deployment for debian:11 (bookworm)

Face

• [fea] Added new fields to fulltext index config (keep_diacritics, min_word_part_size and word_part_delimiters)
• [fix] Fixed ms measure for statistics column titles

v5.5.0

Core

• [fea] Added support for INNER JOIN (as filters) in UPDATE/DELETE-queries, including queries with self-joins (UPDATE ns1 SET v=1 INNER JOIN ns1 ON ns1.idx IN ns1.allowed_ids INNER JOIN ns2 ON ns1.prices = ns2.price_id)
• [fea] Added support for mixed field types (scalars + arrays) into multifield DISTINCT
• [fea] Added support for JOINs between null-values
• [fix] Fixed multi-fields sort by composite indexes with tree-type (i.e. cases with ORDER BY tree_composite_1, other_filed)
• [fix] Fixed PK index validation: now it can't be created, if there are duplicated values in the target field

Vector indexes

• [fea] Change auto-embedding API for more embedding flexibility and future chunking support
• [fix] Fixed data-race in distance calculation in multithread HNSW index
• [fix] Fixed crash in QueryResults containing combination of multiple null/non-null vectors fields
• [fix] Fixed error handling in case of inappropriate index update

Fulltext

• [fea] Added specific handling for composite words with delimiters (e.g., resident's, that may be splitted into resident and s, or Biot–Savart, that may be splitted into Biot and Savart). Check WordPartDelimiters and MinWordPartSize config fields. Default value of the ExtraWordSymbols was also changed due to this feature
• [fix] Fixed phrase search with composite fulltext indexes

Replication

• [fix] Fixed logical race in async replication role switch of the target follower (may lead to unnecessary resync)
• [fix] Added check for the replication role in DropNamespace-call (now follower's namespaces can't be deleted by user, if replication is active)

Go connector

• [fea] Change interface of NewReindex()-call. Now it returns current status in error to force user to check DB's status
• [fix] Changed SetDefaultQueryDebug() for better corner cases handling

Reindexer server

• [fea] Support multifield DISTINCT in proto-schemas
• [fix] Fixed integer types support in Protobuf interface. This breaks compatibility with old protobuf clients and requires repeated client generation for the new schema

Reindexer tool

• [fea] Added -n/--namespaces options to specify namespaces list, that will be restored from the dump file
• [fix] Fixed erros output during dump restoration process

Face

• [fea] Added validation of URL field in Auto-embedding Config form
• [fea] Added blocking of is_no_column field for the composite field type
• [fea] Added measures conversion for "String waiting to be deleted size" field
• [fea] Removed default value for radius field in vector indexes config
• [fix] Fixed error that appeared on statistics reset