Manticore Search 7.0.0: Smarter Search, Real-Time Syncing, and Enhanced Performance

We’re thrilled to announce the release of Manticore Search 7.0.0, packed with groundbreaking features, performance optimizations, and critical improvements to elevate your search experience. From fuzzy search capabilities to seamless Kafka integration, this release empowers developers to build faster, more resilient, and user-friendly applications.

---

🚨 Important Upgrade Notes

Before diving into the new features, please review these breaking changes:

1. Per-Table Binlog (#879): Perform a clean shutdown before upgrading.
2. Replication Protocol Updates (#1789, #2308): Follow cluster restart guidelines to avoid downtime.
3. Master / agent Protocol Update (#2468): If you are running Manticore Search in a distributed environment with multiple instances, make sure to first upgrade the agents, then the masters.

For full details and more breaking changes, refer to the changelog.

---

🔍 Fuzzy Search & Autocomplete

Say goodbye to typos! The new Fuzzy Search handles spelling mistakes by matching similar terms, while Autocomplete predicts queries in real time. Perfect for e-commerce and dynamic search bars.

mysql> call autocomplete('gra', 'test');
+-----------+
| query     |
+-----------+
| gradually |
| grow      |
| grew      |
| angry     |
| draw      |
| hungry    |
| brave     |
+-----------+

mysql> SELECT * FROM mytable WHERE MATCH('someting') OPTION fuzzy=1, layouts='us,ua', distance=2;
+------+-------------+
| id   | content     |
+------+-------------+
|    1 | something   |
|    2 | some thing  |
+------+-------------+
2 rows in set (0.00 sec)

Both new features are also accessible via the JSON interface.

---

Scroll for Pagination (#2811)

Efficiently navigate large datasets with the Scroll option, ideal for deep pagination without performance penalties.

SELECT weight(), id FROM test WHERE match('hello') limit 20
OPTION scroll='eyJvcmRlcl9ieV9zdHIiOiJ3ZWlnaHQoKSBkZXNjLCBpZCBhc2MiLCJvcmRlcl9ieSI6W3siYXR0ciI6IndlaWdodCgpIiwiZGVzYyI6dHJ1ZSwidmFsdWUiOjEyODEsInR5cGUiOiJpbnQifSx7ImF0dHIiOiJpZCIsImRlc2MiOmZhbHNlLCJ2YWx1ZSI6MiwidHlwZSI6ImludCJ9XX0=';

---

⚡ Real-Time Data Syncing from Kafka

Sync data seamlessly from Kafka. Ingest streaming data into Manticore in real time, enabling instant searchability for logs, metrics, or user-generated content.

CREATE SOURCE kafka
(id bigint, term text, abbrev '$abbrev' text, GlossDef json)
type='kafka'
broker_list='kafka:9092'
topic_list='my-data'
consumer_group='manticore'
num_consumers='2'
batch=50

CREATE TABLE destination_kafka
(id bigint, name text, short_name text, received_at text, size multi);

CREATE MATERIALIZED VIEW view_table
TO destination_kafka AS
SELECT id, term as name, abbrev as short_name, UTC_TIMESTAMP() as received_at, GlossDef.size as size FROM kafka

---

📊 Performance: Secondary Indexes for JSON (#1928)

Accelerate queries that filter by JSON attributes with secondary indexes for json.

ALTER TABLE users ADD COLUMN profile JSON;
ALTER TABLE users ADD secondary_index profile_json ON (profile);

There's also a new command SHOW TABLE INDEXES to inspect indexes:

mysql> SHOW TABLE test INDEXES;
+------------------------------+--------+---------+---------+
| Name                         | Type   | Enabled | Percent |
+------------------------------+--------+---------+---------+
| j['addresses']               | uint32 | 1       | 100     |
| j['addresses']['a1']         | uint32 | 1       | 100     |
| j['addresses']['a2']         | uint32 | 1       | 100     |
| j['factor']                  | uint32 | 1       | 100     |
| j['int_arr']                 | uint32 | 1       | 100     |
| j['tags']                    | uint32 | 1       | 100     |
| id                           | int64  | 1       | 100     |
| j['price']                   | float  | 1       | 100     |
| j['addresses']['a1']['id']   | string | 1       | 100     |
| j['addresses']['a1']['name'] | string | 1       | 100     |
| j['addresses']['a2']['id']   | string | 1       | 100     |
| j['addresses']['a2']['name'] | string | 1       | 100     |
| j['arr']                     | string | 1       | 100     |
| j['str']                     | string | 1       | 100     |
| j['tags']['1']               | string | 1       | 100     |
| j['tags']['2']               | string | 1       | 100     |
+------------------------------+--------+---------+---------+
16 rows in set (0.00 sec)

---

📊 Performance: Non-Blocking Updates & Merges (#2361)

No more query delays during disk chunk merges! Updates and searches now run uninterrupted while disk chunks are optimized.

---

📊 Performance: Automatic disk chunk flush for RT tables (#2787)

Now, Manticore automatically flushes a RAM chunk to a disk chunk, preventing performance issues caused by the lack of optimizations in RAM chunks, which could sometimes lead to instability depending on chunk size.

You can control it per-table or for the whole table with the diskchunk_flush_write_timeout setting.

---

🌍 Jieba Chinese Tokenization (#931)

Achieve precise Chinese text analysis with Jieba integration.

CREATE TABLE products(title text, price float) charset_table = 'cont' morphology = 'jieba_chinese'

---

And many more

The updates highlighted above are just a part of the many improvements included in Manticore 7.0.0. Please read about:

🚀 7 major changes
✅ 49 minor changes
🐞 63 bug fixes

in the changelog.

---

🚀 Get Started Today

Upgrade to Manticore Search 7.0.0 to leverage these powerful new features. For a full list of changes, visit the changelog.

We’d love to hear from you!

• Join the discussion on our Community Forum
• Report issues or suggest features on GitHub
• Chat with us on Slack
• Email us directly at [e-mail]

Special thanks to contributors @subnix, @animetosho, Alexey Ivanov and all who helped shape this release! ❤️