#storage #indexing #query #database

shortcut

an indexed, queryable column-based storage system

15 stable releases (3 major)

4.1.3 Dec 4, 2018
4.1.2 Jan 30, 2017
4.1.0 Dec 1, 2016
4.0.0 Nov 28, 2016
1.1.0 Jun 14, 2016

#13 in Database implementations

Download history 14/week @ 2018-10-23 84/week @ 2018-11-06 31/week @ 2018-11-13 28/week @ 2018-11-20 29/week @ 2018-11-27 125/week @ 2018-12-04 21/week @ 2018-12-11 16/week @ 2018-12-18 177/week @ 2018-12-25 1/week @ 2019-01-01 20/week @ 2019-01-08 3/week @ 2019-01-15

178 downloads per month

MIT/Apache

33KB
639 lines

shortcut

Crates.io Documentation Build Status

This create provides an indexed, queryable column-based storage system.

The storage system is, fundamentally, row-based storage, where all rows have the same number of columns. All columns are the same "type", but given that they can be enum types, you can effectively use differently typed values. Data is stored in a BTreeMap<usize, Vec<T>>, where the outermost BTreeMap is dynamically sized (and may be re-allocated as more rows come in), whereas the innermost Vec is expected to never change. The map index is an autoincremented row identifier similar to the one used by SQLite: https://www.sqlite.org/lang_createtable.html#rowid.

What makes this crate interesting is that it also allows you to place indices on columns for fast lookups. These indices are automatically updates whenever the dataset changes, so that queries continue to return correct results. Indices should conform to either the EqualityIndex trait or the RangeIndex trait. As you would expect, the former allows speeding up exact lookups, whereas the latter can also perform efficient range queries.

Queries are performed over the dataset by calling find with a set of Conditions that will be ANDed together. OR is currently not supported --- issue multiple quieries instead. Each Condition represents a value comparison against the value in a single column. The system automatically picks what index to use to satisfy the query, using a heuristic based on the expected number of rows returned for that column for each index.

Known limitations

  • The set of match operations is currently fairly limited.
  • The system currently provides an add/remove-only abstraction (i.e., no edit).

No runtime deps