Secondary index

SDS secondary index supports local and global secondary indexes. Currently, both indexes are strongly consistent.

Local secondary index

Entity group key must be defined when using local secondary index, which consists of one or more table attributes. The index is divided into three categories: Lazy, Eager, and Immutable.

• Lazy index - Write index record as well while writing time, but it does not immediately delete an invalid index record. An invalid index record will be deleted at time of reading it. Writing efficiency is high, while reading efficiency is lower. Therefore, this type of indexing is great for situations that ask for more writing, less reading. It does not support projection attributes or unique indexes.

• Eager index - Write index record as well while writing time. It also immediately deletes an invalid index record, to ensure all indexes read are valid. Writing efficiency is low, while reading efficiency is high. Therefore, this type of index is great for situations that ask for less writing, more reading. It supports Unique Index. It can also define a set of attributes as a projection, and is stored together with the index (It can be taken as a copy of the corresponding attribute in the main record; Attribute values remain strongly consistent).

• Immutable index - This type of index is suitable for non-overwritten data. Data will not be modified once written (user's own guarantee is required). Since there is no overwriting, there will not be invalid index records. Both writing and reading efficiencies are relatively high. This is highly recommended. This type of index supports projection but does not support unique index.

Index definition LocalSecondaryIndexSpec

• indexSchema: List< KeySpec > Specify which attributes are to be indexed, and which encoding method is to be used when the key is recorded as an index (asc/desc)

• consistencyMode: SecondaryIndexConsistencyMode defines the index type, LAZY, EAGER, or IMMUTABLE

• Projections: List. This is a list of property names that defines which properties of the master record are included in the index record. This can only be set if the index type is EAGER or IMMUTABLE

• Unique: Boolean determines whether it is a unique index. Only if index type is EAGER can this be set as true

Quota calculation for read-write indexes

For details of additional quota calculations generated by reading and writing different types of indexes, see Quota description

Add index

A local secondary index can be added when creating a table. You can also add a local secondary index to an existing table. Only EAGER type, non-unique index can be added to an existing table, which requires re-indexing by using Tools

Limitation

To use local secondary index, it must define entity group key. Index record can only index the records of the same entity group key, so startKey and stopKey must be within the same entity group key when using scan to read data. That means, all the entity group keys are specified, and their key attribute values are equal.

Global secondary index

Table that establishes the global secondary index may not define the entity group key. The index is considered global due to that the index can query data of the entire table. Index data and original data of the global secondary index physically exist in two different tables. The entity group key, primary key, quota, pre-fragment, etc. of the index table can be independently defined.

Index definition Globalsecondaryindexspec

• indexEntityGroup: EntityGroupSpec (optional) is part of index's key property. This part can hash the index table to avoid accessing hotspots. For setting the entity group key to enable hash, you need to avoid scanning as cross-entity group key in the index table.

• indexPrimaryKey: List< KeySpec > (optional) is part of index's key property. This cannot be left empty while indexEntityGroup is empty. The uncovered rowkey property in primary table will be added in proper order to the primary key of the global index table.

• Projections: List (optional) is a list of property names that define which of the properties of the master record are included in the index record. If it is empty, the index table will only contain indexEntityGroup, indexPrimaryKey, and key properties of the original table.

• consistencyLevel: ConsistencyLevel (required). Data consistency level for index table and original table is of STRONG/EVENTUAL/WEAK; the default is STRONG. Currently, only STRONG is supported.

• Unique: Boolean: (required). Determine if it is a unique index; default is set as non-unique index. Currently, only non-unique index is supported.

• Throughput: ProvisionThroughput (required). Read/write quotas for index table in primary cluster.

• slaveThroughput: ProvisionThroughput (optional). Read/write quotas of the index table in the standby cluster. If the quotas are not set, then they are equal to the read/write quotas of master cluster.

• exceededThroughput: ProvisionThroughput (optional). Maximum number of read/write quotas the index table can issue for primary cluster. If not set, they will be defaulted to 1.5 times the read/write quotas of primary cluster.

• exceededSlaveThroughput: ProvisionThroughput (optional). Maximum number of read/write quotas the index table can over-send to standby cluster. If not set, the default is 1.5 times the read/write quota of the standby cluster.

• preSplits: int: An index table that only supports hash distribution for entity group. The range is [ 1, 256 ].

Quota calculation for read-write indexes

For put/delete operations to a table with global secondary index, it needs to consume the original table write quota for the amount of data written in the unit + 1 additional capacity unit. If this involves updating the index table, it will also need to consume the original table’s capacity unit for reading old records. When updating the index, it needs to consume the capacity unit of index table data + 1 additional capacity unit.

Scan/get is computed based on actual capacity units occupied by read data. If it is query on index data during scan, it consumes the quota of index table. If it is query on original table, it consumes the quota of original table.

Limitation

A table cannot have local secondary index and global secondary index at the same time. Stream cannot be used for a table with global secondary index. Query on global secondary index can only retrieve projection attributes.

Difference

Consider the requirements of the application when determining which type of index to use. The following table shows the major differences between local secondary index and global secondary index: