Talos Data Storage Model

Data Storage Model

The organization system for Talos's data storage is as shown in this diagram.

• Logical Structure: As shown in the upper left corner of the diagram, data exists in the form of a Topic, which is logically divided into several partitions. The Topic in the diagram has eight partitions, with partition IDs of [0, 7].

• Physical Structure: Each partition on the hard drive is organized into several files. In the diagram, Partition 7 is organized into three files: File 1, File 2, and File 3.

• Message Organization Structure: When the user writes a Talos message in a file, it is added to the file through the append method. In the diagram, Partition 7, File 3 is organized message by message, each message includes a header and content. After the 1000th message is put into Partition 7, it is then appended into File 3.

• Offset Meaning: Offset indicates what number a message is within the entire partition. In the diagram, the messages stored in File 3 have offsets ranging from [700, 1000].