Real-time tables and table versions

Real-time tables in Beneath have some unique properties that you can learn more about below.

Streams are replicated in a streaming log, a data warehouse and an operational data store

Records written to Beneath are automatically stored in a streaming log, a data warehouse and an operational data store. That means you can consume and query tables in the way that makes the most sense for your use case, including streaming changes, running OLAP queries, and running fast filter queries. You can learn a lot more about this feature in Unified data system.

Tables have a schema and a unique key

All tables have an associated schema that defines the fields of its records. The schema must also define a key comprised of one or more columns (also known as a primary key or unique key). Beneath rejects records that do not adhere to the table’s schema. See Schema definition for more details.

Tables can have multiple versions, known as instances

Records are not stored directly in a table, but rather in a table instance, which a table can have many of. Each instance has an associated version number, which is a positive integer.

Every table instance has two boolean flags: a) It can be marked as finalized, which indicates that no more records will be written to it, and b) it can be marked as primary, which makes it the default instance for the table.

The following list gives some examples that clarify the usefulness of table instances:

• A table that represents unbounded data (such as trades on a stock exchange) will have one unfinalized table instance that grows forever.
• A table that represents batch-updated data (such as the results of an SQL query that runs once every day) will have a new table instance for every batch. When a new batch has been completely written, the table instance is marked as finalized. Old table instances for previous batches are deleted.
• A table that represents static data (such as reference data that never changes) will have one finalized table instance that never changes.

All records have an associated timestamp

Every record in a table instance has an associated timestamp. When you write a record, you can specify a custom timestamp or use the current timestamp of the Beneath server (the default). The timestamp comes in handy when several records with the same key are written to a table instance. In these cases, the record with the highest timestamp is added in the operational data index.