Space and Time HTAP Data Warehouse
What it is and why we built it
HTAP stands for hybrid transactional/analytic processing, and refers to a database that can process both online transactions (OLTP) and online analytics (OLAP) in a single cluster. Traditionally, an app developer has to deploy a transactional database to power their application, a data warehouse to run analytics, and ETL tools to move data between them. At Space and Time, we built the first decentralized HTAP data warehouse to provide devs with the tools they need to power apps, APIs, analytics, data science, and ML/AI in a single decentralized environment.
How it works
The data warehouse is the backbone of the Space and Time network. It’s a decentralized, Web3-native HTAP engine that enables trustlessness, scalability, and blazing fast performance for any data workload.
The Space and Time data warehouse is composed of multiple clusters operated in a permissionless manner by a network of node operators. The various data warehouse clusters across the Space and Time data fabric are the workhorses of the Space and Time system. They are responsible for performing the five major operations of data:
- Data Ingestion - saving data from external sources
- Data Transport - warehouse to warehouse data transfer
- Data Storage - persistent save of data with view to any point in time
- Data Transformation - data cleaning, aggregations, multi-source data joins
- Data Serving- easy and performant access to data, intelligent caching, creation of data APIs
In order to perform all of these operations within the context of a single warehouse node, we need HTAP. HTAP data stores are in vogue, and for good reason. They promise, with good workload management, a system which can adapt to any workload that they're tasked with processing. This is exactly what Space and Time requires as an end-to-end data platform. Warehouse nodes must be generic enough to serve high-throughput ingestion one minute, and then in the next aggregate over the terabytes just ingested. For this, you need an HTAP system.
The Space and Time Validator layer is responsible for telling the data warehouse whether a query is transactional or analytic. The user executes queries through this abstraction as if connected to a single, infinitely-available cluster. Behind the scenes, each request is routed to the appropriate engine (OLTP or OLAP). The complexity of HTAP is completely abstracted away from the user experience. All you have to do is run a query, and we handle the rest.
What it enables
Scalable OLTP faster than PostgreSQL
OLTP is SxT's ability to deliver low-latency queries that power realtime applications and APIs. Developers need to perform extremely fast lookups to power their applications. Space and Time is multi-node, decentralized, and scales easily past 1 terabyte, providing row-based OLTP in-memory cache at cluster scale that with extreme throughput.
Realtime OLAP cheaper than Snowflake
OLAP is SxT’s ability to handle complex queries that power insights, dashboards, ML, and data science. Developers need an OLAP system that has access to fresh data. Data analysts, dashboard developers, and realtime dapps need online analytics. Space and Time provides column-based GPU-accelerated OLAP with elastic scale-out that’s not only realtime, but also fresh.
