EigenDA Introduces LittDB: A Custom Database for Cloud Scale Data Availability

EigenDA has announced the introduction of LittDB, a specialized database designed to tackle the challenges of data availability in blockchain applications. This development comes as a response to the limitations found in traditional databases like LevelDB and BadgerDB, according to EigenCloud.

Challenges with Existing Databases

EigenDA initially relied on LevelDB, an open-source key-value store developed by Google, which has been widely used in applications such as Ethereum’s Geth. However, as the demands for higher performance increased, LevelDB’s throughput limitations and lack of back pressure mechanisms became significant hurdles. The database struggled with high write and read latencies, often falling behind under heavy loads and consuming excessive memory, leading to operational inefficiencies.

Attempts to switch to other databases, like BadgerDB and PebbleDB, revealed similar flaws. Although BadgerDB offered some improvements in speed, its garbage collection system faltered under sustained high write loads, resulting in crashes due to disk space exhaustion. These challenges highlighted the need for a database specifically optimized for EigenDA’s unique workload requirements.

Introducing LittDB

LittDB was developed in-house to address these issues, focusing on high throughput and low latency while eliminating unnecessary features like data mutability and transactions. By doing so, LittDB achieves superior performance compared to general-purpose databases. Its zero-copy architecture, where data is never moved or copied post-write, significantly enhances performance by eliminating the need for compaction, a resource-intensive process in traditional databases.

This architectural decision results in predictable performance and efficient resource utilization, making LittDB well-suited for long-term, high-volume operations. It supports high-performance writes, low-latency reads, TTL-based automatic data expiration, and can handle large data volumes efficiently.

Benchmarking Performance

Benchmark tests conducted on an AWS m5.16xlarge instance demonstrated LittDB’s superior performance. The tests showed that LittDB’s write throughput was 1500 times faster than LevelDB and three times faster than BadgerDB, with stable performance irrespective of the data volume stored. Similarly, LittDB excelled in read latency, offering significantly faster retrieval times compared to its counterparts.

Furthermore, LittDB’s efficient memory and CPU utilization were highlighted, with its design ensuring minimal resource consumption even under peak loads. These results underscore LittDB’s capability to meet the demands of scalable blockchain infrastructure without the compromises associated with general-purpose databases.

By focusing on specific requirements and eliminating unnecessary complexities, EigenDA’s LittDB sets a new standard for database performance in blockchain applications, paving the way for future innovations in cloud-scale data availability solutions.

Image source: Shutterstock