Millions of Transactions per Second on a Single Machine: A Case for a Virtualized Database and Scale-In
Random-access memory is getting cheaper: one server hosting 512 GB of RAM is a reality. A new class of devices, such as non-volatile, byte-addressable DIMM chips (NVM DIMM, e.g., Intel 3D-XPoint™), expand a single machine to tens of terabytes of memory with near-DRAM latencies.
Would adding more RAM and replacing an existing database engine with an in-memory one run faster? Yes, it would, but not by that much: In most cases, processing speeds can be doubled. However, they are not reduced from the anticipated hours to minutes, or less.
In this talk, we’ll introduce a new type of software architecture where an application server and a database server share memory. Thus, data objects are stored only once, and overhead communication between the two servers is eliminated. Effectively, an application’s memory heap is altered with durable software transaction memory, while the physical database the application accesses is virtualized to match its conceptual data model.
We will illustrate how this can:
- Lift existing applications to new performance levels
- Leverage the new hardware for storage-class memory
- Make an existing application server function: specifically, how it is different from stored procedures, affinity collocation and language runtimes hosted in the database
- Justify scale-in and explore the pros and cons: understand how this architecture scales.
