林玉山 (Yu-Shan Lin)
一位專長開發分散式及資料庫系統的軟體工程師與資訊工程博士
目前正在作為消防替代役服一年兵役中
sam123456777 [AT] gmail.com
yslin.eng [AT] gmail.com
2021 年 5 月 19 日 攝於 台灣新竹市國立清華大學
一位專長開發分散式及資料庫系統的軟體工程師與資訊工程博士
目前正在作為消防替代役服一年兵役中
sam123456777 [AT] gmail.com
yslin.eng [AT] gmail.com
2021 年 5 月 19 日 攝於 台灣新竹市國立清華大學
Shan-Hung Wu, Tsai-Yu Feng, Meng-Kai Liao, Shao-Kan Pi, and Yu-Shan Lin
Deterministic database systems, a type of NewSQL database systems, have been shown to yield high throughput on a cluster of commodity machines while ensuring the strong consistency between replicas, provided that the data can be well-partitioned on these machines. However, data partitioning can be suboptimal for many reasons in real-world applications. In this paper, we present T-Part, a transaction execution engine that partitions transactions in a deterministic database system to deal with the unforeseeable workloads or workloads whose data are hard to partition. By modeling the dependency between transactions as a T-graph and continuously partitioning that graph, T-Part allows each transaction to know which later transactions on other machines will read its writes so that it can push forward the writes to those later transactions immediately after committing. This forward-pushing reduces the chance that the later transactions stall due to the unavailability of remote data. We implement a prototype for T-Part. Extensive experiments are conducted and the results demonstrate the effectiveness of T-Part.
Yu-Shan Lin, Shao-Kan Pi, Meng-Kai Liao, Ching Tsai, Aaron Elmore, and Shan-Hung Wu
Recent deterministic database systems have achieved high scalability and high availability in distributed environments given OLTP workloads. However, modern OLTP applications usually have changing workloads or access patterns, so how to make the resource provisioning elastic to the changing workloads becomes an important design goal for a deterministic database system. Live migration, which moves the specified data from a source machine to a destination node while continuously serving the incoming transactions, is a key technique required for the elasticity. In this paper, we present MgCrab, a live migration technique for a deterministic database system, that leverages the determinism to maintain the consistency of data on the source and destination nodes at very low cost during a migration period. We implement MgCrab on an open-source database system. Extensive experiments were conducted and the results demonstrate the effectiveness of MgCrab.
Yu-Shan Lin, Ching Tsai, Tz-Yu Lin, Yun-Sheng Chang, and Shan-Hung Wu
Deterministic database systems have been shown to significantly improve the availability and scalability of a distributed database system deployed on a shared-nothing architecture across WAN while ensuring strong consistency. However, their scalability and performance advantages highly depend on the quality of data partitioning due to the reduced flexibility in transaction processing. Although a deterministic database system can employ workload driven data (re-)partitioning and live data migration algorithms to partition data, we found that the effectiveness of these algorithms is limited in complex real-world environments due to the unpredictability of machine workloads. In this paper, we present Hermes, a deterministic database system prototype that, for the first time, does not rely on sophisticated data partitioning to achieve high scalability and performance. Hermes employs a novel transaction routing mechanism that jointly optimizes the balance of machine workloads, data (re-)partitioning, and live data migration by looking into the queued transactions to be executed in the near future. We conducted extensive experiments which show that Hermes is able to yield 29% to 137% increase in transaction throughput as compared to the state-of-the-art systems under complex real-world workloads.
這個演講介紹 VanillaDB，一個教學用途的關聯式資料庫系統 (RDBMS)。一開始會先給大家一些動機，讓大家了解為什麼要看這種系統的程式碼，以及介紹一些關於 RDBMS 的背景知識，像是 ACID。接著，這個演講會帶聽眾深入了解一個查詢 (Query) 在 VanillaDB 是怎麼被執行的，以及一些開發這類系統會遇到的問題。最後，演講會揭露整個 VanillaDB 專案的全貌，以及我們的下一步。
這個演講將會簡單地帶大家入門 RDBMS 的運作原理。一開始會先介紹 Query 的一天，接著介紹開發 storage engine 時可能會要考慮甚麼。最後帶大家了解如何實作 RDBMS 最重要的 ACID 特性。
