Design And Implementation Of KV-based Distributed Transaction System

In traditional database systems,transaction concurrency control ensures that changes to data by applications have ACID properties.With the widespread use of distributed systems,distributed transaction technology has been widely adopted in fields such as storage and databases.However,increasingly complex application scenarios have placed higher demands on the flexibility,scalability,and performance of distributed transaction systems.Traditional concurrency control methods use a single transaction as the minimum control unit,and each method is suitable for different scenarios,making real-time adjustment difficult in constantly changing business environments.At the same time,under the serializable isolation level,traditional concurrency control methods often experience low concurrency and high rollback rates,leading to low transaction processing efficiency.In cloud-native environments,cloud storage services naturally have high availability characteristics,but traditional distributed transaction systems have not utilized this feature for data storage.Instead,their existing data replication wastes a large amount of storage and bandwidth resources.Additionally,as the amount of data and partitions increases,crosspartition transactions can negatively impact system performance.To address these challenges,this article designs and implements a distributed transaction system based on key-value(KV)pairs,which includes the following key technologies:· Adaptive concurrency control mechanism based on transaction characteristics.This mechanism dynamically selects the appropriate concurrency control method based on real-time transaction size,key-value conflict rate,and other characteristics at the granularity of data items,addressing the limitations of traditional concurrency control methods in a constantly changing business environment.· Transaction dependency analysis method.This thesis designs and implements a method for discovering transactions that do not comply with serializable isolation levels by analyzing the dependencies between transactions generated by concurrent access to the same key-value.By replacing the validation stage in traditional meth-ods,the proposed solution increases the concurrency level between transactions.Additionally,it adopts early validation during execution to reduce the rollback over-head of large transactions.· Unbundled architecture for distributed transaction systems.This thesis designs and implements a KV-based distributed transaction system that separates storage and transaction processing.It provides KV transaction interfaces for applications with various cloud storage facilities in the same way.This decouples storage and trans-action processing and allows each component to scale horizontally.The prototype,Azino,is implemented based on the brpc framework and is evaluated in a distributed environment against YCSB benchmark.By analyzing the system’s throughput,latency,and other metrics in multiple scenarios,Azino can deliver a higher commit rate and lower rollback rate compared to traditional methods as well as good scalability.