Research On Design And Optimization Of Edge-terminal Oriented Open-channel Flash File Systems

With rapid developments of the Internet of Things,artificial intelligence,and high-speed mobile networks,the types and number of terminal devices increase dramatically.Meanwhile,processed edge-terminal data grows explosively.Due to the high storage density,low static power consumption,large capacity,small size,and anti-vibration of flash,they are widely used in edge-terminal storage systems.However,with massive data,traditional schemes are gradually unable to cope with the requirements for large-capacity,high-efficiency,and low-cost storage services of edge-terminal devices.The open-channel flash memory architecture fully combines the characteristics of file systems and flash hardware and brings new opportunities for developing flash file systems by eliminating the semantic barriers among storage layers.However,the direct management of flash hardware by software also brings new challenges to file system design.Therefore,in view of the challenges and opportunities faced by flash memory storage systems in the edge-terminal environment,this thesis studies the design and optimization of file systems based on open-channel flash memory.First,this thesis studies the extensible and low-overhead software-managed embedded flash file system,which can reduce the memory footprint of space management,file management,and namespace management.The proposed file system also achieves the fast booting and wear-leveling of flash memory.Second,by utilizing novel persistent memory,the middle layer of embedded flash file systems is designed to improve their portability and adaptability.Finally,a hybrid open-channel flash memory scheme based on small capacity persistent memory is proposed to reduce the metadata write amplification caused by copy-on-write operation and improve the performance of file systems.The detailed contributions of this thesis are summarized below.(1)Aiming at the high memory footprint,small managed capacity,and long mounting time of existing embedded flash file systems,this thesis designs an extensible low-overhead flash file system,ELOFS,for resource-scarce embedded devices.To reduce the memory footprint of space management,ELOFS adopts a large-grained partition mechanism without recording redundant flash block states.To reduce the memory footprint of file management,ELOFS organizes directories and data files according to access characteristics of embedded devices.To improve the efficiency of namespace retrieval,ELOFS utilizes an efficient path tree to speed up path resolutions.(2)Aiming at the mismatch of the logical operation granularity of traditional embedded flash file systems and the physical granularity of large-capacity flash memory hardware,the difficulty of transplanting,and the high overhead after manual adaptation,this thesis designs the non-volatile middleware of the embedded flash file system,NV-middle.Between the existing embedded file system logical I/O interfaces and the flash hardware I/O interfaces,NV-middle utilizes persistent memory to build an aggregated writing tree to cache small-grained writes of the file system.In addition,NV-middle adopts the delayed erasure mechanism to provide a transparent small-grained logical erase function for the upper-layer system and establishes a page cache mechanism to reduce the performance overhead caused by the read amplification.(3)Block-based file systems such as Btrfs use extent trees to organize file pages to reduce space management overhead and use a copy-on-write mechanism to ensure data consistency during synchronizations.However,copy-on-write operations lead to extents fragmentation,which violates merits of extents.Recursive metadata page updates also degrade the performance of file systems.This thesis fully exploits the advantages of open-channel flash memory and proposes an open-channel file system architecture with small-capacity persistent memory.This thesis designs a hybrid fine-grained copy-on-write and journaling mechanism(Hy Fi M),to avoid the fragmentation of the extent tree and the propagation of updates caused by the copy-on-write operation of the logical layer,and improve the read/write performance of file systems.This thesis implements the proposed techniques in Linux and evaluates them from various aspects.The experimental results show that the proposed ELOFS can provide decent performance with minimum memory footprint;the proposed NV-middle realizes the seamless adaptation of existing embedded file systems to large flash volume;the proposed Hy Fi M can effectively reduce the extent fragmentation caused by copy-on-write operations and improve system performance.