Redundancy Elimination Technology Based On Repartition And Quality Modulation

With the arrival of the big data era, the need for storage capacity hasbeen on dramatic rise. Although per unit storage cost has declined in thepast decade, the net cost of storing the vast amount of informationproduced daily is still significant. Thus, optimizing storage usage is everybit as important as before.Every file on disk wastes some space due to internal fragmentation.Different block sizes cause different amount of space waste. Such wastecan be significant if the combinations of file and block sizes are notcarefully coordinated. In addition, many types of files contain moreinformation than is necessary for the intended usage of the file and theenvironment they are used. For example, a high resolution image file maybe an over kill for display on a small screen device such as a small laptopor mobile phone. If the quality of the files is modulated according to theneed and environment, further space reduction can be achieved.This paper proposes a waste reduction method through diskre-partition and automatic quality modulation. By adjusting the block sizeand re-partition the disk, and at the same time modulate file qualityaccording to the need and environment, we can reduce space waste andachieve higher storage usage. In our disk re-partition method, only themetadata and a small portion of the file data need to be altered ormanipulated and3%-9%spaces can be saved. We applied the modulationmethod to Microsoft Office PowerPoint and PDF (Portable DocumentFormat) files and were able to achieve16%-50%reduction of space waste.