Modern file systems, and even storage systems, might have built-in deduplication, but common file systems still do not. So checking for redundant data and do deduplication when possible might save disk space.
Once up on a a time, there was a system, were we had this 6TB spool of binary files on an production ext4 file-system, and the volume was running out of disk space. The owner of the data thought it likely that there were duplicates in the vast ... [continue reading]