How much can be recovered from a single-disk ZFS pool?

I did some more tests in continuation of a previous write-up on recoverability of ZFS.

Test setup

The test is pretty simple - create a single-disk ZFS pool, fill it with data, make a disk image. Then, start overwriting data with zeros, working from the beginning towards the end of the disk. At some preset points, pause overwrite and attempt recovery (using Klennet ZFS Recovery build 735). From the recovery output, measure two parameters:

• Number of files recovered with correct checksums. Since the files are more or less all of the similar size, the number of files also reflects the total size of recoverable data.
• Number of files names recovered. ZFS stores directories separate from file location metadata, so it is interesting to watch file names gradually disappearing.

I tested two datasets, one really small, and one close to 1 TB. Numbers for these two datasets are as follows

	Small sample	Large sample
Pool size	17.3 GB	899 GB
Size of files	12 GB	683 GB
Percent full	68%	77%
Number of files	2966	129400

Results

The charts show percentage of files recovered vs. percentage of the entire pool space overwritten.

• Brown line is how many files were recovered;
• green line is how many file names were recovered;
• red dashed vertical line marks how much disk space is occupied;
• black dashed diagonal line marks hypothetical one-to-one relationship, where overwriting one percent of disk space causes loss of exactly one percent of files.

Overwritten vs. recovered data, small test.

Overwritten vs. recovered data, large test.

What do these charts show?

• As you overwrite the pool more, more data is lost. That's not entirely unexpected.
• At about one-third of the pool, the last usable set of metadata is overwritten, and no useful data can be recovered.
• Exactly how many files remain recoverable depends on the filesystem layout, that is, on the history of the filesystem and write patterns.
• If the damage is limited to 1% or so of the pool, the entire pool can be recovered.

For practical purposes, this means

• If only the partition tables are deleted, everything still can be recovered.
• If the pool is overwritten with a new blank pool, everything still can be recovered.
• To make the pool entirely unrecoverable, about one-third of its disk space must be overwritten.

Created Monday, February 25, 2019