ZFS recovery - understanding the results

Result view

Once ZFS pool is processed, the results are grouped in the following way:

  • Datasets are grouped as top-level directories, one directory per dataset.
    • If there are snapshots associated with a dataset, all the snapshots are integrated into the same directory as multiple file versions.
    • Past or overwritten file versions found outside the snapshots are also integrated as file versions.
  • If the dataset cannot be associated with a name and cannot be determined to be a snapshot of some other dataset, it is placed in a separate directory with a generic name.
  • There is a separate top-level directory for ZVOLs. Wherever ZVOL name can be identified, it is used as a file name. Otherwise a generic name is generated. Since ZVOLs are stored as files, versions of ZVOLs are naturally stored as versions of the same file. If there are no ZVOLs, this directory is not created.

Checksum verification

After the scan is complete, Klennet Recovery reads every file and calculates ZFS checksums for its contents. This is roughly equivalent to ZFS scrub. The checksum verification takes a long time and determines which files are recoverable and which are not, with one exception described below.

A special case of a partially written file

When you are copying a big file onto a ZFS pool, the filesystem naturally records several versions of the file as it is being written. Let's say there are two versions of the file. The first version contains only half the data, and the second contains all the data. However, both versions have good checksums. This is because, from the filesystem perspective, there is nothing wrong with the first version. You asked the filesystem to write the first half of the file, and the filesystem complied. The data is there, intact. The filesystem did not know that more data was incoming. Therefore, both versions are marked valid by checksum verification, but the half-written version is useless.

Merging multiple file versions

If there are multiple damaged versions of a large file, chances are different parts of the data are damaged in different versions. In case of ZVOLs, multiple versions of a ZVOL, when combined, can sometimes produce a recoverable disk image. This is never a perfect recovery, since the output is a patchwork of data from different points in time. The resulting image typicallly requires additional recovery work.

Pick a ZVOL, right-click it, and select Combine multiple ZFS versions. You will be asked for the location to write the file to and then the copy will start.

  1. The copy process tries to read the block of data from the latest version of the ZVOL.
  2. If the read fails, the copier tries the previous version.
  3. Failing that, it tries all available versions in turn, going from latest (newest) to oldest.
  4. Finally, if no version contains readable data, zeros are written to the output.

The process is repeated for each block of data. The speed is generally low and depends on how many versions there are, and how many fallbacks are required to reach a valid block.