Klennet Storage Software

Photo Recovery with Klennet Carver

Typical photo recovery involves files deleted from a camera memory card or a formatted memory card. These situations are effectively identical as far as recovery is concerned: memory cards using FAT filesystem and deletion or format clears the allocation table entries. The consequence is that you can no longer find the file on the media.

Fragmentation and JPEG file structure

Files which are stored contiguously on the media can be recovered easily by most any data recovery software, either commercial or free (like ZAR or PhotoRec). The software will look through each sector for a file header and then capture data between the two file headers.

If the file is not contiguous on the media, this does not work. Only the first part of the file is recovered, up to the end of the first fragment. Depending on where the fragment ends and what happens to be on the media after it, visible effects vary.

JPEG files normally contain small thumbnail image close to the beginning of file, and the layout of the file looks like this:

Typical JPEG file layout

Headers are smallest, thumbnail is small, and most of the file is occupied by high-resolution image.

JPEG files won't open

If the first fragment is small enough not to include compression parameters, the file will not open at all and different software you can try will produce different error messages.

  • Decode error! JPEG datastream contains no image.
  • Could not complete your request because an unknown or invalid JPEG marker type is found.
  • Could not complete your request because a SOFn, DQT, or DHT JPEG marker is missing before a JPEG SOS marker.
IrfanView error message on a bad JPEG

Images missing bottom or top part

If the file is missing the end, the high-resolution image will be only partially available. Depending on the file format, either top or bottom part will be missing. What you might not expect is that because the thumbnail is OK, the file looks good in Windows Explorer, but when you open it, you will see the damage.

JPEG file layout with its end cut off Sample JPEG file with its end cut off

Note that in the above example the data in file is physically stored in a bottom-to-top order, so the top part is missing, not the bottom part as one might expect.

Mixed images and other artefacts

Files with some extra data in the middle produce a different kind of artefacts, while still looking good in thumbnail view.

JPEG file layout with extra data in the middle Sample JPEG file with extra data in the middle

Last not least, two or more files can mix to produce strange-looking results.

JPEG file layout with last fragment from another file Two JPEG file fragments mixed

Let's reiterate that you need to open the file to see that it is damaged. In all the above cases, the thumbnail view is OK, but the high-resolution image is damaged. On the example below, the same thumbnail of the good image is shown for all of the damage variations displayed above.

Good thumbnail images displayed for damaged files

How often are JPEG files fragmented?

Based on 2017 real-world data, approximately one in ten JPEG files on a digital camera memory card is be fragmented. Higher resolution files (more megapixels) tend to be fragmented more, up to as much as one in three depending on a camera model. Memory cards already in use for a long time also tend to have more fragmented files.

Picture recovery with Klennet Carver

Standard data recovery software will recover files as they are stored on disk. When the file is fragmented, the damaged file will be recovered, even though the thumbnail may look good. Klennet Carver will identify and reassemble fragments of the files, remove spurious data where required, and stitch together multiple fragments belonging to a single file.

To reassemble fragmented files, Klennet Carver identifies the fragments and then essentially tries all possible combinations of these framgemnts (if you want more technical details, read here). If all the fragments of the file are available, the file is eventually recovered. If one of the fragments is overwritten, the JPEG file is recovered up to this overwritten fragment.

This additional recovery capability comes at a cost. There are millions of fragment combinations possible and even after all possible optimizations are applied, one still needs massive amount of computing power to looks through these combinations and find correct ones. Examining all possible combinations takes several hours for a typical 32 GB or 64 GB memory card when done by a reasonably modern (as of 2018) i5 or i7 processor.

Photo recovery vs. JPEG file repair

There are services and software that attempt to repair damaged JPEG files. While having its use, file repair is limited to the data which is already in the files. If the files are fragmented, some data is probably appended to a different file, or it may be in the area of the memory card which was not considered a part of any of the files. One needs to analyze the entire memory to find all the fragments, and photo recovery algorithms taking entire card as the input are on average more effective than file repair algorithms taking separate damaged files as input.

Summary

Sometimes, JPEG or other image files on memory cards are fragmented (stored in several non-adjacent fragments). While the number of fragmented files varies across memory cards, about one in five high-resolution image files is fragmented. If data recovery is required, traditional recovery software can't recover these fragmented files. While thumbnails may look good, when you try to open the file, it is either cut in half or exhibits some other weird artifacts. Klennet Carver can recover fragmented files by using a slow and complicated process involving finding, matching, and merging file fragments from the media back as they were in the original file.

I have a low volume mailing list, for news and tips, which I send out once or twice a month.
Subscribe if you are interested.