Klennet Storage Software

USB adapters silently change sector size

It is certainly not the first time I'm ranting about USB adapters, but the topic seems pretty damn infinite.

So the problem is:

I have a USB 3 enclosure, a 1 TB hard disk drive from a different computer, and my own computer with USB 3 ports. When I put the hard drive in the enclosure, and connect it via USB, my computer can see the windows partition but not open it. When I attach the drive directly to the my computer's motherboard SATA, I can open the Windows folders and files. It doesn't appear to be a permissions or size issue. Anyone know why this is?

Yeah, yeah...

4K sectors

Modern hard drives use one of the two sector sizes: 512 bytes per sector, which is more or less a standard, or 4096 bytes per sector. Drives with 4096 bytes per sector are often called 4K, 4K-sector, or Advanced Format (AF) drives. There are further terms, like 4Kn and 512e, but these are not relevant now.

Why is this needed? Historically, hard drives used 512-byte sectors. At the same time, MBR partitioning used 32 bits to store locations on disk and partition sizes. The maximum amount of disk space one can address with 32-bit pointers and 512-byte sectors is 2 TB. As storage sizes increased back in the day, hard drives started to bump into this 2 TB limit. There is little gain in having a 4 TB hard drive if one can only use 2 TB of it because of MBR limitations. There are two obvious ways to go above the 2 TB limit: either use 48- or 64-bit addresses, or increase sector size. GPT partitioning uses 64 bits for its addresses, and can be used for large drives. Windows XP, however, still in wide use at that time, was not capable of reading GPT. So, the sector size was increased by the factor of eight, from 512 bytes to 4096 bytes, pushing the MBR address space up to 16 TB.

There is one important thing about using different sector sizes for partition tables and filesystems. Filesystems typically have their block size independent from the physical sector size. Some store the sector size used to create the filesystem in the boot sector (FAT, NTFS), other specify their block size in bytes (EXT). On the contrary, partition tables (both MBR and GPT) use physical sector size to compute their addresses. Therefore, changing the physical sector size invalidates partition tables.

Why does USB enclosure matter?

Certain chips used for USB-to-SATA conversion in some USB enclosures will convert the sector size of the drive to 4096 bytes, regardless of the drive's own physical sector size. Probably, this is done for compatibility with Windows XP, but I'm not quite sure. If you put a drive with 512 bytes per sector into such an enclosure, it will be presented to the PC as having 4096 bytes per sector. The PC will then create partition tables based on 4K sectors. If you then remove the drive from the enclosure and connect it directly to a SATA port, the drive will present its own 512 bytes per sector, thus invalidating its partition tables. Windows is careful enough to know there is something wrong with partition tables, and it will not modify such a drive.

The obvious fix if you need the data is to put the drive back into the same or similar enclosure. If you do not need the data, repartition the drive, reformat it, and it is ready for use.

As far as I know, there is no configuration setting on these enclosures. The enclosure either does the conversion, or it does not.

TL;DR - What are the consequences?

If your USB enclosure silently converts 512-byte sectors to 4096-byte sectors, then

  • If you bring a hard drive from another PC, and that hard drive uses 512-byte sectors, you will not be able to read that drive while it is connected through the enclosure. This is exactly the issue encountered in the opening quotation.
  • If you put a blank 512-byte sector drive in the enclosure and format it, you will then need the same or similar enclosure to access the drive. You will not be able to use it on SATA or with an enclosure which does not do sector size translation.

Created Saturday, April 14, 2018

I have a low volume mailing list, for news and tips, which I send out once or twice a month.
Subscribe if you are interested.