Windows ConfidentialA Brief and Incomplete History of FAT32

Raymond Chen

Why did classic Windows 95 limit local hard drives to 2GB? Some people would lead you to believe that the FAT file system team said, "Y’know, the FAT file system supports hard drives of indefinite size, but let’s impose an artificial cap just to make people’s lives more frustrating." In reality, the 2GB limit was the result of a combination of several pre-existing limitations. The maximum size of a cluster was 32KB, and the maximum number of clusters on a single drive was a shade under 65535. When you multiply this out, you find that the maximum drive size was a tick under 2GB.

You could, with some tweaking, get the cluster size up to 64KB on Windows NT®, but you would quickly regret this. If you took this chance, you’d find that many apps would end up overflowing some internal 16-bit computations and producing garbage results. This issue was particularly common among application setup programs, which ended up telling you that your freshly formatted 1GB drive was out of disk space.

Increasing the maximum cluster size doesn’t really help anyway. Even though it raises the limit on the maximum size of a drive, it does so only in a crude way. Since the cluster is the unit of storage for files on FAT drives, increasing the size of a cluster merely increases the granularity of file storage. The total number of files you can store on the drive doesn’t change; the result is simply that each file can be bigger. If all you do is increase the cluster size, there is still a limit of 65535 or so files per drive. It’s the cluster count that you have to increase.

In the FAT file system, the number of clusters per drive was limited to around 65535 because the file allocation table—the object after which the FAT file system was named—consisted of an array of 16-bit cluster numbers. With Windows® 95 OSR2, FAT32 was introduced, raising the maximum number of clusters into the millions by expanding the file allocation table to an array of 32-bit values. Note, however, that the maximum number of clusters is 0x0FFFFFFF—not the full 32-bit value of 0xFFFFFFFF. Since this is limited to a 28-bit value, programs that manipulate FAT32 file allocation tables can use the top four bits as tag bits in their internal data structures. Losing this capacity at the top end has no impact on the theoretical maximum FAT32 volume size. That’s because the FAT32 volume size is capped by the 0xFFFFFFFF maximum sector count, and at 512 bytes per sector, this results in a theoretical maximum FAT32 volume size of 2TB. (While we’re on the topic of FAT32, the maximum size of a single file is 4GB, and the maximum number of files per directory is 65535.) Long before you hit the theoretical maximum volume size, you will reach the practical limits.

FAT used linear searching and threaded file allocation information in a linked list. All these linear-time data structures were acceptable when the FAT file system was developed—at a time when floppy drives were the primary storage device. Today, however, multi-gigabyte drives are the norm and these linear-time algorithms fall apart due to the large amount of disk access required to carry out a single operation. Furthermore, the high cluster count means a ridiculously large file allocation table; as a result, even the simple act of computing how much free disk space is available can take over a minute. That’s why the first time you type the dir command, there is a long pause at the end of the directory listing: the file system is busy calculating the disk free space to display at the bottom of the directory listing. (And don’t forget that Windows 95 needed to support FAT32 when running in real mode. I certainly don’t envy the engineer who had to implement the FAT32 file system in 16-bit assembly language under extremely tight memory constraints.)

At some point you have to say, "Enough is enough." After considerable discussion, 32GB was decided upon as the arbitrary cutoff point. Windows 2000 and higher offer only NTFS when formatting larger drives. Note, however, that Windows 2000 and higher will use a FAT32 drive larger than 32GB; they simply won’t create one. (For a 32GB FAT32 drive, it takes 4 megabytes of disk I/O to compute the amount of free space.)

I remember back when FAT32 was being developed, the project lead came to my office and had me read through his engineering notebook and sign and date every page. In part, this was an opportunity for me to go over his design and discuss it with him. But primarily it was to establish the date of the invention for defensive purposes.

One final note of trivia for your next geek cocktail party: the original name for FAT32 was "BigFAT".

Raymond Chen’s Web site, blogs.msdn.com/oldnewthing, deals with Windows history and Win32 programming. He can spell xerophthalmia but not accommodate.

© 2008 Microsoft Corporation and CMP Media, LLC. All rights reserved; reproduction in part or in whole without permission is prohibited.