Low-Level vs High-Level Formatting

Many people do not distinguish low-level formatting (also called physical formatting) from high-level formatting (also called logical formatting).

Even though hard drives can be very small, they still contain millions of bits and therefore need to be organized so that information can be located. This is the purpose of the file system. The surface of each cylinder, which is originally uniform, is divided during formatting (by divided we mean of course that the surface particles are magnetised...) into little parcels that can be more easily located.

Low-level formatting

The purpose of low-level formatting is to divide the disk surface into basic elements:

• tracks
• sectors
• cylinders

Remember that a hard drive consists of several circular platters rotating around an axis and covered on either side by a magnetic oxide which, since it is polarised, can be used to store data.

The tracks are the concentric areas written on both sides of a platter.

Finally, these tracks are divided into pieces called sectors.

There are millions of tracks and each has around 60 to 120 sectors.

A cylinder refers to all the data located on the same track of different platters (i.e. vertically on top of each other) as this forms a "cylinder" of data in space.

Physical formatting therefore consists in organizing the surface of each platter into entities called trackers and sectors, by polarising the disk areas using the write heads. Tracks are numbered starting from 0, then the heads polarise concentrically the surface of the platters. When the head goes from one track to the next, it leaves a gap. Each track is itself organized into sectors (numbered starting from 1) and separated by gaps. Each of these sectors starts with an area reserved for system information called a prefix and ends with an area called a suffix
The purpose of low-level formatting is therefore to prepare the disk surface to receive data (and therefore does not depend on the operating system) and to mark "defective sectors" using tests performed by the manufacturer.

When you buy a hard drive, it has already undergone low-level formatting. 

SO YOU DO NOT NEED TO PERFORM LOW-LEVEL FORMATTING!

High-level formatting

Logical formatting occurs after the low-level formatting. It creates a file system on the disks that will allow an operating system (DOS, Windows 95, Linux, OS/2, Windows NT, ...) to use the disk space to store and access files.

Operating systems use different file systems, so the type of logical formatting will depend on the operating system you install. So, if you format your disk with a single file system, this naturally limits the number and type of operating systems that you can install (in fact, you can only install operating systems that use the same file system).

Fortunately, there is a solution to this problem which is to create partitions. Each of the partitions can effectively have its own file system, and you can therefore install different types of operating systems.

Types of Hard Drive Partitions and Drive Partitioning Operations

Disk partition

A partition is a part or section of a hard disk, or, in other words, a storage space unit of the highest level on it. You can have only one partition, occupying 100% of your HD, or divide your hard disk into multiple partitions (each for a certain purpose), determine their size, install the operating and file system you want. As a result, your single physical hard drive will turn into several smaller logical disks.

Actually a hard disk can exist on your machine without any partitions on it, but it’ll be pointless because you won’t be able to implement even basic tasks, say, install an operating system and thus store and access your data properly. It need be added that you can install the majority of Windows OSes onto a non-partitioned disk - in this case the operating system will implement partitioning itself in the course of installation.

Types of hard drive partitions

Primary Partition is a partition that is needed to store and boot an operating system, though applications and user data can reside there as well, and what’s more, you can have a primary partition without any operating system on it. There can be up to a maximum of four primary partitions on a single hard disk, with only one of them set as active (see “Active partition”).

Active (boot) partition is a primary partition that has an operating system installed on it. It is used for booting your machine. If you have a single primary partition, it is regarded as active. If you have more than one primary partition, only one of them is marked active (in a given PC session).

Extended partition can be sub-divided into logical drives and is viewed as a container for logical drives, where data proper is located. An extended partition is not formatted or assigned a drive letter. The extended partition is used only for creating a desired number of logical partitions.

Logical drive is created within an extended partition. A logical partition is a way to extend the initial limitation of four partitions. An extended partition can contain up to 24 logical partitions (you’re limited by the number of drive letters and the amount of hard drive space available for creating drives; of course, it’s senseless to use 24 partitions on a system in most cases, because it will be a data organization nightmare). Logical partitions are used for storing data mainly, they can be formatted and assigned drive letters; their details are listed in the extended partition’s table - EMBR (Extended Master Boot Record).

Drive partitioning operations

There are 2 types of partition operations: basic and advanced. The former are more frequent, but you are likely to need to perform the latter ones either. So, let’s analyze both types.

Basic partition operations:

• Create partition
• Increase/decrease partition size
• Increase free space

If a partition’s capacity is not sufficient for storing extra data on it, you might need to increase its space. The increase free space wizard will help you fulfill this task.

• Move partition

In what cases do you move partitions? First, if you want to change the letter order assigned by the OS. Then in case you need to work with some older OS that can boot from binary partitions located at the beginning of the disk, you also perform this kind of operation. Speeding up partition operations and changing partition configuration are also implemented by means of moving partitions.

• Copy partition

You perform this operation when you create a partition backup or want to move all data from an old disk to a new one.

• Delete partition

After a partition is deleted, its space is added to unallocated disk space. It can be used for a new partition or to resize an existing partition.

• Delete partition and destroy data (Wipe process)

To securely wipe out data stored on the deleted partition use special partitioning tools, which include powerful hard disk/partition wiping utilities.

• Split partition

You can split a partition in two or create an empty partition from another partition’s free space.

• Merge partitions

You can merge two partitions, even if their file systems are different. All data will stay intact and reside on the resulting partition.

• Explore partition

You can explore and manage partition contents before configuring operations on that partition.

• Changing partition labels

The partition label is a name assigned to a partition for easier recognition, for example, “System”, “Data”, etc.

• Format partition

To organize a file system that supports files and folders data storage, you must format a partition.

• Checking hard disk partitions for errors
• Defragmenting a partition

Defragmentation is reorganizing file storage on a hard disk partition so that parts of files are not spread about the disk, which allows the red head to move less across the disk and thus, increases PC and server performance.

Advanced partition operations:

• Changing partition letter

Some operating systems assign letters to hard disk partitions at startup. Connecting an additional disk as well as creating or deleting a partition on existing disks might change your system configuration. As a result, some applications might stop working or user files might not be opened. To avoid this, you can change letters assigned by the operating system.

• Converting a file system

Some file systems do not support large-sized partitions, files, or disks. They may also have a limited root size. To improve these characteristics you can use this particular feature.

Supported conversions:

FAT 16→ FAT 32

FAT 16→NTFS

FAT 32→FAT 16

FAT 32→NTFS

NTFS→FAT 16

NTFS→FAT 32

Ext2→Ext3

Ext3→Ext2

• Hiding a partition

This option is necessary for protecting important information from unauthorized or casual access. Partition software usually allows you to hide both primary and logical partitions.

• Unhiding a partition

This feature lets the operating system see the partition, assign a letter and provide access to its files.

• Setting active partition

Since a hard disk can have only one active partition, the latter must be set.

• Resizing a root

The FAT 16 partition is located in a special place and has a limited size. Advanced partitioning tools enable you to change the size of existing partitions.

• Changing cluster size

A file occupies at least one cluster (one unit of disk space allocation for files and directories). Smaller clusters reduce slack disk space.

• Changing partition type

This operation is performed to change a hexadecimal value that defines the file and operating system suitable for a partition.

What is Mac OS HFS+ format ?

The Mac OS Extended Format (HFS+) is a volume format compatible with Mac OS 8.1 and higher, including Mac OS X. In terms of volume, folder, and file size limits, it is similar to the Macintosh Standard Format (HFS). 
HFS+ or HFS Plus is a file system developed by Apple Computer Inc. to replace their Hierarchical File System (HFS) as the primary file system used in Macintosh computers (or other systems running Mac OS). It is also one of the formats used by the iPod music player. HFS+ is also referred to as Mac OS Extended Volume Hard Drive Format, where its predecessor, HFS, is also referred to as Mac OS Standard.

HFS+ is an improved version of HFS, supporting much larger files (block addresses are 32-bit length instead of 16-bit) and using Unicode (instead of Mac OS Roman or any of several other character sets) for naming the items (files, folders) - names which are also character encoded in UTF-16 and normalized to a form very nearly the same as Unicode Normalization Form D (NFD) (which means that precomposed characters like é are decomposed in the HFS+ filename and therefore count as two characters and UTF-16 implies that characters from outside the Basic Multilingual Plane - often seldom used and characters from ancient writing systems - also count as two characters in an HFS+ filename). HFS Plus permits filenames up to 255 UTF-16 characters in length, and n-forked files similar to NTFS, though until recently, almost no software takes advantage of forks other than the data fork and resource fork. HFS+ also uses a full 32-bit allocation mapping table, rather than HFS’s 16 bits. This was a serious limitation of HFS, meaning that no disk could support more than 65,536 allocation blocks under HFS. When disks were small, this was of little consequence, but as larger-capacity drives became available, it meant that the smallest amount of space that any file could occupy (a single allocation block) became excessively large, wasting significant amounts of space. For example, on a 1 GB disk, the allocation block size under HFS is 16 KB, so even a 1 byte file would take up 16 KB of disk space.

How do I know a hard disk is HFS+ format?

Select the hard disk in the Finder and choose the Get Info command in the File menu. The format information will specify either Mac OS Standard (hfs) or Mac OS Extended (hfs+). 

What are the differences between HFS+ format and HFS format?

The most visible difference between HFS+ format and HFS format, other than the minimum file size, is the maximum number of files. 

An HFS+ volume contains five special files:

1. Catalog file - Describes the folder and file hierarchy of the volume. It is organized as a "balanced tree" for fast and efficient searches
2. Allocation file - Specifies whether an allocation block is free (similar to $Bitmap in NTFS). This is stored in a bitmap, specifying a free allocation block with a "clear bit"
3. Attributes file - Contains attribute information regarding files or folders
4. Extents overflow file - Additional extents (contiguous allocation blocks allocated to forks) are stored in a b-tree in this file
5. Startup file - Allows computers to boot that do have built in support for HFS+ file systems

What is FAT and NTFS ?

FAT Definition:

FAT, short for File Allocation Table (used by hard disks, disks, Smartmedia™, Compact Flash™, Memory Stick and other), 
A table that the operating system uses to locate files on a disk. Due to fragmentation, a file may be divided into many sections that are scattered around the disk. The FAT keeps track of all these pieces. 
It is a file system that was created by Microsoft in 1977. FAT ( also know as FAT, FAT16, and FAT32 ) is still in use today as the preferred file system for floppy drive media and portable, high capacity storage devices like flash drives.

FAT was the primary file system used in all of Microsoft's consumer operating systems from MS-DOS to Windows ME. The New Technology File System (NTFS) is the primary file system on Microsoft's newer operating systems but FAT is still a supported option.

Below is more information on the versions of the FAT file system:

• FAT12 - The initial version of the FAT file system, FAT12 was introduced in 1977, even before MS-DOS, and was the primary file system for Microsoft operating systems up to MS-DOS 4.0. FAT12 supports drive sizes up to 32MB.
  
• FAT16 - The second implementation of FAT was FAT16, introduced in 1981. FAT16 was the primary file system for MS-DOS 4.0 up to Windows 95. FAT16 supports drive sizes up to 2GB. It was designed originally to handle files on a floppy drive, and has had minor modifications over the years so it can handle hard disks, and even file names longer than the original limitation of 8.3 characters, but it's still the lowest common denominator. The biggest advantage of FAT16 is that it is compatible across a wide variety of operating systems, including Windows 95/98/Me, OS/2, Linux, and some versions of UNIX. The biggest problem of FAT16 is that it has a fixed maximum number of clusters per partition, so as hard disks get bigger and bigger, the size of each cluster has to get larger. In a 2–GB partition, each cluster is 32 kilobytes, meaning that even the smallest file on the partition will take up 32 KB of space. FAT16 also doesn't support compression, encryption, or advanced security using access control lists.
  
• FAT32 - FAT32 is the latest version of the FAT file system. It was introduced in 1996 for Windows 95 Service Pack 2 users and was the primary file system for consumer Windows versions through Windows ME. FAT32 supports drive sizes up to 8TB.
  FAT32 shares all of the other limitations of FAT16, and adds an important additional limitation—many operating systems that can recognize FAT16 will not work with FAT32—most notably Windows NT, but also Linux and UNIX as well. Now this isn't a problem if you're running FAT32 on a Windows XP computer and sharing your drive out to other computers on your network—they don't need to know (and generally don't really care) what your underlying file system is.

New Technology File System (NTFS):

NTFS Short for NT File System, is a high-performance and self-healing file system used in Windows Vista, Windows XP, 2003, 2000, NT and Windows 7, which supports file-level security, compression and auditing. It also supports large volumes and powerful storage solution such as RAID. NTFS supports hard drive sizes up to 256TB.

NTFS supersedes the FAT file system as the preferred file system for Windows operating systems. NTFS has several improvements over FAT and HPFS (High Performance File System) such as improved support for metadata and the use of advanced data structures to improve performance, reliability, and disk space utilization, plus additional extensions such as security access control lists (ACL) and file system journaling.

The most important new feature of NTFS is the ability to encrypt files and folders to protect your sensitive data.

The Advantages of NTFS:

The NTFS file system, introduced with first version of Windows NT, is a completely different file system from FAT. It provides for greatly increased security, file–by–file compression, quotas, and even encryption. It is the default file system for new installations of Windows XP, and if you're doing an upgrade from a previous version of Windows, you'll be asked if you want to convert your existing file systems to NTFS. Don't worry. If you've already upgraded to Windows XP and didn't do the conversion then, it's not a problem. You can convert FAT16 or FAT32 volumes to NTFS at any point. Just remember that you can't easily go back to FAT or FAT32 (without reformatting the drive or partition), not that I think you'll want to.

The NTFS file system is generally not compatible with other operating systems installed on the same computer, nor is it available when you've booted a computer from a floppy disk. For this reason, many system administrators, myself included, used to recommend that users format at least a small partition at the beginning of their main hard disk as FAT. This partition provided a place to store emergency recovery tools or special drivers needed for reinstallation, and was a mechanism for digging yourself out of the hole you'd just dug into. But with the enhanced recovery abilities built into Windows XP (more on that in a future column), I don't think it's necessary or desirable to create that initial FAT partition.

Partition types: Primary / Extended / Logical

Primary partitions is a partition that is needed to store and boot an operating system, In general, you would install the operating system in a primary partition. A primary partition can be set "active", which allows the computer to locate the operating system that needs to be started. Only 1 primary partition can be active, but more primary partitions can exist. If more than 1 primary partition is set active, the computer will not boot. Also, when no primary partition is set active, the computer will not boot. Only the partition table in the MBR can contain primary partitions.

Extended partition can be sub-divided into logical drives and is viewed as a container for logical drives, where data proper is located. An extended partition is not formatted or assigned a drive letter. The extended partition is used only for creating a desired number of logical partitions..

Logical partitions are the partitions that are created in the extended partition area. A logical partition is a way to extend the initial limitation of four partitions. An extended partition can contain up to 24 logical partitions.
Logical partitions are used for storing data mainly, they can be formatted and assigned drive letters; their details are listed in the extended partition’s table - EMBR (Extended Master Boot Record). 


Note that when using Dynamic disks these rules don't apply anymore.