SG_FORMAT(8)

NAME
sg_format - format or resize a SCSI disk (perhaps change its block size)

SYNOPSIS
sg_format [--cmplst=0|1] [--count=COUNT] [--dcrt] [--early] [--format] [--help] [--long] [--pfu=PFU] [--pinfo] [--resize] [--rto_req] [--security] [--six] [--size=SIZE] [--verbose] [--version] [--wait] DEVICE

DESCRIPTION

Not all SCSI direct access devices need to be formatted and some have vendor specific formatting procedures. SCSI disks with rotating media are probably the largest group that do support a 'standard' format operation. They are typically factory formatted to a block size of 512 bytes with the largest number of blocks that the manufacturer recommends. The manufacturer's recommendation typically leaves aside a certain number of tracks, spread across the media, for reassignment of logical block addresses during the life of the disk.

This utility can format modern SCSI disks and potentially change their block size (if permitted) and the block count (i.e. number of accessible blocks on the media also known as "resizing"). Resizing a disk to less than the manufacturer's recommended block count is sometimes called "short stroking" (see NOTES section). Resizing the block count while not changing the block size may not require a format operation. The SBC-2 standard (see www.t10.org) has obsoleted the "format device" mode page. Many of the low level details found in that mode page are now left up to the discretion of the manufacturer.

When this utility is used without options (apart from a device name) it prints out the existing block size and block count derived from two sources. These two sources are a block descriptor in the response to a MODE SENSE command and the response to a READ CAPACITY command. The reason for this double check is to detect a "format corrupt" state (see NOTES section).

Recent SBC-3 drafts add several "protection types" to the "protection information" introduced in the SBC-2 standard. See the "protection information" section (section 4.17 in draft SBC-3 rev 11). 8 bytes of protection information are added to each block (a 2 byte "logical block guard" (CRC), a 2 byte "logical block application guard", and a 4 byte "logical block reference tag"). A device that supports protection information sets the "PROTECT" bit in its standard INQUIRY response. The "FMTPINFO" and "RTO_REQ" bits in the FORMAT command cdb plus the "Protection Field Usage" in the parameter header are associated with protection information and can be set by this utility.

OPTIONS
Arguments to long options are mandatory for short options as well. The options are arranged in alphabetical order based on the long option name.

-C, --cmplst=0 | 1
    sets the CMPLST ("complete list") bit in the FORMAT cdb to 0 or 1. The default is 1 in which case the existing GLIST (grown list) is ignored. If the value is 0 then the existing GLIST is taken into account. See the LISTS section below. Active when the --format option is given. In most cases this bit should be left set; some MO disk drives need this bit cleared. The SCSI to ATA Translation (SAT) standard (prior draft: sat-r09) requires this bit to be cleared.

-c, --count=COUNT
    where COUNT is the number of blocks to be formatted or media to be resized to. Can be used with either --format or --resize. With --format this option need not be given in which case it is assumed to be zero. With --format the interpretation of COUNT is:
      (COUNT > 0) : only format the first COUNT blocks and READ
                    CAPACITY will report COUNT blocks after format
      (COUNT = 0) and block size unchanged : use existing block count
      (COUNT = 0) and block size changed : recommended maximum block
                            count for new block size
      (COUNT = -1) : use recommended maximum block count
      (COUNT < -1) : illegal
    With --resize this option must be given and COUNT has this interpretation:
      (COUNT > 0) : after resize READ CAPACITY will report COUNT
                    blocks
      (COUNT = 0) : after resize READ CAPACITY will report 0 blocks
      (COUNT = -1) : after resize READ CAPACITY will report its
                     maximum number of blocks
      (COUNT < -1) : illegal
    In both cases if the given COUNT exceeds the maximum number of blocks (for the block size) then the disk reports an error. See NOTES section below.

-D, --dcrt
    this option sets the DCRT bit in the FORMAT command's parameter list header. It will "disable certification". Certification verifies that blocks are usable during the format process. Using this option may speed the format. The default action of this utility (i.e. when this option is not given) is to clear the DCRT bit thereby requesting "media certification". When the DCRT bit is set, the FOV bit must also be set hence sg_format does that.

-e, --early
    this option is active when --format is given. The default action of this utility is to poll the disk every 30 seconds to determine the progress of the format operation until it is finished. When this option is given this utility will exit "early" as soon as the format has commenced. Then the user can monitor the progress of the ongoing format operation with other utilities (e.g. sg_turs(8) or sg_requests(8)). This option and --wait cannot both be given.

-F, --format
    issue a SCSI FORMAT command. This will destroy all the data held on the media. This option is required to change the block size of a disk. The user is given a 10 second count down to ponder the wisdom of doing this, during which time control-C (amongst other Unix commands) can be used to kill this process before it does any damage. See NOTES section for implementation details and EXAMPLES section for typical use.

-h, --help
    print out the usage information then exit.

-l, --long
    the default action of this utility is to assume 32 bit logical block addresses. With 512 byte block size this permits almost 2 terabytes (almost 2 ** 41 bytes) on a single disk. This option selects commands and parameters that allow for 64 bit logical block addresses. Specifically this is the "longlba" flag in the MODE SENSE (10) command and READ CAPACITY (16) rather than READ CAPACITY (10). When a disk supports "protection information" then this option may also be useful.

-P, --pfu=PFU
    sets the "Protection Field Usage" field in the parameter block associated with a FORMAT command to PFU. The default value is 0, the only other defined value currently is 1. Used together with --pinfo and --rto_req to specify the "protection type" to format the disk to (see SBC-3).

-p, --pinfo
    instructs a format operation to add an extra 8 bytes of protection information by setting the FMTPINFO bit in the FORMAT command cdb. Default action is not to format with protection information. Has no action unless --format is given.

-r, --resize
    rather than format the disk, it can be resized. This means changing the number of blocks on the device reported by the READ CAPACITY command. This option should be used with the --count=COUNT option. The contents of all logical blocks on the media remain unchanged when this option is used. This means that any resize operation can be reversed. This option cannot be used together with either --format or a --size=SIZE whose argument is different to the existing block size.

-R, --rto_req
    instructs a format to enable application client ownership of the "logical block reference tag" field (i.e. the RTO_REQ bit in the FORMAT cdb). The default action is to disable application client ownership of that field. Has no action unless both --format and --pinfo are given.

-S, --security
    sets the "Security Initialization" (SI) bit in the FORMAT command's initialization pattern descriptor within the parameter list. According to SBC-3 the default initialization pattern "shall be written using a security erasure write technique".

-6, --six
    Use 6 byte variants of MODE SENSE and MODE SELECT. The default action is to use the 10 byte variants. Some MO drives need this option set when doing a format.

-s, --size=SIZE
    where SIZE is the block size (i.e. number of bytes in each block) to format the device to. The default value is whatever is currently reported by the block descriptor in a MODE SENSE command. This option is only active when the --format option is also given. If the block size given by this option is different from the current value then a MODE SELECT command is used to change it prior to the FORMAT command being started (as recommended in the draft standard). Recent SCSI disks usually have 512 byte sectors by default and allow up to 16 bytes extra in a sector (i.e. 528 byte sectors). If the given size in unacceptable to the disk, most likely an "Invalid field in parameter list" message will appear in sense data (requires the use of '-v' to decode sense data).

-v, --verbose
    increase the level of verbosity, (i.e. debug output). "-vvv" gives the maximum debug output.

-V, --version
    print the version string and then exit.

-w, --wait
    this option only has an effect when used together with the --format option. The default format action is to set the "IMMED" bit in the FORMAT UNIT command's (short) parameter header. If this option (i.e. --wait) is given then the "IMMED" bit is not set. If --wait is gievn the FORMAT UNIT command waits until the format operation completes before returning its response. This can be several hours on large disks. This utility sets a four hour timeout on such a FORMAT UNIT command.

LISTS
The SBC-3 draft (revision 11) defines PLIST, CLIST, DLIST and GLIST in section 4.9 on "Medium defects". Briefly, the PLIST is the "primary" list of manufacturer detected defects, the CLIST ("certification" list) contains those detected during the format operation, the DLIST is a list of defects that can be given to the format operation. The GLIST is the grown list which starts in the format process as CLIST+DLIST and can "grow" later due to automatic reallocation (see the ARRE and AWRE bits in the read-write error recovery mode page (see sdparm(8))) and use of the SCSI REASSIGN BLOCKS command (see sg_reassign(8)).

The CMPLST bit (controlled by the --cmplst=0|1 option) determines whether the existing GLIST, when the format operation is invoked, is taken into account. The sg_format utility sets the FOV bit to zero which causes DPRY=0, so the PLIST is taken into account, and DCRT=0, so the CLIST is generated and used during the format process.

The sg_format utility does not permit a user to provide a defect list (i.e. DLIST). All protection information options are off by default.

NOTES
The SBC-2 standard states that the REQUEST SENSE command should be used for obtaining a progress indication when the format command is underway. However, tests on a selection of recent disks shows that TEST UNIT READY commands yield progress indications (but not REQUEST SENSE commands). So the current version of this utility uses TEST UNIT READY commands to poll the disk to find out the progress of the format. A new option may be required to handle this when disks catch up.

When the --format option is given then there is a 10 second window during which the user is invited to abort sg_format. This is just prior the SCSI FORMAT UNIT command being issued. If the --wait option is not given then the SCSI FORMAT UNIT command is issued with the IMMED bit set which causes the SCSI command to return after it has started the format operation. The --early option will cause sg_format to exit at that point. Otherwise the DEVICE is polled every 30 seconds with TEST UNIT READY commands until it reports an "all clear" (i.e. the format operation has completed). Normally these polling commands will result in a progress indicator (expressed as a percentage) being output to the screen. If the user gets bored watching the progress report then sg_format process can be terminated (e.g. with control-C) without affecting the format operation which continues. However a bus or device reset (or a power cycle) may well cause the device to become "format corrupt".

When the --format and --wait options are both given then this utility may take a long time to return. In this case care should be taken not to send any other SCSI commands to the disk as it may not respond leaving those commands queued behind the active format command. This may cause a timeout in the OS driver (in a lot shorter period than 4 hours applicable to some format operations). This may result in the OS resetting the disk leaving the format operation incomplete. This may leave the disk in a "format corrupt" state requiring another format to remedy the situation.

When the block size (i.e. the number of bytes in each block) is changed on a disk two SCSI commands must be sent: a MODE SELECT to change the block size followed by a FORMAT command. If the MODE SELECT command succeeds and the FORMAT fails then the disk may be in a state that the draft standard calls "format corrupt". A block descriptor in a subsequent MODE SENSE will report the requested new block size while a READ CAPACITY command will report the existing (i.e. different) block size. Alternatively the READ CAPACITY command may fail, reporting the device is not ready, potentially requiring a format. The solution to this situation is to do a format again (and this time the new block size does not have to be given) or change the block size back to the original size.

The SBC-2 standard states that the block count can be set back to the manufacturer's maximum recommended value in a format or resize operation. This can be done by placing an address of 0xffffffff (or the 64 bit equivalent) in the appropriate block descriptor field to a MODE SELECT command. In signed (two's complement) arithmetic that value corresponds to '-1'. So a --count=-1 causes the block count to be set back to the manufacturer's maximum recommended value. To see exactly which SCSI commands are being executed and parameters passed add "-vvv" to the sg_format command line.

Short stroking is a technique to trade off capacity for performance. Disk performance is usually highest on the outer tracks (i.e. lower logical block addresses) so by resizing or reformatting a disk to a smaller capacity, average performance will usually be increased.

Other utilities may be useful in finding information associated with formatting. These include sg_inq(8) to fetch standard INQUIRY information (e.g. the PROTECT bit) and to fetch the extended INQUIRY VPD page (e.g. RTO and GRD_CHK bits). The sdparm(8) utility can be used to access and potentially change the now obsolete format mode page.

scsiformat is another utility available for formatting SCSI disks with linux. It dates from 1997 (most recent update) and may be useful for disks whose firmware is of that vintage.

The COUNT value is a number which may be followed by one of these multiplicative suffixes: c C *1; w W *2; b B *512; k K KiB *1,024; KB *1,000; m M MiB *1,048,576; MB *1,000,000 . This pattern continues for "G", "T" and "P". Also a suffix of the form "x<n>" multiplies the leading number by <n>. Alternatively numerical values can be given in hexadecimal preceded by either "0x" or "0X" (or with a trailing "h" or "H"). When hex numbers are given, multipliers cannot be used.

EXAMPLES
In the first example below simply find out the existing block count and size derived from two sources: a block descriptor in a MODE SELECT command response and from the response of a READ CAPACITY commands. No changes are made:

   sg_format /dev/sdm

Now a simple format, leaving the block count and size as they were previously. The FORMAT command is executed in IMMED mode and the device is polled every 30 seconds to print out a progress indication:

   sg_format --format /dev/sdm

Now the same format, but waiting (passively) until the format operation is complete:

   sg_format --format --wait /dev/sdm

Next is a format in which the block size is changed to 520 bytes and the block count is set to the manufacturer's maximum value (for that block size). Note, not all disks support changing the block size:

   sg_format --format --size=520 /dev/sdm

Now a resize operation so that only the first 0x10000 (65536) blocks on a disk are accessible. The remaining blocks remain unaltered.

   sg_format --resize --count=0x10000 /dev/sdm

Now resize the disk back to its normal (maximum) block count:

   sg_format --resize --count=-1 /dev/sdm

EXIT STATUS
The exit status of sg_format is 0 when it is successful. Otherwise see the sg3_utils(8) man page. Unless the --wait option is given, the exit status may not reflect the success of otherwise of the format. Using sg_turs(8) and sg_readcap(8) after the format operation may be wise.

AUTHORS
Written by Grant Grundler, James Bottomley and Douglas Gilbert.

REPORTING BUGS
Report bugs to <dgilbert at interlog dot com>.

COPYRIGHT
Copyright © 2005-2007 Grant Grundler, James Bottomley and Douglas Gilbert
This software is distributed under the GPL version 2. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

SEE ALSO
sg_turs(8), sg_requests(8), sg_inq(8), sg_modes(8), sg_vpd(8) sg_reassign(8) [all in sg3_utils], sdparm(8), scsiformat (old)