Knowledge (XXG)

Native Command Queuing

Source 📝

17: 215:
One lesser-known feature of NCQ is that, unlike its ATA TCQ predecessor, it allows the host to specify whether it wants to be notified when the data reaches the disk's platters, or when it reaches the disk's buffer (on-board cache). Assuming a correct hardware implementation, this feature allows data
178:
while it is serving other (possibly new) requests under I/O pressure. Since the algorithms used inside drives' firmware for NCQ dispatch ordering are generally not publicly known, this introduces another level of uncertainty for hardware/firmware performance. Tests at Google around 2008 have shown
44:
to internally optimize the order in which received read and write commands are executed. This can reduce the amount of unnecessary drive head movement, resulting in increased performance (and slightly decreased wear of the drive) for workloads where multiple simultaneous read/write requests are
158:
A 2004 test with the first-generation NCQ drive (Seagate 7200.7 NCQ) found that while NCQ increased IOMeter performance, desktop application performance decreased. One review in 2010 found improvements on the order of 9% (on average) with NCQ enabled in a series of Windows multitasking tests.
84:
engine. NCQ's implementation is preferable because the drive has more accurate knowledge of its performance characteristics and is able to account for its rotational position. Both NCQ and TCQ have a maximum queue length of 32 outstanding commands. Because the ATA TCQ is rarely used,
250:(NVMe) standard also supports command queuing, in a form optimized for SSDs. NVMe allows multiple queues for a single controller and device, allowing at the same time much higher depths for each queue, which more closely matches how the underlying SSD hardware works. 144:
controller) even if AHCI is present on the host bus adapter, which makes initial setup more tedious and conversions of existing installations relatively difficult as most controllers cannot operate their ports in mixed AHCI–SATA/IDE/legacy mode.
92:
For NCQ to be enabled, it must be supported and enabled in the SATA host bus adapter and in the hard drive itself. The appropriate driver must be loaded into the operating system to enable NCQ on the host bus adapter.
179:
that NCQ can delay an I/O for up to 1–2 seconds. A proposed workaround is for the operating system to artificially starve the NCQ queue sooner in order to satisfy low-latency applications in a timely manner.
20:
NCQ allows the drive itself to determine the optimal order in which to retrieve outstanding requests. This may, as here, allow the drive to fulfill all requests in fewer rotations and thus less time.
243:
NCQ also enables the SSD controller to complete commands concurrently (or partly concurrently, for example using pipelines) where the internal organisation of the device enables such processing.
120:
also natively support AHCI, but their AHCI support (via the msahci service) must be manually enabled via registry editing if controller support was not present during their initial install.
174:. There is no mechanism in NCQ for the host to specify any sort of deadlines for an I/O, like how many times a request can be ignored in favor of others. In theory, a queued request can be 69:
bus device protocols to interact with the operating system. The resulting high CPU overhead and negligible performance gain contributed to a lack of market acceptance for ATA TCQ.
679: 626: 321:""Re: DragonFly-2.3.1.165.g25822 master sys/dev/disk/ahci Makefile TODO ahci.c ahci.h ahci_attach.c ahci_cam.c ahci_dragonfly.c ahci_dragonfly.h atascsi.h"" 282: 80:
parameters during its command sequence whereas TCQ interrupts the CPU during command queries and requires it to modulate the ATA host bus adapter's
240:'s X25-E Extreme solid-state drive uses NCQ to ensure that the drive has commands to process while the host system is busy processing CPU tasks. 554: 97: 72:
NCQ differs from TCQ in that, with NCQ, each command is of equal importance, but NCQ's host bus adapter also programs its own
66: 455: 476: 293: 569: 689: 201:
SSDs, the queue depth was even increased to support a maximum of 65,535 queues with up to 65,535 commands each.
444: 394: 373: 304: 512: 216:
consistency to be guaranteed when the disk's on-board cache is used in conjunction with system calls like
183: 65:(TCQ). ATA's attempt at integrating TCQ was constrained by the requirement that ATA host bus adapters use 62: 320: 197:
profit significantly from being able to queue multiple commands for parallel workloads. For PCIe-based
81: 77: 73: 708: 683: 175: 435: 46: 479: 236:
where the drive encounters latency on the host, rather than the other way around. For example,
693: 550: 546: 233: 221: 194: 538: 427: 125: 89:(and the IDE mode of some chipsets) usually only support one outstanding command per port. 41: 653: 334: 316: 259: 702: 539: 346: 264: 163: 113: 101: 439: 100:(AHCI), which allows operating systems to universally control them and enable NCQ. 608:"Intel's X25-E Extreme solid-state drive - Now with single-level cell flash memory" 105: 86: 58: 611: 247: 210: 187: 570:
http://www.seagate.com/docs/pdf/whitepaper/D2c_tech_paper_intc-stx_sata_ncq.pdf
418:
Yu, Y. J.; Shin, D. I.; Eom, H.; Yeom, H. Y. (2010). "NCQ vs. I/O scheduler".
136:
require the installation of a vendor-specific driver (similar to installing a
133: 37: 516: 190:
is disabled when NCQ is enabled, resulting in slower sequential performance.
431: 121: 117: 25: 607: 132:
drives (with their supporting firmware). Older operating systems such as
305:"SATA II Native Command Queuing Overview", Intel Whitepaper, April 2003. 220:. The associated write flag, which is also borrowed from SCSI, is called 338: 633: 586: 374:"Seagate's Barracuda 7200.7 NCQ hard drive - The Tech Report - Page 13" 109: 491: 395:"Multitasking with Native Command Queuing - The Tech Report - Page 5" 171: 582: 237: 217: 167: 15: 627:"AHCI and NVMe as Interfaces for SATA Express Devices – Overview" 198: 166:, decreasing performance; this has been observed in practice on 141: 137: 456:"hard drive - Poor Linux software RAID 5 performance with NCQ" 129: 16: 339:"ahci(4) — Advanced Host Controller Interface for Serial ATA" 354:"ahci - Advanced Host Controller Interface for Serial ATA". 355: 517:"Disks from the Perspective of a File System - ACM Queue" 492:"Mark Lord: Re: Lower HD transfer rate with NCQ enabled?" 162:
NCQ can negatively interfere with the operating system's
294:Volume 1 of the final draft of the ATA-7 standard 680:Serial ATA and the 7 Deadly Sins of Parallel ATA 176:delayed by the drive an arbitrary amount of time 108:support AHCI natively since version 2.6.19, and 104:has supported AHCI with NCQ since 2.3 in 2009. 283:PDF white paper on NCQ from Intel and Seagate 8: 211:Disk buffer § Force Unit Access (FUA) 475:Gwendal Grignou, NCQ Emulation, FLS'08 413: 411: 275: 112:fully supports AHCI since version 8.0. 57:Native Command Queuing was preceded by 182:On some drives' firmware, such as the 124:'s AHCI enables not only NCQ but also 45:outstanding, most often occurring in 7: 606:Gasior, Geoff (November 23, 2008). 376:. The Tech Report. 17 December 2004 98:Advanced Host Controller Interface 14: 545:. Packt Publishing Ltd. p.  541:PostgreSQL 9.0: High Performance 397:. The Tech Report. 3 August 2005 96:Many newer chipsets support the 581:Jonathan Corbet (2010-08-18). 357:DragonFly On-Line Manual Pages 1: 625:Dave Landsman (2013-08-09). 583:"The end of block barriers" 420:ACM Transactions on Storage 725: 232:NCQ is also used in newer 208: 36:) is an extension of the 432:10.1145/1714454.1714456 654:"NVM Express Overview" 537:Gregory Smith (2010). 513:Marshall Kirk McKusick 76:engine with CPU-given 63:Tagged Command Queuing 30:Native Command Queuing 21: 477:talk summary (p. 109) 19: 49:-type applications. 343:BSD Cross Reference 494:. LKML. 2007-04-03 234:solid-state drives 228:Solid-state drives 195:solid-state drives 40:protocol allowing 22: 556:978-1-84951-031-8 222:Force Unit Access 716: 690:Hype vs. Reality 668: 667: 665: 664: 650: 644: 643: 641: 640: 631: 622: 616: 615: 603: 597: 596: 594: 593: 578: 572: 567: 561: 560: 544: 534: 528: 527: 525: 524: 509: 503: 502: 500: 499: 488: 482: 473: 467: 466: 464: 463: 452: 446: 443: 415: 406: 405: 403: 402: 391: 385: 384: 382: 381: 370: 364: 361: 350: 331: 325: 324: 313: 307: 302: 296: 291: 285: 280: 149:Hard disk drives 42:hard disk drives 724: 723: 719: 718: 717: 715: 714: 713: 699: 698: 676: 671: 662: 660: 652: 651: 647: 638: 636: 629: 624: 623: 619: 605: 604: 600: 591: 589: 580: 579: 575: 568: 564: 557: 536: 535: 531: 522: 520: 519:. Queue.acm.org 511: 510: 506: 497: 495: 490: 489: 485: 474: 470: 461: 459: 454: 453: 449: 417: 416: 409: 400: 398: 393: 392: 388: 379: 377: 372: 371: 367: 353: 333: 332: 328: 315: 314: 310: 303: 299: 292: 288: 281: 277: 273: 256: 230: 213: 207: 156: 151: 82:third party DMA 74:first party DMA 55: 12: 11: 5: 722: 720: 712: 711: 701: 700: 697: 696: 694:Tom's Hardware 687: 675: 674:External links 672: 670: 669: 658:nvmexpress.org 645: 617: 598: 573: 562: 555: 529: 504: 483: 468: 458:. Server Fault 447: 407: 386: 365: 363: 362: 335:Matthew Dillon 326: 319:(2009-06-04). 317:Matthew Dillon 308: 297: 286: 274: 272: 269: 268: 267: 262: 260:I/O scheduling 255: 252: 229: 226: 206: 203: 155: 152: 150: 147: 61:'s version of 54: 51: 13: 10: 9: 6: 4: 3: 2: 721: 710: 707: 706: 704: 695: 691: 688: 685: 684:Lost Circuits 681: 678: 677: 673: 659: 655: 649: 646: 635: 628: 621: 618: 613: 609: 602: 599: 588: 584: 577: 574: 571: 566: 563: 558: 552: 548: 543: 542: 533: 530: 518: 514: 508: 505: 493: 487: 484: 481: 478: 472: 469: 457: 451: 448: 445: 441: 437: 433: 429: 425: 421: 414: 412: 408: 396: 390: 387: 375: 369: 366: 359: 358: 352: 351: 348: 347:DragonFly BSD 344: 340: 336: 330: 327: 322: 318: 312: 309: 306: 301: 298: 295: 290: 287: 284: 279: 276: 270: 266: 265:Write barrier 263: 261: 258: 257: 253: 251: 249: 244: 241: 239: 235: 227: 225: 223: 219: 212: 204: 202: 200: 196: 191: 189: 185: 180: 177: 173: 169: 165: 164:I/O scheduler 160: 153: 148: 146: 143: 139: 135: 131: 127: 123: 119: 115: 114:Windows Vista 111: 107: 106:Linux kernels 103: 102:DragonFly BSD 99: 94: 90: 88: 83: 79: 75: 70: 68: 64: 60: 52: 50: 48: 43: 39: 35: 31: 27: 18: 692:analysis at 661:. Retrieved 657: 648: 637:. Retrieved 620: 601: 590:. Retrieved 576: 565: 540: 532: 521:. Retrieved 507: 496:. Retrieved 486: 471: 460:. Retrieved 450: 423: 419: 399:. Retrieved 389: 378:. Retrieved 368: 356: 342: 329: 311: 300: 289: 278: 245: 242: 231: 214: 205:Safety (FUA) 192: 186:circa 2007, 181: 161: 157: 95: 91: 87:Parallel ATA 71: 59:Parallel ATA 56: 33: 29: 23: 612:Tech Report 248:NVM Express 154:Performance 128:support on 709:Serial ATA 663:2014-11-26 639:2013-10-02 592:2015-06-27 523:2014-01-11 498:2014-01-11 462:2014-01-11 401:2014-01-11 380:2014-01-11 271:References 209:See also: 188:read-ahead 134:Windows XP 38:Serial ATA 686:(for TCQ) 184:WD Raptor 122:Windows 7 118:Windows 7 26:computing 703:Category 440:14414608 426:: 1–37. 337:(2009). 254:See also 634:SATA-IO 587:LWN.net 224:(FUA). 110:FreeBSD 53:History 553:  480:slides 438:  172:RAID-5 47:server 630:(PDF) 436:S2CID 238:Intel 218:fsync 193:SATA 170:with 168:Linux 551:ISBN 246:The 199:NVMe 142:SCSI 138:RAID 126:TRIM 116:and 682:at 428:doi 140:or 130:SSD 78:DMA 67:ISA 34:NCQ 24:In 705:: 656:. 632:. 610:. 585:. 549:. 547:78 515:. 434:. 422:. 410:^ 345:. 341:. 28:, 666:. 642:. 614:. 595:. 559:. 526:. 501:. 465:. 442:. 430:: 424:6 404:. 383:. 360:. 349:. 323:. 32:(

Index


computing
Serial ATA
hard disk drives
server
Parallel ATA
Tagged Command Queuing
ISA
first party DMA
DMA
third party DMA
Parallel ATA
Advanced Host Controller Interface
DragonFly BSD
Linux kernels
FreeBSD
Windows Vista
Windows 7
Windows 7
TRIM
SSD
Windows XP
RAID
SCSI
I/O scheduler
Linux
RAID-5
delayed by the drive an arbitrary amount of time
WD Raptor
read-ahead

Text is available under the Creative Commons Attribution-ShareAlike License. Additional terms may apply.