Lustre (file system)

1402:

aggregate I/O bandwidth scale with the number of OSTs a file is striped over. Also, since the locking of each object is managed independently for each OST, adding more stripes (one per OST) scales the file I/O locking capacity of the file proportionately. Each file created in the filesystem may specify different layout parameters, such as the stripe count (number of OST objects making up that file), stripe size (unit of data stored on each OST before moving to the next), and OST selection, so that performance and capacity can be tuned optimally for each file. When many application threads are reading or writing to separate files in parallel, it is optimal to have a single stripe per file, since the application is providing its own parallelism. When there are many threads reading or writing a single large file concurrently, then it is optimal to have at least one stripe on each OST to maximize the performance and capacity of that file.

1523:, the clients will request locks that are always an integer multiple of the page size (4096 bytes on most clients). When a client is requesting an extent lock the OST may grant a lock for a larger extent than originally requested, in order to reduce the number of lock requests that the client makes. The actual size of the granted lock depends on several factors, including the number of currently granted locks on that object, whether there are conflicting write locks for the requested lock extent, and the number of pending lock requests on that object. The granted lock is never smaller than the originally requested extent. OST extent locks use the Lustre FID of the object as the resource name for the lock. Since the number of extent lock servers scales with the number of OSTs in the filesystem, this also scales the aggregate locking performance of the filesystem, and of a single file if it is striped over multiple OSTs. 1864: 992:(PFL) is the first feature to use composite layouts, but the implementation is flexible for use with other file layouts such as mirroring and erasure coding. The NRS Token Bucket Filter (TBF) server-side scheduler has implemented new rule types, including RPC-type scheduling and the ability to specify multiple parameters such as JobID and NID for rule matching. Tools for managing ZFS snapshots of Lustre filesystems have been added, to simplify the creation, mounting, and management of MDT and OST ZFS snapshots as separate Lustre 4535: 216: 142: 1079:

RDMA router nodes. The PFL functionality was enhanced with Self-Extending Layouts (SEL) to allow file components to be dynamically sized, to better deal with flash OSTs that may be much smaller than disk OSTs within the same filesystem. The release also included a number of smaller improvements, such as balancing DNE remote directory creation across MDTs, using Lazy-size-on-MDT to reduce the overhead of "lfs find", directories with 10M files per shard for ldiskfs, and bulk RPC sizes up to 64 MB.

1139:, where the metadata server controls all of the block allocation, the Lustre metadata server is only involved in pathname and permission checks, and is not involved in any file I/O operations, avoiding I/O scalability bottlenecks on the metadata server. The ability to have multiple MDTs in a single filesystem is a new feature in Lustre 2.4, and allows directory subtrees to reside on the secondary MDTs, while 2.7 and later allow large single directories to be distributed across multiple MDTs as well. 1364:

launched is not a Linux client. Liblustre allowed data movement directly between application space and the Lustre OSSs without requiring an intervening data copy through the kernel, thus providing access from computational processors to the Lustre file system directly in a constrained operating environment. The liblustre functionality was deleted from Lustre 2.7.0 after having been disabled since Lustre 2.6.0, and was untested since Lustre 2.3.0.

1389:

on the OSTs. When a client opens a file, the file open operation transfers a set of object identifiers and their layout from the MDS to the client, so that the client can directly interact with the OSS node(s) that hold the object(s). This allows the client(s) to perform I/O in parallel across all of the OST objects in the file without further communication with the MDS, avoiding contention from centralized block and lock management.

1413:(PFL) feature uses composite layouts to improve file IO performance over a wider range of workloads, as well as simplify usage and administration. For example, a small PFL file can have a single stripe on flash for low access overhead, while larger files can have many stripes for high aggregate bandwidth and better OST load balancing. The composite layouts are further enhanced in the 2.11 release with the 6418: 46: 1475:

will automatically be created as a remote directory one of the available MDTs (selected in sequential order) to balance space usage and load across servers. If the free space on the MDTs becomes imbalanced (more than 5% difference in free space and inodes) then the clients will bias new subdirectory creation toward MDTs with more free space in order to restore balance.

742:(LFSCK), which allows verification of the distributed filesystem state between data and metadata servers while the filesystem is mounted and in use; and Distributed Namespace Environment (DNE), formerly Clustered Metadata (CMD), which allows the Lustre metadata to be distributed across multiple servers. Development also continued on ZFS-based back-end 1878: 1284:. When the MDS filename lookup is complete and the user and client have permission to access and/or create the file, either the layout of an existing file is returned to the client or a new file is created on behalf of the client, if requested. For read or write operations, the client then interprets the file layout in the 1638:

by a disk cache. Once a file is archived, it can be released from the main filesystem, leaving only a stub that references the archive copy. If a released file is opened, the Coordinator blocks the open, sends a restore request to a copytool, and then completes the open once the copytool has completed restoring the file.

921:(HSM). A core requirement in enterprise environments, HSM allows customers to easily implement tiered storage solutions in their operational environment. This release is the current OpenSFS-designated Maintenance Release branch of Lustre. The most recent maintenance version is 2.5.3 and was released in September 2014. 750:. These features were in the Lustre 2.2 through 2.4 community release roadmap. In November 2011, a separate contract was awarded to Whamcloud for the maintenance of the Lustre 2.x source code to ensure that the Lustre code would receive sufficient testing and bug fixing while new features were being developed. 1624:

Lustre file system high availability features include a robust failover and recovery mechanism, making server failures and reboots transparent. Version interoperability between successive minor versions of the Lustre software enables a server to be upgraded by taking it offline (or failing it over to

1429:

component of a PFL file to be stored directly on the MDT with the inode. This reduces overhead for accessing small files, both in terms of space usage (no OST object is needed) as well as network usage (fewer RPCs needed to access the data). DoM also improves performance for small files if the MDT is

1165:

The MDT, OST, and client may be on the same node (usually for testing purposes), but in typical production installations these devices are on separate nodes communicating over a network. Each MDT and OST may be part of only a single filesystem, though it is possible to have multiple MDTs or OSTs on a

1050:

to allow the LNet Multi-Rail feature from Lustre 2.10 to better handle network faults when a node has multiple network interfaces. The Lazy Size on MDT (LSOM) feature allows storing an estimate of the file size on the MDT for use by policy engines, filesystem scanners, and other management tools that

964:

shared filesystem without having a single set of UID/GIDs for all client nodes. The subdirectory mount feature allows clients to mount a subset of the filesystem namespace from the MDS. This release also added support for up to 16 MiB RPCs for more efficient I/O submission to disk, and added the

928:

Lustre 2.7, released in March 2015, added LFSCK functionality to verify DNE consistency of remote and striped directories between multiple MDTs. Dynamic LNet Config adds the ability to configure and modify LNet network interfaces, routes, and routers at runtime. A new evaluation feature was added for

924:

Lustre 2.6, released in July 2014, was a more modest release feature wise, adding LFSCK functionality to do local consistency checks on the OST as well as consistency checks between MDT and OST objects. The NRS Token Bucket Filter (TBF) policy was added. Single-client IO performance was improved over

888:

Lustre 2.2, released in March 2012, focused on providing metadata performance improvements and new features. It added parallel directory operations allowing multiple clients to traverse and modify a single large directory concurrently, faster recovery from server failures, increased stripe counts for

865:

Lustre 1.8.0, released in May 2009, provided OSS Read Cache, improved recovery in the face of multiple failures, added basic heterogeneous storage management via OST Pools, adaptive network timeouts, and version-based recovery. It was a transition release, being interoperable with both Lustre 1.6 and

1675:

Policy Engine: watches filesystem Changelogs for new files to archive, applies policies to release files based on age or space usage, and communicates with MDT and Coordinator. The Policy Engine can also trigger actions like migration between, purge, and removal. The most commonly used policy engine

1637:

Lustre provides the capability to have multiple storage tiers within a single filesystem namespace. It allows traditional HSM functionality to copy (archive) files off the primary filesystem to a secondary archive storage tier. The archive tier is typically a tape-based system, that is often fronted

1518:

locks. Clients can be granted overlapping read extent locks for part or all of the file, allowing multiple concurrent readers of the same file, and/or non-overlapping write extent locks for independent regions of the file. This allows many Lustre clients to access a single file concurrently for both

1474:

In Lustre 2.15, the client LMV implements round-robin and space balanced default directory layouts, so that clients can use a large number of MDTs in a single filesystem more effectively. When a new subdirectories is created near the root of the filesystem (the top 3 directory levels by default), it

1078:

storage on the client nodes while keeping the files part of the global filesystem namespace, and OST Overstriping which allows files to store multiple stripes on a single OST to better utilize fast OSS hardware. As well, the LNet Multi-Rail Network Health functionality was improved to work with LNet

1641:

In addition to external storage tiering, it is possible to have multiple storage tiers within a single filesystem namespace. OSTs of different types (e.g. HDD and SSD) can be declared in named storage pools. The OST pools can be selected when specifying file layouts, and different pools can be used

1388:

data structure contains basic information about each file, such as where the data contained in the file is stored. The Lustre file system also uses inodes, but inodes on MDTs point to one or more OST objects associated with the file rather than to data blocks. These objects are implemented as files

1022:

storage used on OSTs. As well, the LNet Dynamic Discovery feature allows auto-configuration of LNet Multi-Rail between peers that share an LNet network. The LDLM Lock Ahead feature allows appropriately modified applications and libraries to pre-fetch DLM extent locks from the OSTs for files, if the

963:

mechanism as Kerberos to provide client and server node authentication, and RPC message integrity and security (encryption). The Nodemap feature allows categorizing client nodes into groups and then mapping the UID/GID for those clients, allowing remotely administered clients to transparently use a

1628:

Lustre MDSes are configured as an active/passive pair exporting a single MDT, or one or more active/active MDS pairs with DNE exporting two or more separate MDTs, while OSSes are typically deployed in an active/active configuration exporting separate OSTs to provide redundancy without extra system

1470:

operations, the entries from each directory shard are returned to the client sorted in the local MDT directory hash order, and the client performs a merge sort to interleave the filenames in hash order so that a single 64-bit cookie can be used to determine the current offset within the directory.

1401:

in chunks typically 1 MB or larger. Striping a file over multiple OST objects provides significant performance benefits if there is a need for high bandwidth access to a single large file. When striping is used, the maximum file size is not limited by the size of a single target. Capacity and

1086:

Lustre 2.15 was released on June 16, 2022 and includes three main features. Client Directory Encryption expands on the fscrypt data encryption in the 2.14 release to also allow file and directory names to be encrypted on the client before network transfer and persistent storage on the MDT. DNE MDT

913:

can now be used as the backing filesystem for both MDT and OST storage. The LFSCK feature added the ability to scan and verify the internal consistency of the MDT FID and LinkEA attributes. The Network Request Scheduler (NRS) adds policies to optimize client request processing for disk ordering or

1642:

within a single PFL file layout. Files can be migrated between storage tiers either manually or under control of the Policy Engine. Since Lustre 2.11, it is also possible to mirror a file to different OST pools with a FLR file layout, for example to pre-stage files into flash for a computing job.

1300:

After the initial lookup of the file layout, the MDS is not normally involved in file IO operations since all block allocation and data IO is managed internally by the OST. Clients do not directly modify the objects or data on the OST filesystems, but instead delegate this task to OSS nodes. This

1296:

being operated on and executes one or more parallel read or write operations directly to the OSS nodes that hold the data objects. With this approach, bottlenecks for client-to-OSS communications are eliminated, so the total bandwidth available for the clients to read and write data scales almost

897:

as the backing file system. The Lustre File System ChecK (LFSCK) feature can verify and repair the MDS Object Index (OI) while the file system is in use, after a file-level backup/restore or in case of MDS corruption. The server-side IO statistics were enhanced to allow integration with batch job

741:

awarded a contract for Lustre feature development to Whamcloud. This contract covered the completion of features, including improved Single Server Metadata Performance scaling, which allows Lustre to better take advantage of many-core metadata server; online Lustre distributed filesystem checking

1450:

When a client initially mounts a filesystem, it is provided the 128-bit Lustre File Identifier (FID, composed of the 64-bit Sequence number, 32-bit Object ID, and 32-bit Version) of the root directory for the mountpoint. When doing a filename lookup, the client performs a lookup of each pathname

1363:

applications with direct filesystem access. Liblustre was a user-level library that allows computational processors to mount and use the Lustre file system as a client. Using liblustre, the computational processors could access a Lustre file system even if the service node on which the job was

1082:

Lustre 2.14 was released on February 19, 2021 and includes three main features. Client Data Encryption implements fscrypt to allow file data to be encrypted on the client before network transfer and persistent storage on the OST and MDT. OST Pool Quotas extends the quota framework to allow the

1321:

In a typical Lustre installation on a Linux client, a Lustre filesystem driver module is loaded into the kernel and the filesystem is mounted like any other local or network filesystem. Client applications see a single, unified filesystem even though it may be composed of tens to thousands of

968:

interface to allow clients to provide I/O hints to the servers to prefetch file data into server cache or flush file data from server cache. There was improved support for specifying filesystem-wide default OST pools, and improved inheritance of OST pools in conjunction with other file layout

988:) on a client and server to increase aggregate I/O bandwidth. Individual files can use composite file layouts that are constructed of multiple components, which are file regions based on the file offset, that allow different layout parameters such as stripe count, OST pool/storage type, etc. 1233:

storage instead of hardware RAID devices. The Lustre OSS and MDS servers read, write, and modify data in the format imposed by the backing filesystem and return this data to the clients. This allows Lustre to take advantage of improvements and features in the underlying filesystem, such as

725:

In December 2010, Oracle announced that they would cease Lustre 2.x development and place Lustre 1.8 into maintenance-only support, creating uncertainty around the future development of the file system. Following this announcement, several new organizations sprang up to provide support and

1451:

component by mapping the parent directory FID Sequence number to a specific MDT via the FID Location Database (FLDB), and then does a lookup on the MDS managing this MDT using the parent FID and filename. The MDS will return the FID for the requested pathname component along with a

1210:, Cray SeaStar and RapidArray networks were also supported, but these network drivers were deprecated when these networks were no longer commercially available, and support was removed completely in Lustre 2.8. Lustre will take advantage of remote direct memory access ( 1604:

of two or more network interfaces between a client and server to improve bandwidth. The LNet interface types do not need to be the same network type. In 2.12 Multi-Rail was enhanced to improve fault tolerance if multiple network interfaces are available between peers.

1083:

assignment and enforcement of quotas on the basis of OST storage pools. DNE Auto Restriping can now adjust how many MDTs a large directory is striped over based on size thresholds defined by the administrator, similar to Progressive File Layouts for directories.

1655:

Copytool: handles data motion and metadata updates. There are different copytools to interface with different archive systems. A generic POSIX copytool is available for archives that provide a POSIX-like front-end interface. Copytools are also available for the

1507:(xattrs, since Lustre 2.5). A client can fetch multiple metadata lock bits for a single inode with a single RPC request, but currently they are only ever granted a read lock for the inode. The MDS manages all modifications to the inode in order to avoid lock 1087:

space balancing automatically balances new directory creation across MDTs in the filesystem in round-robin and/or based on available inodes and space, which in turn helps distribute client metadata workload over MDTs more evenly. For applications using the

1367:

In Linux Kernel version 4.18, the incomplete port of the Lustre client was removed from the kernel staging area in order to speed up development and porting to newer kernels. The out-of-tree Lustre client and server is still available for RHEL, SLES, and

1130:

devices per Lustre filesystem that stores namespace metadata, such as filenames, directories, access permissions, and file layout. The MDT data is stored in a local disk filesystem. However, unlike block-based distributed filesystems, such as

1030:

Lustre 2.12 was released on December 21, 2018 and focused on improving Lustre usability and stability, with improvements the performance and functionality of the FLR and DoM features added in Lustre 2.11, as well as smaller changes to NRS

1150:

devices. Depending on the server's hardware, an OSS typically serves between two and eight OSTs, with each OST managing a single local disk filesystem. The capacity of a Lustre file system is the sum of the capacities provided by the

877:. However, Lustre 1.8.6 and later clients can interoperate with Lustre 2.0 and later servers. The Metadata Target (MDT) and OST on-disk format from 1.8 can be upgraded to 2.0 and later without the need to reformat the filesystem. 884:

6 and increased the maximum ext4-based OST size from 24 TB to 128 TB, as well as a number of performance and stability improvements. Lustre 2.1 servers remained inter-operable with 1.8.6 and later clients.

892:

Lustre 2.3, released in October 2012, continued to improve the metadata server code to remove internal locking bottlenecks on nodes with many CPU cores (over 16). The object store added a preliminary ability to use

2454:

Wang, Teng; Byna, Suren; Lockwood, Glenn K.; Snyder, Shane; Carns, Philip; Kim, Sunggon; Wright, Nicholas J. (May 2019). "A Zoom-in Analysis of I/O Logs to Detect Root Causes of I/O Performance Bottlenecks".

1392:

If only one OST object is associated with an MDT inode, that object contains all the data in the Lustre file. When more than one object is associated with a file, data in the file is "striped" in chunks in a

909:. Distributed Namespace Environment (DNE) allows horizontal metadata capacity and performance scaling for 2.4 clients, by allowing subdirectory trees of a single namespace to be located on separate MDTs. 3196: 1051:

can more efficiently make decisions about files without a fully accurate file sizes or blocks count without having to query the OSTs for this information. This release also added the ability to manually

6908: 1234:

compression and data checksums in ZFS. Clients do not have any direct access to the underlying storage, which ensures that a malfunctioning or malicious client cannot corrupt the filesystem structure.

1455:

lock. Once the MDT of the last directory in the path is determined, further directory operations (for non-striped directories) will normally take place on that MDT, avoiding contention between MDTs.

1055:

an existing directory across multiple MDTs, to allow migration of directories with large numbers of files to use the capacity and performance of several MDS nodes. The Lustre RPC data checksum added

1719:

supercomputers and large multi-cluster sites. Six of the top 10 and more than 60 of the top 100 supercomputers use Lustre file systems. These include the 700PB 13 TB/s Orion filesystem for the

1535: 914:

fairness. Clients can optionally send bulk RPCs up to 4 MB in size. Client-side software was updated to work with Linux kernels up to version 3.6, and is still interoperable with 1.8 clients.

1329:(MPP) installations, computational processors can access a Lustre file system by redirecting their I/O requests to a dedicated I/O node configured as a Lustre client. This approach is used in the 4457: 2798: 4392: 1458:

For DNE striped directories, the per-directory layout stored on the parent directory provides a hash function and a list of MDT directory FIDs across which the directory is distributed. The

925:

the previous releases. This release also added a preview of DNE striped directories, allowing single large directories to be stored on multiple MDTs to improve performance and scalability.

2546:

Saini, Subhash; Rappleye, Jason; Chang, Johnny; Barker, David; Mehrotra, Piyush; Biswas, Rupak (Dec 2012). "I/O performance characterization of Lustre and NASA applications on Pleiades".

1629:

overhead. In single-MDT filesystems, the standby MDS for one filesystem is the MGS and/or monitoring node, or the active MDS for another file system, so no nodes are idle in the cluster.

2880: 788:

also established the Lustre Community Portal, a technical site that provides a collection of information and documentation in one area for reference and guidance to support the Lustre

880:

Lustre 2.1, released in September 2011, was a community-wide initiative in response to Oracle suspending development on Lustre 2.x releases. It added the ability to run servers on

5988: 1499:

for the file, using FID as the resource name. The metadata locks are split into separate bits that protect the lookup of the file (file owner and group, permission and mode, and

466:

32bitapi, acl, checksum, flock, lazystatfs, localflock, lruresize, noacl, nochecksum, noflock, nolazystatfs, nolruresize, nouser_fid2path, nouser_xattr, user_fid2path, user_xattr

2618: 1791: 1442:

feature allows the component size to be dynamic during write so that it can cope with individual (flash) OSTs running out of space before the whole filesystem is out of space.

4129: 3353: 1301:

approach ensures scalability for large-scale clusters and supercomputers, as well as improved security and reliability. In contrast, shared block-based filesystems such as

1625:

a standby server), performing the upgrade, and restarting it, while all active jobs continue to run, experiencing a delay while the backup server takes over the storage.

586:

clusters to large-scale, multi-site systems. Since June 2005, Lustre has consistently been used by at least half of the top ten, and more than 60 of the top 100 fastest

3326: 2440: 2406: 2339: 854:

Lustre 1.6.0, released in April 2007, allowed mount configuration (“mountconf”) allowing servers to be configured with "mkfs" and "mount", allowed dynamic addition of

730:, EUROPEAN Open File Systems (EOFS) and others. By the end of 2010, most Lustre developers had left Oracle. Braam and several associates joined the hardware-oriented 2726:"Sun Microsystems Expands High Performance Computing Portfolio with Definitive Agreement to Acquire Assets of Cluster File Systems, Including the Lustre File System" 959:

Lustre 2.9 was released in December 2016 and included a number of features related to security and performance. The Shared Secret Key security flavour uses the same

3200: 1612:

networks in excess of 100 MB/s, throughput up to 11 GB/s using InfiniBand enhanced data rate (EDR) links, and throughput over 11 GB/s across

6680: 6923: 5970: 2941: 1772: 681: 4461: 1811: 940:

Lustre 2.8, released in March 2016, finished the DNE striped directory feature, including support for migrating directories between MDTs, and cross-MDT

734:

when it acquired the assets of ClusterStor, while Barton, Dilger, and others formed software startup Whamcloud, where they continued to work on Lustre.

2806: 1852:

offers Amazon FSx for Lustre, a fully managed service, making it easy to launch and run high-performance file systems cost effectively in their cloud.

869:

Lustre 2.0, released in August 2010, was based on significant internally restructured code to prepare for major architectural advancements. Lustre 2.x

1822:

Commercial technical support for Lustre is often bundled along with the computing system or storage hardware sold by the vendor. Some vendors include

232: 2888: 4396: 3402: 1590: 1070:

Lustre 2.13 was released on December 5, 2019 and added a new performance-related features Persistent Client Cache (PCC), which allows direct use of

840:

2.6, and had a "size glimpse" feature to avoid lock revocation on files undergoing write, and client side data write-back cache accounting (grant).

1157:

that access and use the data. Lustre presents all clients with a unified namespace for all of the files and data in the filesystem, using standard

3523: 2633: 4679: 4654: 3450: 3043: 2644: 1784: 1334: 827: 747: 4265: 2576: 2474: 1795: 3263: 2725: 1869: 1503:(ACL)), the state of the inode (directory size, directory contents, link count, timestamps), layout (file striping, since Lustre 2.4), and 3974: 2211: 2180: 1531:

The communication between the Lustre clients and servers is implemented using Lustre Networking (LNet), which was originally based on the

1253:, and tells clients the layout of the object(s) that make up each regular file. MDTs and OSTs currently use either an enhanced version of 1007:

file layouts for improved availability in case of storage or server failure and/or improved performance with highly concurrent reads. The

5911: 5386: 833:

Lustre 1.0.0 was released in December 2003, and provided basic Lustre filesystem functionality, including server failover and recovery.

4417: 4024: 6918: 6725: 5901: 5072: 4881: 3815: 2322: 811: 685: 3123: 6928: 5523: 4444:"Fujitsu Releases World's Highest-Performance File System – FEFS scalable file system software for advanced x86 HPC cluster systems" 2522: 195: 4136: 3357: 2003: 5189: 4911: 4871: 4154: 1043: 3499: 1756: 632:. This makes Lustre file systems a popular choice for businesses with large data centers, including those in industries such as 6634: 4111: 2274: 2018: 1846:, and others. It is also possible to get software-only support for Lustre file systems from some vendors, including Whamcloud. 1799: 1657: 1036: 937:

mapping for clients with different administrative domains, along with improvements to the DNE striped directory functionality.

918: 583: 2829: 2389: 6903: 5963: 3330: 2703: 1744: 1519:

read and write, avoiding bottlenecks during file I/O. In practice, because Linux clients manage their data cache in units of

1276:

kernel that connects the client to the server(s). Upon initial mount, the client is provided a File Identifier (FID) for the

385:

Per Metadata Target (MDT): 4 billion files (ldiskfs backend), 256 trillion files (ZFS backend), up to 128 MDTs per filesystem

656:, and finance. The I/O performance of Lustre has widespread impact on these applications and has attracted broad attention. 1438:

feature allows a single component to have multiple stripes on one OST to further improve parallelism of locking, while the

6913: 6715: 4968: 2770: 1807: 1724: 1573:(RDMA) is used for data and metadata transfer between nodes when provided by the underlying networks, such as InfiniBand, 1562: 1463: 1326: 1237:

An OST is a dedicated filesystem that exports an interface to byte ranges of file objects for read/write operations, with

35: 3087: 2674: 718:. In November 2008, Braam left Sun Microsystems, and Eric Barton and Andreas Dilger took control of the project. In 2010 6655: 6516: 5906: 5322: 4896: 3844: 2138: 1697:

Released: A stub inode exists on an MDT, but the data objects have been removed and the only copy exists in the archive.

1574: 1532: 1352: 1179: 117: 4290: 2972: 1645:

HSM includes some additional Lustre components to manage the interface between the primary filesystem and the archive:

1491:

style, protects the integrity of each file's data and metadata. Access and modification of a Lustre file is completely

1309:

allow direct access to the underlying storage by all of the clients in the filesystem, which requires a large back-end

6834: 6442: 5827: 5675: 5607: 4901: 4708: 4443: 3311: 1570: 1211: 1092: 503: 101: 75: 4183: 3730: 3167: 2755: 700:

acquired the assets of Cluster File Systems Inc. including its "intellectual property". Sun included Lustre with its

4616: 2257: 999:

Lustre 2.11 was released in April 2018 and contains two significant new features, and several smaller features. The

6552: 5712: 5702: 5692: 5084: 4674: 4647: 3917: 3786: 3379: 2600: 2376: 1661: 712: 701: 669: 579: 191: 6869: 803:. DDN organized the new acquisition as an independent division, reviving the Whamcloud name for the new division. 768:

In February 2013, Xyratex Ltd., announced it acquired the original Lustre trademark, logo, website and associated

672:(CMU) at the time. Braam went on to found his own company Cluster File Systems in 2001, starting from work on the 6850: 5956: 3017: 2218: 859: 4201: 4093: 3554: 6839: 6598: 6593: 6496: 6383: 5777: 5630: 5533: 5478: 5353: 5209: 4978: 3999: 2949: 1720: 1593:

BXI. High availability and recovery features enable transparent recovery in conjunction with failover servers.

1504: 1484: 1246: 1096: 673: 595: 155: 4055: 3481: 2354: 905:

Lustre 2.4, released in May 2013, added a considerable number of major features, many funded directly through

4367: 4342: 3141: 3101: 2076: 1313:

attached to all clients, and increases the risk of filesystem corruption from misbehaving/defective clients.

582:(version 2 only) and provides high performance file systems for computer clusters ranging in size from small 6491: 6398: 6165: 5876: 5832: 5814: 5513: 5503: 4958: 3500:"Demonstrating the Improvement in the Performance of a Single Lustre Client from Version 1.8 to Version 2.6" 2911: 2547: 1100: 1023:

application knows (or predicts) that this file extent will be modified in the near future, which can reduce

945: 902:

to track per-job statistics. Client-side software was updated to work with Linux kernels up to version 3.0.

641: 559: 486: 471: 179: 2855: 1466:, which will handle further operations on that file in an identical manner to a non-striped directory. For 6898: 6685: 6403: 6373: 5722: 5687: 5625: 5104: 4924: 4788: 4718: 3065: 3002: 1728: 1569:) networks, and allows simultaneous availability across multiple network types with routing between them. 1520: 1394: 599: 242: 3410: 3222: 613:

Lustre file systems are scalable and can be part of multiple computer clusters with tens of thousands of

6639: 6557: 5858: 5767: 5717: 5660: 5406: 5376: 5327: 5279: 5179: 5152: 5029: 4919: 4832: 4723: 4640: 2097: 1764: 1515: 1421:

redundancy for these files as well as improved read performance. The Lustre 2.11 release also added the

1356: 1238: 826:

Lustre file system was first installed for production use in March 2003 on the MCR Linux Cluster at the

769: 603: 31: 4608: 4219: 3949: 2498:

Gunasekaran, Raghul; Oral, Sarp; Hill, Jason; Miller, Ross; Wang, Feiyi; Leverman, Dustin (Nov 2015).

2117: 6212: 6153: 5932: 5893: 5707: 5437: 5426: 5338: 5042: 4859: 4849: 4805: 4770: 3704: 3531: 2648: 2242: 1891: 1835: 1613: 1551: 1539: 1514:

File data locks are managed by the OST on which each object of the file is striped, using byte-range

1310: 1241:

locks to protect data consistency. An MDT is a dedicated filesystem that stores inodes, directories,

1187: 1056: 1000: 843:

Lustre 1.4.0, released in November 2004, provided protocol compatibility between versions, could use

781: 772:

from Oracle. In June 2013, Intel began expanding Lustre usage beyond traditional HPC, such as within

432:

modification (mtime), attribute modification (ctime), access (atime), delete (dtime), create (crtime)

3458: 3242: 2163: 2050: 1409:

was added to allow files to have different layout parameters for different regions of the file. The

1104: 889:

a single file (across up to 2000 OSTs), and improved single-client directory traversal performance.

6486: 6454: 5937: 5883: 5822: 5411: 5079: 5020: 4936: 3978: 1849: 1839: 1508: 1500: 1269: 1250: 1161:

semantics, and allows concurrent and coherent read and write access to the files in the filesystem.

1032: 1011:(DoM) feature allows small (few MiB) files to be stored on the MDT to leverage typical flash-based 953: 800: 482: 174: 6417: 4258: 1538:. Disk storage is connected to the Lustre MDS and OSS server nodes using direct attached storage ( 231: 6603: 5927: 5789: 5752: 5665: 5391: 5381: 5366: 5307: 5137: 4800: 4793: 4780: 4733: 4237: 2987: 2733: 2582: 2528: 2480: 2307: 1221:

devices, though will work with any block devices. Since Lustre 2.4, the MDT and OST can also use

1207: 1012: 815: 758: 719: 677: 614: 515: 453: 3127: 4547: 2184: 1265:/DMU for back-end data storage to store files/objects using the open source ZFS-on-Linux port. 1107:(UDSP) allows setting interface selection policies for nodes with multiple network interfaces. 6803: 6783: 6738: 5742: 5682: 5518: 5204: 5174: 5166: 5035: 4931: 4906: 4728: 4078: 2612: 2572: 2518: 2470: 2434: 2400: 2333: 2323:"Accelerating AI at-scale with Selene DGXA100 SuperPOD and Lustre Parallel Filesystem Storage" 1760: 1431: 1417:(FLR) feature, which allows a file to have multiple overlapping layouts for a file, providing 993: 563: 271: 4603: 2421: 6818: 6788: 6768: 6562: 6327: 6202: 6074: 6069: 5979: 5762: 5670: 5468: 5118: 5062: 4751: 4482: 4421: 4032: 2564: 2556: 2510: 2462: 2289: 1609: 1601: 1344: 956:

authentication and RPC encryption over the network, and performance improvements for LFSCK.

934: 715: 697: 645: 575: 567: 542: 186: 162: 4622: 1008: 6773: 6511: 6084: 6079: 6064: 6059: 5871: 5784: 5292: 4686: 4328: 1823: 1492: 1217:

The storage used for the MDT and OST backing filesystems is normally provided by hardware

1064: 1047: 1024: 1016: 930: 689: 1694:

Dirty: The primary copy of the file has been modified and differs from the archived copy.

792:

community. On April 8, 2014, Ken Claffey announced that Xyratex/Seagate was donating the

141: 4534: 2458:

2019 19th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGRID)

972:

Lustre 2.10 was released in July 2017 and has a number of significant improvements. The

215: 45: 6695: 6437: 6388: 6284: 6195: 6190: 6148: 6054: 6049: 6044: 6039: 5772: 5757: 5697: 4875: 4746: 1748: 1652:

Agent: runs a copytool to copy data from primary storage to the archive and vice versa.

1277: 1004: 743: 1649:

Coordinator: receives archive and restore requests and dispatches them to agent nodes.

1040: 6892: 6743: 6307: 6302: 6297: 6268: 6185: 5802: 5747: 5421: 5361: 5220: 3425: 3288: 3197:"OpenSFS Announces Collaborative Effort to Support Lustre 2.1 Community Distribution" 2484: 2456: 1543: 881: 587: 525: 2586: 2532: 2500:"Comparative I/O workload characterization of two leadership class storage clusters" 1003:(FLR) feature expands on the 2.10 PFL implementation, adding the ability to specify 17: 6748: 6368: 6180: 6175: 6091: 6034: 5866: 5574: 5162: 5129: 5067: 5047: 1883: 1669: 1060: 837: 649: 547: 167: 989: 757:, after Whamcloud won the FastForward DOE contract to prepare Lustre for use with 328:

file, directory, hardlink, symlink, block special, character special, socket, FIFO

126: 3868: 3613: 2499: 1973: 1892:

List of file systems, the distributed parallel fault-tolerant file system section

664:

The Lustre file system architecture was started as a research project in 1999 by

6700: 6624: 6481: 6378: 6222: 6207: 5564: 4663: 3583: 1776: 1768: 1495:

among all of the clients. Metadata locks are managed by the MDT that stores the

1462:(LMV) on the client hashes the filename and maps it to a specific MDT directory 789: 708: 665: 633: 607: 260: 4483:"Exascaler: Massively Scalable, High Performance, Lustre File System Appliance" 814:

Lustre BOF that the Lustre trademark had been transferred to them jointly from

6813: 6572: 6536: 6501: 6459: 6263: 6258: 6253: 6227: 6121: 6116: 5842: 5640: 3044:"With New RFP, OpenSFS to Invest in Critical Open Source Technologies for HPC" 2560: 2466: 2370: 1859: 1732: 1558: 1171: 977: 844: 784:

tools, addressing Lustre technical debt, and parallel file system incubators.

653: 637: 629: 2142: 2004:"Frontier supercomputer debuts as world's fastest, breaking exascale barrier" 6705: 6588: 6531: 6358: 6160: 6101: 6096: 5797: 5538: 5416: 5147: 5015: 5010: 2568: 2514: 1780: 1740: 1677: 1665: 1582: 1360: 1330: 1175: 1015:

storage for lower latency and reduced IO contention, instead of the typical

981: 941: 917:

Lustre 2.5, released in October 2013, added the highly anticipated feature,

2607:. Cluster File Systems, Inc. Archived from the original on August 12, 2007. 1281: 780:

announced request for proposals (RFP) to cover Lustre feature development,

419:

All bytes except NUL ('\0') and '/' and the special file names "." and ".."

2799:"Xyratex Advances Lustre® Initiative, Assumes Ownership of Related Assets" 1554:(SAN) technologies, which is independent of the client-to-server network. 1059:

integrated data checksums from the client to the kernel block layer, SCSI

862:(SMP) servers, and provided free space management for object allocations. 796:

domain back to the user community, and this was completed in March, 2015.

704:

hardware offerings, with the intent to bring Lustre technologies to Sun's

6793: 6778: 6733: 6690: 6629: 6393: 6363: 6322: 6317: 6312: 6170: 6141: 6136: 6131: 6126: 5615: 5559: 5528: 5317: 5142: 4998: 4891: 4844: 4738: 1566: 1348: 1214:) transfers, when available, to improve throughput and reduce CPU usage. 1195: 1125: 985: 973: 726:

development in an open community development model, including Whamcloud,

622: 618: 3886: 3755: 3673: 3643: 2942:"OpenSFS and Whamcloud Sign Lustre Community Tree Development Agreement" 1943: 722:, by way of its acquisition of Sun, began to manage and release Lustre. 6710: 6526: 6506: 6471: 6248: 6217: 6111: 6106: 6024: 5569: 5543: 5370: 5003: 4973: 4864: 4820: 4627: 3928: 3894: 3763: 3681: 3651: 3621: 3591: 3562: 3296: 3274: 1951: 1843: 1834:. Vendors selling storage hardware with bundled Lustre support include 1831: 1826:(as the HP StorageWorks Scalable File Share, circa 2004 through 2008), 1700:

Lost: the archive copy of the file has been lost and cannot be restored

1488: 1418: 1203: 1170:

layer can use several types of network interconnects, including native

1136: 1091:

GPU Direct Storage interface (GDS), the Lustre client can do zero-copy

949: 906: 807: 785: 777: 762: 738: 731: 727: 858:(OSTs), enabled Lustre distributed lock manager (LDLM) scalability on 6808: 5579: 5488: 5483: 5287: 4988: 4510: 4242: 3953: 2881:"Whamcloud Signs Multi-Year Lustre Development Contract With OpenSFS" 1716: 1398: 1369: 1191: 1088: 1019: 960: 799:

In June 2018, the Lustre team and assets were acquired from Intel by

773: 591: 4393:"French Atomic Energy Group Expands HPC File System to 11 Petabytes" 4371: 4346: 4003: 3289:"A Novel Network Request Scheduler for a Large Scale Storage System" 3264:"A Novel Network Request Scheduler for a Large Scale Storage System" 3145: 6464: 6447: 5948: 6660: 6521: 6432: 6348: 6243: 6029: 6019: 6014: 6009: 5837: 5635: 5620: 5597: 5592: 5587: 5498: 5493: 5431: 5312: 5265: 5260: 5253: 5248: 5243: 5238: 5194: 5184: 5089: 5057: 4951: 4946: 4941: 4837: 4765: 4713: 2830:"Bojanic & Braam Getting Lustre Band Back Together at Xyratex" 2029: 1914: 1877: 1736: 1578: 1547: 1496: 1385: 1273: 1242: 1183: 1158: 1075: 899: 830:, the third-largest supercomputer in the Top500 list at the time. 754: 693: 571: 478: 510:

Yes (network, storage with ZFS 0.8+, fscrypt with Lustre 2.14.0+)

30:"Cluster File Systems" redirects here. For the generic term, see 6798: 6476: 6353: 6332: 5508: 5473: 5463: 5458: 5396: 5332: 5302: 5297: 5233: 5228: 5199: 5108: 5099: 5052: 4993: 4963: 4854: 4825: 4815: 4810: 4775: 4757: 4496: 4307: 2549:

2012 19th International Conference on High Performance Computing

2420:

Steve Crusan, Brock Johnson, Hudson River Trading (2022-05-10),

2205: 2203: 2201: 1827: 1803: 1752: 1586: 1511:

and is currently the only node that gets write locks on inodes.

1381: 1306: 1302: 1254: 1230: 1226: 1218: 1199: 1132: 1071: 848: 5952: 4636: 4632: 4552:

Internal workings of Lustre file system and its core subsystems

1703:

No Release: the file should not be released from the filesystem

6567: 5114: 5094: 4983: 4885: 4569: 3199:. Open Scalable File Systems. February 8, 2011. Archived from 1262: 1222: 910: 894: 705: 626: 1802:, Brazilian National Laboratory of Scientific Computing, and 1536:

Portals network programming application programming interface

765:

then transitioned contracts for Lustre development to Intel.

4574: 3003:"DOE doles out cash to AMD, Whamcloud for exascale research" 2388:

James Beal, Pavlos Antoniou, Sanger Institute (2021-05-20).

1290:

maps the file logical offset and size to one or more objects

4586: 4564: 4056:"Catamount Software Architecture with Dual Core Extensions" 1691:

Archive: A full copy exists on the archive side of the HSM.

4528: 3816:"Overstriping: Extracting Maximum Shared File Performance" 2321:

Julie Bernauer, Prethvi Kashinkunti, NVIDIA (2021-05-20).

2308:"Nvidia AI supercomputer shows its Lustre in Oracle cloud" 2275:"Cray to Provide NOAA with Two AMD-Powered Supercomputers" 1434:-based, while the OSTs are disk-based. In Lustre 2.13 the 1340:

Another approach used in the early years of Lustre is the

1289: 976:(LMR) feature allows bonding multiple network interfaces ( 206: 4155:"Lustre Networking Technologies: Ethernet vs. Infiniband" 4025:"DataDirect Selected As Storage Tech Powering BlueGene/L" 2054: 2017:

Oracle Corporation / Intel Corporation (August 4, 2002).

1198:, and other proprietary network technologies such as the 1095:

read and write from the storage server directly into the

2771:"Whamcloud aims to make sure Lustre has a future in HPC" 2217:. Lawrence Livermore National Laboratory. Archived from 1166:

single node that are part of different filesystems. The

4612: 4591: 4257:

Aurélien Degrémont; Thomas Leibovici (April 16, 2009).

3731:"T10PI End-to-End Data Integrity Protection for Lustre" 3482:"Lustre QoS Based on NRS Policy of Token Bucket Filter" 3327:"Video: New Lustre 2.5 Release Offers HSM Capabilities" 3066:"Seagate Donates Lustre.org Back to the User Community" 1557:

LNet can use many commonly used network types, such as

1280:

of the mountpoint. When the client accesses a file, it

1268:

The client mounts the Lustre filesystem locally with a

1115:

A Lustre file system has three major functional units:

6909:

Distributed file systems supported by the Linux kernel

4308:"LNCC – Laboratório Nacional de Computação Científica" 4291:"The Orion File System: Configuration and Performance" 3977:. Government Computer News. 2008-10-26. Archived from 2732:. Sun Microsystems. September 12, 2007. Archived from 2507:

Proceedings of the 10th Parallel Data Storage Workshop

1688:

Exist: Some copy, possibly incomplete exists in a HSM.

1446:

Metadata objects and DNE remote or striped directories

944:

and rename. As well, it included improved support for

1585:, as well as proprietary high-speed networks such as 793: 578:. Lustre file system software is available under the 6870:"Cray Moves to Acquire the Seagate ClusterStor Line" 1792:

National Energy Research Scientific Computing Center

1297:

linearly with the number of OSTs in the filesystem.

621:(PB) of storage on hundreds of servers, and tens of 6827: 6761: 6724: 6669: 6648: 6617: 6581: 6545: 6425: 6341: 6236: 6002: 5995: 5920: 5892: 5857: 5813: 5731: 5653: 5606: 5552: 5447: 5352: 5278: 5218: 5161: 5128: 4695: 3950:"NVIDIA Magnum IO GPUDirect Storage Overview Guide" 2988:"Intel gobbles Lustre file system expert Whamcloud" 1405:In the Lustre 2.10 release, the ability to specify 847:networks, and could exploit extents/mballoc in the 539: 534: 524: 514: 502: 492: 470: 462: 452: 444: 436: 428: 423: 413: 405: 397: 389: 377: 369: 361: 353: 345: 340: 332: 324: 316: 311: 303: 288: 278: 266: 256: 248: 238: 201: 185: 173: 161: 151: 116: 100: 74: 52: 4259:"Lustre HSM Project—Lustre User Advanced Seminars" 1684:HSM also defines new states for files including: 598:, as well as previous top supercomputers such as 590:in the world, including the world's No. 1 ranked 3124:"Lustre Helps Power Third Fastest Supercomputer" 2617:: CS1 maint: bot: original URL status unknown ( 2019:"Lustre* Software Release 2.x Operations Manual" 1202:Gemini interconnect. In Lustre 2.3 and earlier, 836:Lustre 1.2.0, released in March 2004, worked on 4458:"High Throughput Storage Solutions with Lustre" 3088:"DDN Breathes New Life Into Lustre File System" 2183:. Oak Ridge National Laboratory. Archived from 1998: 1996: 1994: 1790:There are also large Lustre filesystems at the 1027:for multiple clients writing to the same file. 4548:Understanding Lustre Internals, Second Edition 3382:. Scientific Computing World. 12 November 2013 3312:"OpenSFS Announces Availability of Lustre 2.5" 2006:. Oak Ridge National Laboratory. May 30, 2022. 1739:Advanced Institute for Computational Science, 284:Andreas Dilger, Eric Barton (HPC), Phil Schwan 5964: 4648: 4296:. Oak Ridge National Laboratory. May 3, 2023. 3845:"Spillover Space: Self-Extending Layouts HLD" 3354:"Lustre Gets Business Class Upgrade with HSM" 3102:"Lustre Trademark Released to User Community" 2682:Slides for presentation at Cluster World 2003 357:700 PB (production), over 16 EB (theoretical) 8: 3310:Prickett Morgan, Timothy (5 November 2013). 3018:"Intel Carves Mainstream Highway for Lustre" 2805:. Xyratex. February 19, 2013. Archived from 2439:: CS1 maint: multiple names: authors list ( 2405:: CS1 maint: multiple names: authors list ( 2338:: CS1 maint: multiple names: authors list ( 2098:"Open-source Lustre gets supercomputing nod" 1680:, but other policy engines can also be used. 1372:distro kernels, as well as vanilla kernels. 1322:individual servers and MDT/OST filesystems. 298: 224: 6681:Common Development and Distribution License 3142:"MCR Linux Cluster Xeon 2.4 GHz – Quadrics" 2973:"Intel Purchases Lustre Purveyor Whamcloud" 2919:Slides for Supercomputing 2011 presentation 2887:. HPC Wire. August 16, 2011. Archived from 2675:"The Ultra-Scalable HPTC Lustre Filesystem" 1706:No Archive: the file should not be archived 6673: 6416: 5999: 5971: 5957: 5949: 5735: 5451: 5215: 4701: 4655: 4641: 4633: 4533: 2077:"Lustre File System, Version 2.4 Released" 1225:for the backing filesystem in addition to 1146:nodes that store file data on one or more 728:Open Scalable File Systems, Inc. (OpenSFS) 682:Accelerated Strategic Computing Initiative 297: 230: 223: 214: 140: 40: 4395:. HPCwire.com. 2012-06-15. Archived from 4182:Aurélien Degrémont (September 17, 2013). 4135:. HEPiX Forum, April 2007. Archived from 3814:Patrick Farrell, Whamcloud (April 2019). 2856:"Whamcloud Staffs up for Brighter Lustre" 1938: 1936: 1608:LNet provides end-to-end throughput over 1397:manner across the OST objects similar to 3843:Patrick Farrell, Cray (March 15, 2019). 2634:"Lustre, The Inter-Galactic File System" 2256:Andreas Dilger, Whamcloud (2019-06-20). 1908: 1906: 1099:memory to avoid an extra data copy from 873:cannot interoperate with 1.8 or earlier 6860: 2243:"Orion: Frontier's Massive File System" 1902: 753:In July 2012 Whamcloud was acquired by 680:at CMU. Lustre was developed under the 320:Hash, Interleaved Hash with DNE in 2.7+ 3001:Timothy Prickett Morgan (2012-07-11). 2986:Timothy Prickett Morgan (2012-07-16). 2793: 2791: 2756:"Oracle has Kicked Lustre to the Curb" 2645:Lawrence Livermore National Laboratory 2610: 2432: 2398: 2331: 1913:Corbet, Jonathon (December 17, 2003). 1785:Lawrence Livermore National Laboratory 1335:Lawrence Livermore National Laboratory 1103:memory and extra processing overhead. 828:Lawrence Livermore National Laboratory 748:Lawrence Livermore National Laboratory 83:2.15.5 (latest maintenance release), 4418:"HP StorageWorks Scalable File Share" 2702:Britta Wülfing (September 13, 2007). 1796:Pacific Northwest National Laboratory 1633:HSM (Hierarchical Storage Management) 1282:performs a filename lookup on the MDS 7: 4200:Thomas Stibor (September 20, 2016). 4079:"Linux Kernel 4.18rc1 release notes" 3559:Lustre-announce mailing list archive 3042:Brueckner, Rich (21 February 2013). 2355:"Long distance Lustre Communication" 1870:Free and open-source software portal 34:. For the programming language, see 2910:Galen Shipman (November 18, 2011). 2828:Rich Brueckner (November 9, 2010). 2704:"Sun Assimilates Lustre Filesystem" 2372:Cambridge-1: A NVIDIA Success Story 1249:, controls file access permissions/ 1229:, allowing them to effectively use 684:Path Forward project funded by the 4238:"Lustre/HSM Google Drive copytool" 4031:. October 15, 2004. Archived from 3916:Sébastien Buisson (May 11, 2022). 3401:Jones, Peter (September 9, 2014). 3175:Presentation to Sun HPC Consortium 2854:Rich Brueckner (January 4, 2011). 2288:Julian Kunkel, DKRZ (2017-06-15). 2118:"Xyratex Captures Oracle's Lustre" 686:United States Department of Energy 25: 6924:Free special-purpose file systems 4420:. Hewlett-Packard. Archived from 4271:from the original on May 25, 2010 4236:Stéphane Thiell (11 April 2021). 4153:Caldwell, Blane (March 9, 2016). 3528:HPDD-discuss mailing list archive 3455:HPDD-discuss mailing list archive 3407:HPDD-discuss mailing list archive 3329:. Inside Big Data. Archived from 3168:"Lustre Roadmap and Future Plans" 3086:Daniel Robinson (June 27, 2018). 2632:Peter J. Braam (August 4, 2002). 562:, generally used for large-scale 440:2^34 bits (ext4), 2^64 bits (ZFS) 4592:EOFS – European Open File System 4162:OLCF Lustre Center of Excellence 4117:. Lustre User Group, April 2016. 3787:"Lustre Persistent Client Cache" 3729:Shuichi Ihara, DDN (June 2018). 3016:Nicole Hemsoth (June 12, 2013). 1876: 1862: 1425:(DoM) feature, which allows the 594:supercomputer in November 2022, 81:2.15.0 (latest major release), 44: 27:Parallel distributed file system 4331:. www.nas.nasa.gov. 2008-08-18. 4092:Andreas Dilger (May 20, 2021). 3553:Jones, Peter (March 16, 2016). 3522:Jones, Peter (March 13, 2015). 3457:. July 30, 2014. Archived from 3166:Peter Bojanic (June 15, 2008). 2940:Whamcloud (November 15, 2011). 2673:R. Kent Koeninger (June 2003). 1800:Texas Advanced Computing Center 1658:High Performance Storage System 919:Hierarchical Storage Management 761:systems in the 2018 timeframe. 625:per second (TB/s) of aggregate 4218:Robert Read (March 24, 2015). 4184:"LUSTRE/HSM BINDING IS THERE!" 4130:"Lustre Experience at CEA/DIF" 4128:Lafoucrière, Jacques-Charles. 3785:Li Xi, Whamcloud (June 2018). 3424:Morrone, Chris (Dec 7, 2015). 3104:. InsideHPC. November 24, 2019 1745:National Supercomputing Center 1715:Lustre is used by many of the 1376:Data objects and file striping 379: 1: 6716:Open Source University Meetup 6625:Sun Microsystems Laboratories 4343:"TOP500 List – November 2006" 4202:"TSM Copytool for Lustre HSM" 3480:Ihara, Shuichi (2014-10-14). 3426:"Retired Release Terminology" 2769:J. Leidel (August 20, 2010). 2139:"Post-K (Fugaku) Information" 1808:Tokyo Institute of Technology 1806:in North America, in Asia at 1725:Oak Ridge National Laboratory 1105:User Defined Selection Policy 36:Lustre (programming language) 6868:Black, Doug (28 July 2017). 4460:. 2015-04-14. Archived from 4370:. TOP500.Org. Archived from 4345:. TOP500.Org. Archived from 4112:"Multi-Rail LNet for Lustre" 4099:. Open Scalable Filesystems. 4002:. 2011-05-10. Archived from 3144:. Top500.Org. Archived from 2921:. Open Scalable File Systems 2079:. Open Scalable File Systems 1818:Commercial technical support 1452: 1327:massively parallel processor 1293: 1123:nodes that have one or more 6835:The Network is the Computer 5828:Filesystem-level encryption 2971:Joab Jackson (2012-07-16). 2306:Chris Mellor (2023-05-02). 2258:"Lustre: The Next 20 Years" 2120:. HPCWire. 21 February 2013 1596:Since Lustre 2.10 the LNet 1571:Remote Direct Memory Access 1487:(LDLM), implemented in the 1286:logical object volume (LOV) 1148:object storage target (OST) 1144:object storage server (OSS) 6945: 6035:SPARCstation/server/center 4675:Comparison of file systems 4609:Hewlett Packard Enterprise 2461:. IEEE. pp. 102–111. 1353:Catamount operating system 810:and EOFS announced at the 702:high-performance computing 670:Carnegie Mellon University 580:GNU General Public License 225:Cluster File Systems, Inc. 29: 6919:Sun Microsystems software 6848: 6676: 6517:Sun Secure Global Desktop 6414: 5986: 5738: 5454: 4704: 4670: 4604:DataDirect Networks (DDN) 4598:Hardware/software vendors 4368:"TOP500 List – June 2006" 4220:"Lustre HSM in the Cloud" 4054:Suzanne M. Kelly (2006). 3869:"Lustre 2.13.0 Changelog" 3614:"Lustre 2.10.0 Changelog" 3356:. HPCwire. Archived from 2758:. Inside HPC. 2011-01-10. 2561:10.1109/HiPC.2012.6507507 2467:10.1109/CCGRID.2019.00021 2390:"Update on Secure Lustre" 2353:Raj Gautam (2019-05-15). 2212:"ZFS on Linux for Lustre" 2141:. Fujitsu. Archived from 1039:, and JobStats. It added 860:symmetric multiprocessing 365:32 PB (ext4), 16 EB (ZFS) 307:December, 2003 with Linux 229: 112: 70: 43: 6929:Distributed file systems 6840:Write once, run anywhere 6384:System Service Processor 5778:Extended file attributes 5479:Compact Disc File System 4329:"Pleiades Supercomputer" 4094:"Lustre 2.15 and Beyond" 3918:"Client Data Encryption" 3703:Li Xi, DDN (June 2018). 3584:"Lustre 2.9.0 Changelog" 3126:. DSStar. Archived from 2245:. insideHPC. 2023-04-03. 1974:"Lustre 2.15.5 released" 1485:distributed lock manager 1247:extended file attributes 6399:Ultra Port Architecture 5877:Installable File System 4511:"Amazon FSx for Lustre" 3555:"Lustre 2.8.0 released" 3524:"Lustre 2.7.0 released" 3451:"Lustre 2.6.0 released" 3403:"Lustre 2.5.3 released" 2555:. IEEE. pp. 1–10. 2515:10.1145/2834976.2834985 2509:. ACM. pp. 31–36. 2277:. HPC Wire. 2020-02-24. 2181:"Titan System Overview" 1460:Logical Metadata Volume 1411:Progressive File Layout 990:Progressive File Layout 946:Security-Enhanced Linux 776:. For 2013 as a whole, 642:artificial intelligence 566:. The name Lustre is a 560:distributed file system 180:Distributed file system 107:2.15.64 / 2024-06-21 6686:Java Community Process 6553:StorageTek 5800 System 6404:Visual Instruction Set 4925:TiVo Media File System 4789:Encrypting File System 3975:"Lustre to run on ZFS" 2164:"Supercomputer Fugaku" 1773:University of Illinois 1721:Frontier supercomputer 1662:Tivoli Storage Manager 1589:Aries and Gemini, and 1121:metadata servers (MDS) 856:object storage targets 674:InterMezzo file system 558:is a type of parallel 87:; 3 months ago 58:; 20 years ago 56:December 16, 2003 6904:Computer file systems 6640:Project Looking Glass 4920:Macintosh File System 4581:Community foundations 4575:Lustre (OpenSFS) wiki 4565:Lustre Community wiki 3243:"Lustre 2.2 Released" 3223:"Lustre 2.1 Released" 1915:"Lustre 1.0 released" 1440:Self-Extending Layout 1415:File Level Redundancy 1384:disk file system, an 1168:Lustre Network (LNet) 1001:File Level Redundancy 770:intellectual property 696:. In September 2007, 668:, who was a staff of 370:File size granularity 32:Clustered file system 6914:Network file systems 6599:Visualization System 6594:Constellation System 5933:GUID Partition Table 5280:Distributed parallel 5028:Shared File System ( 2809:on September 7, 2016 1836:Hitachi Data Systems 1614:100 Gigabit Ethernet 1600:(MR) feature allows 1552:storage area network 1294:locks the file range 851:on-disk filesystem. 782:parallel file system 85:/ June 28, 2024 18:Cluster File Systems 6487:Java Desktop System 5938:Apple Partition Map 5884:Virtual file system 5823:Access-control list 4937:NetWare File System 3897:. February 19, 2021 3875:. December 5, 2019. 3684:. December 21, 2018 3461:on October 17, 2014 3413:on October 17, 2014 3360:on 17 December 2013 2891:on January 25, 2013 2641:Presentation slides 2310:. Blocks and Files. 1850:Amazon Web Services 1840:DataDirect Networks 1814:, and many others. 1509:resource contention 1505:extended attributes 1501:access control list 1355:on systems such as 1041:LNet Network Health 898:schedulers such as 617:nodes, hundreds of 406:Max directory depth 390:Max filename length 300: 226: 6374:Modular Datacenter 5989:Acquired by Oracle 5928:Master Boot Record 5753:Data deduplication 5392:Google File System 5308:Google File System 4794:Extent File System 4756:Byte File System ( 4615:(including former 3766:. December 5, 2019 3705:"Lazy Size on MDS" 3594:. December 7, 2016 3333:on 8 December 2013 3177:. Sun Microsystems 2736:on October 2, 2007 2210:Brian Behlendorf. 2026:Instruction Manual 1292:. The client then 1063:, and T10-enabled 1046:2019-02-12 at the 806:In November 2019, 759:exascale computing 720:Oracle Corporation 516:Data deduplication 398:Max dirname length 317:Directory contents 292:Lustre file system 6857: 6856: 6804:Procom Technology 6784:Lighthouse Design 6757: 6756: 6739:Andy Bechtolsheim 6558:StorageTek SL8500 6412: 6411: 6289:UltraSPARC III Cu 6166:Blade workstation 5946: 5945: 5853: 5852: 5743:Case preservation 5649: 5648: 5348: 5347: 5274: 5273: 5036:Smart File System 4570:Lustre (DDN) wiki 4559:Information wikis 3498:Uselton, Andrew. 3352:Hemsoth, Nicole. 3325:Brueckner, Rich. 3314:. EnterpriseTech. 2952:on April 27, 2012 2578:978-1-4673-2371-0 2476:978-1-7281-0912-1 2423:Lustre in Finance 2057:on March 31, 2001 1919:Linux Weekly News 1620:High availability 1550:) or traditional 1407:composite layouts 1380:In a traditional 1359:, which provided 952:) on the client, 688:, which included 564:cluster computing 553: 552: 543:operating systems 296: 295: 272:Boulder, Colorado 222: 221: 16:(Redirected from 6936: 6884: 6883: 6881: 6880: 6865: 6789:Montalvo Systems 6769:Afara Websystems 6674: 6420: 6213:Java Workstation 6203:SPARC Enterprise 6000: 5980:Sun Microsystems 5973: 5966: 5959: 5950: 5763:Execute in place 5736: 5469:Boot File System 5452: 5216: 4752:Boot File System 4702: 4657: 4650: 4643: 4634: 4537: 4532: 4531: 4529:Official website 4515: 4514: 4507: 4501: 4500: 4497:"Lustre Support" 4493: 4487: 4486: 4479: 4473: 4472: 4470: 4469: 4454: 4448: 4447: 4440: 4434: 4433: 4431: 4429: 4424:on June 12, 2008 4414: 4408: 4407: 4405: 4404: 4389: 4383: 4382: 4380: 4379: 4364: 4358: 4357: 4355: 4354: 4339: 4333: 4332: 4325: 4319: 4318: 4316: 4315: 4304: 4298: 4297: 4295: 4287: 4281: 4280: 4278: 4276: 4270: 4263: 4254: 4248: 4247: 4233: 4227: 4226: 4224: 4215: 4209: 4208: 4206: 4197: 4191: 4190: 4188: 4179: 4173: 4172: 4170: 4168: 4159: 4150: 4144: 4143: 4141: 4134: 4125: 4119: 4118: 4116: 4107: 4101: 4100: 4098: 4089: 4083: 4082: 4075: 4069: 4068: 4066: 4065: 4060: 4051: 4045: 4044: 4042: 4040: 4035:on June 14, 2013 4021: 4015: 4014: 4012: 4011: 3996: 3990: 3989: 3987: 3986: 3971: 3965: 3964: 3962: 3960: 3946: 3940: 3939: 3937: 3935: 3922: 3913: 3907: 3906: 3904: 3902: 3887:"Release 2.14.0" 3883: 3877: 3876: 3865: 3859: 3858: 3856: 3854: 3849: 3840: 3834: 3833: 3831: 3829: 3820: 3811: 3805: 3804: 3802: 3800: 3791: 3782: 3776: 3775: 3773: 3771: 3756:"Release 2.13.0" 3752: 3746: 3745: 3743: 3741: 3735: 3726: 3720: 3719: 3717: 3715: 3709: 3700: 3694: 3693: 3691: 3689: 3674:"Release 2.12.0" 3670: 3664: 3663: 3661: 3659: 3644:"Release 2.11.0" 3640: 3634: 3633: 3631: 3629: 3610: 3604: 3603: 3601: 3599: 3580: 3574: 3573: 3571: 3569: 3550: 3544: 3543: 3541: 3539: 3534:on April 2, 2015 3530:. Archived from 3519: 3513: 3512: 3510: 3509: 3504: 3495: 3489: 3488: 3486: 3477: 3471: 3470: 3468: 3466: 3447: 3441: 3440: 3438: 3436: 3422: 3420: 3418: 3409:. Archived from 3398: 3392: 3391: 3389: 3387: 3376: 3370: 3369: 3367: 3365: 3349: 3343: 3342: 3340: 3338: 3322: 3316: 3315: 3307: 3301: 3300: 3285: 3279: 3278: 3268: 3260: 3254: 3253: 3251: 3250: 3245:. Yahoo! Finance 3239: 3233: 3232: 3230: 3229: 3219: 3213: 3212: 3210: 3208: 3193: 3187: 3186: 3184: 3182: 3172: 3163: 3157: 3156: 3154: 3153: 3138: 3132: 3131: 3120: 3114: 3113: 3111: 3109: 3098: 3092: 3091: 3083: 3077: 3076: 3074: 3072: 3062: 3056: 3055: 3053: 3051: 3039: 3033: 3032: 3030: 3028: 3013: 3007: 3006: 2998: 2992: 2991: 2983: 2977: 2976: 2968: 2962: 2961: 2959: 2957: 2948:. Archived from 2937: 2931: 2930: 2928: 2926: 2916: 2912:"OpenSFS Update" 2907: 2901: 2900: 2898: 2896: 2877: 2871: 2870: 2868: 2866: 2851: 2845: 2844: 2842: 2840: 2825: 2819: 2818: 2816: 2814: 2795: 2786: 2785: 2783: 2781: 2766: 2760: 2759: 2752: 2746: 2745: 2743: 2741: 2722: 2716: 2715: 2713: 2711: 2706:. Linux Magazine 2699: 2693: 2692: 2690: 2688: 2679: 2670: 2664: 2663: 2661: 2659: 2653: 2647:. Archived from 2638: 2629: 2623: 2622: 2616: 2608: 2597: 2591: 2590: 2569:2060/20130001600 2554: 2543: 2537: 2536: 2504: 2495: 2489: 2488: 2451: 2445: 2444: 2438: 2430: 2428: 2417: 2411: 2410: 2404: 2396: 2394: 2385: 2379: 2373: 2368: 2362: 2361: 2359: 2350: 2344: 2343: 2337: 2329: 2327: 2318: 2312: 2311: 2303: 2297: 2296: 2294: 2290:"Lustre at DKRZ" 2285: 2279: 2278: 2271: 2265: 2264: 2262: 2253: 2247: 2246: 2239: 2233: 2232: 2230: 2229: 2223: 2216: 2207: 2196: 2195: 2193: 2192: 2177: 2171: 2170: 2168: 2160: 2154: 2153: 2151: 2150: 2135: 2129: 2128: 2126: 2125: 2114: 2108: 2107: 2105: 2104: 2094: 2088: 2087: 2085: 2084: 2073: 2067: 2066: 2064: 2062: 2053:. Archived from 2047: 2041: 2040: 2038: 2036: 2023: 2014: 2008: 2007: 2000: 1989: 1988: 1986: 1985: 1970: 1964: 1963: 1961: 1959: 1944:"Release 2.15.0" 1940: 1931: 1930: 1928: 1926: 1910: 1886: 1881: 1880: 1872: 1867: 1866: 1865: 1610:Gigabit Ethernet 1602:link aggregation 1436:OST Overstriping 1423:Data-on-Metadata 1357:Sandia Red Storm 1333:installation at 967: 737:In August 2011, 716:operating system 698:Sun Microsystems 646:machine learning 568:portmanteau word 414:Allowed filename 381: 301: 234: 227: 218: 213: 210: 208: 163:Operating system 144: 139: 136: 134: 132: 130: 128: 95: 93: 88: 66: 64: 59: 48: 41: 21: 6944: 6943: 6939: 6938: 6937: 6935: 6934: 6933: 6889: 6888: 6887: 6878: 6876: 6867: 6866: 6862: 6858: 6853: 6844: 6823: 6774:Cobalt Networks 6753: 6720: 6665: 6644: 6613: 6577: 6541: 6421: 6408: 6337: 6292:UltraSPARC IIIi 6232: 5991: 5982: 5977: 5947: 5942: 5916: 5888: 5872:File system API 5849: 5809: 5785:File change log 5727: 5703:Record-oriented 5676:Self-certifying 5645: 5602: 5548: 5443: 5344: 5270: 5214: 5157: 5124: 4697: 4691: 4687:Unix filesystem 4666: 4661: 4600: 4583: 4561: 4544: 4527: 4526: 4523: 4518: 4509: 4508: 4504: 4495: 4494: 4490: 4481: 4480: 4476: 4467: 4465: 4456: 4455: 4451: 4442: 4441: 4437: 4427: 4425: 4416: 4415: 4411: 4402: 4400: 4391: 4390: 4386: 4377: 4375: 4366: 4365: 4361: 4352: 4350: 4341: 4340: 4336: 4327: 4326: 4322: 4313: 4311: 4306: 4305: 4301: 4293: 4289: 4288: 4284: 4274: 4272: 4268: 4261: 4256: 4255: 4251: 4235: 4234: 4230: 4222: 4217: 4216: 4212: 4204: 4199: 4198: 4194: 4186: 4181: 4180: 4176: 4166: 4164: 4157: 4152: 4151: 4147: 4139: 4132: 4127: 4126: 4122: 4114: 4110:Shehata, Amir. 4109: 4108: 4104: 4096: 4091: 4090: 4086: 4077: 4076: 4072: 4063: 4061: 4058: 4053: 4052: 4048: 4038: 4036: 4023: 4022: 4018: 4009: 4007: 4000:"ZFS on Lustre" 3998: 3997: 3993: 3984: 3982: 3973: 3972: 3968: 3958: 3956: 3948: 3947: 3943: 3933: 3931: 3920: 3915: 3914: 3910: 3900: 3898: 3885: 3884: 3880: 3867: 3866: 3862: 3852: 3850: 3847: 3842: 3841: 3837: 3827: 3825: 3818: 3813: 3812: 3808: 3798: 3796: 3789: 3784: 3783: 3779: 3769: 3767: 3754: 3753: 3749: 3739: 3737: 3733: 3728: 3727: 3723: 3713: 3711: 3707: 3702: 3701: 3697: 3687: 3685: 3672: 3671: 3667: 3657: 3655: 3654:. April 3, 2018 3642: 3641: 3637: 3627: 3625: 3624:. July 13, 2017 3612: 3611: 3607: 3597: 3595: 3582: 3581: 3577: 3567: 3565: 3552: 3551: 3547: 3537: 3535: 3521: 3520: 3516: 3507: 3505: 3502: 3497: 3496: 3492: 3484: 3479: 3478: 3474: 3464: 3462: 3449: 3448: 3444: 3434: 3432: 3423: 3416: 3414: 3400: 3399: 3395: 3385: 3383: 3378: 3377: 3373: 3363: 3361: 3351: 3350: 3346: 3336: 3334: 3324: 3323: 3319: 3309: 3308: 3304: 3287: 3286: 3282: 3266: 3262: 3261: 3257: 3248: 3246: 3241: 3240: 3236: 3227: 3225: 3221: 3220: 3216: 3206: 3204: 3203:on May 23, 2011 3195: 3194: 3190: 3180: 3178: 3170: 3165: 3164: 3160: 3151: 3149: 3140: 3139: 3135: 3122: 3121: 3117: 3107: 3105: 3100: 3099: 3095: 3085: 3084: 3080: 3070: 3068: 3064: 3063: 3059: 3049: 3047: 3041: 3040: 3036: 3026: 3024: 3015: 3014: 3010: 3005:. The Register. 3000: 2999: 2995: 2990:. The Register. 2985: 2984: 2980: 2970: 2969: 2965: 2955: 2953: 2939: 2938: 2934: 2924: 2922: 2914: 2909: 2908: 2904: 2894: 2892: 2879: 2878: 2874: 2864: 2862: 2853: 2852: 2848: 2838: 2836: 2827: 2826: 2822: 2812: 2810: 2797: 2796: 2789: 2779: 2777: 2768: 2767: 2763: 2754: 2753: 2749: 2739: 2737: 2724: 2723: 2719: 2709: 2707: 2701: 2700: 2696: 2686: 2684: 2677: 2672: 2671: 2667: 2657: 2655: 2654:on May 15, 2013 2651: 2636: 2631: 2630: 2626: 2609: 2599: 2598: 2594: 2579: 2552: 2545: 2544: 2540: 2525: 2502: 2497: 2496: 2492: 2477: 2453: 2452: 2448: 2431: 2426: 2419: 2418: 2414: 2397: 2392: 2387: 2386: 2382: 2371: 2369: 2365: 2357: 2352: 2351: 2347: 2330: 2325: 2320: 2319: 2315: 2305: 2304: 2300: 2292: 2287: 2286: 2282: 2273: 2272: 2268: 2260: 2255: 2254: 2250: 2241: 2240: 2236: 2227: 2225: 2221: 2214: 2209: 2208: 2199: 2190: 2188: 2179: 2178: 2174: 2166: 2162: 2161: 2157: 2148: 2146: 2137: 2136: 2132: 2123: 2121: 2116: 2115: 2111: 2102: 2100: 2096: 2095: 2091: 2082: 2080: 2075: 2074: 2070: 2060: 2058: 2049: 2048: 2044: 2034: 2032: 2021: 2016: 2015: 2011: 2002: 2001: 1992: 1983: 1981: 1972: 1971: 1967: 1957: 1955: 1954:. June 16, 2022 1942: 1941: 1934: 1924: 1922: 1912: 1911: 1904: 1900: 1882: 1875: 1868: 1863: 1861: 1858: 1824:Hewlett-Packard 1820: 1810:, in Europe at 1713: 1635: 1622: 1529: 1481: 1448: 1378: 1319: 1272:driver for the 1113: 1048:Wayback Machine 1025:lock contention 974:LNet Multi-Rail 965: 824: 822:Release history 690:Hewlett-Packard 662: 648:, oil and gas, 541: 505: 494: 473: 445:Date resolution 415: 354:Max volume size 346:Min volume size 281: 274: 205: 147: 135:/lustre-release 125: 108: 102:Preview release 96: 91: 89: 86: 62: 60: 57: 53:Initial release 39: 28: 23: 22: 15: 12: 11: 5: 6942: 6940: 6932: 6931: 6926: 6921: 6916: 6911: 6906: 6901: 6891: 6890: 6886: 6885: 6859: 6855: 6854: 6849: 6846: 6845: 6843: 6842: 6837: 6831: 6829: 6825: 6824: 6822: 6821: 6816: 6811: 6806: 6801: 6796: 6791: 6786: 6781: 6776: 6771: 6765: 6763: 6759: 6758: 6755: 6754: 6752: 6751: 6746: 6741: 6736: 6730: 6728: 6722: 6721: 6719: 6718: 6713: 6708: 6703: 6698: 6696:OpenOffice.org 6693: 6688: 6683: 6677: 6671: 6667: 6666: 6664: 6663: 6658: 6652: 6650: 6646: 6645: 6643: 6642: 6637: 6632: 6627: 6621: 6619: 6615: 6614: 6612: 6611: 6606: 6601: 6596: 6591: 6585: 6583: 6579: 6578: 6576: 6575: 6570: 6565: 6560: 6555: 6549: 6547: 6543: 6542: 6540: 6539: 6534: 6529: 6524: 6519: 6514: 6509: 6504: 6499: 6494: 6489: 6484: 6479: 6474: 6469: 6468: 6467: 6457: 6452: 6451: 6450: 6440: 6435: 6429: 6427: 6423: 6422: 6415: 6413: 6410: 6409: 6407: 6406: 6401: 6396: 6391: 6389:SPARC T series 6386: 6381: 6376: 6371: 6366: 6361: 6356: 6351: 6345: 6343: 6339: 6338: 6336: 6335: 6330: 6325: 6320: 6315: 6310: 6305: 6300: 6295: 6294: 6293: 6290: 6285:UltraSPARC III 6282: 6281: 6280: 6277: 6276:UltraSPARC IIi 6274: 6273:UltraSPARC IIe 6266: 6261: 6256: 6251: 6246: 6240: 6238: 6234: 6233: 6231: 6230: 6225: 6220: 6215: 6210: 6205: 6200: 6199: 6198: 6193: 6188: 6183: 6178: 6168: 6163: 6158: 6157: 6156: 6146: 6145: 6144: 6139: 6134: 6129: 6124: 6119: 6114: 6109: 6099: 6094: 6089: 6088: 6087: 6082: 6077: 6072: 6067: 6062: 6057: 6052: 6047: 6042: 6032: 6027: 6022: 6017: 6012: 6006: 6004: 5997: 5993: 5992: 5987: 5984: 5983: 5978: 5976: 5975: 5968: 5961: 5953: 5944: 5943: 5941: 5940: 5935: 5930: 5924: 5922: 5918: 5917: 5915: 5914: 5912:Log-structured 5909: 5904: 5898: 5896: 5890: 5889: 5887: 5886: 5881: 5880: 5879: 5869: 5863: 5861: 5855: 5854: 5851: 5850: 5848: 5847: 5846: 5845: 5840: 5830: 5825: 5819: 5817: 5815:Access control 5811: 5810: 5808: 5807: 5806: 5805: 5800: 5792: 5787: 5782: 5781: 5780: 5773:File attribute 5770: 5765: 5760: 5758:Data scrubbing 5755: 5750: 5745: 5739: 5733: 5729: 5728: 5726: 5725: 5720: 5715: 5713:Steganographic 5710: 5705: 5700: 5695: 5693:Log-structured 5690: 5685: 5680: 5679: 5678: 5673: 5668: 5657: 5655: 5651: 5650: 5647: 5646: 5644: 5643: 5638: 5633: 5628: 5623: 5618: 5612: 5610: 5604: 5603: 5601: 5600: 5595: 5590: 5585: 5582: 5577: 5572: 5567: 5562: 5556: 5554: 5550: 5549: 5547: 5546: 5541: 5536: 5531: 5526: 5521: 5516: 5511: 5506: 5501: 5496: 5491: 5486: 5481: 5476: 5471: 5466: 5461: 5455: 5449: 5445: 5444: 5442: 5441: 5434: 5429: 5424: 5419: 5414: 5409: 5404: 5399: 5394: 5389: 5384: 5379: 5374: 5364: 5358: 5356: 5350: 5349: 5346: 5345: 5343: 5342: 5335: 5330: 5325: 5320: 5315: 5310: 5305: 5300: 5295: 5290: 5284: 5282: 5276: 5275: 5272: 5271: 5269: 5268: 5263: 5258: 5257: 5256: 5246: 5241: 5236: 5231: 5225: 5223: 5213: 5212: 5207: 5202: 5197: 5192: 5187: 5182: 5177: 5171: 5169: 5159: 5158: 5156: 5155: 5150: 5145: 5140: 5134: 5132: 5126: 5125: 5123: 5122: 5112: 5102: 5097: 5092: 5087: 5082: 5077: 5076: 5075: 5070: 5060: 5055: 5050: 5045: 5040: 5039: 5038: 5033: 5023: 5018: 5016:Reliance Nitro 5013: 5008: 5007: 5006: 4996: 4991: 4986: 4981: 4976: 4971: 4966: 4961: 4956: 4955: 4954: 4944: 4939: 4934: 4929: 4928: 4927: 4922: 4914: 4909: 4904: 4899: 4894: 4889: 4879: 4876:Classic Mac OS 4869: 4868: 4867: 4857: 4852: 4847: 4842: 4841: 4840: 4830: 4829: 4828: 4823: 4818: 4813: 4803: 4798: 4797: 4796: 4791: 4783: 4778: 4773: 4768: 4763: 4762: 4761: 4754: 4749: 4747:Be File System 4741: 4736: 4731: 4726: 4721: 4716: 4711: 4705: 4699: 4693: 4692: 4690: 4689: 4684: 4683: 4682: 4671: 4668: 4667: 4662: 4660: 4659: 4652: 4645: 4637: 4631: 4630: 4628:Aeon Computing 4625: 4620: 4606: 4599: 4596: 4595: 4594: 4589: 4582: 4579: 4578: 4577: 4572: 4567: 4560: 4557: 4556: 4555: 4554: 4553: 4543: 4540: 4539: 4538: 4522: 4521:External links 4519: 4517: 4516: 4502: 4488: 4474: 4449: 4435: 4409: 4384: 4359: 4334: 4320: 4299: 4282: 4249: 4228: 4210: 4192: 4174: 4145: 4142:on 2012-02-08. 4120: 4102: 4084: 4070: 4046: 4016: 3991: 3966: 3941: 3908: 3878: 3860: 3835: 3806: 3777: 3747: 3721: 3695: 3665: 3635: 3605: 3575: 3545: 3514: 3490: 3472: 3442: 3393: 3371: 3344: 3317: 3302: 3280: 3255: 3234: 3214: 3188: 3158: 3133: 3130:on 2013-02-03. 3115: 3093: 3078: 3057: 3034: 3008: 2993: 2978: 2963: 2932: 2902: 2872: 2846: 2820: 2787: 2761: 2747: 2717: 2694: 2665: 2624: 2592: 2577: 2538: 2523: 2490: 2475: 2446: 2412: 2380: 2363: 2360:. Exxon Mobil. 2345: 2313: 2298: 2280: 2266: 2248: 2234: 2197: 2172: 2155: 2130: 2109: 2089: 2068: 2042: 2009: 1990: 1965: 1932: 1901: 1899: 1896: 1895: 1894: 1888: 1887: 1873: 1857: 1854: 1844:Aeon Computing 1819: 1816: 1749:Tianjin, China 1712: 1709: 1708: 1707: 1704: 1701: 1698: 1695: 1692: 1689: 1682: 1681: 1673: 1653: 1650: 1634: 1631: 1621: 1618: 1528: 1525: 1493:cache coherent 1480: 1477: 1447: 1444: 1377: 1374: 1318: 1317:Implementation 1315: 1278:root directory 1163: 1162: 1152: 1140: 1112: 1109: 823: 820: 744:object storage 666:Peter J. Braam 661: 658: 588:supercomputers 551: 550: 545: 537: 536: 532: 531: 530:Yes (ZFS only) 528: 522: 521: 520:Yes (ZFS only) 518: 512: 511: 508: 500: 499: 498:Yes (ZFS only) 496: 490: 489: 476: 468: 467: 464: 460: 459: 456: 450: 449: 446: 442: 441: 438: 434: 433: 430: 429:Dates recorded 426: 425: 421: 420: 417: 411: 410: 407: 403: 402: 399: 395: 394: 391: 387: 386: 383: 375: 374: 371: 367: 366: 363: 359: 358: 355: 351: 350: 347: 343: 342: 338: 337: 334: 330: 329: 326: 322: 321: 318: 314: 313: 309: 308: 305: 294: 293: 290: 286: 285: 282: 279: 276: 275: 270: 268: 264: 263: 261:Peter J. Braam 258: 254: 253: 250: 246: 245: 240: 236: 235: 220: 219: 203: 199: 198: 189: 183: 182: 177: 171: 170: 165: 159: 158: 153: 149: 148: 146: 145: 122: 120: 114: 113: 110: 109: 106: 104: 98: 97: 80: 78: 76:Stable release 72: 71: 68: 67: 54: 50: 49: 26: 24: 14: 13: 10: 9: 6: 4: 3: 2: 6941: 6930: 6927: 6925: 6922: 6920: 6917: 6915: 6912: 6910: 6907: 6905: 6902: 6900: 6899:2002 software 6897: 6896: 6894: 6875: 6871: 6864: 6861: 6852: 6847: 6841: 6838: 6836: 6833: 6832: 6830: 6826: 6820: 6817: 6815: 6812: 6810: 6807: 6805: 6802: 6800: 6797: 6795: 6792: 6790: 6787: 6785: 6782: 6780: 6777: 6775: 6772: 6770: 6767: 6766: 6764: 6760: 6750: 6747: 6745: 6744:Scott McNealy 6742: 6740: 6737: 6735: 6732: 6731: 6729: 6727: 6723: 6717: 6714: 6712: 6709: 6707: 6704: 6702: 6699: 6697: 6694: 6692: 6689: 6687: 6684: 6682: 6679: 6678: 6675: 6672: 6668: 6662: 6659: 6657: 6654: 6653: 6651: 6647: 6641: 6638: 6636: 6633: 6631: 6628: 6626: 6623: 6622: 6620: 6616: 6610: 6607: 6605: 6602: 6600: 6597: 6595: 6592: 6590: 6587: 6586: 6584: 6580: 6574: 6571: 6569: 6566: 6564: 6561: 6559: 6556: 6554: 6551: 6550: 6548: 6544: 6538: 6535: 6533: 6530: 6528: 6525: 6523: 6520: 6518: 6515: 6513: 6510: 6508: 6505: 6503: 6500: 6498: 6495: 6493: 6490: 6488: 6485: 6483: 6480: 6478: 6475: 6473: 6470: 6466: 6463: 6462: 6461: 6458: 6456: 6453: 6449: 6446: 6445: 6444: 6441: 6439: 6436: 6434: 6431: 6430: 6428: 6424: 6419: 6405: 6402: 6400: 6397: 6395: 6392: 6390: 6387: 6385: 6382: 6380: 6377: 6375: 6372: 6370: 6367: 6365: 6362: 6360: 6357: 6355: 6352: 6350: 6347: 6346: 6344: 6340: 6334: 6331: 6329: 6326: 6324: 6321: 6319: 6316: 6314: 6311: 6309: 6308:UltraSPARC T2 6306: 6304: 6303:UltraSPARC T1 6301: 6299: 6298:UltraSPARC IV 6296: 6291: 6288: 6287: 6286: 6283: 6278: 6275: 6272: 6271: 6270: 6269:UltraSPARC II 6267: 6265: 6262: 6260: 6257: 6255: 6252: 6250: 6247: 6245: 6242: 6241: 6239: 6235: 6229: 6226: 6224: 6221: 6219: 6216: 6214: 6211: 6209: 6206: 6204: 6201: 6197: 6194: 6192: 6189: 6187: 6184: 6182: 6179: 6177: 6174: 6173: 6172: 6169: 6167: 6164: 6162: 6159: 6155: 6152: 6151: 6150: 6147: 6143: 6140: 6138: 6135: 6133: 6130: 6128: 6125: 6123: 6120: 6118: 6115: 6113: 6110: 6108: 6105: 6104: 6103: 6100: 6098: 6095: 6093: 6090: 6086: 6083: 6081: 6078: 6076: 6073: 6071: 6068: 6066: 6063: 6061: 6058: 6056: 6053: 6051: 6048: 6046: 6043: 6041: 6038: 6037: 6036: 6033: 6031: 6028: 6026: 6023: 6021: 6018: 6016: 6013: 6011: 6008: 6007: 6005: 6001: 5998: 5994: 5990: 5985: 5981: 5974: 5969: 5967: 5962: 5960: 5955: 5954: 5951: 5939: 5936: 5934: 5931: 5929: 5926: 5925: 5923: 5919: 5913: 5910: 5908: 5905: 5903: 5902:Cryptographic 5900: 5899: 5897: 5895: 5891: 5885: 5882: 5878: 5875: 5874: 5873: 5870: 5868: 5865: 5864: 5862: 5860: 5856: 5844: 5841: 5839: 5836: 5835: 5834: 5831: 5829: 5826: 5824: 5821: 5820: 5818: 5816: 5812: 5804: 5801: 5799: 5796: 5795: 5793: 5791: 5788: 5786: 5783: 5779: 5776: 5775: 5774: 5771: 5769: 5766: 5764: 5761: 5759: 5756: 5754: 5751: 5749: 5748:Copy-on-write 5746: 5744: 5741: 5740: 5737: 5734: 5730: 5724: 5721: 5719: 5716: 5714: 5711: 5709: 5706: 5704: 5701: 5699: 5696: 5694: 5691: 5689: 5686: 5684: 5681: 5677: 5674: 5672: 5669: 5667: 5664: 5663: 5662: 5659: 5658: 5656: 5652: 5642: 5639: 5637: 5634: 5632: 5629: 5627: 5624: 5622: 5619: 5617: 5614: 5613: 5611: 5609: 5605: 5599: 5596: 5594: 5591: 5589: 5586: 5583: 5581: 5578: 5576: 5573: 5571: 5568: 5566: 5563: 5561: 5558: 5557: 5555: 5551: 5545: 5542: 5540: 5537: 5535: 5532: 5530: 5527: 5525: 5522: 5520: 5517: 5515: 5512: 5510: 5507: 5505: 5502: 5500: 5497: 5495: 5492: 5490: 5487: 5485: 5482: 5480: 5477: 5475: 5472: 5470: 5467: 5465: 5462: 5460: 5457: 5456: 5453: 5450: 5446: 5440: 5439: 5435: 5433: 5430: 5428: 5425: 5423: 5420: 5418: 5415: 5413: 5410: 5408: 5405: 5403: 5400: 5398: 5395: 5393: 5390: 5388: 5385: 5383: 5380: 5378: 5375: 5372: 5368: 5365: 5363: 5360: 5359: 5357: 5355: 5351: 5341: 5340: 5336: 5334: 5331: 5329: 5326: 5324: 5321: 5319: 5316: 5314: 5311: 5309: 5306: 5304: 5301: 5299: 5296: 5294: 5291: 5289: 5286: 5285: 5283: 5281: 5277: 5267: 5264: 5262: 5259: 5255: 5252: 5251: 5250: 5247: 5245: 5242: 5240: 5237: 5235: 5232: 5230: 5227: 5226: 5224: 5222: 5221:wear leveling 5217: 5211: 5208: 5206: 5203: 5201: 5198: 5196: 5193: 5191: 5188: 5186: 5183: 5181: 5178: 5176: 5173: 5172: 5170: 5168: 5164: 5160: 5154: 5151: 5149: 5146: 5144: 5141: 5139: 5136: 5135: 5133: 5131: 5127: 5120: 5116: 5113: 5110: 5106: 5103: 5101: 5098: 5096: 5093: 5091: 5088: 5086: 5083: 5081: 5078: 5074: 5071: 5069: 5066: 5065: 5064: 5061: 5059: 5056: 5054: 5051: 5049: 5046: 5044: 5041: 5037: 5034: 5031: 5027: 5026: 5024: 5022: 5019: 5017: 5014: 5012: 5009: 5005: 5002: 5001: 5000: 4997: 4995: 4992: 4990: 4987: 4985: 4982: 4980: 4977: 4975: 4972: 4970: 4967: 4965: 4962: 4960: 4957: 4953: 4950: 4949: 4948: 4945: 4943: 4940: 4938: 4935: 4933: 4930: 4926: 4923: 4921: 4918: 4917: 4915: 4913: 4910: 4908: 4905: 4903: 4900: 4898: 4895: 4893: 4890: 4887: 4883: 4880: 4877: 4873: 4870: 4866: 4863: 4862: 4861: 4858: 4856: 4853: 4851: 4848: 4846: 4843: 4839: 4836: 4835: 4834: 4831: 4827: 4824: 4822: 4819: 4817: 4814: 4812: 4809: 4808: 4807: 4804: 4802: 4799: 4795: 4792: 4790: 4787: 4786: 4784: 4782: 4779: 4777: 4774: 4772: 4769: 4767: 4764: 4759: 4755: 4753: 4750: 4748: 4745: 4744: 4742: 4740: 4737: 4735: 4732: 4730: 4727: 4725: 4722: 4720: 4717: 4715: 4712: 4710: 4707: 4706: 4703: 4700: 4694: 4688: 4685: 4681: 4678: 4677: 4676: 4673: 4672: 4669: 4665: 4658: 4653: 4651: 4646: 4644: 4639: 4638: 4635: 4629: 4626: 4624: 4621: 4618: 4614: 4610: 4607: 4605: 4602: 4601: 4597: 4593: 4590: 4588: 4585: 4584: 4580: 4576: 4573: 4571: 4568: 4566: 4563: 4562: 4558: 4551: 4550: 4549: 4546: 4545: 4542:Documentation 4541: 4536: 4530: 4525: 4524: 4520: 4513:. 2019-06-11. 4512: 4506: 4503: 4499:. 2018-11-27. 4498: 4492: 4489: 4485:. 2015-04-14. 4484: 4478: 4475: 4464:on 2015-06-27 4463: 4459: 4453: 4450: 4446:. 2015-06-13. 4445: 4439: 4436: 4423: 4419: 4413: 4410: 4399:on 2012-02-04 4398: 4394: 4388: 4385: 4374:on 2012-05-12 4373: 4369: 4363: 4360: 4349:on 2012-05-10 4348: 4344: 4338: 4335: 4330: 4324: 4321: 4309: 4303: 4300: 4292: 4286: 4283: 4267: 4260: 4253: 4250: 4245: 4244: 4239: 4232: 4229: 4221: 4214: 4211: 4203: 4196: 4193: 4185: 4178: 4175: 4163: 4156: 4149: 4146: 4138: 4131: 4124: 4121: 4113: 4106: 4103: 4095: 4088: 4085: 4080: 4074: 4071: 4057: 4050: 4047: 4034: 4030: 4026: 4020: 4017: 4006:on 2011-12-05 4005: 4001: 3995: 3992: 3981:on 2012-10-02 3980: 3976: 3970: 3967: 3955: 3951: 3945: 3942: 3930: 3926: 3919: 3912: 3909: 3896: 3892: 3888: 3882: 3879: 3874: 3870: 3864: 3861: 3846: 3839: 3836: 3824: 3817: 3810: 3807: 3795: 3788: 3781: 3778: 3765: 3761: 3757: 3751: 3748: 3736:. Lustre Wiki 3732: 3725: 3722: 3710:. Lustre Wiki 3706: 3699: 3696: 3683: 3679: 3675: 3669: 3666: 3653: 3649: 3645: 3639: 3636: 3623: 3619: 3615: 3609: 3606: 3593: 3589: 3585: 3579: 3576: 3564: 3560: 3556: 3549: 3546: 3533: 3529: 3525: 3518: 3515: 3501: 3494: 3491: 3483: 3476: 3473: 3460: 3456: 3452: 3446: 3443: 3431: 3427: 3412: 3408: 3404: 3397: 3394: 3381: 3375: 3372: 3359: 3355: 3348: 3345: 3332: 3328: 3321: 3318: 3313: 3306: 3303: 3298: 3294: 3290: 3284: 3281: 3276: 3272: 3265: 3259: 3256: 3244: 3238: 3235: 3224: 3218: 3215: 3202: 3198: 3192: 3189: 3181:September 23, 3176: 3169: 3162: 3159: 3148:on 2012-04-20 3147: 3143: 3137: 3134: 3129: 3125: 3119: 3116: 3103: 3097: 3094: 3089: 3082: 3079: 3067: 3061: 3058: 3045: 3038: 3035: 3027:September 23, 3023: 3019: 3012: 3009: 3004: 2997: 2994: 2989: 2982: 2979: 2974: 2967: 2964: 2956:September 23, 2951: 2947: 2946:Press release 2943: 2936: 2933: 2925:September 23, 2920: 2913: 2906: 2903: 2895:September 23, 2890: 2886: 2885:Press release 2882: 2876: 2873: 2865:September 18, 2861: 2857: 2850: 2847: 2839:September 23, 2835: 2831: 2824: 2821: 2813:September 18, 2808: 2804: 2803:Press release 2800: 2794: 2792: 2788: 2780:September 23, 2776: 2772: 2765: 2762: 2757: 2751: 2748: 2740:September 23, 2735: 2731: 2730:Press release 2727: 2721: 2718: 2710:September 23, 2705: 2698: 2695: 2687:September 23, 2683: 2676: 2669: 2666: 2658:September 23, 2650: 2646: 2642: 2635: 2628: 2625: 2620: 2614: 2606: 2602: 2596: 2593: 2588: 2584: 2580: 2574: 2570: 2566: 2562: 2558: 2551: 2550: 2542: 2539: 2534: 2530: 2526: 2524:9781450340083 2520: 2516: 2512: 2508: 2501: 2494: 2491: 2486: 2482: 2478: 2472: 2468: 2464: 2460: 2459: 2450: 2447: 2442: 2436: 2425: 2424: 2416: 2413: 2408: 2402: 2391: 2384: 2381: 2378: 2374: 2367: 2364: 2356: 2349: 2346: 2341: 2335: 2324: 2317: 2314: 2309: 2302: 2299: 2291: 2284: 2281: 2276: 2270: 2267: 2259: 2252: 2249: 2244: 2238: 2235: 2224:on 2014-10-31 2220: 2213: 2206: 2204: 2202: 2198: 2187:on 2018-02-13 2186: 2182: 2176: 2173: 2165: 2159: 2156: 2145:on 2020-06-08 2144: 2140: 2134: 2131: 2119: 2113: 2110: 2099: 2093: 2090: 2078: 2072: 2069: 2061:September 23, 2056: 2052: 2051:"Lustre Home" 2046: 2043: 2031: 2027: 2020: 2013: 2010: 2005: 1999: 1997: 1995: 1991: 1979: 1975: 1969: 1966: 1953: 1949: 1945: 1939: 1937: 1933: 1920: 1916: 1909: 1907: 1903: 1897: 1893: 1890: 1889: 1885: 1879: 1874: 1871: 1860: 1855: 1853: 1851: 1847: 1845: 1841: 1837: 1833: 1829: 1825: 1817: 1815: 1813: 1809: 1805: 1801: 1797: 1793: 1788: 1786: 1782: 1778: 1774: 1770: 1766: 1762: 1758: 1754: 1750: 1746: 1742: 1738: 1734: 1730: 1726: 1722: 1718: 1710: 1705: 1702: 1699: 1696: 1693: 1690: 1687: 1686: 1685: 1679: 1674: 1671: 1667: 1663: 1659: 1654: 1651: 1648: 1647: 1646: 1643: 1639: 1632: 1630: 1626: 1619: 1617: 1615: 1611: 1606: 1603: 1599: 1594: 1592: 1588: 1584: 1580: 1576: 1572: 1568: 1564: 1560: 1555: 1553: 1549: 1545: 1541: 1537: 1534: 1526: 1524: 1522: 1517: 1512: 1510: 1506: 1502: 1498: 1494: 1490: 1486: 1478: 1476: 1472: 1469: 1465: 1461: 1456: 1454: 1445: 1443: 1441: 1437: 1433: 1428: 1424: 1420: 1416: 1412: 1408: 1403: 1400: 1396: 1390: 1387: 1383: 1375: 1373: 1371: 1365: 1362: 1358: 1354: 1350: 1346: 1343: 1338: 1336: 1332: 1328: 1323: 1316: 1314: 1312: 1308: 1304: 1298: 1295: 1291: 1288:layer, which 1287: 1283: 1279: 1275: 1271: 1266: 1264: 1260: 1256: 1252: 1248: 1244: 1240: 1235: 1232: 1228: 1224: 1220: 1215: 1213: 1209: 1205: 1201: 1197: 1193: 1189: 1185: 1181: 1177: 1173: 1169: 1160: 1156: 1153: 1149: 1145: 1141: 1138: 1134: 1129: 1127: 1122: 1118: 1117: 1116: 1110: 1108: 1106: 1102: 1098: 1094: 1090: 1084: 1080: 1077: 1073: 1068: 1066: 1062: 1058: 1054: 1049: 1045: 1042: 1038: 1034: 1028: 1026: 1021: 1018: 1014: 1010: 1006: 1002: 997: 995: 991: 987: 983: 979: 975: 970: 962: 957: 955: 951: 947: 943: 938: 936: 932: 926: 922: 920: 915: 912: 908: 903: 901: 896: 890: 886: 883: 882:Red Hat Linux 878: 876: 872: 867: 863: 861: 857: 852: 850: 846: 841: 839: 834: 831: 829: 821: 819: 817: 813: 809: 804: 802: 797: 795: 791: 787: 783: 779: 775: 771: 766: 764: 760: 756: 751: 749: 745: 740: 735: 733: 729: 723: 721: 717: 714: 710: 707: 703: 699: 695: 691: 687: 683: 679: 675: 671: 667: 659: 657: 655: 651: 647: 643: 639: 635: 631: 628: 624: 620: 616: 611: 609: 605: 601: 597: 593: 589: 585: 581: 577: 573: 570:derived from 569: 565: 561: 557: 549: 546: 544: 538: 533: 529: 527: 526:Copy-on-write 523: 519: 517: 513: 509: 507: 501: 497: 491: 488: 484: 480: 477: 475: 469: 465: 461: 457: 455: 451: 447: 443: 439: 435: 431: 427: 422: 418: 412: 408: 404: 400: 396: 392: 388: 384: 376: 372: 368: 364: 362:Max file size 360: 356: 352: 348: 344: 339: 335: 331: 327: 323: 319: 315: 310: 306: 302: 291: 287: 283: 277: 273: 269: 265: 262: 259: 255: 251: 247: 244: 241: 237: 233: 228: 217: 212: 204: 200: 197: 193: 190: 188: 184: 181: 178: 176: 172: 169: 166: 164: 160: 157: 154: 150: 143: 138: 124: 123: 121: 119: 115: 111: 105: 103: 99: 84: 79: 77: 73: 69: 55: 51: 47: 42: 37: 33: 19: 6877:. Retrieved 6873: 6863: 6762:Acquisitions 6749:Vinod Khosla 6608: 6563:Open Storage 6161:Blade server 6092:SPARCclassic 5867:File manager 5436: 5401: 5337: 5163:Flash memory 5130:Optical disc 5068:soft updates 5048:Soup (Apple) 4698:non-rotating 4664:File systems 4505: 4491: 4477: 4466:. Retrieved 4462:the original 4452: 4438: 4428:December 13, 4426:. Retrieved 4422:the original 4412: 4401:. Retrieved 4397:the original 4387: 4376:. Retrieved 4372:the original 4362: 4351:. Retrieved 4347:the original 4337: 4323: 4312:. Retrieved 4302: 4285: 4273:. Retrieved 4252: 4241: 4231: 4213: 4195: 4177: 4165:. Retrieved 4161: 4148: 4137:the original 4123: 4105: 4087: 4073: 4062:. Retrieved 4049: 4037:. Retrieved 4033:the original 4028: 4019: 4008:. Retrieved 4004:the original 3994: 3983:. Retrieved 3979:the original 3969: 3957:. Retrieved 3944: 3932:. Retrieved 3924: 3911: 3901:February 19, 3899:. Retrieved 3890: 3881: 3872: 3863: 3851:. Retrieved 3838: 3826:. Retrieved 3822: 3809: 3797:. Retrieved 3793: 3780: 3768:. Retrieved 3759: 3750: 3738:. Retrieved 3724: 3712:. Retrieved 3698: 3688:February 11, 3686:. Retrieved 3677: 3668: 3656:. Retrieved 3647: 3638: 3626:. Retrieved 3617: 3608: 3596:. Retrieved 3587: 3578: 3566:. Retrieved 3558: 3548: 3536:. Retrieved 3532:the original 3527: 3517: 3506:. Retrieved 3493: 3475: 3463:. Retrieved 3459:the original 3454: 3445: 3433:. Retrieved 3429: 3415:. Retrieved 3411:the original 3406: 3396: 3384:. Retrieved 3380:"Lustre 2.5" 3374: 3362:. Retrieved 3358:the original 3347: 3335:. Retrieved 3331:the original 3320: 3305: 3299:. June 2009. 3292: 3283: 3277:. June 2009. 3270: 3258: 3247:. Retrieved 3237: 3226:. Retrieved 3217: 3207:December 13, 3205:. Retrieved 3201:the original 3191: 3179:. Retrieved 3174: 3161: 3150:. Retrieved 3146:the original 3136: 3128:the original 3118: 3106:. Retrieved 3096: 3081: 3069:. Retrieved 3060: 3048:. Retrieved 3037: 3025:. Retrieved 3021: 3011: 2996: 2981: 2966: 2954:. Retrieved 2950:the original 2945: 2935: 2923:. Retrieved 2918: 2905: 2893:. Retrieved 2889:the original 2884: 2875: 2863:. Retrieved 2859: 2849: 2837:. Retrieved 2833: 2823: 2811:. Retrieved 2807:the original 2802: 2778:. Retrieved 2774: 2764: 2750: 2738:. Retrieved 2734:the original 2729: 2720: 2708:. Retrieved 2697: 2685:. Retrieved 2681: 2668: 2656:. Retrieved 2649:the original 2640: 2627: 2605:old web site 2604: 2595: 2548: 2541: 2506: 2493: 2457: 2449: 2422: 2415: 2383: 2366: 2348: 2316: 2301: 2283: 2269: 2251: 2237: 2226:. Retrieved 2219:the original 2189:. Retrieved 2185:the original 2175: 2158: 2147:. Retrieved 2143:the original 2133: 2122:. Retrieved 2112: 2101:. Retrieved 2092: 2081:. Retrieved 2071: 2059:. Retrieved 2055:the original 2045: 2033:. Retrieved 2025: 2012: 1982:. Retrieved 1980:. 2024-06-28 1977: 1968: 1956:. Retrieved 1947: 1923:. Retrieved 1918: 1884:Linux portal 1848: 1821: 1789: 1714: 1683: 1670:Google Drive 1644: 1640: 1636: 1627: 1623: 1616:interfaces. 1607: 1597: 1595: 1556: 1530: 1513: 1482: 1473: 1467: 1459: 1457: 1449: 1439: 1435: 1426: 1422: 1414: 1410: 1406: 1404: 1391: 1379: 1366: 1341: 1339: 1324: 1320: 1299: 1285: 1267: 1258: 1236: 1216: 1167: 1164: 1154: 1147: 1143: 1142:One or more 1128:target (MDT) 1124: 1120: 1119:One or more 1114: 1111:Architecture 1085: 1081: 1069: 1061:host adapter 1052: 1029: 998: 971: 969:parameters. 958: 939: 927: 923: 916: 904: 891: 887: 879: 874: 870: 868: 866:Lustre 2.0. 864: 855: 853: 842: 838:Linux kernel 835: 832: 825: 805: 798: 767: 752: 736: 724: 678:Coda project 663: 650:life science 612: 555: 554: 548:Linux kernel 483:POSIX.1e ACL 267:Headquarters 239:Company type 168:Linux kernel 82: 6701:OpenSolaris 6604:Grid Engine 6582:Performance 6512:Java System 6482:OpenWindows 6223:Cobalt Qube 6208:JavaStation 5833:Permissions 5448:Specialized 4680:distributed 4167:December 6, 3925:Lustre Wiki 3891:Lustre Wiki 3873:Lustre Wiki 3853:December 5, 3828:December 5, 3823:Lustre Wiki 3799:December 5, 3794:Lustre Wiki 3770:December 5, 3760:Lustre Wiki 3740:December 5, 3714:December 5, 3678:Lustre Wiki 3648:Lustre Wiki 3618:Lustre Wiki 3598:December 8, 3588:Lustre Wiki 3465:October 21, 3435:January 18, 3430:Lustre Wiki 3417:October 21, 3386:11 December 3364:11 December 3337:11 December 3293:Lustre Wiki 3271:Lustre Wiki 3108:December 5, 3071:9 September 3046:. insideHPC 2975:. PC World. 1948:Lustre Wiki 1769:Blue Waters 1711:Deployments 1483:The Lustre 1395:round-robin 1065:hard drives 1057:SCSI T10-PI 1009:Data-on-MDT 994:mountpoints 790:open source 709:file system 634:meteorology 504:Transparent 495:compression 493:Transparent 474:permissions 472:File system 6893:Categories 6879:2017-12-01 6819:Tarantella 6814:StorageTek 6537:VirtualBox 6502:StarOffice 6264:UltraSPARC 6259:SuperSPARC 6254:microSPARC 6237:Processors 6228:Cobalt RaQ 6149:Enterprise 5859:Interfaces 5843:Sticky bit 5723:Versioning 5688:Journaling 5631:Rubberhose 5427:SMB (CIFS) 5219:host-side 4619:employees) 4468:2015-04-14 4403:2012-06-15 4378:2008-08-29 4353:2008-08-29 4314:2015-05-27 4064:2016-02-16 4010:2011-11-25 3985:2011-02-23 3628:October 3, 3508:2014-10-18 3249:2012-05-08 3228:2012-02-02 3152:2023-12-14 2860:Inside HPC 2834:Inside HPC 2775:Inside HPC 2395:. OpenSFS. 2328:. OpenSFS. 2295:. OpenSFS. 2263:. HPCIODC. 2228:2020-06-23 2191:2013-09-19 2169:. Fujitsu. 2149:2020-06-23 2124:2014-10-18 2103:2014-10-18 2083:2014-10-18 1984:2024-06-28 1978:lustre.org 1898:References 1733:K Computer 1598:Multi-Rail 1565:(commonly 1559:InfiniBand 1527:Networking 1351:using the 1172:InfiniBand 978:InfiniBand 845:InfiniBand 794:lustre.org 654:rich media 638:simulation 630:throughput 506:encryption 463:Attributes 437:Date range 416:characters 409:4096 bytes 312:Structures 304:Introduced 280:Key people 152:Written in 129:.whamcloud 118:Repository 92:2024-06-28 63:2003-12-16 6706:OpenSPARC 6670:Community 6649:Education 6532:GlassFish 6359:Fireplane 5718:Synthetic 5661:Clustered 5608:Encrypted 5539:OverlayFS 5148:ISO 13490 4724:Amiga OFS 4719:Amiga FFS 4310:. Lncc.br 3568:March 28, 3538:March 15, 3050:1 October 2601:"Company" 2485:195832257 2429:, OpenSFS 1925:March 15, 1921:. LWN.net 1781:Blue Gene 1767:at ORNL, 1741:Tianhe-1A 1678:RobinHood 1666:Amazon S3 1583:Omni-Path 1468:readdir() 1361:userspace 1342:liblustre 1331:Blue Gene 1176:Omni-Path 1155:Client(s) 984:, and/or 982:Omni-Path 942:hard link 623:terabytes 619:petabytes 584:workgroup 540:Supported 401:255 bytes 393:255 bytes 325:File type 6851:Category 6794:MySQL AB 6779:Gridware 6734:Bill Joy 6691:NetBeans 6635:Fortress 6630:picoJava 6618:Research 6426:Software 6364:LOM port 6323:SPARC T5 6318:SPARC T4 6313:SPARC T3 5996:Hardware 5803:Symbolic 5732:Features 5708:Semantic 5616:eCryptfs 5560:configfs 5529:SquashFS 5417:POHMELFS 5318:OrangeFS 5143:ISO 9660 5063:UFS/UFS2 5011:Reliance 4999:ReiserFS 4845:Files-11 4739:bcachefs 4696:Disk and 4266:Archived 4029:HPC Wire 3959:June 17, 3934:June 17, 3658:April 4, 3022:HPC Wire 2613:cite web 2587:14323627 2533:15844745 2435:citation 2401:cite web 2334:cite web 2035:June 25, 1958:June 17, 1856:See also 1838:(2012), 1787:(LLNL). 1727:(ORNL), 1660:(HPSS), 1567:Ethernet 1419:RAID 0+1 1349:Cray XT3 1325:On some 1208:Quadrics 1196:Ethernet 1126:metadata 1053:restripe 1044:Archived 1005:mirrored 986:Ethernet 954:Kerberos 711:and the 596:Frontier 424:Features 382:of files 333:Bootable 289:Products 6874:HPCWire 6828:Slogans 6711:OpenJDK 6546:Storage 6507:iPlanet 6472:SunView 6438:Solaris 6379:Neptune 6249:MB86900 6025:Sun386i 6003:Systems 5921:Layouts 5907:Default 5570:debugfs 5544:UnionFS 5438:more... 5371:OpenAFS 5339:more... 5004:Reiser4 4974:OpenZFS 4865:HAMMER2 4821:ext3cow 4801:Episode 4617:Xyratex 4587:OpenSFS 3929:OpenSFS 3895:OpenSFS 3764:OpenSFS 3682:OpenSFS 3652:OpenSFS 3622:OpenSFS 3592:OpenSFS 3563:OpenSFS 3297:OpenSFS 3275:OpenSFS 2377:YouTube 1952:OpenSFS 1842:(DDN), 1832:Fujitsu 1777:Sequoia 1771:at the 1743:at the 1735:at the 1664:(TSM), 1489:OpenVMS 1479:Locking 1347:on the 1345:library 1259:ldiskfs 1257:called 1204:Myrinet 1174:verbs, 1013:RAID-10 966:ladvise 950:SELinux 907:OpenSFS 875:servers 871:clients 849:ldiskfs 816:Seagate 808:OpenSFS 786:OpenSFS 778:OpenSFS 763:OpenSFS 739:OpenSFS 732:Xyratex 713:Solaris 676:in the 660:History 608:Sequoia 576:cluster 487:SELinux 257:Founder 249:Founded 243:Private 209:.lustre 202:Website 187:License 90: ( 61: ( 6809:SavaJe 6726:People 6609:Lustre 6492:Studio 6279:Gemini 5794:Links 5768:Extent 5698:Object 5666:Global 5584:specfs 5580:procfs 5575:kernfs 5553:Pseudo 5534:UMSDOS 5489:Davfs2 5484:cramfs 5422:Hadoop 5402:Lustre 5288:BeeGFS 5254:NILFS2 4989:QNX4FS 4952:NILFS2 4860:HAMMER 4850:Fossil 4623:NetApp 4275:May 5, 4243:GitHub 4039:May 9, 3954:NVIDIA 2585: 2575: 2531: 2521: 2483: 2473: 1783:/L at 1775:, and 1761:Jaguar 1729:Fugaku 1717:TOP500 1668:, and 1581:, and 1533:Sandia 1516:extent 1399:RAID 0 1370:Ubuntu 1239:extent 1192:TCP/IP 1182:, and 1089:NVIDIA 1020:RAID-6 961:GSSAPI 774:Hadoop 615:client 600:Fugaku 592:TOP500 556:Lustre 341:Limits 299:Lustre 192:GPL v2 6661:BlueJ 6589:Cloud 6522:MySQL 6433:SunOS 6349:Sun4d 6342:Other 6244:SPARC 6196:X4500 6191:T2000 6154:10000 6102:Ultra 6097:Netra 6030:Sun-4 6020:Sun-3 6015:Sun-2 6010:Sun-1 5894:Lists 5838:Modes 5683:Flash 5654:Types 5636:SSHFS 5621:EncFS 5598:WinFS 5593:tmpfs 5588:sysfs 5565:devfs 5499:FTPFS 5494:EROFS 5432:SSHFS 5313:OCFS2 5266:UBIFS 5261:YAFFS 5249:NILFS 5244:LogFS 5239:JFFS2 5195:EROFS 5185:exFAT 5090:Xiafs 5073:WAPBL 5058:UBIFS 4969:OneFS 4947:NILFS 4942:Next3 4932:MINIX 4838:exFAT 4766:Btrfs 4734:AthFS 4714:AdvFS 4294:(PDF) 4269:(PDF) 4262:(PDF) 4223:(PDF) 4205:(PDF) 4187:(PDF) 4158:(PDF) 4140:(PDF) 4133:(PDF) 4115:(PDF) 4097:(PDF) 4059:(PDF) 3921:(PDF) 3848:(PDF) 3819:(PDF) 3790:(PDF) 3734:(PDF) 3708:(PDF) 3503:(PDF) 3485:(PDF) 3267:(PDF) 3171:(PDF) 2915:(PDF) 2678:(PDF) 2652:(PDF) 2637:(PDF) 2583:S2CID 2553:(PDF) 2529:S2CID 2503:(PDF) 2481:S2CID 2427:(PDF) 2393:(PDF) 2358:(PDF) 2326:(PDF) 2293:(PDF) 2261:(PDF) 2222:(PDF) 2215:(PDF) 2167:(PDF) 2030:Intel 2022:(PDF) 1765:Titan 1737:RIKEN 1579:iWARP 1548:iSCSI 1521:pages 1497:inode 1464:shard 1427:first 1386:inode 1274:Linux 1261:, or 1243:POSIX 1184:iWARP 1159:POSIX 1151:OSTs. 1137:PanFS 1076:NVRAM 900:SLURM 755:Intel 694:Intel 604:Titan 572:Linux 535:Other 479:POSIX 454:Forks 349:32 MB 6799:Pixo 6656:SCPs 6497:Java 6477:NeWS 6465:ZFS+ 6448:NIS+ 6394:SPOT 6369:MBus 6354:SBus 6333:MAJC 6328:Rock 6186:E25K 6171:Fire 5798:Hard 5790:Fork 5671:Grid 5524:MVFS 5519:NOVA 5514:LTFS 5509:Lnfs 5504:FUSE 5474:CDfs 5464:AXFS 5459:Aufs 5397:GPFS 5382:Coda 5333:Xsan 5323:PVFS 5303:GFS2 5298:CXFS 5293:Ceph 5234:JFFS 5229:CHFS 5210:NVFS 5200:F2FS 5190:TFAT 5175:APFS 5165:and 5109:z/OS 5100:Xsan 5085:WAFL 5080:VxFS 5053:Tux3 5043:SNFS 5025:SFS 4994:ReFS 4964:NTFS 4916:MFS 4902:HTFS 4897:HPFS 4892:HFS+ 4855:GPFS 4826:ext4 4816:ext3 4811:ext2 4785:EFS 4776:CXFS 4771:CVFS 4758:z/VM 4743:BFS 4729:APFS 4709:ADFS 4613:Cray 4430:2016 4277:2018 4169:2019 4041:2012 3961:2022 3936:2022 3903:2021 3855:2019 3830:2019 3801:2019 3772:2019 3742:2019 3716:2019 3690:2019 3660:2018 3630:2017 3600:2016 3570:2016 3540:2015 3467:2014 3437:2016 3419:2014 3388:2013 3366:2013 3339:2013 3209:2016 3183:2013 3110:2019 3073:2014 3052:2013 3029:2013 2958:2013 2927:2013 2897:2013 2867:2013 2841:2013 2815:2013 2782:2013 2742:2013 2712:2013 2689:2013 2660:2013 2619:link 2573:ISBN 2519:ISBN 2471:ISBN 2441:link 2407:link 2340:link 2063:2013 2037:2024 1960:2022 1927:2015 1828:ATOS 1804:NASA 1779:and 1763:and 1753:LUMI 1731:and 1591:Atos 1587:Cray 1575:RoCE 1561:and 1382:Unix 1307:OCFS 1305:and 1303:GPFS 1255:ext4 1251:ACLs 1245:and 1231:JBOD 1227:ext4 1219:RAID 1212:RDMA 1200:Cray 1188:OFED 1186:via 1180:RoCE 1135:and 1133:GPFS 1093:RDMA 1074:and 1072:NVMe 812:SC19 692:and 644:and 606:and 574:and 378:Max 373:4 KB 252:2001 211:.org 196:LGPL 175:Type 137:.git 131:.com 6573:ZFS 6568:QFS 6527:xVM 6460:ZFS 6455:NFS 6443:NIS 6218:Ray 6181:15K 6176:12K 6075:IPX 6070:IPC 5641:ZFS 5626:EFS 5412:NFS 5407:NCP 5387:DFS 5377:AFP 5367:AFS 5354:NAS 5328:QFS 5205:JFS 5180:FAT 5167:SSD 5153:UDF 5138:HSF 5119:Sun 5115:ZFS 5105:zFS 5095:XFS 5021:RFS 4984:QFS 4979:PFS 4959:NSS 4912:LFS 4907:JFS 4886:MVS 4882:HFS 4872:HFS 4833:FAT 4806:ext 4781:DFS 2565:hdl 2557:doi 2511:doi 2463:doi 2375:on 1812:CEA 1757:CSC 1755:at 1747:in 1723:at 1676:is 1563:TCP 1540:SAS 1453:DLM 1432:SSD 1311:SAN 1270:VFS 1263:ZFS 1223:ZFS 1194:on 1101:CPU 1097:GPU 1037:HSM 1033:TBF 1017:HDD 935:GID 931:UID 911:ZFS 895:ZFS 801:DDN 746:at 706:ZFS 627:I/O 602:, 448:1 s 380:no. 207:www 133:/fs 127:git 6895:: 6872:. 6142:80 6137:60 6132:30 6127:24 6122:10 6085:ZX 6080:LX 6065:20 6060:10 5362:9P 5030:VM 4611:/ 4264:. 4240:. 4160:. 4027:. 3952:. 3927:. 3923:. 3893:. 3889:. 3871:. 3821:. 3792:. 3762:. 3758:. 3680:. 3676:. 3650:. 3646:. 3620:. 3616:. 3590:. 3586:. 3561:. 3557:. 3526:. 3453:. 3428:. 3405:. 3295:. 3291:. 3273:. 3269:. 3173:. 3020:. 2944:. 2917:. 2883:. 2858:. 2832:. 2801:. 2790:^ 2773:. 2728:. 2680:. 2643:. 2639:. 2615:}} 2611:{{ 2603:. 2581:. 2571:. 2563:. 2527:. 2517:. 2505:. 2479:. 2469:. 2437:}} 2433:{{ 2403:}} 2399:{{ 2336:}} 2332:{{ 2200:^ 2028:. 2024:. 1993:^ 1976:. 1950:. 1946:. 1935:^ 1917:. 1905:^ 1830:, 1798:, 1794:, 1759:, 1751:, 1577:, 1546:, 1544:FC 1542:, 1337:. 1206:, 1190:, 1178:, 1067:. 1035:, 996:. 980:, 818:. 652:, 640:, 636:, 610:. 485:, 481:, 458:No 336:No 194:, 6882:. 6117:5 6112:2 6107:1 6055:5 6050:4 6045:2 6040:1 5972:e 5965:t 5958:v 5373:) 5369:( 5121:) 5117:( 5111:) 5107:( 5032:) 4888:) 4884:( 4878:) 4874:( 4760:) 4656:e 4649:t 4642:v 4471:. 4432:. 4406:. 4381:. 4356:. 4317:. 4279:. 4246:. 4225:. 4207:. 4189:. 4171:. 4081:. 4067:. 4043:. 4013:. 3988:. 3963:. 3938:. 3905:. 3857:. 3832:. 3803:. 3774:. 3744:. 3718:. 3692:. 3662:. 3632:. 3602:. 3572:. 3542:. 3511:. 3487:. 3469:. 3439:. 3421:. 3390:. 3368:. 3341:. 3252:. 3231:. 3211:. 3185:. 3155:. 3112:. 3090:. 3075:. 3054:. 3031:. 2960:. 2929:. 2899:. 2869:. 2843:. 2817:. 2784:. 2744:. 2714:. 2691:. 2662:. 2621:) 2589:. 2567:: 2559:: 2535:. 2513:: 2487:. 2465:: 2443:) 2409:) 2342:) 2231:. 2194:. 2152:. 2127:. 2106:. 2086:. 2065:. 2039:. 1987:. 1962:. 1929:. 1672:. 948:( 933:/ 156:C 94:) 65:) 38:. 20:)

Text is available under the Creative Commons Attribution-ShareAlike License. Additional terms may apply.