Knowledge (XXG)

Maildir

Source đź“ť

954:
overwrite existing mails.¶ Step 2 is pointless because there's a race condition between steps 2 and 3. PID/host combination by itself should already guarantee that it never finds such a file. If it does, something's broken and the stat() check won't help since another process might be doing the same thing at the same time, and you end up writing to the same file in tmp/, causing the mail to get corrupted.¶ In step 4 the link() would fail if an identical file already existed in the Maildir, right? Wrong. The file may already have been moved to cur/ directory, and since it may contain any number of flags by then you can't check with a simple stat() anymore if it exists or not.¶ Step 2 was pointed out to be useful if clock had moved backwards. However, this doesn't give any actual safety guarantees because an identical base filename could already exist in cur/. Besides if the system was just rebooted, the file in tmp/ could probably be even overwritten safely (assuming it wasn't already link()ed to new/).¶ So really, all that's important in not getting mails overwritten in your Maildir is step 1: Always create filenames that are guaranteed to be unique. Forget about the 2 second waits and such that the Qmail's man page talks about.
477:. The "2" specifies the version of the information that follows the comma. "2" is the only currently officially specified version, "1" being an experimental version. The specification defines flags that show whether the message has been read, deleted and so on: the initial (capital) letter of "Passed", "Replied", "Seen", "Trashed", "Draft", and "Flagged". Applications often choose to supplement this very limited set of flags, for example notmuch offers flag synchronization in addition to arbitrary user-defined flags, while Dovecot uses lowercase letters to match 26 IMAP keywords, which may include keywords such as $ 953:
All this trouble is rather pointless. Only the first step is what really guarantees that the mails won't get overwritten, the rest just sounds nice. Even though they might catch a problem once in a while, they give no guaranteed protection and will just as easily pass duplicate filenames through and
118:
extension to the Maildir format to support subfolders and mail quotas. Maildir++ directories contain subdirectories with names that start with a '.' (dot) which are also Maildir++ folders. The extension complies with the original Maildir specification, which allows for subdirectories in addition to
42:
circa 1995, with a major goal of eliminating the need for program code to handle file locking and unlocking through use of the local filesystem. Maildir design reflects the fact that the only operations valid for an email message is that it be created, deleted or have its status changed in some way.
205:
By 2003, the recommendations had been further amended to require that instead of the PID and counter, the middle part of the filename should be created by "concatenating enough of the following strings to guarantee uniqueness" even in the face of multiple simultaneous deliveries to the same maildir
510:
As there is currently no agreement on what character this alternative separator should be, there can be interoperability difficulties between different Maildir-supporting programs on these systems. However, not all Maildir-related software needs to know what the separator character is, because not
511:
all Maildir-related software needs to be able to read or modify the flags of a message ("read", "replied to" etc.); software that merely delivers to a Maildir or archives old messages from it based only on date, should work no matter what separator is in use. If only the
295:
is (in hexadecimal) the UNIX device number of this file. Unfortunately, device numbers aren't always available through NFS. (Device numbers are also not helpful with the standard UNIX filesystem: a maildir has to be within a single UNIX device for
201:
By 2000, the qmail author recommended in an updated specification to append the value of a per-process counter to the PID, whose value should be incremented after each delivery. The rate-limiting recommendation to "wait two seconds" was dropped.
406:. Any leftover file will eventually be deleted. This sequence guarantees that a maildir-reading program will not see a partially written message. There can be multiple programs reading a maildir at the same time. They range from 238:
system call, which reports the number of times that the system has been booted. Together with #, this guarantees uniqueness; unfortunately, most operating systems don't support
87:
subdirectory temporarily stores e-mail messages that are in the process of being delivered. This subdirectory may also store other kinds of temporary files.
941: 515:
needs to read or modify message flags, and only one MUA is used, then non-standard alternative separators may be used without interoperability problems.
1093: 465:
technique is non-atomic and may result in duplicated messages. An informational suffix is appended to filenames at this stage. It consists of a
782: 834: 746: 484:
Although Maildir was intended to allow lockless usage, in practice some software that uses Maildirs also uses locks, such as Dovecot.
1061: 478: 438: 411: 47: 1260: 772: 504: 147:
that delivers an email message into a Maildir. The mail delivery agent creates a new file with a unique filename in the
808: 583: 38:
with a unique name, and each mail folder is a file system directory containing these files. Maildir was designed by
1223: 912: 1011: 866: 60: 937: 223:
system call, which returns a number that increases by 1 every time it is called, starting from 0 after reboot.
151:
directory. At the time of its invention guaranteeing unique filenames efficiently was difficult. The original
500: 94:
subdirectory stores messages that have been delivered, but have not yet been seen by any mail application.
1089: 981: 713: 666: 361:
had made no further changes to the 2003 filename generation recommendations. On modern POSIX systems,
678: 672: 434: 415: 965: 904: 858: 742: 718: 705: 590: 537: 524: 466: 358: 281: 140: 111: 39: 602: 555: 531: 503:) can use a non-standard alternative separator, such as ";" or "-". It is often trivial to patch 390: 351: 186: 166: 1137: 889: 1240: 778: 496: 265:. Unfortunately, some operating systems don't include cryptographic random number generators. 474: 144: 1191: 830: 750: 969: 656: 492:
The Maildir standard can only be implemented on systems that accept colons in filenames.
407: 985: 622: 426:, which may or may not be aware of the maildir structure. Readers should never look in 362: 1254: 1085: 933: 347: 197:
create a file with the unique filename and write the message contents to the new file
35: 469:(to separate the unique part of the filename from the actual information), a "2", a 512: 374:
The delivery process stores the message in the maildir by creating and writing to
280:
number of this file. Unfortunately, inode numbers aren't always available through
1114: 1228: 637: 419: 31: 101:
subdirectory stores messages that have already been seen by mail applications.
346:
This 2003 algorithm was criticised in 2006 as being unnecessarily complex by
617: 573: 395: 179: 160: 721:– experimental and “disabled by default because there are still many bugs” 804: 1138:
mutt maildir support: workaround for filesystems that don't accept colons
608: 597: 543: 173: 457:
new messages". This moving needs to be done using the atomic filesystem
684: 648: 613: 367: 998:
That specification requires that the action of the function be atomic.
1245: 908: 862: 23: 394:, which is atomic in many systems. Alternatively, it can be done by 616:, a Maildir-aware mail-retrieval and delivery agent alternative to 441:
server, or a mail user agent acting locally) finds messages in the
46: 700: 694: 567: 561: 470: 423: 277: 178:
concatenate the above three values into a string separated by the
152: 27: 540:
SMTP and IMAP server, for which the Maildir++ format was invented
1218: 689: 642: 549: 410:(MUAs), which access the server's file system directly, through 669:
previously the official GNOME mail reader (prior to Evolution)
418:
servers acting on behalf of remote MUAs, to utilities such as
341:
is (in decimal) the number of deliveries made by this process.
627: 495:
Systems that don't allow colons in filenames (this includes
1167: 1037: 631: 1149: 257:
is (in hexadecimal) the output of the operating system's
234:
is (in hexadecimal) the output of the operating system's
219:
is (in hexadecimal) the output of the operating system's
710:
Notmuch (fast, global-search and tag-based email system)
34:, rather than in a database. Each message is assigned a 191:
reports that the filename exists, then wait two seconds
645:, synchronises mailboxes, supporting Maildir and IMAP4 564:
SMTP server, for which the Maildir format was invented
315:
is (in decimal) the microsecond counter from the same
579: 194:
go to previous step until the filename does not exist
651:, a minimal Maildir-aware MDA implemented in Haskell 1115:"mailbox — Manipulate mailboxes in various formats" 453:. It is just a means to notify the user "you have 433:When a cognizant maildir-reading process (either a 798: 796: 794: 1150:"aerc - the world's best email client homepage" 737: 735: 899: 897: 853: 851: 261:system call or an equivalent source, such as 8: 663:aerc (efficient and extensible email client) 319:used for the left part of the unique name. 885: 883: 67:) usually has three subdirectories named 938:"Maildir Mailbox Format: Mail Delivery'" 634:mailboxes between any number of replicas 45: 731: 519:Software that supports Maildir directly 582:, SMTP and IMAP server implemented in 7: 110:Sam Varshavchik, the author of the 1012:"Management of maildir structures" 357:As of November 2023, qmail author 26:format is a common way of storing 14: 1090:"Maildir Mailbox Format: Locking" 402:and then unlinking the file from 1010:Sam Varshavchik (25 July 2016). 488:File-system compatibility issues 412:Internet Message Access Protocol 155:algorithm for unique names was: 114:and other software, defined the 1096:from the original on 2024-06-24 944:from the original on 2024-06-24 915:from the original on 2003-04-01 869:from the original on 2000-09-02 837:from the original on 2024-05-29 811:from the original on 2024-04-17 388:. The moving can be done using 381:, and then moving this file to 365:can be safely created with the 330:is (in decimal) the process ID. 1168:"Notmuch mail system homepage" 1062:"notmuch 0.38.3 documentation" 1038:"Notmuch mail system homepage" 507:to use a different separator. 1: 505:free and open-source software 276:is (in hexadecimal) the UNIX 890:Dovecot Wiki: maildir format 681:, official GNOME mail client 206:from one or more processes: 499:and some configurations of 1277: 1224:MH Message Handling System 675:a curses-based mail reader 182:; this is the new filename 829:Varshavchik, Sam (2011). 803:Varshavchik, Sam (2009). 259:unix_cryptorandomnumber() 1192:"Maildir in Thunderbird" 546:The original SMTP server 1241:manual page for maildir 501:Novell Storage Services 481:or user-defined flags. 1246:maildir specifications 909:"Using maildir format" 863:"Using maildir format" 771:Blum, Richard (2001). 344: 51: 1261:Email storage formats 905:Bernstein., Daniel J. 859:Bernstein., Daniel J. 461:, as the alternative 240:unix_sequencenumber() 221:unix_sequencenumber() 208: 49: 1119:Python documentation 743:Bernstein, Daniel J. 726:Notes and references 580:Stalwart Mail Server 416:Post Office Protocol 371:C library function. 777:. Sams Publishing. 719:Mozilla Thunderbird 538:Courier Mail Server 141:mail delivery agent 135:Technical operation 112:Courier Mail Server 40:Daniel J. Bernstein 167:process identifier 52: 50:Internal structure 784:978-0-672-32114-6 697:, KDE mail reader 497:Microsoft Windows 350:, the creator of 244:unix_bootnumber() 236:unix_bootnumber() 172:read the current 165:read the current 159:read the current 1268: 1206: 1205: 1203: 1202: 1188: 1182: 1181: 1179: 1178: 1164: 1158: 1157: 1146: 1140: 1135: 1129: 1128: 1126: 1125: 1111: 1105: 1104: 1102: 1101: 1082: 1076: 1075: 1073: 1072: 1058: 1052: 1051: 1049: 1048: 1034: 1028: 1027: 1025: 1023: 1007: 1001: 1000: 995: 993: 978: 972: 963: 957: 956: 950: 949: 930: 924: 923: 921: 920: 901: 892: 887: 878: 877: 875: 874: 855: 846: 845: 843: 842: 826: 820: 819: 817: 816: 800: 789: 788: 768: 762: 761: 759: 758: 749:. Archived from 739: 630:, synchronising 463:link-then-unlink 460: 452: 444: 429: 408:mail user agents 405: 401: 393: 387: 380: 370: 359:Daniel Bernstein 318: 303: 299: 264: 260: 245: 241: 237: 222: 189: 180:period character 150: 100: 93: 86: 78: 74: 70: 66: 1276: 1275: 1271: 1270: 1269: 1267: 1266: 1265: 1251: 1250: 1237: 1215: 1210: 1209: 1200: 1198: 1190: 1189: 1185: 1176: 1174: 1172:notmuchmail.org 1166: 1165: 1161: 1148: 1147: 1143: 1136: 1132: 1123: 1121: 1113: 1112: 1108: 1099: 1097: 1084: 1083: 1079: 1070: 1068: 1060: 1059: 1055: 1046: 1044: 1042:notmuchmail.org 1036: 1035: 1031: 1021: 1019: 1009: 1008: 1004: 991: 989: 980: 979: 975: 970:Wayback Machine 964: 960: 947: 945: 932: 931: 927: 918: 916: 903: 902: 895: 888: 881: 872: 870: 857: 856: 849: 840: 838: 828: 827: 823: 814: 812: 802: 801: 792: 785: 770: 769: 765: 756: 754: 741: 740: 733: 728: 660: 594: 591:Delivery agents 528: 521: 490: 458: 450: 442: 427: 403: 399: 389: 382: 375: 366: 363:temporary files 316: 301: 297: 262: 258: 243: 239: 235: 220: 187: 148: 137: 108: 98: 91: 84: 76: 72: 68: 64: 57: 17: 12: 11: 5: 1274: 1272: 1264: 1263: 1253: 1252: 1249: 1248: 1243: 1236: 1235:External links 1233: 1232: 1231: 1226: 1221: 1214: 1211: 1208: 1207: 1183: 1159: 1141: 1130: 1106: 1086:Sirainen, Timo 1077: 1066:notmuch-config 1053: 1029: 1018:(Mailing list) 1002: 986:The Open Group 973: 958: 934:Sirainen, Timo 925: 893: 879: 847: 821: 790: 783: 763: 730: 729: 727: 724: 723: 722: 716: 711: 708: 703: 698: 692: 687: 682: 676: 670: 664: 659: 654: 653: 652: 646: 640: 635: 625: 620: 611: 606: 605:delivery agent 600: 593: 588: 587: 586: 577: 571: 565: 559: 553: 547: 541: 535: 527: 522: 520: 517: 489: 486: 445:directory, it 385:uniquefilename 378:uniquefilename 343: 342: 331: 320: 317:gettimeofday() 305: 285: 266: 247: 224: 199: 198: 195: 192: 183: 176: 170: 163: 136: 133: 107: 104: 103: 102: 95: 88: 56: 55:Specifications 53: 30:messages on a 15: 13: 10: 9: 6: 4: 3: 2: 1273: 1262: 1259: 1258: 1256: 1247: 1244: 1242: 1239: 1238: 1234: 1230: 1227: 1225: 1222: 1220: 1217: 1216: 1212: 1197: 1193: 1187: 1184: 1173: 1169: 1163: 1160: 1155: 1154:aerc-mail.org 1151: 1145: 1142: 1139: 1134: 1131: 1120: 1116: 1110: 1107: 1095: 1091: 1087: 1081: 1078: 1067: 1063: 1057: 1054: 1043: 1039: 1033: 1030: 1017: 1016:courier-users 1013: 1006: 1003: 999: 987: 983: 977: 974: 971: 967: 966:Archive index 962: 959: 955: 943: 939: 935: 929: 926: 914: 910: 906: 900: 898: 894: 891: 886: 884: 880: 868: 864: 860: 854: 852: 848: 836: 832: 825: 822: 810: 806: 799: 797: 795: 791: 786: 780: 776: 775: 767: 764: 753:on 1997-10-12 752: 748: 744: 738: 736: 732: 725: 720: 717: 715: 712: 709: 707: 704: 702: 699: 696: 693: 691: 688: 686: 683: 680: 677: 674: 671: 668: 665: 662: 661: 658: 655: 650: 647: 644: 641: 639: 636: 633: 629: 626: 624: 621: 619: 615: 612: 610: 607: 604: 601: 599: 596: 595: 592: 589: 585: 581: 578: 575: 572: 569: 566: 563: 560: 557: 554: 551: 548: 545: 542: 539: 536: 533: 530: 529: 526: 523: 518: 516: 514: 508: 506: 502: 498: 493: 487: 485: 482: 480: 476: 472: 468: 464: 456: 449:move them to 448: 440: 436: 431: 425: 421: 417: 413: 409: 397: 392: 386: 379: 372: 369: 364: 360: 355: 353: 349: 348:Timo Sirainen 340: 336: 332: 329: 325: 321: 314: 310: 306: 294: 290: 286: 283: 279: 275: 271: 267: 256: 252: 248: 233: 229: 225: 218: 214: 210: 209: 207: 203: 196: 193: 190: 184: 181: 177: 175: 171: 168: 164: 162: 158: 157: 156: 154: 146: 142: 134: 132: 130: 126: 122: 117: 113: 105: 96: 89: 82: 81: 80: 63:(often named 62: 54: 48: 44: 41: 37: 33: 29: 25: 22: 16:E-mail format 1199:. Retrieved 1195: 1186: 1175:. Retrieved 1171: 1162: 1153: 1144: 1133: 1122:. Retrieved 1118: 1109: 1098:. Retrieved 1080: 1069:. Retrieved 1065: 1056: 1045:. Retrieved 1041: 1032: 1020:. Retrieved 1015: 1005: 997: 990:. Retrieved 976: 961: 952: 946:. Retrieved 928: 917:. Retrieved 871:. Retrieved 861:(c. 2000) . 839:. Retrieved 824: 813:. Retrieved 805:"maildir(5)" 773: 766: 755:. Retrieved 751:the original 747:"maildir(5)" 657:Mail readers 632:notmuch mail 525:Mail servers 509: 494: 491: 483: 473:and various 462: 454: 446: 432: 398:the file to 396:hard-linking 384: 377: 373: 356: 345: 338: 334: 327: 323: 312: 308: 292: 288: 273: 269: 263:/dev/urandom 254: 250: 231: 227: 216: 212: 204: 200: 138: 128: 124: 120: 115: 109: 58: 20: 18: 1229:MIX (email) 1196:mozilla.org 831:"Maildir++" 714:Pine/Alpine 638:OfflineIMAP 576:SMTP server 570:SMTP server 558:SMTP server 552:SMTP server 534:IMAP server 32:file system 1201:2020-12-06 1177:2019-06-22 1124:2023-06-19 1100:2024-08-09 1071:2024-04-17 1047:2019-06-22 948:2024-08-09 919:2018-11-23 873:2018-11-23 841:2024-08-09 815:2024-08-09 757:2018-11-23 59:A Maildir 907:(2003) . 679:Evolution 618:Fetchmail 574:OpenSMTPD 304:to work.) 161:Unix time 116:Maildir++ 106:Maildir++ 61:directory 1255:Category 1213:See also 1094:Archived 982:"rename" 942:Archived 913:Archived 867:Archived 835:Archived 809:Archived 745:(1995). 649:Attomail 628:muchsync 609:maildrop 598:procmail 544:Sendmail 459:rename() 337:, where 326:, where 311:, where 302:rename() 291:, where 272:, where 253:, where 230:, where 215:, where 174:hostname 1022:26 July 992:23 July 968:at the 774:Postfix 685:GNUMail 614:getmail 603:Dovecot 556:Postfix 532:Dovecot 479:MDNSent 368:mkstemp 352:Dovecot 145:program 65:Maildir 21:Maildir 988:. 2013 781:  391:rename 298:link() 188:stat() 75:, and 24:e-mail 701:mailx 695:KMail 667:Balsa 643:isync 568:MeTA1 562:qmail 475:flags 471:comma 467:colon 424:rsync 278:inode 169:(PID) 153:qmail 143:is a 28:email 1219:mbox 1024:2016 994:2016 779:ISBN 706:Mutt 690:Gnus 673:Cone 584:Rust 550:Exim 447:must 439:IMAP 422:and 420:biff 383:new/ 376:tmp/ 300:and 242:and 127:and 97:The 90:The 83:The 36:file 19:The 623:fdm 513:MUA 451:cur 443:new 437:or 435:POP 428:tmp 414:or 404:tmp 400:new 282:NFS 185:if 149:tmp 129:cur 125:new 121:tmp 99:cur 92:new 85:tmp 79:. 77:cur 73:new 69:tmp 1257:: 1194:. 1170:. 1152:. 1117:. 1092:. 1088:. 1064:. 1040:. 1014:. 996:. 984:. 951:. 940:. 936:. 911:. 896:^ 882:^ 865:. 850:^ 833:. 807:. 793:^ 734:^ 430:. 354:. 139:A 131:. 123:, 71:, 1204:. 1180:. 1156:. 1127:. 1103:. 1074:. 1050:. 1026:. 922:. 876:. 844:. 818:. 787:. 760:. 455:X 339:n 335:n 333:Q 328:n 324:n 322:P 313:n 309:n 307:M 293:n 289:n 287:V 284:. 274:n 270:n 268:I 255:n 251:n 249:R 246:. 232:n 228:n 226:X 217:n 213:n 211:#

Index

e-mail
email
file system
file
Daniel J. Bernstein

directory
Courier Mail Server
mail delivery agent
program
qmail
Unix time
process identifier
hostname
period character
stat()
inode
NFS
Timo Sirainen
Dovecot
Daniel Bernstein
temporary files
mkstemp
rename
hard-linking
mail user agents
Internet Message Access Protocol
Post Office Protocol
biff
rsync

Text is available under the Creative Commons Attribution-ShareAlike License. Additional terms may apply.

↑