Knowledge (XXG)

Parchive

Source đź“ť

197:
most useful in version 1 where the parity volumes were much larger than the short index files. These larger parity volumes contain the actual recovery data along with a duplicate copy of the information in the index files (which allows them to be used on their own to verify the integrity of the data files if there is no small index file available).
230:
In January 2002, Howard Fukada proposed that a new Par2 specification should be devised with the significant changes that data verification and repair should work on blocks of data rather than whole files, and that the algorithm should switch to using 16 bit numbers rather than the 8 bit numbers that
144:
is a discontinued experimental version developed by MultiPar author Yutaka Sawada. The original SourceForge Parchive project has been inactive since April 30, 2015. A new PAR3 specification has been worked on since April 28, 2019 by PAR2 specification author Michael Nahas. An alpha version of the
196:
Because the index files were so small, they minimized the amount of extra data that had to be downloaded from Usenet to verify that the data files were all present and undamaged, or to determine how many parity volumes were required to repair any damage or reconstruct any missing files. They were
188:
With the introduction of Parchive, parity files could be created that were then uploaded along with the original data files. If any of the data files were damaged or lost while being propagated between Usenet servers, users could download parity files and use them to reconstruct the damaged or
238:
and par2cmdline. Abandoned since 2004, Paul Houle created phpar2 to supersede par2cmdline. Yutaka Sawada created MultiPar to supersede QuickPar. MultiPar uses par2j.exe (which is partially based on par2cmdline's optimization techniques) to use as MultiPar's backend engine.
231:
PAR1 used. Michael Nahas and Peter Clements took up these ideas in July 2002, with additional input from Paul Nettle and Ryan Gallagher (who both wrote Par1 clients). Version 2.0 of the Parchive specification was published by Michael Nahas in September 2002.
363:
A discussion on a new format started in the GitHub issue section of the maintained fork par2cmdline on January 29, 2019. The discussion led to a new format which is also named as Par3. The new Par3 format's specification is
341:
Par2 specification supports up to 32,768 source blocks and up to 65,535 recovery blocks. Input files are split into multiple equal-sized blocks so that recovery files do not need to be the size of the largest input file.
200:
In July 2001, Tobias Rieper and Stefan Wehlus proposed the Parity Volume Set specification, and with the assistance of other project members, version 1.0 of the specification was published in October 2001. Par1 used
219:
The recovery files had to be the size of the largest input file, so it did not work well when the input files were of various sizes. (This limited its usefulness when not paired with the proprietary RAR compression
551:, and uses Yutaka Sawada's par2j.exe as the PAR2 backend. MultiPar supports multiple languages by Unicode. The name of MultiPar was derived from "multi-lingual PAR client". MultiPar is also verified to work with 161:
was not designed to transmit arbitrary binary data. Another limitation, which was acceptable for conversations but not for files, was that messages were normally fairly short in length and limited to 7-bit
368:, but remains being an alpha draft as of January 28, 2022. The specification is written by Michael Nahas, the author of Par2 specification, with the help from Yutaka Sawada, animetosho and malaire. 326:
indicates the number of the first recovery block within the PAR2 file. If an index file of a download states that 4 blocks are missing, the easiest way to repair the files would be by downloading
360:
The Par3 specification was originally planned to be published as an enhancement over the Par2 specification. However, to date, it has remained closed source by specification owner Yutaka Sawada.
711:
The original idea behind this project was to provide a tool to apply the data-recovery capability concepts of RAID-like systems to the posting and recovery of multi-part archives on Usenet.
291:
given all of the other original files and any one of the parity volumes. Alternatively, it is possible to recreate two missing files from any two of the parity volumes and so forth.
189:
missing files. Parchive included the construction of small index files (*.par in version 1 and *.par2 in version 2) that do not contain any recovery data. These indexes contain
399:
Supports "incremental backups", where a user creates recovery files for some file or folder, change some data, and create new recovery files reusing some of the older files.
563:, and may work with other operating systems too. Although the Par2 components are (or will be) open source, the MultiPar GUI on top of them is currently not open source. 773: 526:) â€” Produce, verify and repair par2 files recursively, both on the command line as well as with the aid of a graphical user interface. It is available in the 798: 671: 687: 1023: 1081: 1045: 748: 723: 690:, reply #3, by Yutaka Sawada: "Their formal title are "Parity Volume Set Specification 1.0" and "Parity Volume Set Specification 2.0." 202: 678:
reply #3, by Yutaka Sawada: "Their formal title are "Parity Volume Set Specification 1.0" and "Parity Volume Set Specification 2.0."
523: 911: 873: 126: 974: 575: â€” advanced par2cmdline with multithreading and highly optimized assemblercode (about 66% faster than QuickPar 0.9.1) 185:. Large files were broken up to reduce the effect of a corrupted download, but the unreliable nature of Usenet remained. 638: 777: 544: 802: 499: 449: 1007: 453: 205:
to create new recovery files. Any of the recovery files can be used to rebuild a missing file from an incomplete
1148: 1143: 1069: 667: 445: 378:
Supports packing small files into one block, as well as deduplication when a block appears in multiple files.
117:, and accidental or malicious damage. Despite the name, Parchive uses more advanced techniques (specifically 352:
Directory support is included in the PAR2 specification, but most or all implementations do not support it.
844: 365: 407: 118: 349:
is mentioned in the PAR2 specification as an option, most PAR2 implementations do not support Unicode.
145:
PAR3 specification has been published on January 29, 2022 while the program itself is being developed.
393: 887: 527: 596: 226:
It was strongly tied to Usenet and it was felt that a more general tool might have a wider audience.
36: 1085: 1019: 1015: 1000: 995: 389: 1049: 932: 752: 727: 645:
are capable of integrating parity data into their formats for error detection and correction:
953: 877: 223:
The recovery algorithm had a bug, due to a flaw in the academic paper on which it was based.
114: 863: 620: 1153: 1011: 675: 552: 516:
library. Does not support verifying or repair, it can currently only create PAR2 archives.
157:. Usenet was originally designed for informal conversations, and the underlying protocol, 106: 465: 1118: 178: 91: 651:– RAID levels at and above RAID 5 make use of parity data to detect and repair errors. 1137: 642: 475: 461: 275:), which is CRC type file with no recovery blocks, and a number of "parity volumes" ( 190: 95: 1128: 996:"Finding movies (or TV shows): Recovering missing RAR files with PAR and PAR2 files" 591: 479: 859: 371:
The new format claims to have multiple advantages over the Par2 format, including:
153:
Parchive was intended to increase the reliability of transferring files via Usenet
79: 55: 51: 1123: 193:
that can be used to quickly identify the target files and verify their integrity.
101:
Parchive was originally written to solve the problem of reliable file sharing on
248: 67: 890: 867: 1113: 491: 122: 973:
Nahas, Michael; Clements, Peter; Nettle, Paul; Gallagher, Ryan (2003-05-11).
954:"A Tutorial on Reed-Solomon Coding for Fault-Tolerance in RAID-like Systems" 823: 626: 502:, or later. Currently merged into BlackIkeEagle's fork and maintained there. 322:
in the filename indicates how many blocks it contains, and the number after
212:
Version 1 became widely used on Usenet, but it did suffer some limitations:
154: 519: 615: 485: 578: 566: 235: 234:
Peter Clements then went on to write the first two Par2 implementations,
206: 170: 110: 87: 700: 572: 17: 513: 505: 452:) â€” a concurrent (multithreaded) version of par2cmdline 0.4 using 346: 845:"Parity Volume Set Specification 3.0 [2022-01-28 ALPHA DRAFT]" 283:, etc.). Given all of the original files except for one (for example, 882: 560: 556: 531: 495: 418: 174: 102: 98:
operations that can repair or regenerate corrupted or missing data.
338:, it is identical in function to the small index file used in PAR1. 1108: 471: 169:
Various techniques were devised to send files over Usenet, such as
608: 569:(freeware) â€” unmaintained since 2004, superseded by MultiPar. 382: 163: 648: 403: 182: 158: 933:"Note: Correction to the 1997 Tutorial on Reed-Solomon Coding" 548: 509: 457: 414: 294:
Par1 supports up to a total of 256 source and recovery files.
1119:
Slyck's Guide To The Usenet Newsgroups: PAR & PAR2 Files
799:"PAR3 specification proposal not finished as of April 2011" 581: â€” First PAR implementation, unmaintained since 2001. 105:, but it can be used for protecting any kind of data from 1109:
Parchive project - full specifications and math behind it
847:. Michael Nahas, Yutaka-Sawada, animetosho, and malaire. 402:
Supports more error correction code algorithms (such as
385:
file names, file permissions, hard links and soft links.
251:
are incompatible. (However, many clients support both.)
547:(freeware)  â€” Builds upon QuickPar's features and 512:) is a high performance, multithreaded PAR2 client and 388:
Supports embedding PAR data inside other formats, like
302:
Par2 files generally use this naming/extension system:
494:
is another multithreaded version of par2cmdline using
47: 35: 999: 375:Supports more than 2 files and more than 2 blocks. 334:is also acceptable. There is also an index file 216:It was restricted to handle at most 255 files. 688:Re: Correction to Parchive on Knowledge (XXG) 668:Re: Correction to Parchive on Knowledge (XXG) 8: 774:"Risk of undetectable intended modification" 30: 1129:UsenetReviewz's guide to opening par files 931:Plank, James S.; Ding, Ying (April 2003). 271:, the Parchive consists of an index file ( 881: 1082:"contacted you, asking about sourcecode" 287:), it is possible to create the missing 660: 430:Supports hard links and symbolic links. 177:. Later Usenet software allowed 8 bit 912:"Parity Volume Set Specification v1.0" 181:, which permitted new techniques like 29: 975:"Parity Volume Set Specification 2.0" 701:"Parchive: Parity Archive Volume Set" 474: â€” (obsolete). Available in the 7: 749:"Question about your usage of PAR3" 460:based CPUs. It is available in the 952:Plank, James S. (September 1997). 330:. However, due to the redundancy, 140:is mature for widespread use, and 25: 876:. p. 5. sec. 2.2. 488:maintained fork by BlackIkeEagle. 94:, with the capability to perform 42:.par, .par2, .p??, (.par3 future) 1124:Guide to repair files using PAR2 1070:Working on Ubuntu 18.04 via wine 1046:"MultiPar works with PCBSD 9.0" 824:"Parchive: Parity Archive Tool" 76:Parity Volume Set Specification 869:Network News Transfer Protocol 724:"possibility of new PAR3 file" 611:conforming operating systems: 1: 910:Nahas, Michael (2001-10-14). 203:Reed–Solomon error correction 1114:Introduction to PAR and PAR2 1002:Steal this File Sharing Book 994:Wang, Wallace (2004-10-25). 639:Comparison of file archivers 318:, etc. The number after the 424:Supports empty directories. 417:hash function in Par2 with 1170: 427:Supports file permissions. 1008:San Francisco, California 247:Versions 1 and 2 of the 74:, and formally known as 456:. Only compatible with 332:filename.vol007+06.PAR2 328:filename.vol003+04.PAR2 316:filename.vol007+06.PAR2 312:filename.vol003+04.PAR2 308:filename.vol001+02.PAR2 304:filename.vol000+01.PAR2 623:, a frontend for par2. 119:error correction codes 82:system that produces 528:Python Package Index 472:Original par2cmdline 408:sparse random matrix 259:For Par1, the files 366:published on GitHub 32: 27:Type of file format 674:2014-10-14 at the 121:) than simplistic 37:Filename extension 1025:978-1-59327-050-6 864:"Character Codes" 862:(February 1986). 592:MacPAR deLuxe 4.2 61: 60: 16:(Redirected from 1161: 1097: 1096: 1094: 1093: 1084:. Archived from 1078: 1072: 1067: 1061: 1060: 1058: 1057: 1048:. Archived from 1042: 1036: 1035: 1033: 1032: 1006:(1st ed.). 1005: 991: 985: 984: 982: 981: 970: 964: 963: 961: 960: 949: 943: 942: 940: 939: 928: 922: 921: 919: 918: 907: 901: 900: 898: 897: 885: 883:10.17487/RFC0977 855: 849: 848: 841: 835: 834: 832: 831: 820: 814: 813: 811: 810: 801:. Archived from 795: 789: 788: 786: 785: 776:. Archived from 770: 764: 763: 761: 760: 751:. Archived from 745: 739: 738: 736: 735: 726:. Archived from 720: 714: 713: 708: 707: 697: 691: 685: 679: 665: 90:verification of 33: 21: 1169: 1168: 1164: 1163: 1162: 1160: 1159: 1158: 1149:Data management 1144:Archive formats 1134: 1133: 1105: 1100: 1091: 1089: 1080: 1079: 1075: 1068: 1064: 1055: 1053: 1044: 1043: 1039: 1030: 1028: 1026: 1012:No Starch Press 993: 992: 988: 979: 977: 972: 971: 967: 958: 956: 951: 950: 946: 937: 935: 930: 929: 925: 916: 914: 909: 908: 904: 895: 893: 858:Kantor, Brian; 857: 856: 852: 843: 842: 838: 829: 827: 826:. 30 April 2015 822: 821: 817: 808: 806: 797: 796: 792: 783: 781: 772: 771: 767: 758: 756: 747: 746: 742: 733: 731: 722: 721: 717: 705: 703: 699: 698: 694: 686: 682: 676:Wayback Machine 666: 662: 658: 635: 605: 588: 541: 466:par2cmdline-tbb 442: 437: 394:ISO disk images 358: 300: 257: 245: 151: 127:error detection 107:data corruption 43: 28: 23: 22: 15: 12: 11: 5: 1167: 1165: 1157: 1156: 1151: 1146: 1136: 1135: 1132: 1131: 1126: 1121: 1116: 1111: 1104: 1103:External links 1101: 1099: 1098: 1073: 1062: 1037: 1024: 986: 965: 944: 923: 902: 850: 836: 815: 790: 765: 740: 715: 692: 680: 659: 657: 654: 653: 652: 646: 643:file archivers 634: 631: 630: 629: 624: 618: 616:Par2 for KDE 4 604: 601: 600: 599: 594: 587: 584: 583: 582: 576: 570: 564: 540: 537: 536: 535: 517: 503: 492:par2cmdline-mt 489: 483: 469: 441: 440:Multi-Platform 438: 436: 433: 432: 431: 428: 425: 422: 411: 400: 397: 386: 379: 376: 357: 354: 299: 296: 256: 253: 244: 241: 228: 227: 224: 221: 217: 179:Extended ASCII 150: 147: 92:data integrity 72:parity archive 59: 58: 49: 48:Type of format 45: 44: 41: 39: 26: 24: 14: 13: 10: 9: 6: 4: 3: 2: 1166: 1155: 1152: 1150: 1147: 1145: 1142: 1141: 1139: 1130: 1127: 1125: 1122: 1120: 1117: 1115: 1112: 1110: 1107: 1106: 1102: 1088:on 2013-09-26 1087: 1083: 1077: 1074: 1071: 1066: 1063: 1052:on 2013-09-28 1051: 1047: 1041: 1038: 1027: 1021: 1017: 1013: 1009: 1004: 1003: 997: 990: 987: 976: 969: 966: 955: 948: 945: 934: 927: 924: 913: 906: 903: 892: 889: 884: 879: 875: 871: 870: 865: 861: 860:Lapsley, Phil 854: 851: 846: 840: 837: 825: 819: 816: 805:on 2014-03-09 804: 800: 794: 791: 780:on 2014-03-09 779: 775: 769: 766: 755:on 2014-03-09 754: 750: 744: 741: 730:on 2012-07-07 729: 725: 719: 716: 712: 702: 696: 693: 689: 684: 681: 677: 673: 669: 664: 661: 655: 650: 647: 644: 640: 637: 636: 632: 628: 625: 622: 619: 617: 614: 613: 612: 610: 607:Software for 602: 598: 595: 593: 590: 589: 585: 580: 577: 574: 571: 568: 565: 562: 558: 554: 550: 546: 543: 542: 538: 533: 529: 525: 521: 518: 515: 511: 507: 504: 501: 497: 493: 490: 487: 484: 481: 477: 476:FreeBSD Ports 473: 470: 467: 463: 462:FreeBSD Ports 459: 455: 451: 447: 444: 443: 439: 434: 429: 426: 423: 420: 416: 413:Replaced the 412: 409: 405: 401: 398: 395: 391: 387: 384: 380: 377: 374: 373: 372: 369: 367: 361: 355: 353: 350: 348: 343: 339: 337: 336:filename.PAR2 333: 329: 325: 321: 317: 313: 309: 305: 297: 295: 292: 290: 286: 282: 278: 274: 270: 266: 262: 254: 252: 250: 242: 240: 237: 232: 225: 222: 218: 215: 214: 213: 210: 208: 204: 198: 194: 192: 186: 184: 180: 176: 172: 167: 165: 160: 156: 148: 146: 143: 139: 136:is obsolete, 135: 130: 128: 124: 120: 116: 112: 108: 104: 99: 97: 96:data recovery 93: 89: 85: 81: 77: 73: 69: 65: 57: 53: 50: 46: 40: 38: 34: 19: 1090:. Retrieved 1086:the original 1076: 1065: 1054:. Retrieved 1050:the original 1040: 1029:. Retrieved 1001: 989: 978:. Retrieved 968: 957:. Retrieved 947: 936:. Retrieved 926: 915:. Retrieved 905: 894:. Retrieved 868: 853: 839: 828:. Retrieved 818: 807:. Retrieved 803:the original 793: 782:. Retrieved 778:the original 768: 757:. Retrieved 753:the original 743: 732:. Retrieved 728:the original 718: 710: 704:. Retrieved 695: 683: 663: 606: 390:ZIP archives 370: 362: 359: 351: 344: 340: 335: 331: 327: 323: 319: 315: 311: 307: 303: 301: 293: 288: 284: 280: 276: 272: 268: 264: 260: 258: 246: 233: 229: 211: 199: 195: 187: 168: 152: 141: 137: 133: 132:As of 2014, 131: 100: 83: 80:erasure code 75: 71: 63: 62: 56:archive file 52:Erasure code 1014:. pp.  486:par2cmdline 480:par2cmdline 249:file format 191:file hashes 125:methods of 68:portmanteau 1138:Categories 1092:2013-09-21 1056:2012-02-27 1031:2009-09-24 980:2009-10-29 959:2009-10-29 938:2009-10-29 917:2017-06-19 896:2009-10-29 830:2020-05-20 809:2012-07-01 784:2012-07-01 759:2012-07-01 734:2012-07-01 706:2009-10-29 656:References 627:GPar2 2.03 621:PyPar2 1.4 530:system as 478:system as 464:system as 171:uuencoding 155:newsgroups 86:files for 381:Supports 345:Although 672:Archived 633:See also 586:Mac OS X 567:QuickPar 545:MultiPar 532:par2deep 524:LGPL-3.0 520:par2deep 446:par2+tbb 435:Software 243:Versions 236:QuickPar 207:download 111:disc rot 88:checksum 78:) is an 64:Parchive 31:Parchive 18:SmartPAR 1016:164–167 641:– Some 539:Windows 514:Node.js 347:Unicode 267:, ..., 149:History 115:bit rot 1154:Usenet 1022:  597:UnRarX 579:Mirror 573:phpar2 561:Ubuntu 557:TrueOS 555:under 506:ParPar 496:OpenMP 419:BLAKE3 220:tool.) 175:Base64 166:text. 123:parity 103:Usenet 609:POSIX 603:POSIX 500:GPLv2 450:GPLv2 383:UTF-8 324:"vol" 281:f.p02 277:f.p01 273:f.par 164:ASCII 1020:ISBN 874:IETF 649:RAID 559:and 553:Wine 406:and 404:LDPC 356:Par3 298:Par2 255:Par1 183:yEnc 173:and 159:NNTP 142:PAR3 138:PAR2 134:PAR1 891:977 888:RFC 878:doi 549:GUI 510:CC0 458:x86 454:TBB 415:MD5 392:or 320:"+" 84:par 70:of 66:(a 1140:: 1018:. 1010:: 998:. 886:. 872:. 866:. 709:. 670:, 498:, 410:). 314:, 310:, 306:, 289:f2 285:f2 279:, 269:fn 265:f2 263:, 261:f1 209:. 129:. 113:, 109:, 54:, 1095:. 1059:. 1034:. 983:. 962:. 941:. 920:. 899:. 880:: 833:. 812:. 787:. 762:. 737:. 534:. 522:( 508:( 482:. 468:. 448:( 421:. 396:. 20:)

Index

SmartPAR
Filename extension
Erasure code
archive file
portmanteau
erasure code
checksum
data integrity
data recovery
Usenet
data corruption
disc rot
bit rot
error correction codes
parity
error detection
newsgroups
NNTP
ASCII
uuencoding
Base64
Extended ASCII
yEnc
file hashes
Reed–Solomon error correction
download
QuickPar
file format
Unicode
published on GitHub

Text is available under the Creative Commons Attribution-ShareAlike License. Additional terms may apply.

↑