Knowledge

Erasure code

Source đź“ť

33: 258: 1418:
The classical way to recover from failures in storage systems was to use replication. However, replication incurs significant overhead in terms of wasted bytes. Therefore, increasingly large storage systems, such as those used in data centers use erasure-coded storage. The most common form of erasure
1058:
Note that Alice cannot encode her telephone number in just one err-mail, because it contains six characters, and that the maximum length of one err-mail message is five characters. If she sent her phone number in pieces, asking Bob to acknowledge receipt of each piece, at least four messages would
236:
While technically RAID can be seen as a kind of erasure code, "RAID" is generally applied to an array attached to a single host computer (which is a single point of failure), while "erasure coding" generally implies multiple hosts, sometimes called a
1387:
symbol message into a practically infinite encoded form, i.e., they can generate an arbitrary amount of redundancy symbols that can all be used for error correction. Receivers can start decoding after they have received slightly more than
1398:
address the issue of rebuilding (also called repairing) lost encoded fragments from existing encoded fragments. This issue occurs in distributed storage systems where communication to maintain encoded redundancy is a problem.
1481:, 10 MB of user data is divided into ten 1MB blocks. Then, four additional 1 MB parity blocks are created to provide redundancy. This can tolerate up to 4 concurrent failures. The storage overhead here is 14/10 = 1.4X. 1491:
Initially, erasure codes were used to reduce the cost of storing "cold" (rarely-accessed) data efficiently; but erasure codes can also be used to improve performance serving "hot" (more-frequently-accessed) data.
2132:
is a near optimal extension to Luigi Rizzo's work that uses band matrices. Many parameters can be set, like the size of the width of the band and size of the finite field. It also successfully exploits the large
1484:
In the case of a fully replicated system, the 10 MB of user data will have to be replicated 4 times to tolerate up to 4 concurrent failures. The storage overhead in that case will be 50/10 = 5 times.
1495:
RAID N+M divides data blocks across N+M drives, and can recover all the data when any M drives fail. In particular, RAID 7.3 refers to triple-parity RAID, and can recover all the data when any 3 drives fail.
1163: − 1 'on the fly', i.e., distribute the workload evenly between transmission of the symbols. If the receiver wants to do his calculations 'on the fly', he can construct a new polynomial 655: 475: 540: 684: = 2, redundancy symbols may be created by sampling different points along the line between the two original symbols. This is pictured with a simple example, called err-mail: 395: 1915: 1019: 789: 845: 2034:
Dimakis, Alexandros G.; Godfrey, P. Brighten; Wu, Yunnan; Wainwright, Martin J.; Ramchandran, Kannan (September 2010). "Network Coding for Distributed Storage Systems".
915: 880: 276: 1790: 567: 1059:
have to be sent anyway (two from Alice, and two acknowledgments from Bob). So the erasure code in this example, which requires five messages, is quite economical.
1488:
This gives an idea of the lower storage overhead of erasure-coded storage compared to full replication and thus the attraction in today's storage systems.
1034: 923: 222:
As of 2023, modern data storage systems can be designed to tolerate the complete failure of a few disks without data loss, using one of 3 approaches:
1952: 1407:
Erasure coding is now standard practice for reliable data storage. In particular, various implementations of Reed-Solomon erasure coding are used by
1864: 238: 2140: 2005: 1062:
This example is a little bit contrived. For truly generic erasure codes that work over any data set, we would need something other than the
315:
code word symbols are sufficient to recover the original message (i.e., they have optimal reception efficiency). Optimal erasure codes are
1780:. The third Sino-French Workshop on Information and Communication Technologies, SIFWICT 2015, Jun 2015, Nantes, France. ffhal-01162047f 1919: 1055:(5)) he has received. Bob can perform this procedure using any two err-mails, so the erasure code in this example has a rate of 40%. 1423:, an advanced mathematics formula used to enable regeneration of missing data from pieces of known data, called parity blocks. In a ( 1420: 1369:
trade correction capabilities for computational complexity: practical algorithms can encode and decode with linear time complexity.
294: 116: 244:
Compared to block-level RAID systems, object storage erasure coding has some significant differences that make it more resilient.
1839: 1144: − 1). The receiver can now also use polynomial interpolation to recover the lost packets, provided he receives 1615: 316: 1518: 54: 575: 2149:
Developed in 1996, was the first use of FEC "Forward Error correction" on the Internet. It was first used commercially to *
1671: 97: 212: 50: 69: 2089: 2123: 407: 76: 1567: 1753: 1555: 487: 2120:
is a Free Software library implementing Reed-Solomon and Cauchy erasure code techniques with SIMD optimisations.
1995: 1627: 1079: 348: 141: 83: 1604: 1596: 43: 2166: 2053: 1644: 181: 1600: 1320: 65: 1943:. 2016. p. 2: section "Abstract". p. 9: section "Systematic codes". p. 12: section "Regenerating codes". 1868: 1609: 1550: 144:(FEC) code under the assumption of bit erasures (rather than bit errors), which transforms a message of 1940: 1725: 1395: 956: 726: 794: 2058: 1102: 208: 1636:(differs in that the original secret is encrypted and obscured until the decode quorum is reached) 1152:
is less than 2, where b is the number of bits in a symbol, then multiple polynomials can be used.
2134: 2071: 2043: 1328: 708:
Instead of asking Bob to acknowledge the messages she sends, Alice devises the following scheme.
1033: 922: 1365:
symbols to recover the message (where ε>0). Reducing ε can be done at the cost of CPU time.
2001: 1696: 1639: 885: 850: 1591: 2063: 545: 2137:
size of modern CPUs. How it compares to the near optimal codes mentioned above is unknown.
1335: 241:(RAIS). The erasure code allows operations to continue when any one of those hosts stops. 201: 90: 1599:, an MDS code outperforming Reed–Solomon in the maximal number of redundant packets, see 1029:
are the two parts of the telephone number. Now suppose Bob receives "D=777" and "E=851".
1633: 1347: 1283:
and evaluate it to find the lost packets. So both the sender and the receiver require
2160: 2075: 2021: 1530: 1513: 1408: 1372: 691: 687: 207:
There are many different erasure coding schemes. The most popular erasure codes are
133: 1545: 1535: 1324: 1083: 1415:
built into Linux, Microsoft Azure, Facebook cold storage, and Backblaze Vaults.
216: 197: 32: 1981: 941:(5), and then transmits three redundant messages: "C=703", "D=777" and "E=851". 1814: 1582: 328: 2093: 2067: 152:
symbols such that the original message can be recovered from a subset of the
17: 1777: 569:, is erased, it can be easily recovered by summing the remaining variables: 169: 2150: 1097:, but usually a power of 2. The sender numbers the data symbols from 0 to 2151:
stream live video of Sir Arthur C. Clarke in Sri Lanka to UIUC in Indiana
1577: 1540: 1791:"Understanding IBM Spectrum Scale Erasure Code Edition fault tolerance" 1776:
Dimitri Pertin, Alexandre van Kempen, Benoît Parrein, Nicolas Normand.
1459:) chunks are available, one can recover the entire data. This means a ( 542:
is now consistent with regard to the checksum. If one of these values,
1965: 1889: 1840:"Erasure coding vs Raid as a data protection method | Computer Weekly" 1412: 660: 1994:
Xia, Mingyuan; Saxena, Mohit; Blaum, Mario; Pease, David A. (2015).
1586: 1039:
Bob can reconstruct Alice's phone number by computing the values of
2048: 1672:"RAID vs. Erasure Coding. What's the Difference? | Blog | Xinnor" 2129: 1982:"Backblaze Open-sources Reed-Solomon Erasure Coding Source Code" 1571: 1504:
Here are some examples of implementations of the various codes:
1478: 1279:) can be calculated. So we have enough data points to construct 663:
is a widely-used application of the parity check erasure code.
184:. The recovery algorithm expects that it is known which of the 180:
denotes the number of symbols required for recovery, is called
251: 26: 1477:
In RS (10, 4) code, which is used in Facebook for their
2090:"home [Erasure Coding for Distributed Storage Wiki]" 720:= 629, and sends 2 messages – "A=555" and "B=629" – to Bob. 1941:"Erasure Coding for Big-data Systems: Theory and Practice" 1101: − 1 and sends them. He then constructs a 2126:
by Luigi Rizzo describes optimal erasure correction codes
694:
using err-mail. Err-mail works just like e-mail, except
2117: 272: 1815:"Object Storage Erasure Coding vs. Block Storage RAID" 1342:
symbols can be found copied, unencoded, as one of the
650:{\displaystyle v_{e}=-\sum _{i=1,i\neq e}^{k+1}v_{i}.} 1916:"Why erasure coding is the future of data resiliency" 959: 888: 853: 797: 729: 578: 548: 490: 410: 351: 2143:
for regenerating codes and rebuilding erasure codes.
1676:
The Fastest and Most Reliable Software RAID | Xinnor
1247:
has been received successfully. Secondly, if symbol
1078:
The linear construction above can be generalized to
2146: 1447:. The coding is done such that as long as at least 267:
may be too technical for most readers to understand
57:. Unsourced material may be challenged and removed. 1865:"Erasure Code: RAID As It Should Be – Huawei BLOG" 1013: 909: 874: 839: 783: 712:She breaks her telephone number up into two parts 649: 561: 534: 469: 389: 1403:Applications of erasure coding in storage systems 307:Optimal erasure codes have the property that any 1435:data blocks, called "chunks", are encoded into ( 1082:. Additionally, points are now computed over a 690:wants to send her telephone number (555629) to 148:symbols into a longer message (code word) with 1525:Near optimal fountain (rateless erasure) codes 1443:) chunks. The total set of chunks comprises a 701:Messages longer than 5 characters are illegal. 470:{\displaystyle v_{k+1}=-\sum _{i=1}^{k}v_{i}.} 1346:message symbols. (Erasure codes that support 397:, a checksum is computed and appended to the 8: 505: 491: 366: 352: 704:It is very expensive (similar to air-mail). 535:{\displaystyle \{v_{i}\}_{1\leq i\leq k+1}} 1953:"Erasure Encoding—Practice and Principles" 1935: 1933: 1931: 1929: 2057: 2047: 1976: 1974: 958: 887: 852: 796: 728: 638: 622: 599: 583: 577: 553: 547: 508: 498: 489: 458: 448: 437: 415: 409: 390:{\displaystyle \{v_{i}\}_{1\leq i\leq k}} 369: 359: 350: 295:Learn how and when to remove this message 279:, without removing the technical details. 129:Code added to allow recovery of lost data 117:Learn how and when to remove this message 1914:Bhaskaran, Dinesh Kumar (July 6, 2018). 1666: 1664: 1662: 1660: 1467:) RS-encoded storage can tolerate up to 2124:Software FEC in computer communications 2036:IEEE Transactions on Information Theory 1726:"RAID vs Erasure Coding vs Replication" 1656: 333:Parity check is the special case where 1148:symbols successfully. If the order of 239:Redundant Array of Inexpensive Servers 2147:ECIP "Erasure Code Internet Protocol" 1997:A Tale of Two Erasure Codes in {HDFS} 1747: 1745: 1719: 1717: 1323:, with code words constructed over a 1255:has been received successfully, then 698:About half of all the mail gets lost. 277:make it understandable to non-experts 7: 1778:"Comparison of RAID-6 Erasure Codes" 1311:) space for operating 'on the fly'. 55:adding citations to reliable sources 2141:Coding for Distributed Storage wiki 1697:"Ceph.io — Erasure Coding in Ceph" 1419:coding used in storage systems is 723:She constructs a linear function, 25: 1597:Erasure Resilient Systematic Code 1334:Most practical erasure codes are 1155:The sender can construct symbols 1014:{\displaystyle f(i)=a+(b-a)(i-1)} 784:{\displaystyle f(i)=a+(b-a)(i-1)} 2022:""Triple-parity RAID and beyond" 1032: 921: 840:{\displaystyle f(i)=555+74(i-1)} 317:maximum distance separable codes 256: 31: 1616:maximum distance separable code 1319:This process is implemented by 1089:First we choose a finite field 196:Erasure coding was invented by 42:needs additional citations for 1519:Low-density parity-check codes 1350:never use a systematic code). 1191:was received successfully and 1008: 996: 993: 981: 969: 963: 898: 892: 863: 857: 834: 822: 807: 801: 778: 766: 763: 751: 739: 733: 341: + 1. From a set of 1: 213:Low-density parity-check code 2092:. 2017-07-31. Archived from 1338:-- each one of the original 1199:) = 0 when symbol 1863:Kruth, Peter (2023-10-04). 1724:Lee, Brandon (2023-12-26). 945:Bob knows that the form of 2183: 1508:Near optimal erasure codes 1431:) RS code, a given set of 1381:near-optimal erasure codes 1379:) are notable examples of 1367:Near-optimal erasure codes 1359:Near-optimal erasure codes 1354:Near-optimal erasure codes 1239:) = 0 if symbol 1207:was not received. Now let 1124:) is equal to data symbol 326: 1754:"RAID Vs. Erasure Coding" 1361:require (1 + Îµ) 1315:Real world implementation 680:In the simple case where 2068:10.1109/TIT.2010.2054295 1758:www.networkcomputing.com 1628:Forward error correction 1231:). Firstly we know that 1080:polynomial interpolation 929:She computes the values 910:{\displaystyle f(2)=629} 875:{\displaystyle f(1)=555} 142:forward error correction 1383:. They can transform a 1299:)) operations and only 1093:with order of at least 667:Polynomial oversampling 1645:Binary erasure channel 1421:Reed-Solomon (RS) code 1377:rateless erasure codes 1015: 911: 876: 841: 785: 651: 633: 563: 536: 484: + 1 values 471: 453: 391: 156:symbols. The fraction 1562:Optimal erasure codes 1103:(Lagrange) polynomial 1016: 912: 877: 842: 786: 652: 595: 564: 562:{\displaystyle v_{e}} 537: 472: 433: 392: 248:Optimal erasure codes 2000:. pp. 213–226. 1966:"Erasure Coding 101" 1890:"Erasure Coding 101" 1271:) −  1223:) −  957: 886: 851: 795: 727: 576: 546: 488: 408: 349: 182:reception efficiency 51:improve this article 1605:RS(9,2) with 3 bits 1601:RS(4,2) with 2 bits 1307: −  1295: −  672:Example: Err-mail ( 209:Reed-Solomon coding 1922:on August 7, 2020. 1844:ComputerWeekly.com 1610:Regenerating Codes 1592:Reed–Solomon codes 1396:Regenerating codes 1329:Vandermonde matrix 1321:Reed–Solomon codes 1011: 907: 872: 837: 781: 647: 559: 532: 467: 387: 215:(LDPC codes), and 188:symbols are lost. 2007:978-1-931971-20-1 1640:Spelling alphabet 1392:encoded symbols. 1047:from the values ( 305: 304: 297: 127: 126: 119: 101: 16:(Redirected from 2174: 2105: 2104: 2102: 2101: 2086: 2080: 2079: 2061: 2051: 2042:(9): 4539–4551. 2031: 2025: 2020:Adam Leventhal. 2018: 2012: 2011: 1991: 1985: 1978: 1969: 1962: 1956: 1950: 1944: 1939:Rashmi Vinayak. 1937: 1924: 1923: 1918:. Archived from 1911: 1905: 1904: 1902: 1901: 1886: 1880: 1879: 1877: 1876: 1867:. Archived from 1860: 1854: 1853: 1851: 1850: 1836: 1830: 1829: 1827: 1826: 1811: 1805: 1804: 1802: 1801: 1787: 1781: 1774: 1768: 1767: 1765: 1764: 1749: 1740: 1739: 1737: 1736: 1721: 1712: 1711: 1709: 1708: 1693: 1687: 1686: 1684: 1683: 1668: 1574:storage systems. 1556:Triangular codes 1336:systematic codes 1203: <  1128:. He then sends 1036: 1020: 1018: 1017: 1012: 925: 916: 914: 913: 908: 881: 879: 878: 873: 846: 844: 843: 838: 790: 788: 787: 782: 656: 654: 653: 648: 643: 642: 632: 621: 588: 587: 568: 566: 565: 560: 558: 557: 541: 539: 538: 533: 531: 530: 503: 502: 476: 474: 473: 468: 463: 462: 452: 447: 426: 425: 396: 394: 393: 388: 386: 385: 364: 363: 300: 293: 289: 286: 280: 260: 259: 252: 122: 115: 111: 108: 102: 100: 59: 35: 27: 21: 2182: 2181: 2177: 2176: 2175: 2173: 2172: 2171: 2157: 2156: 2114: 2109: 2108: 2099: 2097: 2088: 2087: 2083: 2059:10.1.1.117.6892 2033: 2032: 2028: 2019: 2015: 2008: 1993: 1992: 1988: 1979: 1972: 1963: 1959: 1951: 1947: 1938: 1927: 1913: 1912: 1908: 1899: 1897: 1888: 1887: 1883: 1874: 1872: 1862: 1861: 1857: 1848: 1846: 1838: 1837: 1833: 1824: 1822: 1813: 1812: 1808: 1799: 1797: 1789: 1788: 1784: 1775: 1771: 1762: 1760: 1752:O'Reilly, Jim. 1751: 1750: 1743: 1734: 1732: 1723: 1722: 1715: 1706: 1704: 1695: 1694: 1690: 1681: 1679: 1670: 1669: 1658: 1653: 1624: 1564: 1527: 1510: 1502: 1405: 1375:(also known as 1356: 1317: 1076: 955: 954: 884: 883: 849: 848: 793: 792: 791:, in this case 725: 724: 678: 676: = 2) 669: 634: 579: 574: 573: 549: 544: 543: 504: 494: 486: 485: 454: 411: 406: 405: 401:source values: 365: 355: 347: 346: 331: 325: 301: 290: 284: 281: 273:help improve it 270: 261: 257: 250: 202:Gustave Solomon 194: 172:. The fraction 130: 123: 112: 106: 103: 60: 58: 48: 36: 23: 22: 15: 12: 11: 5: 2180: 2178: 2170: 2169: 2159: 2158: 2155: 2154: 2144: 2138: 2127: 2121: 2113: 2112:External links 2110: 2107: 2106: 2081: 2026: 2013: 2006: 1986: 1970: 1957: 1945: 1925: 1906: 1881: 1855: 1831: 1806: 1782: 1769: 1741: 1713: 1688: 1655: 1654: 1652: 1649: 1648: 1647: 1642: 1637: 1634:Secret sharing 1631: 1623: 1620: 1619: 1618: 1612: 1607: 1594: 1589: 1580: 1575: 1563: 1560: 1559: 1558: 1553: 1548: 1543: 1538: 1533: 1526: 1523: 1522: 1521: 1516: 1509: 1506: 1501: 1498: 1404: 1401: 1373:Fountain codes 1355: 1352: 1348:secret sharing 1316: 1313: 1263:) =  1075: 1072: 1010: 1007: 1004: 1001: 998: 995: 992: 989: 986: 983: 980: 977: 974: 971: 968: 965: 962: 943: 942: 919: 918: 906: 903: 900: 897: 894: 891: 871: 868: 865: 862: 859: 856: 836: 833: 830: 827: 824: 821: 818: 815: 812: 809: 806: 803: 800: 780: 777: 774: 771: 768: 765: 762: 759: 756: 753: 750: 747: 744: 741: 738: 735: 732: 721: 706: 705: 702: 699: 677: 670: 668: 665: 658: 657: 646: 641: 637: 631: 628: 625: 620: 617: 614: 611: 608: 605: 602: 598: 594: 591: 586: 582: 556: 552: 529: 526: 523: 520: 517: 514: 511: 507: 501: 497: 493: 478: 477: 466: 461: 457: 451: 446: 443: 440: 436: 432: 429: 424: 421: 418: 414: 384: 381: 378: 375: 372: 368: 362: 358: 354: 324: 321: 303: 302: 264: 262: 255: 249: 246: 234: 233: 232:Erasure Coding 230: 227: 193: 190: 168:is called the 128: 125: 124: 66:"Erasure code" 39: 37: 30: 24: 14: 13: 10: 9: 6: 4: 3: 2: 2179: 2168: 2167:Coding theory 2165: 2164: 2162: 2152: 2148: 2145: 2142: 2139: 2136: 2131: 2128: 2125: 2122: 2119: 2116: 2115: 2111: 2096:on 2017-07-31 2095: 2091: 2085: 2082: 2077: 2073: 2069: 2065: 2060: 2055: 2050: 2045: 2041: 2037: 2030: 2027: 2023: 2017: 2014: 2009: 2003: 1999: 1998: 1990: 1987: 1983: 1980:Brian Beach. 1977: 1975: 1971: 1967: 1964:Matt Sarrel. 1961: 1958: 1954: 1949: 1946: 1942: 1936: 1934: 1932: 1930: 1926: 1921: 1917: 1910: 1907: 1895: 1891: 1885: 1882: 1871:on 2023-10-04 1870: 1866: 1859: 1856: 1845: 1841: 1835: 1832: 1820: 1816: 1810: 1807: 1796: 1792: 1786: 1783: 1779: 1773: 1770: 1759: 1755: 1748: 1746: 1742: 1731: 1727: 1720: 1718: 1714: 1702: 1698: 1692: 1689: 1677: 1673: 1667: 1665: 1663: 1661: 1657: 1650: 1646: 1643: 1641: 1638: 1635: 1632: 1629: 1626: 1625: 1621: 1617: 1613: 1611: 1608: 1606: 1602: 1598: 1595: 1593: 1590: 1588: 1584: 1581: 1579: 1576: 1573: 1569: 1566: 1565: 1561: 1557: 1554: 1552: 1551:Network Codes 1549: 1547: 1544: 1542: 1539: 1537: 1534: 1532: 1531:Fountain code 1529: 1528: 1524: 1520: 1517: 1515: 1514:Tornado codes 1512: 1511: 1507: 1505: 1499: 1497: 1493: 1489: 1486: 1482: 1480: 1476: 1472: 1470: 1466: 1462: 1458: 1455: +  1454: 1450: 1446: 1442: 1439: +  1438: 1434: 1430: 1426: 1422: 1416: 1414: 1410: 1409:Apache Hadoop 1402: 1400: 1397: 1393: 1391: 1386: 1382: 1378: 1374: 1370: 1368: 1364: 1360: 1353: 1351: 1349: 1345: 1341: 1337: 1332: 1330: 1326: 1322: 1314: 1312: 1310: 1306: 1302: 1298: 1294: 1290: 1286: 1282: 1278: 1274: 1270: 1266: 1262: 1258: 1254: 1251: â‰Ą  1250: 1246: 1242: 1238: 1234: 1230: 1226: 1222: 1218: 1214: 1210: 1206: 1202: 1198: 1194: 1190: 1186: 1182: 1178: 1174: 1170: 1166: 1162: 1158: 1153: 1151: 1147: 1143: 1139: 1135: 1131: 1127: 1123: 1119: 1115: 1111: 1107: 1104: 1100: 1096: 1092: 1087: 1085: 1081: 1073: 1071: 1069: 1065: 1060: 1056: 1054: 1050: 1046: 1042: 1037: 1035: 1030: 1028: 1024: 1005: 1002: 999: 990: 987: 984: 978: 975: 972: 966: 960: 952: 948: 940: 936: 932: 928: 927: 926: 924: 904: 901: 895: 889: 869: 866: 860: 854: 831: 828: 825: 819: 816: 813: 810: 804: 798: 775: 772: 769: 760: 757: 754: 748: 745: 742: 736: 730: 722: 719: 715: 711: 710: 709: 703: 700: 697: 696: 695: 693: 689: 685: 683: 675: 671: 666: 664: 662: 644: 639: 635: 629: 626: 623: 618: 615: 612: 609: 606: 603: 600: 596: 592: 589: 584: 580: 572: 571: 570: 554: 550: 527: 524: 521: 518: 515: 512: 509: 499: 495: 483: 464: 459: 455: 449: 444: 441: 438: 434: 430: 427: 422: 419: 416: 412: 404: 403: 402: 400: 382: 379: 376: 373: 370: 360: 356: 344: 340: 336: 330: 322: 320: 319:(MDS codes). 318: 314: 310: 299: 296: 288: 278: 274: 268: 265:This section 263: 254: 253: 247: 245: 242: 240: 231: 228: 225: 224: 223: 220: 218: 214: 210: 205: 203: 199: 191: 189: 187: 183: 179: 175: 171: 167: 163: 160: =  159: 155: 151: 147: 143: 139: 135: 134:coding theory 121: 118: 110: 99: 96: 92: 89: 85: 82: 78: 75: 71: 68: â€“  67: 63: 62:Find sources: 56: 52: 46: 45: 40:This article 38: 34: 29: 28: 19: 18:Erasure codes 2098:. Retrieved 2094:the original 2084: 2039: 2035: 2029: 2016: 1996: 1989: 1960: 1948: 1920:the original 1909: 1898:. Retrieved 1896:. 2022-04-25 1893: 1884: 1873:. Retrieved 1869:the original 1858: 1847:. Retrieved 1843: 1834: 1823:. Retrieved 1821:. 2021-07-27 1818: 1809: 1798:. Retrieved 1794: 1785: 1772: 1761:. Retrieved 1757: 1733:. Retrieved 1729: 1705:. Retrieved 1703:. 2014-04-07 1700: 1691: 1680:. Retrieved 1678:. 2023-09-03 1675: 1546:Raptor codes 1536:Online codes 1503: 1494: 1490: 1487: 1483: 1474: 1473: 1468: 1464: 1460: 1456: 1452: 1448: 1444: 1440: 1436: 1432: 1428: 1424: 1417: 1406: 1394: 1389: 1384: 1380: 1376: 1371: 1366: 1362: 1358: 1357: 1343: 1339: 1333: 1325:finite field 1318: 1308: 1304: 1300: 1296: 1292: 1288: 1284: 1280: 1276: 1272: 1268: 1264: 1260: 1256: 1252: 1248: 1244: 1240: 1236: 1232: 1228: 1224: 1220: 1216: 1212: 1208: 1204: 1200: 1196: 1192: 1188: 1184: 1183:) if symbol 1180: 1176: 1172: 1168: 1167:, such that 1164: 1160: 1156: 1154: 1149: 1145: 1141: 1137: 1133: 1129: 1125: 1121: 1117: 1113: 1109: 1105: 1098: 1094: 1090: 1088: 1084:finite field 1077: 1074:General case 1067: 1063: 1061: 1057: 1052: 1048: 1044: 1040: 1038: 1031: 1026: 1022: 950: 946: 944: 938: 934: 930: 920: 847:, such that 717: 713: 707: 686: 681: 679: 673: 659: 481: 479: 398: 342: 338: 334: 332: 323:Parity check 312: 308: 306: 291: 282: 266: 243: 235: 221: 206: 195: 185: 177: 173: 165: 161: 157: 153: 149: 145: 138:erasure code 137: 131: 113: 104: 94: 87: 80: 73: 61: 49:Please help 44:verification 41: 1795:www.ibm.com 1471:failures. 1112:) of order 480:The set of 311:out of the 285:August 2023 226:Replication 217:Turbo codes 198:Irving Reed 107:August 2023 2100:2023-08-20 2049:cs/0702015 1900:2024-09-18 1894:MinIO Blog 1875:2024-09-18 1849:2024-09-18 1825:2024-09-18 1819:MinIO Blog 1800:2024-09-18 1763:2024-09-18 1735:2024-09-18 1707:2024-09-18 1682:2024-09-18 1651:References 1614:any other 1583:Tahoe-LAFS 1570:: used in 1116:such that 329:Parity bit 327:See also: 77:newspapers 2076:260559901 2054:CiteSeerX 1585:includes 1070:) given. 1003:− 988:− 937:(4), and 829:− 773:− 758:− 616:≠ 597:∑ 593:− 519:≤ 513:≤ 435:∑ 431:− 380:≤ 374:≤ 204:in 1960. 170:code rate 2161:Category 2135:register 2118:Jerasure 1730:BDRSuite 1622:See also 1578:Parchive 1541:LT codes 1500:Examples 1475:Example: 1451:out of ( 1327:using a 1136:), ..., 1051:(4) and 1021:, where 176:, where 2024:. 2009. 1984:. 2015. 1968:. 2022. 1955:. 2016. 1701:ceph.io 1463:,  1427:,  716:= 555, 345:values 337:=  335:n  271:Please 192:History 91:scholar 2130:Feclib 2074:  2056:  2004:  1630:codes. 1568:Parity 1445:stripe 1413:RAID-6 1411:, the 661:RAID 5 93:  86:  79:  72:  64:  2072:S2CID 2044:arXiv 1243:< 1187:< 953:) is 933:(3), 688:Alice 140:is a 136:, an 98:JSTOR 84:books 2002:ISBN 1587:zfec 1572:RAID 1479:HDFS 1215:) = 1175:) = 1043:and 1025:and 882:and 229:RAID 200:and 174:k’/k 70:news 2064:doi 1603:or 1159:to 905:629 870:555 814:555 692:Bob 275:to 132:In 53:by 2163:: 2070:. 2062:. 2052:. 2040:56 2038:. 1973:^ 1928:^ 1892:. 1842:. 1817:. 1793:. 1756:. 1744:^ 1728:. 1716:^ 1699:. 1674:. 1659:^ 1331:. 1086:. 820:74 219:. 211:, 178:k’ 2153:. 2103:. 2078:. 2066:: 2046:: 2010:. 1903:. 1878:. 1852:. 1828:. 1803:. 1766:. 1738:. 1710:. 1685:. 1469:m 1465:m 1461:k 1457:m 1453:k 1449:k 1441:m 1437:k 1433:k 1429:m 1425:k 1390:k 1385:k 1363:k 1344:n 1340:k 1309:k 1305:n 1303:( 1301:O 1297:k 1293:n 1291:( 1289:n 1287:( 1285:O 1281:r 1277:i 1275:( 1273:q 1269:i 1267:( 1265:p 1261:i 1259:( 1257:r 1253:k 1249:i 1245:k 1241:i 1237:i 1235:( 1233:r 1229:i 1227:( 1225:q 1221:i 1219:( 1217:p 1213:i 1211:( 1209:r 1205:k 1201:i 1197:i 1195:( 1193:q 1189:k 1185:i 1181:i 1179:( 1177:p 1173:i 1171:( 1169:q 1165:q 1161:n 1157:k 1150:F 1146:k 1142:n 1140:( 1138:p 1134:k 1132:( 1130:p 1126:i 1122:i 1120:( 1118:p 1114:k 1110:x 1108:( 1106:p 1099:k 1095:n 1091:F 1068:i 1066:( 1064:f 1053:f 1049:f 1045:b 1041:a 1027:b 1023:a 1009:) 1006:1 1000:i 997:( 994:) 991:a 985:b 982:( 979:+ 976:a 973:= 970:) 967:i 964:( 961:f 951:k 949:( 947:f 939:f 935:f 931:f 917:. 902:= 899:) 896:2 893:( 890:f 867:= 864:) 861:1 858:( 855:f 835:) 832:1 826:i 823:( 817:+ 811:= 808:) 805:i 802:( 799:f 779:) 776:1 770:i 767:( 764:) 761:a 755:b 752:( 749:+ 746:a 743:= 740:) 737:i 734:( 731:f 718:b 714:a 682:k 674:k 645:. 640:i 636:v 630:1 627:+ 624:k 619:e 613:i 610:, 607:1 604:= 601:i 590:= 585:e 581:v 555:e 551:v 528:1 525:+ 522:k 516:i 510:1 506:} 500:i 496:v 492:{ 482:k 465:. 460:i 456:v 450:k 445:1 442:= 439:i 428:= 423:1 420:+ 417:k 413:v 399:k 383:k 377:i 371:1 367:} 361:i 357:v 353:{ 343:k 339:k 313:n 309:k 298:) 292:( 287:) 283:( 269:. 186:n 166:n 164:/ 162:k 158:r 154:n 150:n 146:k 120:) 114:( 109:) 105:( 95:· 88:· 81:· 74:· 47:. 20:)

Index

Erasure codes

verification
improve this article
adding citations to reliable sources
"Erasure code"
news
newspapers
books
scholar
JSTOR
Learn how and when to remove this message
coding theory
forward error correction
code rate
reception efficiency
Irving Reed
Gustave Solomon
Reed-Solomon coding
Low-density parity-check code
Turbo codes
Redundant Array of Inexpensive Servers
help improve it
make it understandable to non-experts
Learn how and when to remove this message
maximum distance separable codes
Parity bit
RAID 5
Alice
Bob

Text is available under the Creative Commons Attribution-ShareAlike License. Additional terms may apply.

↑