919:
When tied values occur, they are each given the average of the ranks that would have been given had no ties occurred. For example, the data set {80,76,34,80,73,80} has values of 80 tied for 4th, 5th, and 6th place; since the mean of {4,5,6} = 5, ranks would be assigned to the raw data values as
1286:
844:
2321:
2081:
275:
1776:
2523:
1023:
1697:
370:
1884:
588:
452:
1963:
193:
1600:
1093:
2667:
1354:
690:
2404:
2784:
2729:
1534:
674:
517:
905:
2998:
927:; however, this effect is small unless there are a large number of ties. To correct for ties, assign ranks to tied values as above and compute the correction factors
2598:
2561:
2442:
2114:
1427:
638:
461:
is 1, then all the judges or survey respondents have been unanimous, and each judge or respondent has assigned the same order to the list of objects or concerns. If
3002:
2181:
873:
2808:
2201:
2154:
2134:
1799:
1620:
1507:
1487:
1467:
1447:
1397:
52:
Suppose, for instance, that a number of people have been asked to rank a list of political concerns, from the most important to the least important. Kendall's
465:
is 0, then there is no overall trend of agreement among the respondents, and their responses may be regarded as essentially random. Intermediate values of
64:
is 0, then there is no overall trend of agreement among the respondents, and their responses may be regarded as essentially random. Intermediate values of
2849:
477:
2216:
1982:
3216:
3197:
3156:
60:
is 1, then all the survey respondents have been unanimous, and each respondent has assigned the same order to the list of concerns. If
2906:"Large-scale group decision-making (LSGDM) for performance measurement of healthcare construction projects: Ordinal Priority Approach"
3240:
3173:
2974:
204:
1708:
3235:
2453:
1976:
In case of tie rank, we need to consider it in the above formula. To correct for ties, we should compute the correction factors,
933:
1628:
72:
290:
1807:
525:
385:
1902:
3144:
Kendall, M. G., & Gibbons, J. D. (1990). Rank correlation methods. New York, NY : Oxford
University Press.
3060:
1281:{\displaystyle W={\frac {12\sum _{i=1}^{n}(R_{i}^{2})-3m^{2}n(n+1)^{2}}{m^{2}n(n^{2}-1)-m\sum _{j=1}^{m}(T_{j})}},}
128:
1539:
2827:
84:
30:
2614:
1375:
In some cases, the importance of the raters (experts) might not be the same as each other. In this case, the
839:{\displaystyle W={\frac {12\sum _{i=1}^{n}(R_{i}^{2})-3r^{2}n\left(p+1\right)^{2}}{\lambda ^{2}n(n^{2}-1)}}.}
1305:
42:
1536:(in real-world situation, the importance of each rater can be different). Indeed, the weight of judges is
2352:
against a null hypothesis of no agreement (i.e. random rankings) is given by
Kendall and Gibbons (1990)
2358:
3084:
2734:
2574:. Results indicated the chi-square method was overly conservative compared to a permutation test when
2675:
76:
3183:
1512:
653:
483:
2326:
If the weights of the raters are equal (the distribution of the weights is uniform), the value of
878:
3133:
3042:
2992:
2790:
test performs approximately as well as the permutation test method, and may be preferred to when
3245:
3212:
3193:
3169:
3152:
2980:
2970:
2943:
2925:
2577:
3178:
Legendre, P (2005) Species
Associations: The Kendall Coefficient of Concordance Revisited.
3123:
3034:
2933:
2917:
2844:
2567:
2531:
2412:
2089:
1402:
1039:
th group of tied ranks, (where a group is a set of values having constant (tied) rank,) and
617:
34:
2159:
2839:
469:
indicate a greater or lesser degree of unanimity among the various judges or respondents.
852:
650:
and each pair of objects is presented together to some judge a total of exactly λ times,
3085:"Kendall's coefficient of concordance W – generalized for randomly incomplete datasets"
2963:
2938:
2905:
2793:
2186:
2139:
2119:
1784:
1605:
1492:
1472:
1452:
1432:
1382:
3229:
3161:
3046:
2854:
38:
603:
3038:
2316:{\displaystyle W_{w}={\frac {12S}{(n^{3}-n)-\sum _{j=1}^{m}\vartheta _{j}T_{j}}}}
2921:
2076:{\displaystyle T_{j}=\sum _{i=1}^{n}(t_{ij}^{3}-t_{ij})\;\;\;\;\;\;\;\forall j}
3128:
3107:
68:
indicate a greater or lesser degree of unanimity among the various responses.
2929:
2984:
3025:
Marozzi, Marco (2014). "Testing for concordance between several criteria".
2947:
41:, and can be used for assessing agreement among raters and in particular
3149:
Nonparametric
Statistics for Non-Statisticians: A Step-by-Step Approach
3137:
1046:
is the number of groups of ties in the set of ranks (ranging from 1 to
79:
values and compare two sequences of outcomes simultaneously, Kendall's
2819:
2348:
In the case of complete ranks, a commonly used significance test for
2969:. Gibbons, Jean Dickinson, 1938- (5th ed.). London: E. Arnold.
2904:
Mahmoudi, Amin; Abbasi, Mehdi; Yuan, Jingfeng; Li, Lingzhi (2022).
2566:
Legendre compared via simulation the power of the chi-square and
1968:
The above formula is suitable when we do not have any tie rank.
1061:
is the correction factor required for the set of ranks for judge
3180:
Journal of
Agricultural, Biological and Environmental Statistics
2823:
1069:
th set of ranks. Note that if there are no tied ranks for judge
2409:
Where the test statistic takes a chi-squared distribution with
2604:
test, as proposed in the original publication introducing the
911:
objects, the formula above is equivalent to the original one.
270:{\displaystyle {\bar {R}}={\frac {1}{n}}\sum _{i=1}^{n}R_{i}.}
2447:
In the case of incomplete rankings (see above), this becomes
1771:{\displaystyle {\bar {R}}={\frac {1}{n}}\sum _{i=1}^{n}R_{i}}
3211:(4th ed.). New York: Marcel Dekker. pp. 476–482.
2518:{\displaystyle \chi ^{2}={\frac {\lambda (n^{2}-1)}{k+1}}W}
1018:{\displaystyle T_{j}=\sum _{i=1}^{g_{j}}(t_{i}^{3}-t_{i}),}
3207:
Gibbons, Jean
Dickinson; Chakraborti, Subhabrata (2003).
2961:
Kendall, Maurice G. (Maurice George), 1907-1983. (1990).
1692:{\displaystyle R_{i}=\sum _{j=1}^{m}\vartheta _{j}r_{ij}}
56:
can be calculated from these data. If the test statistic
16:
Rank correlation statistic used for inter-rater agreement
2818:
Kendall's W and
Weighted Kendall's W are implemented in
2672:
Where the test statistic follows an F distribution with
365:{\displaystyle S=\sum _{i=1}^{n}(R_{i}-{\bar {R}})^{2},}
49:
ranges from 0 (no agreement) to 1 (complete agreement).
1879:{\displaystyle S=\sum _{i=1}^{n}(R_{i}-{\bar {R}})^{2}}
602:
objects, and when the correspondent block design is a
2796:
2737:
2678:
2617:
2580:
2570:
approaches to determining significance for
Kendall's
2534:
2456:
2415:
2361:
2219:
2189:
2162:
2142:
2122:
2092:
1985:
1905:
1810:
1787:
1711:
1631:
1608:
1542:
1515:
1495:
1475:
1455:
1435:
1405:
1385:
1308:
1096:
936:
881:
855:
693:
656:
643:
every object is ranked exactly the same total number
620:
528:
493:
388:
293:
207:
131:
3192:(2nd ed.). New York: McGraw-Hill. p. 266.
3190:
Nonparametric
Statistics for the Behavioral Sciences
604:(n, m, r, p, λ)-design (note the different notation)
2608:statistic by Kendall & Babington Smith (1939):
2962:
2802:
2778:
2723:
2661:
2592:
2555:
2517:
2436:
2398:
2315:
2195:
2175:
2148:
2128:
2108:
2075:
1957:
1878:
1793:
1770:
1691:
1614:
1594:
1528:
1501:
1481:
1461:
1441:
1421:
1391:
1348:
1280:
1017:
899:
867:
838:
668:
632:
583:{\displaystyle {\bar {r}}_{s}={\frac {mW-1}{m-1}}}
582:
511:
446:
364:
269:
187:
3027:Journal of Statistical Computation and Simulation
598:When the judges evaluate only some subset of the
505:
488:
83:makes no assumptions regarding the nature of the
3106:Kendall, M. G.; Babington Smith, B. (Sep 1939).
2600:. Marozzi extended this by also considering the
2203:. With the correction for ties, the formula for
447:{\displaystyle W={\frac {12S}{m^{2}(n^{3}-n)}}.}
87:and can handle any number of distinct outcomes.
37:. It is a normalization of the statistic of the
3188:Siegel, Sidney; Castellan, N. John Jr. (1988).
2873:Dodge (2003): see "concordance, coefficient of"
1958:{\displaystyle W_{w}={\frac {12S}{(n^{3}-n)}}}
1083:With the correction for ties, the formula for
923:The effect of ties is to reduce the value of
476:is linearly related to the mean value of the
188:{\displaystyle R_{i}=\sum _{j=1}^{m}r_{i,j},}
8:
2997:: CS1 maint: multiple names: authors list (
2830:, and other statistical software packages.
2810:is small, as it is computationally simpler.
2116:represents the number of tie ranks in judge
1702:and the mean value of these total ranks is,
118:judges. Then the total rank given to object
1595:{\displaystyle \vartheta _{j}(j=1,2,...,m)}
198:and the mean value of these total ranks is
3166:The Oxford Dictionary of Statistical Terms
3001:) CS1 maint: numeric names: authors list (
2066:
2065:
2064:
2063:
2062:
2061:
2060:
519:possible pairs of rankings between judges
3127:
2937:
2795:
2770:
2742:
2736:
2710:
2683:
2677:
2624:
2616:
2579:
2533:
2483:
2470:
2461:
2455:
2414:
2366:
2360:
2304:
2294:
2284:
2273:
2251:
2233:
2224:
2218:
2188:
2167:
2161:
2141:
2121:
2097:
2091:
2048:
2035:
2027:
2014:
2003:
1990:
1984:
1937:
1919:
1910:
1904:
1870:
1855:
1854:
1845:
1832:
1821:
1809:
1786:
1762:
1752:
1741:
1727:
1713:
1712:
1710:
1680:
1670:
1660:
1649:
1636:
1630:
1607:
1547:
1541:
1520:
1514:
1494:
1474:
1454:
1434:
1410:
1404:
1384:
1337:
1324:
1313:
1307:
1263:
1250:
1239:
1214:
1198:
1186:
1161:
1142:
1137:
1124:
1113:
1103:
1095:
1003:
990:
985:
970:
965:
954:
941:
935:
880:
854:
815:
799:
787:
758:
739:
734:
721:
710:
700:
692:
655:
619:
551:
542:
531:
530:
527:
504:
498:
487:
485:
423:
410:
395:
387:
353:
338:
337:
328:
315:
304:
292:
258:
248:
237:
223:
209:
208:
206:
170:
160:
149:
136:
130:
2183:shows the total number of ties in judge
478:Spearman's rank correlation coefficients
3089:The R Project for Statistical Computing
2866:
2850:Spearman's rank correlation coefficient
2662:{\displaystyle F={\frac {W(m-1)}{1-W}}}
1602:. Then, the total rank given to object
2990:
2786:degrees of freedom. Marozzi found the
1349:{\displaystyle \sum _{j=1}^{m}(T_{j})}
3116:The Annals of Mathematical Statistics
2891:Siegel & Castellan (1988, p. 266)
472:Kendall and Gibbons (1990) also show
7:
3147:Corder, G.W., Foreman, D.I. (2009).
2899:
2897:
1379:should be used. Suppose that object
27:Kendall's coefficient of concordance
3209:Nonparametric Statistical Inference
1298:is the sum of the ranks for object
1035:is the number of tied ranks in the
2067:
1489:judges. Also, the weight of judge
492:
14:
2399:{\displaystyle \chi ^{2}=m(n-1)W}
610:each judge ranks the same number
2882:Gibbons & Chakraborti (2003)
2779:{\displaystyle v_{2}=(m-1)v_{1}}
2724:{\displaystyle v_{1}=n-1-(2/m)}
1781:The sum of squared deviations,
280:The sum of squared deviations,
73:Pearson correlation coefficient
71:While tests using the standard
2763:
2751:
2718:
2704:
2642:
2630:
2495:
2476:
2390:
2378:
2263:
2244:
2057:
2020:
1949:
1930:
1867:
1860:
1838:
1718:
1589:
1553:
1529:{\displaystyle \vartheta _{j}}
1343:
1330:
1269:
1256:
1226:
1207:
1183:
1170:
1148:
1130:
1009:
978:
827:
808:
745:
727:
669:{\displaystyle \lambda \geq 1}
536:
435:
416:
350:
343:
321:
214:
1:
1371:Steps of Weighted Kendall's W
907:so that each judge ranks all
512:{\displaystyle m \choose {2}}
3039:10.1080/00949655.2013.766189
1356:is the sum of the values of
900:{\displaystyle \lambda =r=m}
1449:, where there are in total
676:, a constant for all pairs.
110:, where there are in total
3262:
2922:10.1007/s10489-022-04094-y
3241:Nonparametric statistics
3182:, 10(2), 226–245.
2965:Rank correlation methods
920:follows: {5,3,1,5,2,5}.
85:probability distribution
31:non-parametric statistic
3236:Inter-rater reliability
3129:10.1214/aoms/1177732186
2593:{\displaystyle m<20}
2528:Where again, there are
606:. In other words, when
43:inter-rater reliability
3061:"Weighted Kendall's W"
2804:
2780:
2725:
2663:
2594:
2557:
2556:{\displaystyle df=n-1}
2519:
2438:
2437:{\displaystyle df=n-1}
2400:
2317:
2289:
2197:
2177:
2150:
2130:
2110:
2109:{\displaystyle t_{ij}}
2077:
2019:
1959:
1880:
1837:
1795:
1772:
1757:
1693:
1665:
1616:
1596:
1530:
1503:
1483:
1463:
1443:
1423:
1422:{\displaystyle r_{ij}}
1393:
1350:
1329:
1282:
1255:
1129:
1019:
977:
901:
869:
840:
726:
670:
634:
633:{\displaystyle p<n}
584:
513:
457:If the test statistic
448:
366:
320:
271:
253:
189:
165:
2805:
2781:
2726:
2664:
2595:
2558:
2520:
2439:
2401:
2318:
2269:
2198:
2178:
2176:{\displaystyle T_{j}}
2151:
2131:
2111:
2078:
1999:
1960:
1881:
1817:
1796:
1773:
1737:
1694:
1645:
1617:
1597:
1531:
1504:
1484:
1464:
1444:
1424:
1394:
1351:
1309:
1283:
1235:
1109:
1020:
950:
902:
870:
841:
706:
671:
635:
585:
514:
449:
367:
300:
272:
233:
190:
145:
2910:Applied Intelligence
2794:
2735:
2676:
2615:
2578:
2563:degrees of freedom.
2532:
2454:
2444:degrees of freedom.
2413:
2359:
2217:
2187:
2160:
2140:
2120:
2090:
1983:
1903:
1808:
1785:
1709:
1629:
1606:
1540:
1513:
1493:
1473:
1453:
1433:
1403:
1383:
1377:Weighted Kendall's W
1306:
1094:
934:
879:
853:
691:
654:
618:
614:of objects for some
526:
484:
386:
291:
205:
129:
95:Suppose that object
91:Steps of Kendall's W
77:normally distributed
2916:(12): 13781–13802.
2568:permutation testing
2328:Weighted Kendall's
2205:Weighted Kendall's
2040:
1972:Correction for Ties
1891:Weighted Kendall's
1147:
995:
915:Correction for Ties
868:{\displaystyle p=n}
744:
375:and then Kendall's
2800:
2776:
2721:
2659:
2590:
2553:
2515:
2434:
2396:
2344:Significance Tests
2313:
2193:
2173:
2146:
2126:
2106:
2073:
2023:
1955:
1876:
1791:
1768:
1689:
1612:
1592:
1526:
1499:
1479:
1459:
1439:
1419:
1399:is given the rank
1389:
1346:
1278:
1133:
1015:
981:
897:
865:
836:
730:
666:
630:
580:
497:
444:
362:
267:
185:
99:is given the rank
3218:978-0-8247-4052-8
3199:978-0-07-057357-4
3157:978-0-470-45461-9
3065:www.mathworks.com
2803:{\displaystyle m}
2657:
2510:
2311:
2196:{\displaystyle j}
2149:{\displaystyle i}
2129:{\displaystyle j}
1953:
1863:
1801:, is defined as,
1794:{\displaystyle S}
1735:
1721:
1615:{\displaystyle i}
1502:{\displaystyle j}
1482:{\displaystyle m}
1462:{\displaystyle n}
1442:{\displaystyle j}
1392:{\displaystyle i}
1370:
1273:
831:
594:Incomplete Blocks
578:
539:
503:
439:
346:
231:
217:
90:
3253:
3222:
3203:
3141:
3131:
3108:"The Problem of
3093:
3092:
3081:
3075:
3074:
3072:
3071:
3057:
3051:
3050:
3033:(9): 1843–1850.
3022:
3016:
3013:
3007:
3006:
2996:
2988:
2968:
2958:
2952:
2951:
2941:
2901:
2892:
2889:
2883:
2880:
2874:
2871:
2809:
2807:
2806:
2801:
2785:
2783:
2782:
2777:
2775:
2774:
2747:
2746:
2730:
2728:
2727:
2722:
2714:
2688:
2687:
2668:
2666:
2665:
2660:
2658:
2656:
2645:
2625:
2599:
2597:
2596:
2591:
2562:
2560:
2559:
2554:
2524:
2522:
2521:
2516:
2511:
2509:
2498:
2488:
2487:
2471:
2466:
2465:
2443:
2441:
2440:
2435:
2405:
2403:
2402:
2397:
2371:
2370:
2322:
2320:
2319:
2314:
2312:
2310:
2309:
2308:
2299:
2298:
2288:
2283:
2256:
2255:
2242:
2234:
2229:
2228:
2202:
2200:
2199:
2194:
2182:
2180:
2179:
2174:
2172:
2171:
2155:
2153:
2152:
2147:
2135:
2133:
2132:
2127:
2115:
2113:
2112:
2107:
2105:
2104:
2082:
2080:
2079:
2074:
2056:
2055:
2039:
2034:
2018:
2013:
1995:
1994:
1964:
1962:
1961:
1956:
1954:
1952:
1942:
1941:
1928:
1920:
1915:
1914:
1885:
1883:
1882:
1877:
1875:
1874:
1865:
1864:
1856:
1850:
1849:
1836:
1831:
1800:
1798:
1797:
1792:
1777:
1775:
1774:
1769:
1767:
1766:
1756:
1751:
1736:
1728:
1723:
1722:
1714:
1698:
1696:
1695:
1690:
1688:
1687:
1675:
1674:
1664:
1659:
1641:
1640:
1621:
1619:
1618:
1613:
1601:
1599:
1598:
1593:
1552:
1551:
1535:
1533:
1532:
1527:
1525:
1524:
1508:
1506:
1505:
1500:
1488:
1486:
1485:
1480:
1468:
1466:
1465:
1460:
1448:
1446:
1445:
1440:
1429:by judge number
1428:
1426:
1425:
1420:
1418:
1417:
1398:
1396:
1395:
1390:
1355:
1353:
1352:
1347:
1342:
1341:
1328:
1323:
1287:
1285:
1284:
1279:
1274:
1272:
1268:
1267:
1254:
1249:
1219:
1218:
1203:
1202:
1192:
1191:
1190:
1166:
1165:
1146:
1141:
1128:
1123:
1104:
1024:
1022:
1021:
1016:
1008:
1007:
994:
989:
976:
975:
974:
964:
946:
945:
906:
904:
903:
898:
874:
872:
871:
866:
845:
843:
842:
837:
832:
830:
820:
819:
804:
803:
793:
792:
791:
786:
782:
763:
762:
743:
738:
725:
720:
701:
675:
673:
672:
667:
639:
637:
636:
631:
589:
587:
586:
581:
579:
577:
566:
552:
547:
546:
541:
540:
532:
518:
516:
515:
510:
509:
508:
502:
491:
453:
451:
450:
445:
440:
438:
428:
427:
415:
414:
404:
396:
371:
369:
368:
363:
358:
357:
348:
347:
339:
333:
332:
319:
314:
284:, is defined as
276:
274:
273:
268:
263:
262:
252:
247:
232:
224:
219:
218:
210:
194:
192:
191:
186:
181:
180:
164:
159:
141:
140:
106:by judge number
35:rank correlation
3261:
3260:
3256:
3255:
3254:
3252:
3251:
3250:
3226:
3225:
3219:
3206:
3200:
3187:
3105:
3102:
3097:
3096:
3083:
3082:
3078:
3069:
3067:
3059:
3058:
3054:
3024:
3023:
3019:
3015:Legendre (2005)
3014:
3010:
2989:
2977:
2960:
2959:
2955:
2903:
2902:
2895:
2890:
2886:
2881:
2877:
2872:
2868:
2863:
2840:Maurice Kendall
2836:
2816:
2792:
2791:
2766:
2738:
2733:
2732:
2679:
2674:
2673:
2646:
2626:
2613:
2612:
2576:
2575:
2530:
2529:
2499:
2479:
2472:
2457:
2452:
2451:
2411:
2410:
2362:
2357:
2356:
2346:
2300:
2290:
2247:
2243:
2235:
2220:
2215:
2214:
2185:
2184:
2163:
2158:
2157:
2138:
2137:
2118:
2117:
2093:
2088:
2087:
2044:
1986:
1981:
1980:
1974:
1933:
1929:
1921:
1906:
1901:
1900:
1896:is defined as,
1866:
1841:
1806:
1805:
1783:
1782:
1758:
1707:
1706:
1676:
1666:
1632:
1627:
1626:
1604:
1603:
1543:
1538:
1537:
1516:
1511:
1510:
1491:
1490:
1471:
1470:
1451:
1450:
1431:
1430:
1406:
1401:
1400:
1381:
1380:
1373:
1367:sets of ranks.
1361:
1333:
1304:
1303:
1296:
1259:
1210:
1194:
1193:
1182:
1157:
1105:
1092:
1091:
1078:
1059:
1044:
1033:
999:
966:
937:
932:
931:
917:
877:
876:
851:
850:
811:
795:
794:
772:
768:
767:
754:
702:
689:
688:
684:is defined as
680:Then Kendall's
652:
651:
616:
615:
596:
567:
553:
529:
524:
523:
486:
482:
481:
419:
406:
405:
397:
384:
383:
349:
324:
289:
288:
254:
203:
202:
166:
132:
127:
126:
104:
93:
25:(also known as
17:
12:
11:
5:
3259:
3257:
3249:
3248:
3243:
3238:
3228:
3227:
3224:
3223:
3217:
3204:
3198:
3185:
3176:
3159:
3145:
3142:
3122:(3): 275–287.
3101:
3098:
3095:
3094:
3076:
3052:
3017:
3008:
2975:
2953:
2893:
2884:
2875:
2865:
2864:
2862:
2859:
2858:
2857:
2852:
2847:
2842:
2835:
2832:
2815:
2812:
2799:
2773:
2769:
2765:
2762:
2759:
2756:
2753:
2750:
2745:
2741:
2720:
2717:
2713:
2709:
2706:
2703:
2700:
2697:
2694:
2691:
2686:
2682:
2670:
2669:
2655:
2652:
2649:
2644:
2641:
2638:
2635:
2632:
2629:
2623:
2620:
2589:
2586:
2583:
2552:
2549:
2546:
2543:
2540:
2537:
2526:
2525:
2514:
2508:
2505:
2502:
2497:
2494:
2491:
2486:
2482:
2478:
2475:
2469:
2464:
2460:
2433:
2430:
2427:
2424:
2421:
2418:
2407:
2406:
2395:
2392:
2389:
2386:
2383:
2380:
2377:
2374:
2369:
2365:
2345:
2342:
2324:
2323:
2307:
2303:
2297:
2293:
2287:
2282:
2279:
2276:
2272:
2268:
2265:
2262:
2259:
2254:
2250:
2246:
2241:
2238:
2232:
2227:
2223:
2192:
2170:
2166:
2145:
2125:
2103:
2100:
2096:
2084:
2083:
2072:
2069:
2059:
2054:
2051:
2047:
2043:
2038:
2033:
2030:
2026:
2022:
2017:
2012:
2009:
2006:
2002:
1998:
1993:
1989:
1973:
1970:
1966:
1965:
1951:
1948:
1945:
1940:
1936:
1932:
1927:
1924:
1918:
1913:
1909:
1887:
1886:
1873:
1869:
1862:
1859:
1853:
1848:
1844:
1840:
1835:
1830:
1827:
1824:
1820:
1816:
1813:
1790:
1779:
1778:
1765:
1761:
1755:
1750:
1747:
1744:
1740:
1734:
1731:
1726:
1720:
1717:
1700:
1699:
1686:
1683:
1679:
1673:
1669:
1663:
1658:
1655:
1652:
1648:
1644:
1639:
1635:
1611:
1591:
1588:
1585:
1582:
1579:
1576:
1573:
1570:
1567:
1564:
1561:
1558:
1555:
1550:
1546:
1523:
1519:
1498:
1478:
1458:
1438:
1416:
1413:
1409:
1388:
1372:
1369:
1359:
1345:
1340:
1336:
1332:
1327:
1322:
1319:
1316:
1312:
1294:
1289:
1288:
1277:
1271:
1266:
1262:
1258:
1253:
1248:
1245:
1242:
1238:
1234:
1231:
1228:
1225:
1222:
1217:
1213:
1209:
1206:
1201:
1197:
1189:
1185:
1181:
1178:
1175:
1172:
1169:
1164:
1160:
1156:
1153:
1150:
1145:
1140:
1136:
1132:
1127:
1122:
1119:
1116:
1112:
1108:
1102:
1099:
1076:
1057:
1042:
1031:
1026:
1025:
1014:
1011:
1006:
1002:
998:
993:
988:
984:
980:
973:
969:
963:
960:
957:
953:
949:
944:
940:
916:
913:
896:
893:
890:
887:
884:
864:
861:
858:
847:
846:
835:
829:
826:
823:
818:
814:
810:
807:
802:
798:
790:
785:
781:
778:
775:
771:
766:
761:
757:
753:
750:
747:
742:
737:
733:
729:
724:
719:
716:
713:
709:
705:
699:
696:
678:
677:
665:
662:
659:
648:
641:
629:
626:
623:
595:
592:
591:
590:
576:
573:
570:
565:
562:
559:
556:
550:
545:
538:
535:
507:
501:
496:
490:
455:
454:
443:
437:
434:
431:
426:
422:
418:
413:
409:
403:
400:
394:
391:
379:is defined as
373:
372:
361:
356:
352:
345:
342:
336:
331:
327:
323:
318:
313:
310:
307:
303:
299:
296:
278:
277:
266:
261:
257:
251:
246:
243:
240:
236:
230:
227:
222:
216:
213:
196:
195:
184:
179:
176:
173:
169:
163:
158:
155:
152:
148:
144:
139:
135:
102:
92:
89:
15:
13:
10:
9:
6:
4:
3:
2:
3258:
3247:
3244:
3242:
3239:
3237:
3234:
3233:
3231:
3220:
3214:
3210:
3205:
3201:
3195:
3191:
3186:
3184:
3181:
3177:
3175:
3174:0-19-920613-9
3171:
3167:
3163:
3160:
3158:
3154:
3150:
3146:
3143:
3139:
3135:
3130:
3125:
3121:
3117:
3113:
3111:
3104:
3103:
3099:
3090:
3086:
3080:
3077:
3066:
3062:
3056:
3053:
3048:
3044:
3040:
3036:
3032:
3028:
3021:
3018:
3012:
3009:
3004:
3000:
2994:
2986:
2982:
2978:
2976:0-19-520837-4
2972:
2967:
2966:
2957:
2954:
2949:
2945:
2940:
2935:
2931:
2927:
2923:
2919:
2915:
2911:
2907:
2900:
2898:
2894:
2888:
2885:
2879:
2876:
2870:
2867:
2860:
2856:
2855:Friedman test
2853:
2851:
2848:
2846:
2845:Kendall's tau
2843:
2841:
2838:
2837:
2833:
2831:
2829:
2825:
2821:
2813:
2811:
2797:
2789:
2771:
2767:
2760:
2757:
2754:
2748:
2743:
2739:
2715:
2711:
2707:
2701:
2698:
2695:
2692:
2689:
2684:
2680:
2653:
2650:
2647:
2639:
2636:
2633:
2627:
2621:
2618:
2611:
2610:
2609:
2607:
2603:
2587:
2584:
2581:
2573:
2569:
2564:
2550:
2547:
2544:
2541:
2538:
2535:
2512:
2506:
2503:
2500:
2492:
2489:
2484:
2480:
2473:
2467:
2462:
2458:
2450:
2449:
2448:
2445:
2431:
2428:
2425:
2422:
2419:
2416:
2393:
2387:
2384:
2381:
2375:
2372:
2367:
2363:
2355:
2354:
2353:
2351:
2343:
2341:
2339:
2338:
2332:
2331:
2305:
2301:
2295:
2291:
2285:
2280:
2277:
2274:
2270:
2266:
2260:
2257:
2252:
2248:
2239:
2236:
2230:
2225:
2221:
2213:
2212:
2211:
2209:
2208:
2190:
2168:
2164:
2143:
2123:
2101:
2098:
2094:
2070:
2052:
2049:
2045:
2041:
2036:
2031:
2028:
2024:
2015:
2010:
2007:
2004:
2000:
1996:
1991:
1987:
1979:
1978:
1977:
1971:
1969:
1946:
1943:
1938:
1934:
1925:
1922:
1916:
1911:
1907:
1899:
1898:
1897:
1895:
1894:
1871:
1857:
1851:
1846:
1842:
1833:
1828:
1825:
1822:
1818:
1814:
1811:
1804:
1803:
1802:
1788:
1763:
1759:
1753:
1748:
1745:
1742:
1738:
1732:
1729:
1724:
1715:
1705:
1704:
1703:
1684:
1681:
1677:
1671:
1667:
1661:
1656:
1653:
1650:
1646:
1642:
1637:
1633:
1625:
1624:
1623:
1609:
1586:
1583:
1580:
1577:
1574:
1571:
1568:
1565:
1562:
1559:
1556:
1548:
1544:
1521:
1517:
1496:
1476:
1456:
1436:
1414:
1411:
1407:
1386:
1378:
1368:
1366:
1362:
1338:
1334:
1325:
1320:
1317:
1314:
1310:
1301:
1297:
1275:
1264:
1260:
1251:
1246:
1243:
1240:
1236:
1232:
1229:
1223:
1220:
1215:
1211:
1204:
1199:
1195:
1187:
1179:
1176:
1173:
1167:
1162:
1158:
1154:
1151:
1143:
1138:
1134:
1125:
1120:
1117:
1114:
1110:
1106:
1100:
1097:
1090:
1089:
1088:
1086:
1081:
1079:
1072:
1068:
1064:
1060:
1053:
1049:
1045:
1038:
1034:
1012:
1004:
1000:
996:
991:
986:
982:
971:
967:
961:
958:
955:
951:
947:
942:
938:
930:
929:
928:
926:
921:
914:
912:
910:
894:
891:
888:
885:
882:
862:
859:
856:
833:
824:
821:
816:
812:
805:
800:
796:
788:
783:
779:
776:
773:
769:
764:
759:
755:
751:
748:
740:
735:
731:
722:
717:
714:
711:
707:
703:
697:
694:
687:
686:
685:
683:
663:
660:
657:
649:
646:
642:
627:
624:
621:
613:
609:
608:
607:
605:
601:
593:
574:
571:
568:
563:
560:
557:
554:
548:
543:
533:
522:
521:
520:
499:
494:
479:
475:
470:
468:
464:
460:
441:
432:
429:
424:
420:
411:
407:
401:
398:
392:
389:
382:
381:
380:
378:
359:
354:
340:
334:
329:
325:
316:
311:
308:
305:
301:
297:
294:
287:
286:
285:
283:
264:
259:
255:
249:
244:
241:
238:
234:
228:
225:
220:
211:
201:
200:
199:
182:
177:
174:
171:
167:
161:
156:
153:
150:
146:
142:
137:
133:
125:
124:
123:
121:
117:
113:
109:
105:
98:
88:
86:
82:
78:
74:
69:
67:
63:
59:
55:
50:
48:
44:
40:
39:Friedman test
36:
32:
28:
24:
23:
3208:
3189:
3179:
3165:
3148:
3119:
3115:
3109:
3088:
3079:
3068:. Retrieved
3064:
3055:
3030:
3026:
3020:
3011:
2964:
2956:
2913:
2909:
2887:
2878:
2869:
2817:
2787:
2671:
2605:
2601:
2571:
2565:
2527:
2446:
2408:
2349:
2347:
2336:
2334:
2329:
2327:
2325:
2206:
2204:
2085:
1975:
1967:
1892:
1890:
1888:
1780:
1701:
1509:is shown by
1469:objects and
1376:
1374:
1364:
1357:
1299:
1292:
1290:
1084:
1082:
1074:
1070:
1066:
1062:
1055:
1051:
1050:) for judge
1047:
1040:
1036:
1029:
1027:
924:
922:
918:
908:
848:
681:
679:
644:
611:
599:
597:
480:between all
473:
471:
466:
462:
458:
456:
376:
374:
281:
279:
197:
119:
115:
114:objects and
111:
107:
100:
96:
94:
80:
70:
65:
61:
57:
53:
51:
46:
45:. Kendall's
26:
21:
19:
18:
2340:are equal.
2136:for object
1065:, i.e. the
3230:Categories
3100:References
3070:2022-10-06
2335:Kendall's
2210:becomes,
1080:equals 0.
20:Kendall's
3162:Dodge, Y.
3112:Rankings"
3047:119577430
2993:cite book
2930:1573-7497
2758:−
2702:−
2696:−
2651:−
2637:−
2548:−
2490:−
2474:λ
2459:χ
2429:−
2385:−
2364:χ
2292:ϑ
2271:∑
2267:−
2258:−
2068:∀
2042:−
2001:∑
1944:−
1889:and then
1861:¯
1852:−
1819:∑
1739:∑
1719:¯
1668:ϑ
1647:∑
1545:ϑ
1518:ϑ
1363:over all
1311:∑
1237:∑
1230:−
1221:−
1152:−
1111:∑
997:−
952:∑
883:λ
822:−
797:λ
749:−
708:∑
661:≥
658:λ
647:of times,
572:−
561:−
537:¯
430:−
344:¯
335:−
302:∑
235:∑
215:¯
147:∑
3246:Rankings
3164:(2003).
2985:21195423
2948:36091930
2834:See also
2814:Software
1087:becomes
1054:. Thus,
3168:, OUP.
3151:Wiley,
3138:2235668
2939:9449288
75:assume
29:) is a
3215:
3196:
3172:
3155:
3136:
3045:
2983:
2973:
2946:
2936:
2928:
2820:MATLAB
2086:where
1302:, and
1291:where
1028:where
3134:JSTOR
3043:S2CID
2861:Notes
3213:ISBN
3194:ISBN
3170:ISBN
3153:ISBN
3003:link
2999:link
2981:OCLC
2971:ISBN
2944:PMID
2926:ISSN
2824:SPSS
2731:and
2585:<
2333:and
875:and
625:<
33:for
3124:doi
3035:doi
2934:PMC
2918:doi
1622:is
849:If
122:is
103:i,j
3232::
3132:.
3120:10
3118:.
3114:.
3087:.
3063:.
3041:.
3031:84
3029:.
2995:}}
2991:{{
2979:.
2942:.
2932:.
2924:.
2914:52
2912:.
2908:.
2896:^
2826:,
2822:,
2588:20
2237:12
2156:.
1923:12
1107:12
1073:,
704:12
399:12
3221:.
3202:.
3140:.
3126::
3110:m
3091:.
3073:.
3049:.
3037::
3005:)
2987:.
2950:.
2920::
2828:R
2798:m
2788:F
2772:1
2768:v
2764:)
2761:1
2755:m
2752:(
2749:=
2744:2
2740:v
2719:)
2716:m
2712:/
2708:2
2705:(
2699:1
2693:n
2690:=
2685:1
2681:v
2654:W
2648:1
2643:)
2640:1
2634:m
2631:(
2628:W
2622:=
2619:F
2606:W
2602:F
2582:m
2572:W
2551:1
2545:n
2542:=
2539:f
2536:d
2513:W
2507:1
2504:+
2501:k
2496:)
2493:1
2485:2
2481:n
2477:(
2468:=
2463:2
2432:1
2426:n
2423:=
2420:f
2417:d
2394:W
2391:)
2388:1
2382:n
2379:(
2376:m
2373:=
2368:2
2350:W
2337:W
2330:W
2306:j
2302:T
2296:j
2286:m
2281:1
2278:=
2275:j
2264:)
2261:n
2253:3
2249:n
2245:(
2240:S
2231:=
2226:w
2222:W
2207:W
2191:j
2169:j
2165:T
2144:i
2124:j
2102:j
2099:i
2095:t
2071:j
2058:)
2053:j
2050:i
2046:t
2037:3
2032:j
2029:i
2025:t
2021:(
2016:n
2011:1
2008:=
2005:i
1997:=
1992:j
1988:T
1950:)
1947:n
1939:3
1935:n
1931:(
1926:S
1917:=
1912:w
1908:W
1893:W
1872:2
1868:)
1858:R
1847:i
1843:R
1839:(
1834:n
1829:1
1826:=
1823:i
1815:=
1812:S
1789:S
1764:i
1760:R
1754:n
1749:1
1746:=
1743:i
1733:n
1730:1
1725:=
1716:R
1685:j
1682:i
1678:r
1672:j
1662:m
1657:1
1654:=
1651:j
1643:=
1638:i
1634:R
1610:i
1590:)
1587:m
1584:,
1581:.
1578:.
1575:.
1572:,
1569:2
1566:,
1563:1
1560:=
1557:j
1554:(
1549:j
1522:j
1497:j
1477:m
1457:n
1437:j
1415:j
1412:i
1408:r
1387:i
1365:m
1360:j
1358:T
1344:)
1339:j
1335:T
1331:(
1326:m
1321:1
1318:=
1315:j
1300:i
1295:i
1293:R
1276:,
1270:)
1265:j
1261:T
1257:(
1252:m
1247:1
1244:=
1241:j
1233:m
1227:)
1224:1
1216:2
1212:n
1208:(
1205:n
1200:2
1196:m
1188:2
1184:)
1180:1
1177:+
1174:n
1171:(
1168:n
1163:2
1159:m
1155:3
1149:)
1144:2
1139:i
1135:R
1131:(
1126:n
1121:1
1118:=
1115:i
1101:=
1098:W
1085:W
1077:j
1075:T
1071:j
1067:j
1063:j
1058:j
1056:T
1052:j
1048:n
1043:j
1041:g
1037:i
1032:i
1030:t
1013:,
1010:)
1005:i
1001:t
992:3
987:i
983:t
979:(
972:j
968:g
962:1
959:=
956:i
948:=
943:j
939:T
925:W
909:n
895:m
892:=
889:r
886:=
863:n
860:=
857:p
834:.
828:)
825:1
817:2
813:n
809:(
806:n
801:2
789:2
784:)
780:1
777:+
774:p
770:(
765:n
760:2
756:r
752:3
746:)
741:2
736:i
732:R
728:(
723:n
718:1
715:=
712:i
698:=
695:W
682:W
664:1
645:r
640:,
628:n
622:p
612:p
600:n
575:1
569:m
564:1
558:W
555:m
549:=
544:s
534:r
506:)
500:2
495:m
489:(
474:W
467:W
463:W
459:W
442:.
436:)
433:n
425:3
421:n
417:(
412:2
408:m
402:S
393:=
390:W
377:W
360:,
355:2
351:)
341:R
330:i
326:R
322:(
317:n
312:1
309:=
306:i
298:=
295:S
282:S
265:.
260:i
256:R
250:n
245:1
242:=
239:i
229:n
226:1
221:=
212:R
183:,
178:j
175:,
172:i
168:r
162:m
157:1
154:=
151:j
143:=
138:i
134:R
120:i
116:m
112:n
108:j
101:r
97:i
81:W
66:W
62:W
58:W
54:W
47:W
22:W
Text is available under the Creative Commons Attribution-ShareAlike License. Additional terms may apply.