128:
significant (F(1,146)=54.835, p = 9.636e-12 *** ) negative correlation between the number of open reviews and the time since the start of the GA cup. This shouldn't come as a surprise, the GA cup narrows down the number of participants as time goes on resulting in fewer competitors, however this also seems to mean there are fewer reviewers as well. While a good way to reduce the backlog in the short term, it seems the GA Cups do not encourage continued participation in GA reviewing which is likely what contributes to the inevitable backlog increase after a GA cup.
166:
I also investigated whether the GA cup led to significantly more passes or fails. I did not find evidence that this was the case. The number of passed or failed nominations was divided by the total number of closed reviews for that day producing a daily rate of acceptance and a daily rate of failure.
99:
There is also a rather sharp increase in the backlog in between the two cups. It is unclear whether this increase is due to limited numbers of reviewers, an increase in nominations, or some combination of the two. While more investigation is needed, there are some hints in the data. Over the two year
118:
Daily data points were used and chosen as the last revision of that day. The period of interest was the first GA cup (1 October 2014 to 26 February 2015). The control was the period in between the first and second GA cup (27 February 2015 to 30 June 2015). Nominations were counted in the same way as
162:
A t-test was also performed on the daily number of closed nominations during the cup (n=149, M=7.03) and during the inter-cup period (n=125, M=4.78). There is a highly significant difference between the number of closures during the cup as in between the cups (t(270.9)=5.92, p=9.32x10***). This is
149:
was performed on the number of daily nominations during the cup (n=149, M = 9.4) and in the inter-cup period (n=125, M = 10.6). There was no significant difference between the daily number of nominations during the cup and during the inter-cup period (t(246.52) = -1.13, p = 0.26 n.s.). This should
93:
Despite this, there is not a significant difference between the number of reviews during a cup as when a cup is going on ( t(22.563)=0.31109, p=0.7586 ). This may be because the GA cup rounds tended to finish at the end of the month when samples are taken, or because the number of reviews are the
26:
After learning to build a bot, reading about the new GA cup, and wanting to keep my stats skills from getting rusty, I decided to try and look at the GA backlog and see if there are any trends. The following are some data that I gathered on the GA nomination backlog with some analysis to try and
141:
These data provide a good look into how the GA cup affects the backlog. As these data are more granular than previous runs, they lend themselves to analysis better than sheer numbers or monthly totals do. Unfortunately the methodology makes doing correlation over time a rather difficult task so
104:
negative correlation (r=-0.319 between the number of reviews open and time (F(1,23)=2.61, p=0.1195 n.s.). This may mean that the number of reviewers is shrinking, but the data are limited. An investigation into the efficacy of the GA cup would need more nuanced data than these broad strokes.
127:
There is a highly significant difference ( t(257.94)=-15.026, p < 2.2e-16 *** ) in the number of reviews open during the first GA cup and in the period between GA cups showing that the GA cup is indeed what contributes to reducing the backlog. This effect is limited though as there is a
154:
the cup not before), that there was a positive correlation within the GA cup nominations but it cannot be seen because of the scrambled data, or that the difference is too small to notice. Regardless, this is a good result as it shows that if there
41:
the GA cup has diminishing returns—while more investigation is needed, the cups do not seem to encourage more people to keep reviewing after the cup or after getting knocked out (or at least not at the same
46:
This isn't a finished work, as I plan to update it every so often when I get bored enough to do statistics for fun. Hopefully it can help inform discussions on ways to reduce the lengthy backlog.
171:
when no reviews were closed that day) were omitted. Neither passage nor failure showed any significance. Failure: t(239.69)=0.856, p=0.392 n.s. and
Passage: t(239.69)=-0.856, p=0.392 n.s.
69:
The sampled version was chosen as the last version on the 28th day of the month so that there would be a consistent gap of one month between each sample. Nominations were counted by
96:
Upon further analysis with a much larger sample, it turns out that there is indeed a significant difference in the number of open reviews between GA cup days and non-GA cup days.
91:
There is an obvious correlation between the decline in number of nominations awaiting review and the GA cups, which is good evidence to their efficacy in reducing the backlog.
73:
match to the following expression <# \{\{GAN> meaning that only entries on the page were counted, rather than members of the category (which is how
35:
there seems to be a reducing number of open reviews over time—which could mean fewer reviewers, faster turnaround, or some combination
150:
not be extrapolated from. It could be that the first GA cup had an effect on nominations that lasted afterwards (since the control is from
142:
questions of whether the number of reviews over time is decreasing which was brought up in previous sections is still unresolved.
50:
215:
These are the same because they are reciprocals of each other as (passage / close) + (failure / closure) = closure
101:
225:
77:
70:
168:
28:
17:
58:
94:
same, they just have a faster turn around. There may also be a lack of data points.
206:
Closed nominations are those that were removed from the list as passed or failed.
193:
is the number of days sampled, not the number of nominations in total. Likewise,
27:
quantify why and how the backlog changes, particularly with regards to the
167:
These were then compared by t-test. Non-numerical values (arising from
146:
163:
evidence that no one really needed though: The GA cup does its job.
228:
gives p = 0.1131, which is not significant but much closer.
38:
the GA cups as effective means of reducing the backlog
159:an effect on nominations, it is likely very small.
8:
197:is the mean number of nominations per day
186:
184:
180:
7:
49:The full data can be found on the
24:
1:
132:Nomination and Clearing Rates
246:
31:. The main takeaways are:
83:reaches its numbers).
226:Mann–Whitney U test
100:period, there is a
71:regular expression
57:Last two years at
224:Interestingly, a
237:
229:
222:
216:
213:
207:
204:
198:
188:
169:division by zero
82:
76:
245:
244:
240:
239:
238:
236:
235:
234:
233:
232:
223:
219:
214:
210:
205:
201:
189:
182:
177:
139:
134:
125:
116:
111:
102:non-significant
89:
80:
74:
67:
62:
22:
21:
20:
12:
11:
5:
243:
241:
231:
230:
217:
208:
199:
179:
178:
176:
173:
138:
135:
133:
130:
124:
121:
115:
112:
110:
107:
88:
85:
66:
63:
61:
55:
44:
43:
39:
36:
23:
18:User:Wugapodes
15:
14:
13:
10:
9:
6:
4:
3:
2:
242:
227:
221:
218:
212:
209:
203:
200:
196:
192:
187:
185:
181:
174:
172:
170:
164:
160:
158:
153:
148:
143:
136:
131:
129:
122:
120:
113:
108:
106:
103:
97:
95:
86:
84:
79:
72:
64:
60:
56:
54:
52:
47:
40:
37:
34:
33:
32:
30:
19:
220:
211:
202:
194:
190:
165:
161:
156:
151:
144:
140:
126:
117:
109:First GA Cup
98:
92:
90:
81:}}
75:{{
68:
48:
45:
25:
114:Methodology
78:GAN counter
65:Methodology
51:data page
137:Analysis
123:Analysis
87:Analysis
119:above.
147:t-test
59:WP:GAN
29:GA cup
175:Notes
152:after
42:rate)
16:<
183:^
157:is
145:A
53:.
195:M
191:n
Text is available under the Creative Commons Attribution-ShareAlike License. Additional terms may apply.