198:, is several months out of date. The list of should be regenerated when people update the pages. Practically, this could be done once per day. DustyBot will do this in two stages. The first stage will generate a list of ~10,000 dusty pages from the most recent database dump. This requires tens of thousands of page accesses to search for and disregard disambiguation pages. Fortunately, this only needs to be done when a new database dump is available, which happens once every couple of months. The list will be built over the course of several days, keeping page accesses below 10/min. The second stage will scan this list once per day for the 100 pages that are still dusty, and post that at
239:. I could process pages-articles.xml.bz2 instead, which would eliminate the need to check individual pages, but that would mean downloading a 4 GB file instead of tens or hundreds of MB. The list of 10,000 potentially dusty pages is just the first stage and is not posted to Knowledge (XXG). The pages on that list are either really dusty or have been very recently edited. The second stage goes through that list, weeding out the recently updated ones, until it has a list of 100 pages. That list of checked pages will be posted to Knowledge (XXG).
361:
423:
320:
47:
214:
using the API or ask someone with toolserver access to do a query. Also I'm somewhat confused, it "will generate a list of ~10,000 dusty pages" then scan "for the 100 pages that are still dusty." What will it actually be reporting on
Knowledge (XXG)?
257:. It currently has 364,018 pages, so at 5000 pages per request, it should only take ~29 API requests (the counts aren't always accurate), you can also ask someone to do a query on the toolserver database, or use the categorylinks.sql.gz dump.
202:. Because this bot will only edit Knowledge (XXG) once per day, and will only change one hard coded page, the risk of interfering with other editors is low. I am interested in hearing ideas about how to reduce the number of page accesses.
462:
210:
Which db dump are you using? If the dump doesn't contain the page text you should be able to generate lists of disambig pages from the templates at
21:
254:
87:
82:
324:
Approved for trial (5 days). Please provide a link to the relevant contributions and/or diffs when the trial is complete.
117:
199:
211:
102:
397:
295:
195:
276:
Thank you, your advice to download the category with API requests was very helpful. I have manually updated
434:
409:
387:
376:
354:
340:
307:
289:
271:
248:
229:
191:
145:
97:
92:
405:
372:
350:
331:
303:
285:
262:
244:
220:
132:
77:
393:
277:
157:
17:
253:
Okay, never mind the list of templates, you should just be able to generate a list from
456:
401:
368:
346:
328:
299:
281:
259:
240:
217:
128:
58:
429:
382:
46:
400:. DustyBot updates that page to avoid edit conflicts and page parsing.
236:
441:
The above discussion is preserved as an archive of the debate.
112:
107:
72:
294:
Also, I noticed that there is a date in the table at
447:
Subsequent comments should be made in a new section.
42:
Subsequent comments should be made in a new section.
463:Approved Knowledge (XXG) bot requests for approval
381:What is the point of edits to the /Updated page?
36:The following discussion is an archived debate.
8:
298:that will need to be updated by DustyBot.
166:(e.g. Continuous, daily, one time run)
189:The list of dusty pages, linked from
7:
398:Knowledge (XXG):Maintenance#Reports
28:
255:Category:All disambiguation pages
44:The result of the discussion was
421:
359:
318:
45:
235:I'm using page.sql.gz from the
139:Automatic or Manually Assisted:
200:Knowledge (XXG):Dusty articles
1:
212:MediaWiki:Disambiguationspage
435:03:33, 25 October 2008 (UTC)
410:03:32, 25 October 2008 (UTC)
388:03:25, 25 October 2008 (UTC)
377:17:51, 21 October 2008 (UTC)
355:04:38, 17 October 2008 (UTC)
341:23:21, 16 October 2008 (UTC)
308:16:33, 14 October 2008 (UTC)
290:16:29, 14 October 2008 (UTC)
272:20:47, 13 October 2008 (UTC)
249:15:59, 13 October 2008 (UTC)
230:07:24, 13 October 2008 (UTC)
479:
280:with the generated list.
444:Please do not modify it.
345:Ready to start testing.
39:Please do not modify it.
392:It is transcluded into
326:Whenever you're ready.
367:DustyBot is disabled.
175:Already has a bot flag
394:Knowledge (XXG):DUSTY
278:Knowledge (XXG):DUSTY
158:Knowledge (XXG):DUSTY
22:Requests for approval
146:Programming Language
18:Knowledge (XXG):Bots
296:Maintenance reports
196:Maintenance reports
433:
386:
187:Function Details:
154:Function Summary:
470:
446:
432:
425:
424:
385:
363:
362:
339:
322:
321:
270:
228:
49:
41:
478:
477:
473:
472:
471:
469:
468:
467:
453:
452:
451:
442:
422:
365:Trial complete.
360:
327:
319:
258:
216:
208:
123:
62:
37:
26:
25:
24:
12:
11:
5:
476:
474:
466:
465:
455:
454:
450:
449:
419:
418:
417:
416:
415:
414:
413:
412:
357:
315:
314:
313:
312:
311:
310:
292:
207:
204:
163:Edit period(s)
122:
121:
115:
110:
105:
100:
95:
90:
85:
80:
75:
73:Approved BRFAs
70:
63:
61:
56:
55:
54:
32:
30:
27:
15:
14:
13:
10:
9:
6:
4:
3:
2:
475:
464:
461:
460:
458:
448:
445:
439:
438:
437:
436:
431:
428:
411:
407:
403:
399:
395:
391:
390:
389:
384:
380:
379:
378:
374:
370:
366:
358:
356:
352:
348:
344:
343:
342:
338:
337:
335:
330:
325:
317:
316:
309:
305:
301:
297:
293:
291:
287:
283:
279:
275:
274:
273:
269:
268:
266:
261:
256:
252:
251:
250:
246:
242:
238:
234:
233:
232:
231:
227:
226:
224:
219:
213:
205:
203:
201:
197:
194:
193:
188:
184:
182:
179:
176:
172:
170:
167:
164:
160:
159:
155:
151:
149:
147:
142:
140:
136:
134:
130:
127:
119:
116:
114:
111:
109:
106:
104:
101:
99:
96:
94:
91:
89:
86:
84:
81:
79:
76:
74:
71:
69:
65:
64:
60:
57:
52:
48:
43:
40:
34:
33:
31:
23:
19:
443:
440:
426:
420:
364:
333:
332:
323:
264:
263:
222:
221:
209:
192:SpecialPages
190:
186:
185:
180:
177:
174:
173:
168:
165:
162:
161:
153:
152:
144:
143:
138:
137:
125:
124:
67:
50:
38:
35:
29:
237:10/08 dump
206:Discussion
141:Automatic
113:rights log
103:page moves
427:Approved.
126:Operator:
108:block log
457:Category
402:Wronkiew
369:Wronkiew
347:Wronkiew
300:Wronkiew
282:Wronkiew
241:Wronkiew
129:Wronkiew
83:contribs
59:DustyBot
51:Approved
20: |
156:Update
171:Daily
178:(Y/N)
88:count
16:<
406:talk
396:and
373:talk
351:talk
304:talk
286:talk
245:talk
150:PHP
148:(s):
133:talk
118:flag
98:logs
78:talk
68:BRFA
336:man
329:Mr.
267:man
260:Mr.
225:man
218:Mr.
93:SUL
459::
430:BJ
408:)
383:BJ
375:)
353:)
334:Z-
306:)
288:)
265:Z-
247:)
223:Z-
183:N
135:)
404:(
371:(
349:(
302:(
284:(
243:(
181::
169::
131:(
120:)
66:(
53:.
Text is available under the Creative Commons Attribution-ShareAlike License. Additional terms may apply.
