536:
null-edit will be invalidated and will have to be updated again on the next inclusion. I virtually add some load on the system, meaning... that it will delay treatment of items of the JobQueue. (Think about it, the JobQueue is meant to work when system load is low: "I queue complex queries for later, when I'll have time to do it". If Parser has to update again pages, load goes up, jobs are treated later) Now, I see you coming, you're going to say that even without null-edits, the system itself is going to invalidate those pages, and ask the Parser to work. YES, you're absolutely right. But order does matter.
586:
47:
313:
edits is not going to crash the server, and it would allow the editor to empty a category, redirect it and move on to their next task rather than having to wait a month and then remember to come back and finish the job. I could check with a developer if it would make you happier about the performance issue. — Martin
312:
Yes, the job queue exists for a reason. What I have tried to assert above however, is that in some circumstances it would be beneficial to allow the template programmer to decide to bypass the queue if (s)he thinks there is a significant advantage of finishing the job more quickly. A few hundred null
198:
I do not intend to run this bot on very large categories (say, categories with more than a few thousand members). Waiting for the job queue to complete at off-peak times is more appropriate for these major changes. However forcing the recategorisation will be appropriate in smaller categories, when a
545:
I think that there is no need to try to bypass the system to make things go "faster": you're working from a higher level, and should not, by design, attempt to do this. I you have issues with the current JobQueue, open bug requests, complain noisily to developers and sysadmins. But don't try to "fix
266:
The backlink cache is never forcibly cleared. However, it has an expiry time of an hour, so this should only affect pages where the template is added within an hour of an edit to the template, so it shouldn't affect the majority of pages, and should be fixed if the template is edited again after the
330:
I agree that, if this is considered desirable, it should be done at the MediaWiki level. I am sure this is not the most efficient way of dealing with this issue. Note also that there has been significant development on the job queue recently -- I believe it is significantly more effective than it
262:
I did some testing with my own wiki, and haven't been able to replicate any of the issues described here. When editing a template, the job queue was properly populated and the script used by
Wikimedia to run the jobs as well as the jobs themselves seemed to be working correctly. The only 2 possible
222:
I'm guessing that its not intentional for such changes to take a month. The job queue length is currently only about 20,000. For it to take a month would mean that each job would have to take 2 minutes to run. Over the weekend I plan to do some investigating into the job queue to make sure that its
194:
it has been known to take up to two months for every article to move across. This can be undesirable for several reasons. It results in a long period of flux in which two categories are partially populated, which can cause confusion. It often means that by the time the category is empty, the editor
535:
More important, because I nulledit some articles myself, you recursively queue some more "jobs". Not directly on the JobQueue tho: I don't think that you can do this only null-editing. But because I null edit articles, I invalidate the Parser cache. Meaning that all pages transcluding the page you
406:
Reply to Z-man. Thanks for looking into this in detail. The two months I quoted was an extreme maximum, and indeed a lot of the articles make the transit quite quickly. However, please believe me that it can, and quite often does, take several weeks for all the articles to have moved categories.
390:
I.e. if there is a problem with the job queue taking too long to complete, it is the developers'/sysadmins' jobs to fix it. This is an ugly hack that has the potential to cause significant server load. If it is desirable, as I say, it should be done properly, by the MediaWiki developers.
531:
The articles were updated beforehand, meaning that part of what's queued in the JobQueue is useless: some jobs will just wait to be processed, and when finally its their turn... well, too bad, a bot did the job itself. It means that some articles are queued for
488:
The page you're about to null edit has little views over time. (as in there are no visits during the time it takes to process the page in the job queue). Taking a few hours to be updated is not going to change anything: no one is going to see it in the meantime
164:
on a list of pages (usually all pages in a particular category) in order to force their recategorisation when a populating template is changed, in cases when it is not desirable to wait up to one month (sometimes even more) for the job queue to do it.
298:
My thought is that the job queue exists for a reason, and before we approve a bot to hack around the job queue, it might be nice to ask the developers if they object or if this can be implemented in the queue timer more efficiently.
189:
I do a fair amount of template coding, and quite often templates are placed on pages in order to categorise them. When the template is edited, it can take a long time for the pages to change category: depending on the size of the
206:
to perform this task. Previously I have used the tool in a semi-automated manner by appending a line space to end of the page. The software automatically takes off a line space; thus a null edit is performed. I would expect that
492:
The page you're going to null edit is frequently viewed: during the time it takes for the Job Queue to process the page, some readers or editors are actually going to see a slightly, or completely broken page. Splitting again:
480:
FYI, since the Job Queue got implemented, the null edit script got deleted from the pywikipedia framework. There is a reason behing this. When edits are queued to be done, it's simply useless to do them a few hours before.
452:
Well, funnily enough, yesterday I made a stupid mistake with a template and it emptied about 10 thousand articles in a matter of hours :) So maybe you are right ... but this is far from typical in my experience. — Martin
195:
who changed the template has long forgotten about it and does not go back and redirect or delete the now empty category. In other words, the operation of cleaning up after a change is less likely to be as thorough.
424:
As I said, if it takes a month or more, that's more likely a sign that something is broken than an intentional design, in which case we should be trying to fix it, not hacking around it. Note that the job queue
505:
For me it's quite simple. The more the article is viewed, the more it has chances to be fixed in the mean-time. If it's not viewed a lot, chances that people are going to see the broken thing are low.
627:
429:
broken up until a few months ago, have there been any issues since the beginning of March? (according to the server admin log, it looks like it was fixed around mid-February) Looking at
363:
OverlordQ: there may be times when this is desied, but endorsing it kind of obsoletes the job queue. if this is done a lot, it would pose massive load on the serve3r4s
566:
Thanks for the comments everyone. I am surprised that null edits could be so controversial :) But I accept the reasoning here and will withdraw my request. — Martin
546:
it" from a high level. If you have some coding skills, please have a look at the MediaWiki code, try to come up with improvments, but such bots do no good.
21:
515:
But you have the right to disagree with the previous statement. If so, please think twice about the technical manipulations that you're doing:
87:
541:
And as a result JobQueue gets longer, and it takes longer to process each item. Oh, wait. I thought my aim was to reduce processing time?
208:
82:
117:
433:
now, its down to <700, so in 4 days since my first comment here, its worked through all 20,000 that I mentioned previously.
102:
270:
If the category is dependant on some parser function that doesn't return true when the job is run, it won't be updated.
243:
Hmm, so far I haven't been able to reproduce any issues. There's still a couple more things I want to test though.
344:? If something makes a job easier for editors then that takes priority over server performance concerns. — Martin
97:
596:
578:
560:
465:
447:
419:
395:
383:
As a technical matter, it's our responsibility to keep the system running well enough for what the sites require
374:
356:
335:
325:
307:
289:
257:
237:
214:
I will set up a task list, invite any editors to place requests and prioritise them according to my judgement.
341:
92:
522:
As a consequence it takes a significant time to process articles. But I want articles to be processed faster.
77:
525:
Solution: in the meantime, I do null edits to invalidate the cache myself to "fix" the articles beforehand
161:
58:
499:
The borked thing is seen but people don't know how to fix, or are not willing to fix it. Problem stays.
512:. Knowledge (XXG) is an encyclopedia project, yada, yada, no hurry. No need to fix what's not broken.
496:
Editors see the the issue and try to fix it. They edit the section/purge the article. Problem's gone.
149:
438:
430:
280:
248:
228:
555:
17:
203:
585:
191:
573:
508:
There's really no hurry in fixing a mistake when you know that it is going to be fixed
460:
414:
369:
351:
320:
135:
621:
435:
392:
332:
277:
245:
225:
593:
552:
301:
46:
171:
I intend to run it whenever I receive an appropriate request from an editor.
569:
456:
410:
347:
316:
131:
223:
still populating the queue correctly and that the jobs are run correctly.
199:
job and tidy-up operation can be completed in a relatively short time.
606:
The above discussion is preserved as an archive of the debate.
340:
I don't really understand the concerns here. Have you read
549:
I would strongly recommend not to accept such practices.
112:
107:
72:
612:
Subsequent comments should be made in a new section.
42:
Subsequent comments should be made in a new section.
628:
Withdrawn
Knowledge (XXG) bot requests for approval
477:Ah. Quite surprising to see such requests here.
36:The following discussion is an archived debate.
8:
367:does not consume resources unnecessarily
7:
209:Special:Contributions/Null edit bot
28:
202:I believe I can successfully use
44:The result of the discussion was
584:
45:
365:Have you read the bots policy?
143:Automatic or Manually Assisted:
1:
145:Automatic and unsupervised
644:
597:17:51, 20 April 2009 (UTC)
579:17:34, 20 April 2009 (UTC)
561:17:18, 20 April 2009 (UTC)
466:16:33, 20 April 2009 (UTC)
448:16:30, 20 April 2009 (UTC)
420:14:47, 20 April 2009 (UTC)
396:16:18, 20 April 2009 (UTC)
375:16:14, 20 April 2009 (UTC)
357:15:57, 20 April 2009 (UTC)
336:15:13, 20 April 2009 (UTC)
326:14:51, 20 April 2009 (UTC)
308:05:09, 19 April 2009 (UTC)
290:04:30, 19 April 2009 (UTC)
258:16:40, 17 April 2009 (UTC)
238:04:41, 16 April 2009 (UTC)
528:Two direct consequences:
609:Please do not modify it.
519:The job Queue is crowded
379:A quote from that page:
39:Please do not modify it.
331:was a few months ago.
590:Withdrawn by operator.
263:issues I can see are:
175:Already has a bot flag
51:Withdrawn by operator
22:Requests for approval
362:<Duesentrieb: -->
150:Programming Language
18:Knowledge (XXG):Bots
431:Special:Statistics
211:would stay empty.
158:Function Overview:
577:
464:
418:
355:
324:
187:Function Details:
139:
635:
611:
588:
567:
558:
454:
446:
408:
372:
345:
314:
304:
288:
256:
236:
154:AutoWikiBrowser
129:
49:
41:
643:
642:
638:
637:
636:
634:
633:
632:
618:
617:
616:
607:
556:
434:
370:
302:
276:
244:
224:
220:
169:Edit period(s):
123:
62:
37:
26:
25:
24:
12:
11:
5:
641:
639:
631:
630:
620:
619:
615:
614:
601:
582:
581:
543:
542:
539:
538:
537:
533:
526:
523:
520:
503:
502:
501:
500:
497:
490:
475:
474:
473:
472:
471:
470:
469:
468:
404:
403:
402:
401:
400:
399:
398:
388:
387:
386:
385:(Brion Vibber)
377:
342:WP:PERFORMANCE
328:
293:
292:
273:
272:
271:
268:
267:cache expires.
260:
219:
216:
122:
121:
115:
110:
105:
100:
95:
90:
85:
80:
75:
73:Approved BRFAs
70:
63:
61:
56:
55:
54:
32:
30:
27:
15:
14:
13:
10:
9:
6:
4:
3:
2:
640:
629:
626:
625:
623:
613:
610:
604:
603:
602:
599:
598:
595:
591:
587:
580:
575:
571:
565:
564:
563:
562:
559:
554:
550:
547:
540:
534:
530:
529:
527:
524:
521:
518:
517:
516:
513:
511:
506:
498:
495:
494:
491:
487:
486:
485:
484:A few cases:
482:
478:
467:
462:
458:
451:
450:
449:
445:
444:
442:
437:
432:
428:
423:
422:
421:
416:
412:
405:
397:
394:
389:
384:
381:
380:
378:
376:
373:
368:
364:
360:
359:
358:
353:
349:
343:
339:
338:
337:
334:
329:
327:
322:
318:
311:
310:
309:
306:
305:
297:
296:
295:
294:
291:
287:
286:
284:
279:
274:
269:
265:
264:
261:
259:
255:
254:
252:
247:
242:
241:
240:
239:
235:
234:
232:
227:
217:
215:
212:
210:
205:
200:
196:
193:
188:
184:
182:
179:
176:
172:
170:
166:
163:
159:
155:
153:
151:
146:
144:
140:
137:
133:
127:
119:
116:
114:
111:
109:
106:
104:
101:
99:
96:
94:
91:
89:
86:
84:
81:
79:
76:
74:
71:
69:
65:
64:
60:
59:Null edit bot
57:
52:
48:
43:
40:
34:
33:
31:
23:
19:
608:
605:
600:
589:
583:
551:
548:
544:
514:
509:
507:
504:
483:
479:
476:
440:
439:
426:
382:
366:
361:
300:
282:
281:
250:
249:
230:
229:
221:
213:
201:
197:
186:
185:
180:
177:
174:
173:
168:
167:
157:
156:
148:
147:
142:
141:
125:
124:
67:
50:
38:
35:
29:
160:To perform
218:Discussion
162:null edits
113:rights log
103:page moves
407:— Martin
192:job queue
128:— Martin
126:Operator:
108:block log
622:Category
532:nothing.
510:for sure
83:contribs
20: |
594:Quadell
553:NicDumZ
489:anyway.
303:MBisanz
178:(Y/N)
88:count
16:<
574:talk
570:MSGJ
461:talk
457:MSGJ
415:talk
411:MSGJ
352:talk
348:MSGJ
321:talk
317:MSGJ
152:(s):
136:talk
132:MSGJ
118:flag
98:logs
78:talk
68:BRFA
443:man
436:Mr.
427:was
285:man
278:Mr.
275:--
253:man
246:Mr.
233:man
226:Mr.
204:AWB
183:No
93:SUL
624::
592:–
572:·
459:·
441:Z-
413:·
350:·
319:·
283:Z-
251:Z-
231:Z-
134:·
576:)
568:(
557:~
463:)
455:(
417:)
409:(
393:]
371:Q
354:)
346:(
333:]
323:)
315:(
181::
138:)
130:(
120:)
66:(
53:.
Text is available under the Creative Commons Attribution-ShareAlike License. Additional terms may apply.
