152:
107:
188:
24:
229:
1225:
1339:
Lesser errors: Session loss often occurs using cookie-based authentication methods. Good bot frameworks should handle these automatically by logging in again on getting the assertbotfailed or assertuserfailed API response, but if yours doesn't, you can avoid these problems just by using OAuth. OAuth
1208:
Automate delsort taggings using some custom machine learning model. The model can be trained on basis of delsort tagging done so far by humans. Difficulty here is that unbiased training of the model requires access to content of deleted articles as well. Simply training it on articles that were kept
1335:
Faster: BotPasswords requires at least 3 API calls just to get the bot off the block: one to fetch a login token, another to actually log in, and usually another one to fetch editing tokens. Since OAuth doesn't require any API call to begin the authentication, you just need one API call – to fetch
1318:
If SDZeroBot is unable to save an edit because it is introducing a spam-blacklisted link (which of course isn't the bot's fault since it likely just picked up the text to be added from another place), it identifies the problematic link from the API response, and removes the protocol ("http:" or
1124:, as well as on the /data/project/sdzerobot directory on Toolforge. Even the logs (*.out and *.err files) are publicly visible, which is by default not the case on Toolforge. The jobs.yml file used to schedule the tasks can also be viewed there.
1082:
Good question. Excerpts of articles used on many of SDZeroBot's classification pages are generated using a combination of regex and some slightly more formal parsing methods. The Node.js source code used can be seen
1309:
A good combination of failure monitoring techniques is essential for operating bots that reliably perform a number of tasks without requiring you to spend time and energy on making sure everything is running.
1343:
No need to cache cookies: If your bot task is too frequent (say every 10 minutes), you're likely to have a high login rate unless you cache the login cookies and use them across bot runs. High login rates are
79:
1379:
690:
1280:
The only kinds of errors the above wouldn't handle are the ones that occur even before the javascript code starts executing (such as the file accidentally losing its executable permission) or
1374:
1295:
In addition, for the report pages, this user page lists them above along with their last updated timestamps. Along with the expected frequency of the updates, it is fed into a
932:
908:
878:
1323:
ends up looking like , but it is closest to the original while allowing the edit to go through. Besides, the link was blacklisted anyway so probably shouldn't be clickable.
1044:
1021:
990:
854:
830:
1194:
Consider creating a
Toolforge-hosted web UI for AfD sorting list, so that more columns can be added (whose visibility can be toggled using javascript), based on ideas in
977:
1181:
274:
1187:
If that also doesn't give desired level of accuracy (esp. for non-bio articles), don't actually add the shortdescs to the article, but show it on the sortlists.
1270:
141:
Administrators: If this bot needs to be blocked due to a malfunction, please remember to disable autoblocks so that other
Toolforge bots are not affected.
1195:
1146:
716:
938:
862:: Keeps track of activity of fully automatic bots and reports the ones that are not working. Optionally also notifies the respective operators.
642:
1030:
133:
498:
1364:
859:
1152:
Split the biography list by professions - this can be done just by looking at the other topics the bios have been classified with.
1273:
that traps errors and formats it as an email sent to the tool account, which lands in my inbox. For good measure, there's also a
666:
450:
1289:
546:
1319:"https:") from the link, and then attempts to save the page again. This does mean that a link that was supposed to look like
1010:
1369:
1103:
805:
802:
23:
399:
238:
for shutting off specific bot tasks. If the bot appears to be malfunctioning across multiple tasks, it should be blocked.
1237:
67:
63:
1248:
835:
618:
570:
351:
1106:, but it was all too messy and integrated with a lot of other popups code that I couldn't get it to work standalone.
187:
982:
Consolidate stub tags on page where possible (replace X-stub and Y-stub with X-Y-stub or Y-X-stub if either exists)
594:
375:
326:
914:
733:
730:
423:
779:
776:
31:
886:
474:
298:
75:
925:
922:
1241:
522:
792:
789:
114:
71:
235:
228:
1296:
1215:
Probably using
TemplateStyles, improve the appearance of the tables on very small and very wide screens.
1155:
For the STEM lists, make sections for core articles, STEM biographies, STEM media, STEM companies, ... (
1084:
1053:
151:
1274:
118:
1202:
1133:
1060:
1205:
of AfDs using ORES. Works only for select delsort lists which have a corresponding ORES topic, or
1088:
1255:
Create articlesearch tool – shows excerpts of articles from CirrusSearch queries – use ReactJS
161:
1174:
429:
35:
1156:
54:
1303:
995:
1180:
Explore use of machine learning for this, failing which other methods such as
Trialpears'
291:
288:
106:
1109:
All excerpts are short enough, so that attribution and copyright concerns are avoided.
950:
947:
901:
898:
847:
844:
838:
Notify users of AfD nominations of articles to which they've significantly contributed
587:
584:
563:
560:
539:
536:
515:
512:
491:
488:
1358:
1281:
871:
868:
1234:
Don't duplicate nomination text on AFD sorting report for multi-article nominations.
467:
464:
443:
440:
1096:
709:
706:
683:
680:
416:
413:
1091:. This excerpt generator is also available as a webservice hosted on Toolforge at
659:
656:
635:
632:
368:
365:
611:
608:
392:
389:
344:
341:
319:
316:
766:
763:
206:
50:
129:
214:
210:
70:
edits that would be extremely tedious to do manually, in accordance with the
1306:
which although maintained by SDZeroBot, is not used for monitoring itself.
1121:
218:
1331:
Always use OAuth instead of BotPasswords. There are all these advantages:
1138:
If you're interested in helping out with these tasks, please contact me.
380:
Classification of articles nominated for deletion at AfD by ORES topics
196:
125:
42:
937:
Raise edit requests to keep gadgets in sync with upstream sources per
87:
Administrators: if this bot is malfunctioning or causing harm, please
200:
74:. The bot is approved and currently active – the relevant
1095:
with a horrible bare minimum UI, but a better API endpoint. See the
356:
Classification of articles proposed for PROD deletion by ORES topics
1269:
For each SDZeroBot task, most of the code is in an async function
1166:
Automatically produce short descriptions for articles and drafts.
1149:(1500+ pages), use machine learning to section this list by sport.
305:
304:
Classification of pending AfC submissions by topics predicted by
1120:
source code that drives SDZeroBot is publicly available via the
623:
Annotated listing of articles for which peer review is requested
503:
Lists recently declined AFC drafts with excerpts and other data
1169:
Drafts mostly don't have short descriptions at all. They'd be
223:
101:
18:
1345:
1092:
1073:
Tasks which edit only in the userspace don't require a BRFA.
186:
1380:
Knowledge (XXG) bots with JavaScript source code published
647:
Lists pending AfC submissions with excerpts and other data
1299:
which prints the timestamp in bold red if it's delayed.
479:
Records excerpts of drafts that have been deleted per G13
1348:. Again, with OAuth, you don't have to worry about this.
1244:
figure out ways to better identify bad and good drafts?
671:
Annotated listing of unreferenced BLPs, for Women in Red
599:
Classifies articles awaiting GA review using ORES topics
575:
Lists G13-eligible drafts with descriptions and excerpts
551:
Classifies soon-to-be-G13-eligible drafts by ORES topics
428:
Tracks the status of articles proposed for deletion per
279:
List of user scripts by number of users and active users
998:
geographical stub articles with more specific stub tags
88:
892:: Updates tables with result of specified SQL queries.
1320:
1218:
Create unified lists for PROD and AFD which include
332:
Classification of unreviewed articles by ORES topics
1375:
Knowledge (XXG) bots running on
Wikimedia Toolforge
691:WP:List of Wikipedians by good article nominations
527:Lists drafts that would be-G13-eligible in a week
1212:Big one: identify promising AFC drafts using ML.
1177:. Also useful for the AfD, NPP, and PROD lists.
8:
1247:integrate unreliable source detection using
1102:I initially considered using the code from
455:Tracks conversions of articles to redirects
253:
138:
717:User:SDZeroBot/DYK nomination counts.json
404:Tracks articles being moved to draftspace
16:Knowledge (XXG) editing bot run by SD0001
1142:Split certain sortlists into subtopics.
1093:https://summary-generator.toolforge.org/
1036:to drafts that were moved from mainspace
962:
812:
745:
1275:process-level uncaughtException handler
1013:identifying cycles in the category tree
939:User:SDZeroBot/Gadgets-sync-config.json
643:User:SDZeroBot/Pending AfC submissions
1078:How do you generate article excerpts?
721:List of users by most DYK nominations
275:WP:User scripts/Most imported scripts
7:
1209:at AfD would not give good results.
1351:More secure: OAuth is more secure.
1284:, which are both handled by using
1222:deletion rationale and lead text.
499:User:SDZeroBot/Recent AfC declines
217:. The source code is available on
14:
1260:Tips and tricks for bot operators
205:bot framework, also developed by
1223:
1134:User:SD0001/Scripts § To-do
667:User:SDZeroBot/Unreferenced BLPs
451:User:SDZeroBot/Redirectify Watch
227:
150:
105:
22:
547:User:SDZeroBot/G13 soon sorting
1011:User:SDZeroBot/Category cycles
213:, while the newer ones are in
62:It is used to make repetitive
1:
1346:frowned upon by server admins
400:User:SDZeroBot/Draftify Watch
1238:User:SDZeroBot/Declined AFCs
209:. Most tasks are written in
1365:Active Knowledge (XXG) bots
1249:user:headbomb/unreliable.js
1031:Drafts moved from mainspace
619:User:SDZeroBot/Peer reviews
571:User:SDZeroBot/G13 eligible
352:User:SDZeroBot/PROD sorting
1396:
1314:Handling blacklisted links
1131:
595:User:SDZeroBot/GAN sorting
376:User:SDZeroBot/AfD sorting
327:User:SDZeroBot/NPP sorting
965:
915:User:SDZeroBot/Purge list
913:Purges pages linked from
815:
751:
695:List of users by most GAs
424:User:SDZeroBot/PROD Watch
78:(s) or tasks can be seen
1370:All Knowledge (XXG) bots
1290:Toolforge Jobs framework
1099:for usage instructions.
475:User:SDZeroBot/G13 Watch
1182:bio shortdesc generator
1087:, which also relies on
523:User:SDZeroBot/G13 soon
742:Other continuous tasks
236:User:SDZeroBot/Shutoff
191:
1009:Created the lists at
190:
1340:tokens don't expire.
1089:mwn's wikitext class
959:One-time / on-demand
860:Bot activity monitor
76:request for approval
1265:Monitoring failures
748:
119:Wikimedia Toolforge
1286:--emails onfailure
747:Internal tracking
746:
195:SDZeroBot runs on
192:
1122:GitHub repository
1074:
1070:
1069:
956:
955:
811:
810:
739:
738:
727:19 September 2024
702:18 September 2024
677:19 September 2024
653:19 September 2024
629:17 September 2024
605:19 September 2024
581:19 September 2024
557:16 September 2024
533:19 September 2024
509:19 September 2024
485:19 September 2024
461:19 September 2024
437:18 September 2024
410:17 September 2024
386:19 September 2024
362:19 September 2024
338:19 September 2024
313:19 September 2024
285:15 September 2024
242:
241:
147:
146:
143:
137:
100:
99:
94:
1387:
1287:
1230:
1227:
1226:
1072:
1058:
1052:
1035:
1029:
963:
891:
885:
813:
749:
703:
254:
231:
224:
181:
177:
171:
167:
165:
154:
123:
109:
102:
93:
84:
61:
48:
40:
26:
19:
1395:
1394:
1390:
1389:
1388:
1386:
1385:
1384:
1355:
1354:
1329:
1316:
1285:
1267:
1262:
1228:
1224:
1203:delsort tagging
1136:
1130:
1115:
1080:
1056:
1050:
1033:
1027:
961:
889:
887:Database report
883:
744:
701:
252:
247:
193:
184:
183:
179:
178:
176:in the category
175:
169:
168:
163:
162:
155:
139:
124:It runs on the
85:
83:
59:
46:
38:
17:
12:
11:
5:
1393:
1391:
1383:
1382:
1377:
1372:
1367:
1357:
1356:
1353:
1352:
1349:
1341:
1337:
1328:
1325:
1315:
1312:
1266:
1263:
1261:
1258:
1257:
1256:
1253:
1252:
1251:
1235:
1232:
1216:
1213:
1210:
1206:
1199:
1191:
1190:
1189:
1188:
1185:
1178:
1163:
1162:
1161:
1160:
1153:
1150:
1129:
1126:
1114:
1111:
1079:
1076:
1068:
1067:
1064:
1061:set categories
1047:
1041:
1040:
1037:
1024:
1018:
1017:
1014:
1007:
1003:
1002:
999:
993:
987:
986:
983:
980:
974:
973:
970:
967:
960:
957:
954:
953:
945:
942:
935:
929:
928:
920:
917:
911:
905:
904:
896:
893:
881:
875:
874:
866:
863:
857:
851:
850:
842:
839:
833:
827:
826:
823:
820:
817:
809:
808:
800:
796:
795:
787:
783:
782:
774:
770:
769:
761:
757:
756:
753:
743:
740:
737:
736:
728:
725:
722:
719:
713:
712:
704:
699:
696:
693:
687:
686:
678:
675:
672:
669:
663:
662:
654:
651:
648:
645:
639:
638:
630:
627:
624:
621:
615:
614:
606:
603:
600:
597:
591:
590:
582:
579:
576:
573:
567:
566:
558:
555:
552:
549:
543:
542:
534:
531:
528:
525:
519:
518:
510:
507:
504:
501:
495:
494:
486:
483:
480:
477:
471:
470:
462:
459:
456:
453:
447:
446:
438:
435:
432:
426:
420:
419:
411:
408:
405:
402:
396:
395:
387:
384:
381:
378:
372:
371:
363:
360:
357:
354:
348:
347:
339:
336:
335:Every 12 hours
333:
330:
323:
322:
314:
311:
308:
302:
299:WP:AfC sorting
295:
294:
286:
283:
280:
277:
271:
270:
267:
264:
261:
258:
251:
248:
246:
243:
240:
239:
232:
185:
156:
149:
148:
145:
144:
128:tool account (
110:
98:
97:
95:
68:semi-automated
27:
15:
13:
10:
9:
6:
4:
3:
2:
1392:
1381:
1378:
1376:
1373:
1371:
1368:
1366:
1363:
1362:
1360:
1350:
1347:
1342:
1338:
1334:
1333:
1332:
1326:
1324:
1322:
1313:
1311:
1307:
1305:
1302:There's also
1300:
1298:
1293:
1291:
1283:
1278:
1276:
1272:
1264:
1259:
1254:
1250:
1246:
1245:
1243:
1239:
1236:
1233:
1221:
1217:
1214:
1211:
1207:
1204:
1200:
1197:
1193:
1192:
1186:
1183:
1179:
1176:
1172:
1168:
1167:
1165:
1164:
1158:
1154:
1151:
1148:
1144:
1143:
1141:
1140:
1139:
1135:
1127:
1125:
1123:
1119:
1112:
1110:
1107:
1105:
1100:
1098:
1097:GitHub README
1094:
1090:
1086:
1077:
1075:
1065:
1062:
1055:
1048:
1046:
1043:
1042:
1038:
1032:
1025:
1023:
1020:
1019:
1015:
1012:
1008:
1005:
1004:
1000:
997:
994:
992:
989:
988:
984:
981:
979:
976:
975:
971:
968:
964:
958:
952:
949:
946:
943:
940:
936:
934:
931:
930:
927:
924:
921:
918:
916:
912:
910:
907:
906:
903:
900:
897:
894:
888:
882:
880:
877:
876:
873:
870:
867:
864:
861:
858:
856:
853:
852:
849:
846:
843:
840:
837:
834:
832:
829:
828:
824:
821:
818:
814:
807:
804:
801:
798:
797:
794:
791:
788:
785:
784:
781:
778:
775:
772:
771:
768:
765:
762:
759:
758:
754:
750:
741:
735:
732:
729:
726:
723:
720:
718:
715:
714:
711:
708:
705:
700:
697:
694:
692:
689:
688:
685:
682:
679:
676:
673:
670:
668:
665:
664:
661:
658:
655:
652:
649:
646:
644:
641:
640:
637:
634:
631:
628:
625:
622:
620:
617:
616:
613:
610:
607:
604:
601:
598:
596:
593:
592:
589:
586:
583:
580:
577:
574:
572:
569:
568:
565:
562:
559:
556:
553:
550:
548:
545:
544:
541:
538:
535:
532:
529:
526:
524:
521:
520:
517:
514:
511:
508:
505:
502:
500:
497:
496:
493:
490:
487:
484:
481:
478:
476:
473:
472:
469:
466:
463:
460:
457:
454:
452:
449:
448:
445:
442:
439:
436:
433:
431:
427:
425:
422:
421:
418:
415:
412:
409:
406:
403:
401:
398:
397:
394:
391:
388:
385:
383:Every 4 hours
382:
379:
377:
374:
373:
370:
367:
364:
361:
359:Every 4 hours
358:
355:
353:
350:
349:
346:
343:
340:
337:
334:
331:
328:
325:
324:
321:
318:
315:
312:
310:Every 8 hours
309:
307:
303:
300:
297:
296:
293:
290:
287:
284:
282:Every 2 weeks
281:
278:
276:
273:
272:
268:
265:
262:
259:
256:
255:
249:
244:
237:
233:
230:
226:
225:
222:
220:
216:
212:
208:
204:
203:
199:and uses the
198:
189:
182:
173:
172:
159:
153:
142:
135:
131:
127:
122:
120:
116:
111:
108:
104:
103:
96:
92:
90:
81:
77:
73:
69:
65:
58:
56:
52:
44:
37:
33:
28:
25:
21:
20:
1330:
1317:
1308:
1301:
1294:
1288:while using
1279:
1271:with a catch
1268:
1219:
1170:
1137:
1117:
1116:
1108:
1101:
1081:
1071:
1057:}}
1054:Set category
1051:{{
1034:}}
1028:{{
890:}}
884:{{
836:AfD notifier
799:db-tabulator
201:
194:
174:
170:Award Winner
166:Coolest Tool
160:
157:
140:
112:
86:
49:operated by
32:user account
29:
1336:the tokens.
1175:AfC sorting
1147:sports list
1113:Source code
969:Description
819:Description
266:Last update
260:Description
234:Please use
134:k8s grafana
1359:Categories
1321:Link label
1297:Lua module
1173:useful on
1157:discussion
1132:See also:
1066:On-demand
1001:On demand
972:Frequency
944:Continuous
919:Continuous
895:Continuous
865:Continuous
724:Continuous
215:TypeScript
211:JavaScript
130:k8s-status
72:bot policy
41:that uses
1327:Use OAuth
1201:Automate
1039:One-time
1016:One-time
985:One-time
822:Frequency
773:routerlog
329:+subpages
301:+subpages
263:Frequency
158:SDZeroBot
126:sdzerobot
64:automated
1242:G13 soon
1145:For the
180:Newcomer
117:runs on
89:block it
1049:Adding
1026:Adding
996:Re-sort
430:WP:PROD
250:Reports
197:Node.js
43:Node.js
1304:WP:BAM
1104:popups
760:stream
626:Weekly
554:Weekly
434:Weekly
407:Weekly
257:Report
219:GitHub
207:SD0001
60:
51:SD0001
47:
39:
1128:To do
841:Daily
825:Logs
755:Logs
698:Daily
674:Daily
650:Daily
602:Daily
578:Daily
530:Daily
506:Daily
482:Daily
458:Daily
269:Logs
245:Tasks
113:This
34:is a
30:This
1282:OOMs
1231:PROD
1220:both
1196:here
1171:very
1085:here
1045:BRFA
1022:BRFA
991:BRFA
978:BRFA
966:BRFA
933:BRFA
909:BRFA
879:BRFA
855:BRFA
831:BRFA
816:BRFA
786:gans
306:ORES
164:2020
80:here
55:talk
1118:All
1059:to
951:err
948:out
926:err
923:out
902:err
899:out
872:err
869:out
848:err
845:out
806:err
803:out
793:err
790:out
780:err
777:out
767:err
764:out
752:Job
734:err
731:out
710:err
707:out
684:err
681:out
660:err
657:out
636:err
633:out
612:err
609:out
588:err
585:out
564:err
561:out
540:err
537:out
516:err
513:out
492:err
489:out
468:err
465:out
444:err
441:out
417:err
414:out
393:err
390:out
369:err
366:out
345:err
342:out
320:err
317:out
292:err
289:out
202:mwn
115:bot
66:or
36:bot
1361::
1292:.
1277:.
1240:,
221:.
136:).
132:·
82:.
57:).
1229:Y
1198:.
1184:.
1159:)
1063:.
1006:—
941:.
121:.
91:.
53:(
45:,
Text is available under the Creative Commons Attribution-ShareAlike License. Additional terms may apply.
