437:
31:
113:
If the processes crash now then they will attempt to auto restart and failing that will email me and Rich. It could still "break" in a number of ways which is much harder to quickly see such as the API not returning good data, UDP sockets not binding properly on the bot etc. I'll try and come up with
342:
I've dropped the check to 30min with a 15min threshold. It seems the
Knowledge API was "randomly" timing out causing the bot to not startup or do anything correctly. It seems to be working as expected now however I'll have a poke around the logs and try to figure out what's been going on in a while
323:
Reducing the threshold before a "not running" warning is posted might help too: I know it's important to avoid false alarms, but has there ever been more than 15 mins between reverts when CBNG is up? At present any failures after half past the hour have to wait another hour before they get reported.
95:
Thanks! I'll notify you quicker if I see that it's down again. Some means of automatically alerting yourself when it fails may be a good idea - I suspect that everyone who cares assumed it was offline for retraining or maintenance or something like that. After all, it does provide an invaluable
233:
than 1 seconds (startsecs) 2011-08-18 09:54:22,002 INFO exited: cluebotng_bot (terminated by SIGKILL; not expected) 2011-08-18 09:54:23,006 INFO spawned: 'cluebotng_bot' with pid 4832 2011-08-18 09:54:24,149 INFO success: cluebotng_bot entered RUNNING state, process has stayed up for :
352:
Thanks again - and for the edit summary change. Sorry if I disturbed you - I imagined you sitting at work waiting for the weekend - one never knows... It's a mighty burden you're apparently carrying alone: ensuring that the world's largest encyclopaedia stays free of vandalism :)
238:, the interesting point is the bot is actually running now but looking at the output on every revert the bot is just getting an API error back. I'm going to have a dig though the code and try to figure what on earth is going on or poke Cobi when its day for him later. -
231:
2011-08-18 09:53:47,117 INFO exited: cluebotng_bot (terminated by SIGKILL; not expected) 2011-08-18 09:53:48,150 INFO spawned: 'cluebotng_bot' with pid 4564 2011-08-18 09:53:49,414 INFO success: cluebotng_bot entered RUNNING state, process has stayed up for :
114:
some sort of test that can cover all the components but it is quite complex. Possibly even a cronjob to see if it has made any edits in the past hour would work but would be not be so quick to alert. -
160:
Nice one. Hope it doesn't wake you up in the middle of the night :) Is the updates page for anything other than a visual confirmation that CBNG is still running? If not, I'd suggest that looking at
261:
Oh and the cause of the bot crashing was oom killer killing it - going to see about moving it to a server with 4times more ram but its no straightforward task. I blame php for being so fat ;) -
320:
BTW, it would be good if you could add "up" or "down" to the edit summary of the automated posts made to /running, as this would benefit anyone with the page in their watchlist (i.e. me!)
274:, which I'm sure didn't exist in my Linux days. I guess memory shortage is going to cause all sort of ongoing instabilities, so I hope you can fix that. Could you set /proc/<pid: -->
183:
Nope, it's there as a sanity check and just because I can. Never used the mediawiki package as I've never done any heavy lifting with mediawiki just simple page grabbing/uploading. -
480:
252:
Ok, I fixed an issue with the code that was caused by ipv6 being stupid (the server is dual stacked). Should be functioning fine again now. -
80:
65:
416:
389:
161:
86:
This should be running again, I've also made some changes to how it runs so hopefully it will re-start its self if it crashes. -
492:
476:
209:
Well that was a worst case scenario: CBNG stopped just after 0730 and your script should have alerted you at 0900. —
38:
472:
436:
123:
I'd say a cronjob sounds best - simple and fast enough. Periodically checking the timestamp of its latest edit
292:
I'd rather the bot crash and supervisord restart it a few seconds later than it crash the entire server ;) -
148:
47:
17:
499:
484:
451:
447:
420:
412:
393:
385:
365:
347:
336:
296:
287:
265:
256:
242:
221:
187:
178:
166:
Out of interest, do you ever use
Mediawiki::API to wrap LWP? It supports retries and error reporting. —
155:
139:
118:
108:
90:
443:
408:
381:
468:
404:
377:
317:
Hi Damian - as I'm sure you know, it's down for the second time today. Is this still the oom issue?
496:
360:
344:
331:
293:
282:
262:
253:
239:
216:
184:
173:
152:
134:
115:
103:
87:
75:
354:
325:
276:
271:
210:
167:
128:
97:
69:
46:
If you wish to start a new discussion or revive an old one, please do so on the
275:/oomadj to OOM_DISABLE and let something else break instead? ;) —
144:
25:
227:
Indeed it did, the bot auto restarted as it should have done:
124:
147:
is now running hourly and seems to work ok, updates at
164:
would be just as easy, making it perhaps unnecessary.
59:
60:CBNG hasn't reverted anything for over 4 days
8:
270:Thanks for the updates. I had to look up
373:hi i,ll use the computer tommrow okay
400:erm listening talk to me about stuff
44:Do not edit the contents of this page.
7:
464:i was trying to send him a message
24:
435:
64:Letting you know per the notice
29:
1:
493:Knowledge:No_personal_attacks
68:. I've also alerted Rich. —
500:02:11, 31 August 2011 (UTC)
485:02:08, 31 August 2011 (UTC)
452:12:22, 29 August 2011 (UTC)
421:12:20, 29 August 2011 (UTC)
394:23:58, 28 August 2011 (UTC)
366:14:44, 26 August 2011 (UTC)
348:14:15, 26 August 2011 (UTC)
337:13:07, 26 August 2011 (UTC)
297:15:33, 18 August 2011 (UTC)
288:15:29, 18 August 2011 (UTC)
266:12:24, 18 August 2011 (UTC)
257:12:22, 18 August 2011 (UTC)
243:11:04, 18 August 2011 (UTC)
235:than 1 seconds (startsecs)
222:08:40, 18 August 2011 (UTC)
188:18:27, 17 August 2011 (UTC)
179:18:16, 17 August 2011 (UTC)
156:14:57, 17 August 2011 (UTC)
140:13:36, 17 August 2011 (UTC)
119:12:39, 17 August 2011 (UTC)
109:12:13, 17 August 2011 (UTC)
91:10:05, 17 August 2011 (UTC)
81:09:27, 17 August 2011 (UTC)
516:
343:when I'm actually awake.-
149:User:ClueBot NG/running
127:would work, I think. —
18:User talk:DamianZaremba
96:service these days. —
42:of past discussions.
473:SomedayCameSuddenly
442:meet me in my talk
460:It wasnt vandalism
428:A cupcake for you!
488:
471:comment added by
457:
456:
424:
407:comment added by
397:
380:comment added by
363:
358:
334:
329:
285:
280:
219:
214:
176:
171:
137:
132:
106:
101:
78:
73:
54:
53:
48:current talk page
507:
487:
465:
439:
432:
431:
423:
401:
396:
374:
361:
356:
332:
327:
283:
278:
217:
212:
174:
169:
135:
130:
104:
99:
76:
71:
33:
32:
26:
515:
514:
510:
509:
508:
506:
505:
504:
466:
462:
430:
402:
375:
315:
236:
207:
162:CBNG's contribs
62:
30:
22:
21:
20:
12:
11:
5:
513:
511:
503:
502:
461:
458:
455:
454:
440:
429:
426:
371:
370:
369:
368:
314:
313:CBNG not happy
311:
310:
309:
308:
307:
306:
305:
304:
303:
302:
301:
300:
299:
230:
229:
228:
206:
203:
202:
201:
200:
199:
198:
197:
196:
195:
194:
193:
192:
191:
181:
165:
61:
58:
56:
52:
51:
34:
23:
15:
14:
13:
10:
9:
6:
4:
3:
2:
512:
501:
498:
497:DamianZaremba
494:
491:
490:
489:
486:
482:
478:
474:
470:
459:
453:
449:
445:
441:
438:
434:
433:
427:
425:
422:
418:
414:
410:
406:
398:
395:
391:
387:
383:
379:
367:
364:
359:
351:
350:
349:
346:
345:DamianZaremba
341:
340:
339:
338:
335:
330:
321:
318:
312:
298:
295:
294:DamianZaremba
291:
290:
289:
286:
281:
273:
269:
268:
267:
264:
263:DamianZaremba
260:
259:
258:
255:
254:DamianZaremba
251:
250:
249:
248:
247:
246:
245:
244:
241:
240:DamianZaremba
226:
225:
224:
223:
220:
215:
204:
189:
186:
185:DamianZaremba
182:
180:
177:
172:
163:
159:
158:
157:
154:
153:DamianZaremba
150:
146:
143:
142:
141:
138:
133:
126:
122:
121:
120:
117:
116:DamianZaremba
112:
111:
110:
107:
102:
94:
93:
92:
89:
88:DamianZaremba
85:
84:
83:
82:
79:
74:
67:
57:
49:
45:
41:
40:
35:
28:
27:
19:
467:— Preceding
463:
403:— Preceding
399:
376:— Preceding
372:
322:
319:
316:
237:
208:
190:- from phone
63:
55:
43:
37:
444:Sabrina1908
409:Sabrina1908
382:Sabrina1908
145:This script
125:via the API
36:This is an
272:oom killer
205:Down again
481:contribs
469:unsigned
417:contribs
405:unsigned
390:contribs
378:unsigned
39:archive
16:<
477:talk
448:talk
413:talk
386:talk
357:MALL
328:MALL
279:MALL
213:MALL
170:MALL
151:. -
131:MALL
100:MALL
72:MALL
66:here
362:JIM
333:JIM
284:JIM
234:-->
232:-->
218:JIM
175:JIM
136:JIM
105:JIM
77:JIM
495:-
483:)
479:•
450:)
419:)
415:•
392:)
388:•
353:—
324:—
475:(
446:(
411:(
384:(
355:S
326:S
277:S
211:S
168:S
129:S
98:S
70:S
50:.
Text is available under the Creative Commons Attribution-ShareAlike License. Additional terms may apply.