User talk:Mia

Fixing infobox/Lua errors

I'm putting my notes here so the status is clear while I'm working on fixing things.

GavinRobinson had reported that

Before the upgrade, it seemed to be the number of infoboxes on a manually created page that triggered errors. The first few infoboxes displayed fine but any more didn't display, and the number of error messages was the same as the number of missing infoboxes.

And that:

I've also been getting script errors in manual edits, especially if there are lots of infoboxes on a page. See http://collaborativecollections.org/WorldWarOne/index.php?title=Category:Pages_with_script_errors

However I'm not sure the number of errors is directly related to the number of infoboxes as http://collaborativecollections.org/WorldWarOne/British_Infantry_Arm has ten errors and 7 infoboxes; http://collaborativecollections.org/WorldWarOne/Guards_Division,_UK has matched error and infobox numbers; and http://collaborativecollections.org/WorldWarOne/Coldstream_Guards,_UK has one error and 7 infoboxes. I suspect it might be because some of the infobox templates got out of sync with usage but haven't had time to investigate.

Also NB: $wgShowExceptionDetails may be on at some times, in which case more information will show per error.

I'm still getting signal 24 errors. British Infantry Arm should have 17 infoboxes (see page source). 7 are displaying correctly, 10 are missing and there are 10 error messages.--GavinRobinson (talk) 10:16, 15 January 2015 (PST)

Is it always the same infobox missing? --Mia (talk) 10:43, 15 January 2015 (PST)

I think so. They display in the same order they're entered in the source code. It seems to be always after 7 infoboxes that it breaks down. Category:Pages_with_script_errors also includes some of the infobox templates themselves, so maybe try to fix those first and see if any pages still have problems. A maximum of 7 infoboxes won't be enough as some British regiments have up to 50 battalions. The signal 24 error on pages with more than 7 infoboxes may be a different thing from what was breaking the imports before Christmas. I've manually created all the battalion pages that failed to import, and none of them have script errors.--GavinRobinson (talk) 10:59, 15 January 2015 (PST)

I'm going to quickly try and replicate it with a different infobox before posting on the Scribunto pages. I should probably also try to reduce the number of conditionals as that'd generally reduce load but that's probably not related. This is probably where learning just enough to create infoboxes while in the middle of so many other things will come back to bite me. --Mia (talk) 11:07, 15 January 2015 (PST)

The response I got was 'You set $wgScribuntoEngineConf['luastandalone']['cpuLimit'] too low. Either raise it or unset it.' - it wasn't set so I tried it at 60 seconds, which has sorted out those errors but is slow to load. Caching will help with existing pages but it's still going to need tweaking. Anyway, let me know how you get on. --Mia (talk) 15:30, 15 January 2015 (PST)

Yes, the 24 error is fixed. Pages that had it are now displaying properly, although it looks like they don't disappear from Category:Pages_with_script_errors until they've been edited. Pages with infoboxes are still very slow to save: 57 seconds for Coldstream Guards, UK and 67 seconds after the captcha for 4th Guards Brigade, UK. Talk pages are only taking about 5 seconds to save.--GavinRobinson (talk) 01:14, 16 January 2015 (PST)

I'll remove some of the conditional processing from the infoboxes to see if that helps (but probably not for a few days) --Mia (talk) 03:29, 16 January 2015 (PST)

http://collaborativecollections.org/WorldWarOne/index.php?title=Category:Pages_with_too_many_expensive_parser_function_calls&action=edit&redlink=1 for pages that use 'too many expensive parser functions (like #ifexist)' doesn't exist, which might suggest that it's not the #ifexists, but I'll tweak Template:Infobox military unit, Template:Infobox command structure and Template:Infobox theatre of war --Mia (talk) 16:53, 26 January 2015 (PST)

After removing unnecessary #ifs, parser profiling data for Template:Infobox command structure (generated with the Show Preview button) showed: Expensive parser function count 5/100 though there's only two #ifs on the page. Template:Infobox_military_unit shows the same thing before and after removing one #if from the infobox. However, Template:Infobox theatre of war went from 5/100 to 4/100 when I checked before and after editing out an unnecessary #if.

Pages like 4th_Guards_Brigade,_UK showing 'Expensive parser function count: 0/100' when you view source. --Mia (talk) 12:33, 2 February 2015 (PST)

Import errors

I've successfully imported a batch of 10 pages of Indian battalions. Next I tried a batch of 20. 18 pages imported properly but then it gave a 500 Internal Server Error. This is different from the error message I got before Christmas, which was:

Fatal error: Maximum execution time of 30 seconds exceeded in /home/miaftpuser/collaborativecollections.org/WorldWarOne/extensions/Scribunto/engines/LuaStandalone/LuaStandaloneEngine.php on line 388

That came up after only 6 pages of 32 had imported, so there is an improvement.--GavinRobinson (talk) 00:44, 9 February 2015 (PST)

Thanks Gavin! I've tweaked one php setting for timeout, but if that doesn't improve things I'll try another. It's only a little shared hosting account so it's tricky to beef it up too much. --Mia (talk) 03:59, 9 February 2015 (PST)

I'm still getting the same problem. This time I got 19 pages out of 20 before it threw a 500. Both times, the error came 2 minutes 3 seconds after starting the upload. At this rate, I can get all the Indian battalions imported without too much effort as long as the errors don't get any worse, but I'll need more pages per batch to get through the British.--GavinRobinson (talk) 05:53, 9 February 2015 (PST)

Just tried importing a batch of 14 British record offices. 10 imported then it threw a 500, again at 2 minutes 3 seconds. Record office pages have more infoboxes than Indian battalions but no more than 10 per page. Imported the other 4 in another batch with no problems. I'll see what the limits are for other kinds of unit.--GavinRobinson (talk) 10:58, 18 February 2015 (PST)

Only got 2 British divisions out of a batch of 10 before the 500 error, at 2 minutes 4 seconds. These pages have more infoboxes than most and also 2 revisions per page.--GavinRobinson (talk) 11:05, 18 February 2015 (PST)

The last import error was my fault: I fed it a bad file. So nothing needs investigating there.--GavinRobinson (talk) 10:20, 21 February 2015 (PST)

Ok! Are you still getting the 500 error on other imports? --Mia (talk) 16:45, 22 February 2015 (PST)

Yes, but only if the import process goes over 2 minutes. The number of pages I can get imported in under 2 minutes varies. It seems to be roughly related to the number of infoboxes but not a direct correlation with the total number of infoboxes. I get a vague impression that theatre adds more time than command structure, but too many of either will break it.--GavinRobinson (talk) 00:17, 23 February 2015 (PST)

It might be related to the number of expensive parser functions or the 'Highest expansion depth' rather than time. Each of Template:Infobox theatre of war, Template:Infobox military unit and Template:Infobox command structure show 'Expensive parser function count: 5/100', and two show 'Highest expansion depth: 10/40' (Template:Infobox military unit is 11/40). That said, Lua time usage is still showing as 60 seconds so I'll see what I can tweak. --Mia (talk) 11:36, 23 February 2015 (PST)

The 500 also happens with manual edits if they take 2 minutes 3 seconds to save. Just tried to manually create a page for the biggest British infantry regiment (50+ infoboxes) and it didn't work.--GavinRobinson (talk) 09:00, 27 February 2015 (PST)

Actually I've just found in my notes that some regiment pages took longer than 2:03 to save manual edits and didn't get an error. Lincolnshire Regiment, UK was 2:09; Argyll and Sutherland Highlanders, UK was 2:42. So if any settings have changed between 21st Feb and now they could have made things worse for manual edits.--GavinRobinson (talk) 11:26, 27 February 2015 (PST)

Performance improvements

I've edited in the cache changes suggested in http://wiki.dreamhost.com/MediaWiki_Troubleshooting

Front page before the edits: CPU time usage: 0.044 seconds, Real time usage: 0.043 seconds

Australian units page before the edits: CPU time usage: 0.248 seconds, Real time usage: 0.250 seconds

About the same immediately after but hopefully it'll improve once the cache has built. --Mia (talk) 16:05, 6 February 2015 (PST)

Pages with infoboxes seem to be loading quite fast now but saving edits still very slow. Adding 1 infobox to British Infantry Arm just took 1 minute 43 seconds to save. I might try importing a small batch on Monday and see how far it gets.--GavinRobinson (talk) 09:25, 7 February 2015 (PST)

Yikes, that's crazy! Let me know if it gets any quicker the second time. I've also just increased the time on $wgParserCacheExpireTime to two weeks, but IIRC logged in users won't get the benefits of all cache changes (https://www.mediawiki.org/wiki/Manual%3a%24wgParserCacheExpireTime) --Mia (talk) 10:06, 7 February 2015 (PST)

Another edit to the same page, 1 minute 43 again. Interesting that it's so consistent.--GavinRobinson (talk) 10:56, 7 February 2015 (PST)

Also, a note to myself to check whether there are enough database connections available and or see if there's another database-related bottleneck. --Mia (talk) 10:06, 7 February 2015 (PST)