Difference between revisions of "Mediawiki maintenance"

Revision as of 13:48, 11 January 2013

(See also Mediawiki on biowikifarm (quick introduction/overview of setup) and the Mediawiki installation page (first time installation details).)

NOTE: the instructions below are fully valid for the 1.18/SVN setup. Some steps may differ in the current 1.20/git setup!

Mediawiki maintenance scripts

(See http://www.mediawiki.org/wiki/Manual:Maintenance_scripts)

In /usr/share/mediawiki/phase3/ copy AdminSettings.sample to AdminSettings.php, create a new user wikiadmin for mysql with localhost only, and edit AdminSettings.php accordingly. Then, for each wiki, in /var/www/wikifolder/, a symlink must be created linking to /usr/share/mediawiki/phase3/AdminSettings.php. Since we have a symlinked installation, installations scripts must run like:

 php /var/www/wikifolder/maintenance/rebuildall.php   --conf /var/www/wikifolder/LocalSettings.php
 php /var/www/wikifolder/maintenance/importImages.php --conf /var/www/wikifolder/LocalSettings.php /var/www/wikifolder/media/newimages .jpg .png .svg

Rebuild all is necessary after external text, sql, or xml imports.

Table corruption: rebuildall.php may report that a table, especially searchindex (which is MyISAM even if other tables are InnoDB) is corrupt and needs to be repaired. In phpmyadmin (http://biowikifarm.net/phpmyadmin_v3-3-7), open a sql window for the affected database, and issue: truncate table searchindex;. Running rebuildall.php afterwards concluded the repair in a case we had.

After erroneously localizing commons images not to openmedia but to local wikis, extension Nuke was used to deleted the caching-bot imports, and deleteArchivedFiles to actually delete all archived files, e.g.:

 cd /var/www/v-on/w; php ./maintenance/deleteArchivedFiles.php  --conf ./LocalSettings.php --delete

Fix problems with unparsable image names (see [1]). Needs NOT be run on regular basis, usually no problems:

 cd /var/www/v-k2n/w; php ./maintenance/cleanupImages.php --fix  --conf ./LocalSettings.php
 cd /var/www/v-on/w; php ./maintenance/cleanupImages.php --fix  --conf ./LocalSettings.php
 cd /var/www/v-species/w; php ./maintenance/cleanupImages.php --fix  --conf ./LocalSettings.php
 cd /var/www/v-species/o; php ./maintenance/cleanupImages.php --fix  --conf ./LocalSettings.php
 cd /var/www/v-species/s; php ./maintenance/cleanupImages.php --fix  --conf ./LocalSettings.php

Extension maintenance scripts

Some extensions provide their own maintenance scripts in their own extension folder. On a wiki farm, the execution of these scripts may fail due to the linked directory structure. They need an additional link (needs to be created only once):

 cd /usr/share/mediawiki/; ln -s /usr/share/mediawikistaging/phase3/maintenance
 cd /usr/share/mediawikistaging/; ln -s /usr/share/mediawikistaging/phase3/maintenance

TitleKey now installed (here only example, full set of update commands available further down!):

 cd /var/www/testwiki; php ./extensions/TitleKey/rebuildTitleKeys.php --conf ./LocalSettings.php

MediaWiki version upgrading

To upgrade Mediawiki first run the update on the staging version (= testwiki, http://biowikifarm.net/test):

 # Test for unwanted local changes:
 cd /usr/share/mediawikistaging/; sudo svn status phase3 extensions; 

 # If necessary, reset any unwanted local changes:
 ### cd /usr/share/mediawikistaging/; sudo svn revert -R phase3 extensions; 
 # restore modified icons reverted above:
 ### sudo cp /usr/share/mediawiki/extensions/SemanticMediaWiki/skins/images/smw_button.png ./extensions/SemanticMediaWiki/skins/images/
 ### sudo cp /usr/share/mediawiki/phase3/skins/common/images/poweredby_mediawiki_88x31.png ./phase3/skins/common/images/
 
 # MAIN UPDATING:
 cd /usr/share/mediawikistaging/; sudo svn update phase3 extensions  -r100100
  sudo /usr/sbin/apache2ctl -k graceful && sudo service nginx restart && sudo service php5-fpm restart
 # Optionally other updates:
 sudo apt-get update; sudo apt-get upgrade

Note: when trying to find the precise svn version that is causing a bug within a range of versions (e.g. r100281 to 102570), the following command shows which versions apply to the present SVN folders at all: cd /usr/share/mediawikistaging/phase3 sudo svn log -r100281:102570 -q

After testing and verifying functionality, also update the production version (where "-r100100" indicates to update to revision 52502; this should always be the value of the tested revision from staging / test wiki!):

 # Test for unwanted local changes:
 cd /usr/share/mediawikistaging/; sudo svn status phase3 extensions; 
 
 # If necessary, reset any unwanted local changes:
 ## cd /usr/share/mediawiki/; sudo svn revert -R phase3 extensions; 
 # restore modified icons reverted above:
 ## sudo cp /usr/share/mediawikistaging/extensions/SemanticMediaWiki/skins/images/smw_button.png ./extensions/SemanticMediaWiki/skins/images/
 ## sudo cp /usr/share/mediawikistaging/phase3/skins/common/images/poweredby_mediawiki_88x31.png ./phase3/skins/common/images/
 
 # MAIN UPDATING:
 cd /usr/share/mediawiki; sudo svn update phase3 extensions  # if version then: -r100100
 ## MORE CAREFUL WITH EXT-TRUNK: 
 ## cd /usr/share/mediawiki; sudo svn update ext-trunk # if version then: -r100100
 sudo /usr/sbin/apache2ctl -k graceful && sudo service nginx restart && sudo service php5-fpm restart
 ## ext-LOCAL-svn is usually not necessary, because of automatic svn push
 ## Updating ext-tagged-release not necessary, extensions in tagged releases don't change
 ## to check whether tagged extensions have new versions, go to 
 ##   http://svn.wikimedia.org/svnroot/mediawiki/tags/extensions/
 ## If a tagged extension has to be updated, follow the following pattern:
 cd /usr/share/mediawiki/ext-tagged-release
 sudo svn switch http://svn.wikimedia.org/svnroot/mediawiki/tags/extensions/AdminLinks/REL_0_1_5 AdminLinks
 sudo svn switch http://svn.wikimedia.org/svnroot/mediawiki/tags/extensions/ExternalData/REL_1_3_2 ExternalData
 sudo svn switch http://svn.wikimedia.org/svnroot/mediawiki/tags/extensions/HeaderTabs/REL_0_8_3 HeaderTabs
 sudo svn switch http://svn.wikimedia.org/svnroot/mediawiki/tags/extensions/Maps/REL_1_0_4 Maps
#### 1_0_5 is broken for nnvm.eu-maps!
 sudo svn switch http://svn.wikimedia.org/svnroot/mediawiki/tags/extensions/OpenID/REL_0_9 OpenID
 sudo svn switch http://svn.wikimedia.org/svnroot/mediawiki/tags/extensions/PageObjectModel/REL_0_1_3 PageObjectModel
 sudo svn switch http://svn.wikimedia.org/svnroot/mediawiki/tags/extensions/ReplaceText/REL_0_9_1 ReplaceText
 sudo svn switch http://svn.wikimedia.org/svnroot/mediawiki/tags/extensions/SemanticCompoundQueries/REL_0_3 SemanticCompoundQueries
 sudo svn switch http://svn.wikimedia.org/svnroot/mediawiki/tags/extensions/SemanticDrilldown/REL_1_0_1 SemanticDrilldown
 sudo svn switch http://svn.wikimedia.org/svnroot/mediawiki/tags/extensions/SemanticForms/REL_2_3_2 SemanticForms
 sudo svn switch http://svn.wikimedia.org/svnroot/mediawiki/tags/extensions/SemanticFormsInputs/REL_0_5 SemanticFormsInputs
 sudo svn switch http://svn.wikimedia.org/svnroot/mediawiki/tags/extensions/SemanticMaps/REL_1_0_4 SemanticMaps
#### 1_0_5 is broken for nnvm.eu-maps!
 #OLD: sudo svn switch http://svn.wikimedia.org/svnroot/mediawiki/tags/extensions/SemanticMediaWiki/REL_1_6_2 SemanticMediaWiki
 sudo svn switch http://svn.wikimedia.org/svnroot/mediawiki/tags/extensions/SemanticMediaWiki/REL_1_7_0_2 SemanticMediaWiki
 sudo svn switch http://svn.wikimedia.org/svnroot/mediawiki/tags/extensions/SemanticInternalObjects/REL_0_6_7 SemanticInternalObjects
 #OLD: sudo svn switch http://svn.wikimedia.org/svnroot/mediawiki/tags/extensions/SemanticResultFormats/REL_1_6_2 SemanticResultFormats
 sudo svn switch http://svn.wikimedia.org/svnroot/mediawiki/tags/extensions/SemanticResultFormats/REL_1_7 SemanticResultFormats
 sudo svn switch http://svn.wikimedia.org/svnroot/mediawiki/tags/extensions/SemanticSignup/REL_0_3 SemanticSignup
 sudo svn switch http://svn.wikimedia.org/svnroot/mediawiki/tags/extensions/SocialProfile/REL_1_3 SocialProfile
 sudo svn switch http://svn.wikimedia.org/svnroot/mediawiki/tags/extensions/Validator/REL_0_4_9 Validator
 sudo svn switch http://svn.wikimedia.org/svnroot/mediawiki/tags/extensions/Widgets/REL_0_9_2 Widgets
 ## not necessary with switch, but necessary with checkout: cd /usr/share/mediawiki/ext-tagged-release/Widgets/;
 ##   sudo chgrp www-data compiled_templates; sudo chmod g+w compiled_templates
 sudo /usr/sbin/apache2ctl -k graceful && sudo service nginx restart && sudo service php5-fpm restart

Update, runJobs, rebuildFileCache, refreshMetadata scripts

NOTE: Add gbif-kos!

Occasionally version upgrades of Mediawiki require an upgrade of the database structures. Again, the update.php must be called with symlink-compatible syntax. Examples for mediawiki core and titlekey extension:

 cd /var/www/metawiki;             sudo -u www-data php ./maintenance/update.php  --quick --conf ./LocalSettings.php
 cd /var/www/studienstiftungswiki; sudo -u www-data php ./maintenance/update.php  --quick --conf ./LocalSettings.php
 ### USER TABLE WARNING, 1.19 deletes field required by 1.18!!! ###
 ### cd /var/www/testwiki;             sudo -u www-data php ./maintenance/update.php  --quick --conf ./LocalSettings.php
 cd /var/www/testwiki2;            sudo -u www-data php ./maintenance/update.php  --quick --conf ./LocalSettings.php
 cd /var/www/v-floramalesiana/w;   sudo -u www-data php ./maintenance/update.php  --quick --conf ./LocalSettings.php
 cd /var/www/v-fotoflorabb/w;      sudo -u www-data php ./maintenance/update.php  --quick --conf ./LocalSettings.php
 cd /var/www/v-gbif-d/w;           sudo -u www-data php ./maintenance/update.php  --quick --conf ./LocalSettings.php
 cd /var/www/v-hypergarden/w;      sudo -u www-data php ./maintenance/update.php  --quick --conf ./LocalSettings.php
 cd /var/www/v-ispi/w;   sudo -u www-data php ./maintenance/update.php  --quick --conf ./LocalSettings.php
 cd /var/www/v-k2n/w;    sudo -u www-data php ./maintenance/update.php  --quick --conf ./LocalSettings.php
 cd /var/www/v-k2n/h;    sudo -u www-data php ./maintenance/update.php  --quick --conf ./LocalSettings.php
 cd /var/www/v-k2n/it;   sudo -u www-data php ./maintenance/update.php  --quick --conf ./LocalSettings.php
 cd /var/www/v-k2n/sl;   sudo -u www-data php ./maintenance/update.php  --quick --conf ./LocalSettings.php
 cd /var/www/v-lias/w;   sudo -u www-data php ./maintenance/update.php  --quick --conf ./LocalSettings.php
 cd /var/www/v-nnvm/w;   sudo -u www-data php ./maintenance/update.php  --quick --conf ./LocalSettings.php
 cd /var/www/v-on/w;     sudo -u www-data php ./maintenance/update.php  --quick --conf ./LocalSettings.php
 cd /var/www/v-orowiki/w;sudo -u www-data php ./maintenance/update.php  --quick --conf ./LocalSettings.php
 cd /var/www/v-phytomed/w;       sudo -u www-data php ./maintenance/update.php  --quick --conf ./LocalSettings.php
 cd /var/www/v-phytopathology/w; sudo -u www-data php ./maintenance/update.php  --quick --conf ./LocalSettings.php
 cd /var/www/v-plantnet/terms/e; sudo -u www-data php ./maintenance/update.php  --quick --conf ./LocalSettings.php
 cd /var/www/v-plantnet/terms/f; sudo -u www-data php ./maintenance/update.php  --quick --conf ./LocalSettings.php
 cd /var/www/v-plantnet/uses/e;  sudo -u www-data php ./maintenance/update.php  --quick --conf ./LocalSettings.php
 cd /var/www/v-plantnet/uses/f;  sudo -u www-data php ./maintenance/update.php  --quick --conf ./LocalSettings.php
 cd /var/www/v-rosaceae/w;  sudo -u www-data php ./maintenance/update.php  --quick --conf ./LocalSettings.php
 cd /var/www/v-species/f;   sudo -u www-data php ./maintenance/update.php  --quick --conf ./LocalSettings.php
 cd /var/www/v-species/o;   sudo -u www-data php ./maintenance/update.php  --quick --conf ./LocalSettings.php
 cd /var/www/v-species/s;   sudo -u www-data php ./maintenance/update.php  --quick --conf ./LocalSettings.php
 cd /var/www/v-species/w;   sudo -u www-data php ./maintenance/update.php  --quick --conf ./LocalSettings.php
 cd /var/www/v-workbench/w; sudo -u www-data php ./maintenance/update.php  --quick --conf ./LocalSettings.php
 cd /var/www/v-zsm/w;       sudo -u www-data php ./maintenance/update.php  --quick --conf ./LocalSettings.php
 # START OF TITLEKEY UPDATES:
 cd /var/www/metawiki;             sudo -u www-data php ./extensions/TitleKey/rebuildTitleKeys.php  --quick --conf ./LocalSettings.php
 cd /var/www/studienstiftungswiki; sudo -u www-data php ./extensions/TitleKey/rebuildTitleKeys.php  --quick --conf ./LocalSettings.php
 cd /var/www/testwiki;             sudo -u www-data php ./extensions/TitleKey/rebuildTitleKeys.php  --quick --conf ./LocalSettings.php
 cd /var/www/testwiki2;            sudo -u www-data php ./extensions/TitleKey/rebuildTitleKeys.php  --quick --conf ./LocalSettings.php
 cd /var/www/v-floramalesiana/w;   sudo -u www-data php ./extensions/TitleKey/rebuildTitleKeys.php  --quick --conf ./LocalSettings.php
 cd /var/www/v-fotoflorabb/w;      sudo -u www-data php ./extensions/TitleKey/rebuildTitleKeys.php  --quick --conf ./LocalSettings.php
 cd /var/www/v-gbif-d/w;           sudo -u www-data php ./extensions/TitleKey/rebuildTitleKeys.php  --quick --conf ./LocalSettings.php
 cd /var/www/v-hypergarden/w;      sudo -u www-data php ./extensions/TitleKey/rebuildTitleKeys.php  --quick --conf ./LocalSettings.php
 cd /var/www/v-ispi/w;   sudo -u www-data php ./extensions/TitleKey/rebuildTitleKeys.php  --quick --conf ./LocalSettings.php
 cd /var/www/v-k2n/w;    sudo -u www-data php ./extensions/TitleKey/rebuildTitleKeys.php  --quick --conf ./LocalSettings.php
 cd /var/www/v-k2n/h;    sudo -u www-data php ./extensions/TitleKey/rebuildTitleKeys.php  --quick --conf ./LocalSettings.php
 cd /var/www/v-k2n/it;   sudo -u www-data php ./extensions/TitleKey/rebuildTitleKeys.php  --quick --conf ./LocalSettings.php
 cd /var/www/v-k2n/sl;   sudo -u www-data php ./extensions/TitleKey/rebuildTitleKeys.php  --quick --conf ./LocalSettings.php
 cd /var/www/v-lias/w;   sudo -u www-data php ./extensions/TitleKey/rebuildTitleKeys.php  --quick --conf ./LocalSettings.php
 cd /var/www/v-nnvm/w;   sudo -u www-data php ./extensions/TitleKey/rebuildTitleKeys.php  --quick --conf ./LocalSettings.php
 cd /var/www/v-on/w;     sudo -u www-data php ./extensions/TitleKey/rebuildTitleKeys.php  --quick --conf ./LocalSettings.php
 cd /var/www/v-orowiki/w;sudo -u www-data php ./extensions/TitleKey/rebuildTitleKeys.php  --quick --conf ./LocalSettings.php
 cd /var/www/v-phytomed/w;       sudo -u www-data php ./extensions/TitleKey/rebuildTitleKeys.php  --quick --conf ./LocalSettings.php
 cd /var/www/v-phytopathology/w; sudo -u www-data php ./extensions/TitleKey/rebuildTitleKeys.php  --quick --conf ./LocalSettings.php
 cd /var/www/v-plantnet/terms/e; sudo -u www-data php ./extensions/TitleKey/rebuildTitleKeys.php  --quick --conf ./LocalSettings.php
 cd /var/www/v-plantnet/terms/f; sudo -u www-data php ./extensions/TitleKey/rebuildTitleKeys.php  --quick --conf ./LocalSettings.php
 cd /var/www/v-plantnet/uses/e;  sudo -u www-data php ./extensions/TitleKey/rebuildTitleKeys.php  --quick --conf ./LocalSettings.php
 cd /var/www/v-plantnet/uses/f;  sudo -u www-data php ./extensions/TitleKey/rebuildTitleKeys.php  --quick --conf ./LocalSettings.php
 cd /var/www/v-rosaceae/w;  sudo -u www-data php ./extensions/TitleKey/rebuildTitleKeys.php  --quick --conf ./LocalSettings.php
 cd /var/www/v-species/f;   sudo -u www-data php ./extensions/TitleKey/rebuildTitleKeys.php  --quick --conf ./LocalSettings.php
 cd /var/www/v-species/o;   sudo -u www-data php ./extensions/TitleKey/rebuildTitleKeys.php  --quick --conf ./LocalSettings.php
 cd /var/www/v-species/s;   sudo -u www-data php ./extensions/TitleKey/rebuildTitleKeys.php  --quick --conf ./LocalSettings.php
 cd /var/www/v-species/w;   sudo -u www-data php ./extensions/TitleKey/rebuildTitleKeys.php  --quick --conf ./LocalSettings.php
 cd /var/www/v-workbench/w; sudo -u www-data php ./extensions/TitleKey/rebuildTitleKeys.php  --quick --conf ./LocalSettings.php
 cd /var/www/v-zsm/w;       sudo -u www-data php ./extensions/TitleKey/rebuildTitleKeys.php  --quick --conf ./LocalSettings.php
 # '''runJobs''' in all wikis:
 cd /var/www/metawiki;             sudo -u www-data php ./maintenance/runJobs.php -v --procs 4 --conf ./LocalSettings.php
 cd /var/www/studienstiftungswiki; sudo -u www-data php ./maintenance/runJobs.php -v --procs 4 --conf ./LocalSettings.php
 cd /var/www/testwiki;             sudo -u www-data php ./maintenance/runJobs.php -v --procs 4 --conf ./LocalSettings.php
 cd /var/www/testwiki2;            sudo -u www-data php ./maintenance/runJobs.php -v --procs 4 --conf ./LocalSettings.php
 cd /var/www/v-floramalesiana/w;   sudo -u www-data php ./maintenance/runJobs.php -v --procs 4 --conf ./LocalSettings.php
 cd /var/www/v-fotoflorabb/w;      sudo -u www-data php ./maintenance/runJobs.php -v --procs 4 --conf ./LocalSettings.php
 cd /var/www/v-gbif-d/w;           sudo -u www-data php ./maintenance/runJobs.php -v --procs 4 --conf ./LocalSettings.php
 cd /var/www/v-hypergarden/w;      sudo -u www-data php ./maintenance/runJobs.php -v --procs 4 --conf ./LocalSettings.php
 cd /var/www/v-ispi/w;   sudo -u www-data php ./maintenance/runJobs.php -v --procs 4 --conf ./LocalSettings.php
 cd /var/www/v-k2n/w;    sudo -u www-data php ./maintenance/runJobs.php -v --procs 4 --conf ./LocalSettings.php
 cd /var/www/v-k2n/h;    sudo -u www-data php ./maintenance/runJobs.php -v --procs 4 --conf ./LocalSettings.php
 cd /var/www/v-k2n/it;   sudo -u www-data php ./maintenance/runJobs.php -v --procs 4 --conf ./LocalSettings.php
 cd /var/www/v-k2n/sl;   sudo -u www-data php ./maintenance/runJobs.php -v --procs 4 --conf ./LocalSettings.php
 cd /var/www/v-lias/w;   sudo -u www-data php ./maintenance/runJobs.php -v --procs 4 --conf ./LocalSettings.php
 cd /var/www/v-nnvm/w;   sudo -u www-data php ./maintenance/runJobs.php -v --procs 4 --conf ./LocalSettings.php
 cd /var/www/v-on/w;     sudo -u www-data php ./maintenance/runJobs.php -v --procs 4 --conf ./LocalSettings.php
 cd /var/www/v-orowiki/w;sudo -u www-data php ./maintenance/runJobs.php -v --procs 4 --conf ./LocalSettings.php
 cd /var/www/v-phytomed/w;       sudo -u www-data php ./maintenance/runJobs.php -v --procs 4 --conf ./LocalSettings.php
 cd /var/www/v-phytopathology/w; sudo -u www-data php ./maintenance/runJobs.php -v --procs 4 --conf ./LocalSettings.php
 cd /var/www/v-plantnet/terms/e; sudo -u www-data php ./maintenance/runJobs.php -v --procs 4 --conf ./LocalSettings.php
 cd /var/www/v-plantnet/terms/f; sudo -u www-data php ./maintenance/runJobs.php -v --procs 4 --conf ./LocalSettings.php
 cd /var/www/v-plantnet/uses/e;  sudo -u www-data php ./maintenance/runJobs.php -v --procs 4 --conf ./LocalSettings.php
 cd /var/www/v-plantnet/uses/f;  sudo -u www-data php ./maintenance/runJobs.php -v --procs 4 --conf ./LocalSettings.php
 cd /var/www/v-rosaceae/w;  sudo -u www-data php ./maintenance/runJobs.php -v --procs 4 --conf ./LocalSettings.php
 cd /var/www/v-species/f;   sudo -u www-data php ./maintenance/runJobs.php -v --procs 4 --conf ./LocalSettings.php
 cd /var/www/v-species/o;   sudo -u www-data php ./maintenance/runJobs.php -v --procs 4 --conf ./LocalSettings.php
 cd /var/www/v-species/s;   sudo -u www-data php ./maintenance/runJobs.php -v --procs 4 --conf ./LocalSettings.php
 cd /var/www/v-species/w;   sudo -u www-data php ./maintenance/runJobs.php -v --procs 4 --conf ./LocalSettings.php
 cd /var/www/v-workbench/w; sudo -u www-data php ./maintenance/runJobs.php -v --procs 4 --conf ./LocalSettings.php
 cd /var/www/v-zsm/w;       sudo -u www-data php ./maintenance/runJobs.php -v --procs 4 --conf ./LocalSettings.php
 
 # '''Rebuilding the file cache''' (watch permissions, simulate the www-data user as if written by apache):
 cd /var/www/metawiki;             sudo -u www-data php ./maintenance/rebuildFileCache.php 0 overwrite --conf ./LocalSettings.php
 cd /var/www/studienstiftungswiki; sudo -u www-data php ./maintenance/rebuildFileCache.php 0 overwrite --conf ./LocalSettings.php
 cd /var/www/testwiki;             sudo -u www-data php ./maintenance/rebuildFileCache.php 0 overwrite --conf ./LocalSettings.php
 cd /var/www/testwiki2;            sudo -u www-data php ./maintenance/rebuildFileCache.php 0 overwrite --conf ./LocalSettings.php
 cd /var/www/v-floramalesiana/w;   sudo -u www-data php ./maintenance/rebuildFileCache.php 0 overwrite --conf ./LocalSettings.php
 cd /var/www/v-fotoflorabb/w;      sudo -u www-data php ./maintenance/rebuildFileCache.php 0 overwrite --conf ./LocalSettings.php
 cd /var/www/v-gbif-d/w;           sudo -u www-data php ./maintenance/rebuildFileCache.php 0 overwrite --conf ./LocalSettings.php
 cd /var/www/v-hypergarden/w;      sudo -u www-data php ./maintenance/rebuildFileCache.php 0 overwrite --conf ./LocalSettings.php
 cd /var/www/v-ispi/w;   sudo -u www-data php ./maintenance/rebuildFileCache.php 0 overwrite --conf ./LocalSettings.php
 cd /var/www/v-k2n/w;    sudo -u www-data php ./maintenance/rebuildFileCache.php 0 overwrite --conf ./LocalSettings.php
 cd /var/www/v-k2n/h;    sudo -u www-data php ./maintenance/rebuildFileCache.php 0 overwrite --conf ./LocalSettings.php
 cd /var/www/v-k2n/it;   sudo -u www-data php ./maintenance/rebuildFileCache.php 0 overwrite --conf ./LocalSettings.php
 cd /var/www/v-k2n/sl;   sudo -u www-data php ./maintenance/rebuildFileCache.php 0 overwrite --conf ./LocalSettings.php
 cd /var/www/v-lias/w;   sudo -u www-data php ./maintenance/rebuildFileCache.php 0 overwrite --conf ./LocalSettings.php
 cd /var/www/v-nnvm/w;   sudo -u www-data php ./maintenance/rebuildFileCache.php 0 overwrite --conf ./LocalSettings.php
 cd /var/www/v-on/w;     sudo -u www-data php ./maintenance/rebuildFileCache.php 0 overwrite --conf ./LocalSettings.php
 cd /var/www/v-orowiki/w;sudo -u www-data php ./maintenance/rebuildFileCache.php 0 overwrite --conf ./LocalSettings.php
 cd /var/www/v-phytomed/w;       sudo -u www-data php ./maintenance/rebuildFileCache.php 0 overwrite --conf ./LocalSettings.php
 cd /var/www/v-phytopathology/w; sudo -u www-data php ./maintenance/rebuildFileCache.php 0 overwrite --conf ./LocalSettings.php
 cd /var/www/v-plantnet/terms/e; sudo -u www-data php ./maintenance/rebuildFileCache.php 0 overwrite --conf ./LocalSettings.php
 cd /var/www/v-plantnet/terms/f; sudo -u www-data php ./maintenance/rebuildFileCache.php 0 overwrite --conf ./LocalSettings.php
 cd /var/www/v-plantnet/uses/e;  sudo -u www-data php ./maintenance/rebuildFileCache.php 0 overwrite --conf ./LocalSettings.php
 cd /var/www/v-plantnet/uses/f;  sudo -u www-data php ./maintenance/rebuildFileCache.php 0 overwrite --conf ./LocalSettings.php
 cd /var/www/v-rosaceae/w;  sudo -u www-data php ./maintenance/rebuildFileCache.php 0 overwrite --conf ./LocalSettings.php
 cd /var/www/v-species/f;   sudo -u www-data php ./maintenance/rebuildFileCache.php 0 overwrite --conf ./LocalSettings.php
 cd /var/www/v-species/o;   sudo -u www-data php ./maintenance/rebuildFileCache.php 0 overwrite --conf ./LocalSettings.php
 cd /var/www/v-species/s;   sudo -u www-data php ./maintenance/rebuildFileCache.php 0 overwrite --conf ./LocalSettings.php
 cd /var/www/v-species/w;   sudo -u www-data php ./maintenance/rebuildFileCache.php 0 overwrite --conf ./LocalSettings.php
 cd /var/www/v-workbench/w; sudo -u www-data php ./maintenance/rebuildFileCache.php 0 overwrite --conf ./LocalSettings.php
 cd /var/www/v-zsm/w;       sudo -u www-data php ./maintenance/rebuildFileCache.php 0 overwrite --conf ./LocalSettings.php
 # refreshImageMetadata.php
 cd /var/www/metawiki;             sudo -u www-data php ./maintenance/refreshImageMetadata.php --force --conf ./LocalSettings.php
 cd /var/www/studienstiftungswiki; sudo -u www-data php ./maintenance/refreshImageMetadata.php --force --conf ./LocalSettings.php
 cd /var/www/testwiki;             sudo -u www-data php ./maintenance/refreshImageMetadata.php --force --conf ./LocalSettings.php
 cd /var/www/testwiki2;            sudo -u www-data php ./maintenance/refreshImageMetadata.php --force --conf ./LocalSettings.php
 cd /var/www/v-floramalesiana/w;   sudo -u www-data php ./maintenance/refreshImageMetadata.php --force --conf ./LocalSettings.php
 cd /var/www/v-fotoflorabb/w;      sudo -u www-data php ./maintenance/refreshImageMetadata.php --force --conf ./LocalSettings.php
 cd /var/www/v-gbif-d/w;           sudo -u www-data php ./maintenance/refreshImageMetadata.php --force --conf ./LocalSettings.php
 cd /var/www/v-hypergarden/w;      sudo -u www-data php ./maintenance/refreshImageMetadata.php --force --conf ./LocalSettings.php
 cd /var/www/v-ispi/w;   sudo -u www-data php ./maintenance/refreshImageMetadata.php --force --conf ./LocalSettings.php
 cd /var/www/v-k2n/w;    sudo -u www-data php ./maintenance/refreshImageMetadata.php --force --conf ./LocalSettings.php
 cd /var/www/v-k2n/h;    sudo -u www-data php ./maintenance/refreshImageMetadata.php --force --conf ./LocalSettings.php
 cd /var/www/v-k2n/it;   sudo -u www-data php ./maintenance/refreshImageMetadata.php --force --conf ./LocalSettings.php
 cd /var/www/v-k2n/sl;   sudo -u www-data php ./maintenance/refreshImageMetadata.php --force --conf ./LocalSettings.php
 cd /var/www/v-lias/w;   sudo -u www-data php ./maintenance/refreshImageMetadata.php --force --conf ./LocalSettings.php
 cd /var/www/v-nnvm/w;   sudo -u www-data php ./maintenance/refreshImageMetadata.php --force --conf ./LocalSettings.php
 cd /var/www/v-on/w;     sudo -u www-data php ./maintenance/refreshImageMetadata.php --force --conf ./LocalSettings.php
 cd /var/www/v-orowiki/w;sudo -u www-data php ./maintenance/refreshImageMetadata.php --force --conf ./LocalSettings.php
 cd /var/www/v-phytomed/w;       sudo -u www-data php ./maintenance/refreshImageMetadata.php --force --conf ./LocalSettings.php
 cd /var/www/v-phytopathology/w; sudo -u www-data php ./maintenance/refreshImageMetadata.php --force --conf ./LocalSettings.php
 cd /var/www/v-plantnet/terms/e; sudo -u www-data php ./maintenance/refreshImageMetadata.php --force --conf ./LocalSettings.php
 cd /var/www/v-plantnet/terms/f; sudo -u www-data php ./maintenance/refreshImageMetadata.php --force --conf ./LocalSettings.php
 cd /var/www/v-plantnet/uses/e;  sudo -u www-data php ./maintenance/refreshImageMetadata.php --force --conf ./LocalSettings.php
 cd /var/www/v-plantnet/uses/f;  sudo -u www-data php ./maintenance/refreshImageMetadata.php --force --conf ./LocalSettings.php
 cd /var/www/v-rosaceae/w;  sudo -u www-data php ./maintenance/refreshImageMetadata.php --force --conf ./LocalSettings.php
 cd /var/www/v-species/f;   sudo -u www-data php ./maintenance/refreshImageMetadata.php --force --conf ./LocalSettings.php
 cd /var/www/v-species/o;   sudo -u www-data php ./maintenance/refreshImageMetadata.php --force --conf ./LocalSettings.php
 cd /var/www/v-species/s;   sudo -u www-data php ./maintenance/refreshImageMetadata.php --force --conf ./LocalSettings.php
 cd /var/www/v-species/w;   sudo -u www-data php ./maintenance/refreshImageMetadata.php --force --conf ./LocalSettings.php
 cd /var/www/v-workbench/w; sudo -u www-data php ./maintenance/refreshImageMetadata.php --force --conf ./LocalSettings.php
 cd /var/www/v-zsm/w;       sudo -u www-data php ./maintenance/refreshImageMetadata.php --force --conf ./LocalSettings.php

Note: do not try to update /var/www/wiki-template-staging/ and /var/www/wiki-template/, it won't work, LocalSettings is missing.

For semantic mediawiki extension (SMW) click on "Initialise or upgrade tables" on SpecialPage Semantic Admin:

See also Maintenance scripts above for further scripts.

Switching targets of symbolic links

Old symbolic links can be deleted interactively by prompting on each remove and set anew as follows (example extension SemanticResultFormats):

 cd /usr/share/mediawiki/phase3/extensions
 sudo rm -i SemanticResultFormats;  sudo ln -s ../../extensions2/SemanticResultFormats
 sudo /usr/sbin/apache2ctl -k graceful && sudo service nginx restart && sudo service php5-fpm restart

To document those links, a small bash script create_sorted_extension_list.sh located in both extension folders creates a dated log file sorted by links (the script asks before start logging). It can be started by something like that:

  ./create_sorted_extension_list.sh
    Create sorted extension list (y/n)

Adding a new directory link

Occasionally new directories directly in /phase3/ are introduced by mediawiki.org with version upgrades. New symbolic links need to be created:

# root wiki from staging:
cd /var/www/testwiki/; sudo ln -s /usr/share/mediawikistaging/phase3/resources/ resources
cd /var/www/wiki-template-staging/; sudo ln -s /usr/share/mediawikistaging/phase3/resources/ resources
# root wikis from normal:
cd /var/www/metawiki/; sudo ln -s /usr/share/mediawiki/phase3/resources/ resources
cd /var/www/studienstiftungswiki/; sudo ln -s /usr/share/mediawiki/phase3/resources/ resources
cd /var/www/wiki-template/; sudo ln -s /usr/share/mediawiki/phase3/resources/ resources
# virtual servers
cd /var/www/v-ispi/w/; sudo ln -s /usr/share/mediawiki/phase3/resources/ resources
cd /var/www/v-k2n/w/;  sudo ln -s /usr/share/mediawiki/phase3/resources/ resources
cd /var/www/v-k2n/h/;  sudo ln -s /usr/share/mediawiki/phase3/resources/ resources
cd /var/www/v-k2n/it/; sudo ln -s /usr/share/mediawiki/phase3/resources/ resources
cd /var/www/v-k2n/sl/; sudo ln -s /usr/share/mediawiki/phase3/resources/ resources
cd /var/www/v-lias/w/; sudo ln -s /usr/share/mediawiki/phase3/resources/ resources
cd /var/www/v-on/w/;   sudo ln -s /usr/share/mediawiki/phase3/resources/ resources
cd /var/www/v-phytomed/w/;      sudo ln -s /usr/share/mediawiki/phase3/resources/ resources
cd /var/www/v-phytopathology/w; sudo ln -s /usr/share/mediawiki/phase3/resources/ resources
cd /var/www/v-plantnet/terms/e; sudo ln -s /usr/share/mediawiki/phase3/resources/ resources
cd /var/www/v-plantnet/terms/f; sudo ln -s /usr/share/mediawiki/phase3/resources/ resources
cd /var/www/v-plantnet/uses/e;  sudo ln -s /usr/share/mediawiki/phase3/resources/ resources
cd /var/www/v-plantnet/uses/f;  sudo ln -s /usr/share/mediawiki/phase3/resources/ resources
cd /var/www/v-rosaceae/w; sudo ln -s /usr/share/mediawiki/phase3/resources/ resources
cd /var/www/v-species/f/; sudo ln -s /usr/share/mediawiki/phase3/resources/ resources
cd /var/www/v-species/o/; sudo ln -s /usr/share/mediawiki/phase3/resources/ resources
cd /var/www/v-species/s/; sudo ln -s /usr/share/mediawiki/phase3/resources/ resources
cd /var/www/v-species/w/; sudo ln -s /usr/share/mediawiki/phase3/resources/ resources
cd /var/www/v-workbench/w/; sudo ln -s /usr/share/mediawiki/phase3/resources/ resources

(To remove links one may use the interactive option rm -i, but one must confirm each single file.)

Moving Cache and Media folders

Only limited space exists on the root partition itself. Larger Wiki have their Media and Cache folders relocated to the the /mnt/storage and /mnt/dump partitions, respectively. The cache of the following Wikis is presently in /mnt/dump/var/www/: testwiki-cache; v-ispi-w-cache; v-k2n-w-cache; v-lias-w-cache; v-nnvm-w-cache; v-on-w-cache; v-orowiki-w-cache; v-species-o-cache; v-species-w-cache. Move like:

# create folders on dump, then change owner and group:
cd /mnt/dump/var/www; sudo mkdir v-lias-w-cache; sudo chown www-data *; sudo chgrp www-data *;
# remove current cache, create link to new folder:
cd /var/www/v-lias/w;     sudo rm cache -r; sudo ln -s /mnt/dump/var/www/v-lias-w-cache  cache

For linking media folder (to Storage, not Dump!):

cd /mnt/storage/; sudo mkdir var-www-v-species-w-media;
sudo chown www-data var-www-v-species-w-media; sudo chgrp www-data var-www-v-species-w-media;
sudo cp /var/www/v-species/w/media /mnt/storage/var-www-v-species-w-media
## check and manually delete old media folder! Then create link:
cd /var/www/v-species/w; sudo ln -s /mnt/storage/var-www-v-species-w-media media

General topics

MediaWiki configuration tips

Configuring MediaWiki:Sidebar: New options July 2008, see [2]

It may be useful to copy the contents of http://en.wikipedia.org/wiki/MediaWiki:Common.css and http://en.wikipedia.org/wiki/MediaWiki:Monobook.css into the Mediawiki:Common.css and MediaWiki:Monobook.css pages of each Wiki where you may want to use copied things e.g. templates from wikipedia.

File uploading and Mime types

Uploading files with extensions and Content types (MIME types) usually not allowed on wikis: The default of normally allowed extensions is listed $wgFileExtensions in /includes/DefaultSettings.php (which should never be edited). To allow additional extensions override $wgFileExtensions in LocalSettings.php.

There are also $wgMimeTypeBlacklist and $wgFileBlacklist for file extensions and MIME types that are never allowed for security reasons, even if added to $wgFileExtensions. If needed for special purposes and considered safe, they can be commented out in these blacklists in LocalSettings.php.

If there is still the error message "The file is corrupt or has an incorrect extension. Please check the file and upload again", the reason can be that the file you want to upload has a MIME type for which only certain file extensions are allowed in /includes/mime.types and has an extension not listed there. If considered safe, the extension can be added in that file. The mime type can be determined with the command file -i filename. Note: Files that are changed manually are then excluded from svn update. To drop own changes and get back to the svn version delete the file and then run svn update (see above).

Logging/Debugging

For debugging, you may enable logging of debug and error messages:

# create a log-file somewhere
  sudo touch /var/www/v-k2n/w/log.txt
  sudo chmod 777 /var/www/v-k2n/w/log.txt

# edit LocalSettings.php and add the following line after the line including the DefaultSettings.php:
  $wgDebugLogFile = "/var/www/v-k2n/w/log.txt";

Note, that the logfile will grow fast, so one should switch on the logging only for short periods of time, and delete the logfile after done.

PHP debugging can be doen with xdebug. Details see Mediawiki debugging.

Backup/Restore in general

Backing up the database alone is documented here: http://www.mediawiki.org/wiki/Manual:Backing_up_a_wiki. A more complete script can be found here: http://www.mediawiki.org/wiki/User:Flominator/Backup_MW - this also backups the folder with uploads (media). Based on that source, adapted scripts were created, execute as:

 cd /var/backups/scripts/; ./backup-mw.sh

The additional backup-mw.php script can potentially be used within www to directly create and download a copy of the mediawiki. We are considering this as a measure to build trust.

In addition, to create an xml dump, make sure a link to AdminSettings.php exists in the current wiki folder and the maintenance folder is temporarily copied, then run, e.g.:

 cd /var/www/v-on/w;   sudo -u www-data php ./maintenance/dumpBackup.php  --full --conf ./LocalSettings.php > /var/www/v-orowiki/w/media/full.xml

To restore such a xml-dump, remove tables and create new tables with default values in mysql, then:

 cd /var/www/WIKINAME;  # (replace path with actual wiki path!)
 php ./maintenance/importDump.php /var/backups/wikis/k2n-full.xml --conf ./LocalSettings.php
 php ./maintenance/rebuildrecentchanges.php --conf ./LocalSettings.php    # absolutely necessary
 ## If backlinks etc. do not work, or extensions had to be disabled, use (Very slow!!!!):
 php ./maintenance/rebuildall.php  --quick --conf ./LocalSettings.php

Notes: In our experience, some extensions may create problems during reimport. We found GoogleMaps and DynamicPageList to cause importDump to abort. After disabling these two extensions, the import worked correctly. However, all DPL pages are then still cached without proper DPL rendering even after re-enabling both extensions. Running rebuildall.php fixed this.

Example restore for testing of the wikimediafoundation commons xml-dump:

 cd /var/www/wmf-commons; php ./maintenance/importDump.php  /mnt/wpTest/commonswiki-pages-articles.xml  --conf ./LocalSettings.php

Refresh SemanticMW data

If it becomes necessary to delete all SMW data in a particular wiki and regenerate the data again from the individual pages the following command lines can be used:

 cd /var/www/WIKINAME;  # (replace path with actual wiki path!)
 sudo -u www-data php ./maintenance/runJobs.php -v --procs 4 --conf ./LocalSettings.php
 sudo -u www-data php ./extensions/SemanticMediaWiki/maintenance/SMW_refreshData.php -ftpv --conf ./LocalSettings.php
 sudo -u www-data php ./extensions/SemanticMediaWiki/maintenance/SMW_refreshData.php -v --conf ./LocalSettings.php
 sudo -u www-data php ./maintenance/runJobs.php -v --procs 4 --conf ./LocalSettings.php

Notes:

change to the local directory of the wiki to be refreshed,
make sure all outstanding jobs are completed,
delete all SMW data,
regenerate the data from the individual pages.

The last step can take some time (e.g. about 10 minutes to refresh about 4000 IDs). The wiki seems to work normally during that time but queries may be effected. For that reason it is recommended to implement this operation during a maintenance window.

During step 4 the procedure slows down after about 5 min. and can be sped up by:

interrupting the procedure by using Crtl-C
restarting the last command by specifying the next ID to be processed, e.g.

php ./extensions/SemanticMediaWiki/maintenance/SMW_refreshData.php -v -s 2062 --conf ./LocalSettings.php

if 2061 IDs had been already processed at the time of the interruption.

Difference between revisions of "Mediawiki maintenance"

Revision as of 13:48, 11 January 2013

Contents

Mediawiki maintenance scripts

MediaWiki version upgrading

Update, runJobs, rebuildFileCache, refreshMetadata scripts

Switching targets of symbolic links

Adding a new directory link

Moving Cache and Media folders

General topics

MediaWiki configuration tips

File uploading and Mime types

Logging/Debugging

Backup/Restore in general

Refresh SemanticMW data

Navigation menu

Personal tools

Namespaces

Variants

Views

More

Search

Navigation

Tools