Difference between revisions of "Mediawiki installation/REL1 26"

Revision as of 08:37, 14 November 2019

Install and Update Steps

Updating to a wmf development clone calls often for lot of test and brings surprises when used. It is not recommended. Stable releases are preferred.

Steps in general:

1. update git clone
2. get information on available git_branch_or_tags of extensions (see Bash script ./write_wmf-shared_git-branches-tags2home_ext-tags-and-branches.sh)
3. create shared wiki resource (see Bash script ./create_or_upgrade_shared_wiki-core-resource-structure.sh)
4. update extensions (see Bash script ./create_or_upgrade_shared-wiki-extensions.sh)
   update only skins later on can be done with ./create_or_upgrade_shared-wiki-skins.sh)
5. check old LocalSettings.php for deprecated stuff, check and substitute deprecated variables and set up a new shared settings file, e.g /var/www/MediaWikiCommonSettings_REL1_26.php. Check:
   • https://www.mediawiki.org/wiki/Category:WMF_Releases
   • https://www.mediawiki.org/wiki/Category:MediaWiki_Deprecated_or_obsolete_features
   • e.g. https://www.mediawiki.org/wiki/MediaWiki_1.27/wmf.9
6. use $wgSiteNotice and $wgReadOnly to inform users about update (note that MediaWiki:Sitenotice must contain a "-" to be treated as “unset” at least in newer Wikis)
7. backup database of your v-host-wiki
8. update structure of v-host wiki itself (see ./create_or_upgrade_vhost-wiki.sh)
9. run perhaps composer on the v-host wiki level (if needed at all, but effects all shared wikis!!) and finally maintenance/update.php:
   • set the set up case for the wiki (boolean global settings variable $bwgSetUpAsRichWiki in /var/www/MediaWikiCommonSettings_REL1_26.php)
   • deactivate $wgReadOnly before running update.php
   • cd /path/to/v-hostwiki && sudo -u www-data php ./maintenance/update.php --dbuser wikiadmin --quick --conf ./LocalSettings.php
10. perhaps check cron jobs triggered from biowikifarm extension:CommonsMediaCaching (see below commons media caching or Caching Wikimedia Commons file to local repository)

Update Notes

• a productive v-host Wiki should be updated only if tests on Test Wiki were successful
• maintenance/update.php does not touch shared tables (user etc.) unless you force update.php to do so. It was possible during test phase to share tables also with older wikis (→ sec. Login Settings).
• you may get a blank page after upgrading only the directory structure. One cause can be the old cache, usually maintanance/update.php does this job to clear the cache
• for $wgSiteNotice to work after upgrade page MediaWiki:Sitenotice must contain a minus (“-”) as content to be regarded as empty, otherwise $wgSiteNotice in LocalSettings.php will not be displayed and has no effect.
• to get local upload working MediaWiki:Licenses must contain a list of licenses (e.g. licenses on testwiki2) otherwise local upload is reported to be deactivated.
• add repository names on MediaWiki:Shared-repo-name-openmedia an MediaWiki:Shared-repo-name-specialmedia where the italic part is the value for array key "name" in $wgForeignFileRepos[] = array("name" => "openmedia" /*...*/);
• there seems a BUG in search result layout the sections get all mixed floated for Vector. Add the following to MediaWiki:Vector.css:

/* bugfix MW 1.26.2 */
div.searchresults h2 { clear:both; }

• in some wikis Error creating thumbnail: Unable to save thumbnail to destination occurred:
  1. one solution was to switch on debug mode and resolve from there, it turned out that non-default temporary folder media/temp was missing
  2. a separate wiki with no commons configuration did not scale PNG, solution was to set $wgCustomConvertCommandsByExtension['png'] = "/usr/bin/pngscale %s %d %w %h"; and increase $wgMaxImageArea

Directory Structure

For the different needed features the set up concept is as follows:

/usr/share/mw-wmf-clone/core          (mediawiki)
/usr/share/mw-wmf-clone/extensions    (extensions)
/usr/share/mw-wmf-clone/skins         (skins)

└─ /usr/share/mediawiki26                      v-host-wiki1
   ├── extensions-rich-features/            -> ├── extensions/        
   ├── vendor-rich-features/                -> ├── vendor/            
   ├── composer-rich-features.json          -> ├── composer.json      
   ├── composer-rich-features.local.json    -> ├── composer.local.json (manage composer extensions here!!)
   └── composer-rich-features.lock          -> └── composer.lock      
   
   /usr/share/mediawiki26                      v-host-wiki2
   ├── extensions-simple-features/          -> ├── extensions/        
   ├── vendor-simple-features/              -> ├── vendor/            
   ├── composer-simple-features.json        -> ├── composer.json      
   ├── composer-simple-features.local.json  -> ├── composer.local.json (manage composer extensions here!!)
   └── composer-simple-features.lock        -> └── composer.lock      

Normal folders /extensions or those used by composer (/vendor, composer.json, composer.local.json, composer.lock files) are not used at the moment.

To see what wiki folder links back to a shared wiki you can search for symbolic links (-L without error messages 2>/dev/null) that have the same file or directory as given:

find -L /var/www/ -maxdepth 3 -samefile /usr/share/mediawiki26/index.php -exec dirname '{}' ';' 2>/dev/null \
 | sort | awk 'BEGIN {print "#-------------------------------";print "# Wikis using the same resource:"}{print "#   " NR " -> " $0}'
# extensions setups:
find -L /var/www/ -maxdepth 3 -samefile /usr/share/mediawiki26/extensions-rich-features/ -exec dirname '{}' ';' 2>/dev/null \
 | sort | awk 'BEGIN {print "#-------------------------------";print "# Wikis using the same extensions:"}{print "#   " NR " -> " $0}'
find -L /var/www/ -maxdepth 3 -samefile /usr/share/mediawiki26/extensions-simple-features/ -exec dirname '{}' ';' 2>/dev/null \
 | sort | awk 'BEGIN {print "#-------------------------------";print "# Wikis using the same extensions:"}{print "#   " NR " -> " $0}'

Composer

For composer managed extensions consult Composer (PHP) or Composer (MediaWiki):

• manage needed extensions in the *.local.json not in composer.json
• WARNING: on a shared setup wiki family the setup will not work when there is no actual file vendor/autoload.php, because it checks for a hard coded file (php file git view) so although normal folders /extensions, /vendor are not used now, you have to run once:
  cd /usr/share/mediawiki26 && sudo -u www-data php /usr/local/bin/composer.phar update --no-dev
• on the level of v-host-wiki you can update and manage composer extensions which will update and write back into to the folders of /extensions-simple-features, /vendor-simple-features etc., so updating affects all symlinked wikis then, and depending on the extension (if one requires update.php because it changes the DB) a maintenance update may be needed on all linked v-host wikis
• untested: skin may be managed also by composer

Settings: LocalSettings.php

Needed for symlink setup:

/**
 * If you customize your file layout, set $IP to the directory that contains 
 * the actual MediaWiki files. It will be used as a base to locate files.
 */
putenv ("MW_INSTALL_PATH=/var/www/testwiki2" );
if(getenv( 'MW_INSTALL_PATH' ) ) { 
  $IP = getenv( 'MW_INSTALL_PATH' );
} else {// __FILE__ resolves symlinks but getcwd() does not
  // $GLOBALS['IP'] = dirname( __FILE__ );
  $IP = getcwd( );
}

/**
 * extension directory 
 *
 * needed for maintenance/update.php to work on symlink-setup
 * @since MW 1.25
 */
$wgExtensionDirectory="/var/www/testwiki2/extensions";

Login Settings

Each time a user logs in the password gets written into the data base each time. But along subsequent MediaWiki versions this method was changed, so take care when you run old and new Wikis together:

$wgPasswordDefault ='B'; /* old MD5 hashing on shared Wikifamily with pre MW1.24 and post MW 1.24+ Wikis */

If all wikis are at the newest stage there should be a maintenance script helping to convert all user passwords to the new encrypting method $wgPasswordDefault ='pbkdf2'; at once. Maybe also maintenance/update.php --doshared is appropriate then to update shared tables as well.

You can execute also maintenance/wrapOldPasswords.php to convert the hashing algorithm of all passwords without waiting users to log in.

Extensions

Discussed extensions below: AnnotationGenerator, Charinsert, CiteThisPage, CommonsMediaCaching, ConfirmAccount, Data Transfer, EmailToWiki, Flow, GoogleAPIClient, GoogleLogin, GraphViz, LiquidThreads, Mp3Handler, MultimediaViewer, NiceCategoryList2, PdfHandler, Semantic Forms, Semantic Mediawiki, SyntaxHighlight GeSHi, TimedMediaHandler, Translate, VisualEditor, Widgets, WikimediaMessages

Unexpected problems with extensions during test setup:

• CollapsibleVector (collapsible sidebar not part of skin:Vector any more)
• Cldr → cldr
• SyntaxHighlight GeSHi (extra setup step)
• VisualEditor (did not work at all because of lacking parsoid upgrade, see below)

# copy from old wiki
sudo cp --preserve --recursive /var/www/v-filteredpush/w/extensions/AnnotationGenerator \
  /usr/share/mediawiki26/extensions-rich-features/

# in LocalSettings
require_once( "$IP/extensions/AnnotationGenerator/AnnotationGenerator.php" );

Fatal error: Class 'Symfony\Component\Process\ProcessBuilder' not found in /usr/share/mediawiki27wmf/extensions-simple-features/SyntaxHighlight_GeSHi/SyntaxHighlight_GeSHi.class.php on line 284 

cd /usr/share/mediawiki26/extensions-simple-features/SyntaxHighlight_GeSHi/
sudo -u www-data php /usr/local/bin/composer.phar update --no-dev
cd /usr/share/mediawiki26/extensions-rich-features/SyntaxHighlight_GeSHi/
sudo -u www-data php /usr/local/bin/composer.phar update --no-dev

wfLoadExtension( 'SyntaxHighlight_GeSHi' );

GraphViz::getUploadSubdir: mkdir(/var/www/testwiki/media/graphviz/images/, 16893) failed
- resolved in dev-master

Call to undefined method ForeignDBRepo::fetchImageQuery()

cd /usr/share/mediawiki26/extensions-simple-features
# cd /usr/share/mediawiki26/extensions-rich-features
if [[ -d VisualEditor ]];then sudo mv VisualEditor VisualEditor_backup`date +%Y%m%d%H%M`; fi
sudo git clone --depth 1 --branch REL1_26 https://gerrit.wikimedia.org/r/p/mediawiki/extensions/VisualEditor.git
cd VisualEditor && sudo git submodule update --init --recursive
cd lib/ve && sudo git checkout REL1_26

/usr/lib/parsoid/src/api/localsettings.js
Error loading data from server: (curl error: 7): docserver-http-error: (curl error: 7)

requestRestbase()
  'url' => '/parsoid/local/v1/'
 since 1.26wmf21 changed to
  'url' => '/restbase/local/v1/'

# cd /usr/share/mediawiki26/extensions-simple-features
cd /usr/share/mediawiki26/extensions-rich-features
if [[ -d Widgets ]]; then sudo mv Widgets Widgets_backup`date +%Y%m%d%H%M`; fi
sudo git clone --depth 1 --branch REL1_26 https://gerrit.wikimedia.org/r/p/mediawiki/extensions/Widgets.git
cd Widgets && sudo git submodule init && sudo git submodule update

# cd /usr/share/mediawiki26/extensions-simple-features
cd /usr/share/mediawiki26/extensions-rich-features
if [[ -d GoogleLogin ]];then sudo mv GoogleLogin GoogleLogin_backup`date +%Y%m%d%H%M`; fi
sudo git clone --depth 1 --branch REL1_26 https://gerrit.wikimedia.org/r/p/mediawiki/extensions/GoogleLogin.git

# cd /usr/share/mediawiki26/extensions-simple-features
cd /usr/share/mediawiki26/extensions-rich-features
if [[ -d GoogleAPIClient ]];then sudo mv GoogleAPIClient GoogleAPIClient_backup`date +%Y%m%d%H%M`; fi
sudo git clone --depth 1 --branch REL1_26 https://gerrit.wikimedia.org/r/p/mediawiki/extensions/GoogleAPIClient.git

# cd /usr/share/mediawiki26/extensions-simple-features
cd /usr/share/mediawiki26/extensions-rich-features
if [[ -d NiceCategoryList2 ]];then sudo mv NiceCategoryList2 NiceCategoryList2_backup`date +%Y%m%d%H%M`; fi
  sudo mkdir NiceCategoryList2 && sudo chown www-data:www-data NiceCategoryList2 && cd NiceCategoryList2;
  wget "https://www.mediawiki.org/w/index.php?title=Extension:NiceCategoryList2/3.0&action=raw" --output-document="NiceCategoryList.php"
  sed --in-place '/<\/\?source/d; /<\/\?syntaxhighlight/d;' NiceCategoryList.php # delete lines containing </?source tag

// in LocalSettings.php
require_once ("$IP/extensions/NiceCategoryList2/NiceCategoryList.php");

cd /usr/share/mediawiki26/extensions-simple-features
# cd /usr/share/mediawiki26/extensions-rich-features
if [[ -d CiteThisPage ]];then sudo mv CiteThisPage CiteThisPage_backup`date +%Y%m%d%H%M`; fi
sudo git clone --depth 1 --branch REL1_26 https://gerrit.wikimedia.org/r/p/mediawiki/extensions/CiteThisPage

sudo apt-get install poppler-utils

Skin:Vector

Not sure if composer.local.json can also manage skins

…
"require":{
  "mediawiki/vector-skin":"dev-wmf/1.27.0-wmf.9"
}
…
 

Collpaseable sidbar is not part of skin:Vector but now in extension:CollapsibleVector

 cd /usr/share/mediawiki26/extensions-simple-features
 sudo git clone https://gerrit.wikimedia.org/r/p/mediawiki/extensions/CollapsibleVector.git
 cd /usr/share/mediawiki26/extensions-simple-features/CollapsibleVector
 sudo git checkout origin/REL1_26
 
 cd /usr/share/mediawiki26/extensions-rich-features
 sudo git clone https://gerrit.wikimedia.org/r/p/mediawiki/extensions/CollapsibleVector.git
 cd /usr/share/mediawiki26/extensions-simple-features/CollapsibleVector
 sudo git checkout origin/REL1_26

Skin:naturkunde

cd /usr/share/mediawiki26/skins
sudo git clone https://github.com/MfN-Berlin/mwSkinNaturkunde.git naturkunde
cd naturkunde/
sudo git submodule update --init

# load it in LocalSettigns.php
if (is_dir("$IP/skins/naturkunde")) {
    require_once("$IP/skins/naturkunde/Naturkunde.php");
    # wfLoadSkin('naturkunde'); // wfLoadSkin not yet applicable would need a skins/naturkunde/skin.json
}

CommonsMediaCaching

Biowikifarm extension CommonsMediaCaching is stored in svn, some files need to be at the right place:

• at wiki-folder/maintenance and
• in the cron job system (/etc/cron.d/)

Check that maintenance scripts are in place:

cd /usr/share/mediawiki26/extensions-rich-features/CommonsMediaCaching
# do not overwrite blindly (--interactive)
# file should be owned by www-data (option --preserve)
sudo cp --interactive --preserve downImagesCommons.php /usr/share/mediawiki26/maintenance/
sudo cp --interactive --preserve importImagesCommons.inc /usr/share/mediawiki26/maintenance/
sudo cp --interactive --preserve importImagesCommons.php /usr/share/mediawiki26/maintenance/

Resolve perhaps issues:

• check path settings in cron scripts /etc/cron.d/
• check url of commons wiki (2015 changed from http to only working with https…)
• …

See also Caching Wikimedia Commons file to local repository

Switching a Wiki Setup: Simple to Rich Features

(1) Change settings:

• modify /var/www/MediaWikiCommonSettings_REL1_26.php
  • check the true case for $bwgSetUpAsRichWiki
  • modify perhaps in the semantics section enableSemantics('server-url.org');
  • modify for extension: GlobalUsage at the appropriate section

(2) change link setup accordingly:

• all symbolic links should point afterwards to “…-rich-features-…” something:

cd /var/www/path/to/wiki 
 # do NOT REMOVE FOLDERS, just the setup of symbolic links!!!
 sudo rm --interactive vendor # the prompt should be like: « rm: remove symbolic link ‘vendor’? »
 sudo ln --symbolic /usr/share/mediawiki26/vendor-rich-features              vendor
 sudo rm --interactive extensions # the prompt should be like: « rm: remove symbolic link ‘extensions’? »
 sudo ln --symbolic /usr/share/mediawiki26/extensions-rich-features          extensions
# correct composer. NOTE: symbolic file links can be overridden forcefully, but not folders
 sudo ln --force -s /usr/share/mediawiki26/composer-rich-features.lock       composer.lock
 sudo ln --force -s /usr/share/mediawiki26/composer-rich-features.local.json composer.local.json
 sudo ln --force -s /usr/share/mediawiki26/composer-rich-features.json       composer.json
# check if there are still links to simple feature setups
 ls -l | grep simple

(3) run maintenance/update.php

sudo -u www-data php ./maintenance/update.php --dbuser wikiadmin --quick --conf ./LocalSettings.php

Normally the wiki should be up again.