Mediawiki installation/REL1 26

From Biowikifarm Metawiki
< Mediawiki installation
Revision as of 08:37, 14 November 2019 by Andreas Plank (Talk | contribs) (→‎Extensions: +Extension:CommonsMediaCaching)

Jump to: navigation, search

Install and Update Steps

Updating to a wmf development clone calls often for lot of test and brings surprises when used. It is not recommended. Stable releases are preferred.

Steps in general:

  1. update git clone
  2. get information on available git_branch_or_tags of extensions (see Bash script ./write_wmf-shared_git-branches-tags2home_ext-tags-and-branches.sh)
  3. create shared wiki resource (see Bash script ./create_or_upgrade_shared_wiki-core-resource-structure.sh)
  4. update extensions (see Bash script ./create_or_upgrade_shared-wiki-extensions.sh)
    update only skins later on can be done with ./create_or_upgrade_shared-wiki-skins.sh)
  5. check old LocalSettings.php for deprecated stuff, check and substitute deprecated variables and set up a new shared settings file, e.g /var/www/MediaWikiCommonSettings_REL1_26.php. Check:
  6. use $wgSiteNotice and $wgReadOnly to inform users about update (note that MediaWiki:Sitenotice must contain a "-" to be treated as “unset” at least in newer Wikis)
  7. backup database of your v-host-wiki
  8. update structure of v-host wiki itself (see ./create_or_upgrade_vhost-wiki.sh)
  9. run perhaps composer on the v-host wiki level (if needed at all, but effects all shared wikis!!) and finally maintenance/update.php:
    • set the set up case for the wiki (boolean global settings variable $bwgSetUpAsRichWiki in /var/www/MediaWikiCommonSettings_REL1_26.php)
    • deactivate $wgReadOnly before running update.php
    • cd /path/to/v-hostwiki && sudo -u www-data php ./maintenance/update.php --dbuser wikiadmin --quick --conf ./LocalSettings.php
  10. perhaps check cron jobs triggered from biowikifarm extension:CommonsMediaCaching (see below commons media caching or Caching Wikimedia Commons file to local repository)

Update Notes

  • a productive v-host Wiki should be updated only if tests on Test Wiki were successful
  • maintenance/update.php does not touch shared tables (user etc.) unless you force update.php to do so. It was possible during test phase to share tables also with older wikis (→ sec. Login Settings).
  • you may get a blank page after upgrading only the directory structure. One cause can be the old cache, usually maintanance/update.php does this job to clear the cache
  • for $wgSiteNotice to work after upgrade page MediaWiki:Sitenotice must contain a minus (“-”) as content to be regarded as empty, otherwise $wgSiteNotice in LocalSettings.php will not be displayed and has no effect.
  • to get local upload working MediaWiki:Licenses must contain a list of licenses (e.g. licenses on testwiki2) otherwise local upload is reported to be deactivated.
  • add repository names on MediaWiki:Shared-repo-name-openmedia an MediaWiki:Shared-repo-name-specialmedia where the italic part is the value for array key "name" in $wgForeignFileRepos[] = array("name" => "openmedia" /*...*/);
  • there seems a BUG in search result layout the sections get all mixed floated for Vector. Add the following to MediaWiki:Vector.css:
/* bugfix MW 1.26.2 */
div.searchresults h2 { clear:both; }
  • in some wikis Error creating thumbnail: Unable to save thumbnail to destination occurred:
    1. one solution was to switch on debug mode and resolve from there, it turned out that non-default temporary folder media/temp was missing
    2. a separate wiki with no commons configuration did not scale PNG, solution was to set $wgCustomConvertCommandsByExtension['png'] = "/usr/bin/pngscale %s %d %w %h"; and increase $wgMaxImageArea

Directory Structure

For the different needed features the set up concept is as follows: 

/usr/share/mw-wmf-clone/core          (mediawiki)
/usr/share/mw-wmf-clone/extensions    (extensions)
/usr/share/mw-wmf-clone/skins         (skins)

└─ /usr/share/mediawiki26                      v-host-wiki1
   ├── extensions-rich-features/            -> ├── extensions/        
   ├── vendor-rich-features/                -> ├── vendor/            
   ├── composer-rich-features.json          -> ├── composer.json      
   ├── composer-rich-features.local.json    -> ├── composer.local.json (manage composer extensions here!!)
   └── composer-rich-features.lock          -> └── composer.lock      
   
   /usr/share/mediawiki26                      v-host-wiki2
   ├── extensions-simple-features/          -> ├── extensions/        
   ├── vendor-simple-features/              -> ├── vendor/            
   ├── composer-simple-features.json        -> ├── composer.json      
   ├── composer-simple-features.local.json  -> ├── composer.local.json (manage composer extensions here!!)
   └── composer-simple-features.lock        -> └── composer.lock

Normal folders /extensions or those used by composer (/vendor, composer.json, composer.local.json, composer.lock files) are not used at the moment.

To see what wiki folder links back to a shared wiki you can search for symbolic links (-L without error messages 2>/dev/null) that have the same file or directory as given: 

find -L /var/www/ -maxdepth 3 -samefile /usr/share/mediawiki26/index.php -exec dirname '{}' ';' 2>/dev/null \
 | sort | awk 'BEGIN {print "#-------------------------------";print "# Wikis using the same resource:"}{print "#   " NR " -> " $0}'
# extensions setups:
find -L /var/www/ -maxdepth 3 -samefile /usr/share/mediawiki26/extensions-rich-features/ -exec dirname '{}' ';' 2>/dev/null \
 | sort | awk 'BEGIN {print "#-------------------------------";print "# Wikis using the same extensions:"}{print "#   " NR " -> " $0}'
find -L /var/www/ -maxdepth 3 -samefile /usr/share/mediawiki26/extensions-simple-features/ -exec dirname '{}' ';' 2>/dev/null \
 | sort | awk 'BEGIN {print "#-------------------------------";print "# Wikis using the same extensions:"}{print "#   " NR " -> " $0}'

Composer

For composer managed extensions consult Composer (PHP) or Composer (MediaWiki):

  • manage needed extensions in the *.local.json not in composer.json
  • WARNING: on a shared setup wiki family the setup will not work when there is no actual file vendor/autoload.php, because it checks for a hard coded file (php file git view) so although normal folders /extensions, /vendor are not used now, you have to run once:
    cd /usr/share/mediawiki26 && sudo -u www-data php /usr/local/bin/composer.phar update --no-dev
  • on the level of v-host-wiki you can update and manage composer extensions which will update and write back into to the folders of /extensions-simple-features, /vendor-simple-features etc., so updating affects all symlinked wikis then, and depending on the extension (if one requires update.php because it changes the DB) a maintenance update may be needed on all linked v-host wikis
  • untested: skin may be managed also by composer

Settings: LocalSettings.php

Needed for symlink setup: 

/**
 * If you customize your file layout, set $IP to the directory that contains 
 * the actual MediaWiki files. It will be used as a base to locate files.
 */
putenv ("MW_INSTALL_PATH=/var/www/testwiki2" );
if(getenv( 'MW_INSTALL_PATH' ) ) { 
  $IP = getenv( 'MW_INSTALL_PATH' );
} else {// __FILE__ resolves symlinks but getcwd() does not
  // $GLOBALS['IP'] = dirname( __FILE__ );
  $IP = getcwd( );
}

/**
 * extension directory 
 *
 * needed for maintenance/update.php to work on symlink-setup
 * @since MW 1.25
 */
$wgExtensionDirectory="/var/www/testwiki2/extensions";

Login Settings

Each time a user logs in the password gets written into the data base each time. But along subsequent MediaWiki versions this method was changed, so take care when you run old and new Wikis together: 

$wgPasswordDefault ='B'; /* old MD5 hashing on shared Wikifamily with pre MW1.24 and post MW 1.24+ Wikis */

If all wikis are at the newest stage there should be a maintenance script helping to convert all user passwords to the new encrypting method $wgPasswordDefault ='pbkdf2'; at once. Maybe also maintenance/update.php --doshared is appropriate then to update shared tables as well.

You can execute also maintenance/wrapOldPasswords.php to convert the hashing algorithm of all passwords without waiting users to log in.

Extensions

Discussed extensions below: AnnotationGenerator, Charinsert, CiteThisPage, CommonsMediaCaching, ConfirmAccount, Data Transfer, EmailToWiki, Flow, GoogleAPIClient, GoogleLogin, GraphViz, LiquidThreads, Mp3Handler, MultimediaViewer, NiceCategoryList2, PdfHandler, Semantic Forms, Semantic Mediawiki, SyntaxHighlight GeSHi, TimedMediaHandler, Translate, VisualEditor, Widgets, WikimediaMessages

Unexpected problems with extensions during test setup:

  • CollapsibleVector (collapsible sidebar not part of skin:Vector any more)
  • Cldr → cldr
  • SyntaxHighlight GeSHi (extra setup step)
  • VisualEditor (did not work at all because of lacking parsoid upgrade, see below)
Extension Composer
managed? 		Developer Notes
Extension:AnnotationGenerator no
  • manually created by Joel Sachs, no repository found yet
# copy from old wiki
sudo cp --preserve --recursive /var/www/v-filteredpush/w/extensions/AnnotationGenerator \
  /usr/share/mediawiki26/extensions-rich-features/

# in LocalSettings
require_once( "$IP/extensions/AnnotationGenerator/AnnotationGenerator.php" );
  • special page Special:AnnotationGenerator shows up, but usage is unclear or how it works
SyntaxHighlight_GeSHi yes http://biowikifarm.net/test/MediaWiki:Common.js came: 
Fatal error: Class 'Symfony\Component\Process\ProcessBuilder' not found in /usr/share/mediawiki27wmf/extensions-simple-features/SyntaxHighlight_GeSHi/SyntaxHighlight_GeSHi.class.php on line 284

Solution: needs a composer update in its extension folder (read SyntaxHighlight_GeSHi (documentation)) 

cd /usr/share/mediawiki26/extensions-simple-features/SyntaxHighlight_GeSHi/
sudo -u www-data php /usr/local/bin/composer.phar update --no-dev
cd /usr/share/mediawiki26/extensions-rich-features/SyntaxHighlight_GeSHi/
sudo -u www-data php /usr/local/bin/composer.phar update --no-dev

Then in LocalSettings 

wfLoadExtension( 'SyntaxHighlight_GeSHi' );
Flow no Preferred over extension:LiquidThreads (unmaintained)
  • Problem: $wgNamespaceContentModels[NS_USER_TALK] = CONTENT_MODEL_FLOW_BOARD; disables welcome message by Extension:ConfirmAccount but adds a flow alert: “Welcome to TestWiki2, Testwiki2 User! We're glad you're here.”
  • Problem: “view history” on fresh created user talk page (by Extension:ConfirmAccount) shows not the expected history but only the page title
  • Problem: dependent on VisualEditor, setting can be limited then to:
    $wgFlowEditorList=array('wikitext');
  • Conversion can be done by maintenance scripts and if everything goes fine all threads are converted, the following templates are added as well:
    • template:Archive for converted LQT page (on the archived subpage)
    • template:LQT page converted to Flow (on the board description of the flow page)
LiquidThreads no officially unmaintained, use extension:Flow
  • extension:Flow is supposed to have a maintenance script to convert LiquidThreads
GraphViz yes 
GraphViz::getUploadSubdir: mkdir(/var/www/testwiki/media/graphviz/images/, 16893) failed
- resolved in dev-master
Mp3Handler no External extension was merged into biowikifarm subversion and adjusted to some needs
EmailToWiki no External extension (TODO), see ./get_EmailToWiki_to_extensions_wmf-shared.sh
MultimediaViewer no
  • MultimediaViewer is on top of MediaWiki:Jquery.zoomImage.js (ON) when clicked, only either one of them should be active
  • works well but problem: «Quellübergreifende (Cross-Origin) Anfrage blockiert: Die Gleiche-Quelle-Regel verbietet das Lesen der externen Ressource auf http://species-id.net/o/media/4/4e/Lamium_album_scan.jpg. (Grund: CORS-Kopfzeile 'Access-Control-Allow-Origin' fehlt).»
  • negative but acceptable: only 2-step-zoom compared to MediaWiki:Jquery.zoomImage.js (ON)
  • TODO: NGINX configuration add_header could help to display media from rep OpenMedia, see
    cat /etc/nginx/cors-nginx.conf
  • if deactivated then TimedMediaHandler has a basic UI except for images
TimedMediaHandler no 
Call to undefined method ForeignDBRepo::fetchImageQuery()

Short solution:

Long solution:

Semantic_Mediawiki yes
Semantic_Forms yes
  • custom mandatory form field CSS-layout must be CSS-code!important
VisualEditor no Required an extra git clone (according to VisualEditor documentation), using the normal git archive caused missing resources and files from it's submodule. Untested: possibly https://github.com/Kentzo/git-archive-all would be worth a try to get also submodules, but cloning a flat git got VE loading well at least all resources: 
cd /usr/share/mediawiki26/extensions-simple-features
# cd /usr/share/mediawiki26/extensions-rich-features
if [[ -d VisualEditor ]];then sudo mv VisualEditor VisualEditor_backup`date +%Y%m%d%H%M`; fi
sudo git clone --depth 1 --branch REL1_26 https://gerrit.wikimedia.org/r/p/mediawiki/extensions/VisualEditor.git
cd VisualEditor && sudo git submodule update --init --recursive
cd lib/ve && sudo git checkout REL1_26

Problem: VisualEditor and Parsoid almost develop together. VE was deactivated: 

/usr/lib/parsoid/src/api/localsettings.js
Error loading data from server: (curl error: 7): docserver-http-error: (curl error: 7)

Solution: probably better update Parsoid service as well. See extensions/VisualEditor/ApiVisualEditor.php at line 91 

requestRestbase()
  'url' => '/parsoid/local/v1/'
 since 1.26wmf21 changed to
  'url' => '/restbase/local/v1/'

Different URL handling, hence upgrade of Parsoid is needed for REL1_26 (changed in Git on Saturday, June 13, 2015)

Widgets no Required an extra git clone (according to Widgets documentation), using the normal git archive caused missing resources and files from it’s submodule: 
# cd /usr/share/mediawiki26/extensions-simple-features
cd /usr/share/mediawiki26/extensions-rich-features
if [[ -d Widgets ]]; then sudo mv Widgets Widgets_backup`date +%Y%m%d%H%M`; fi
sudo git clone --depth 1 --branch REL1_26 https://gerrit.wikimedia.org/r/p/mediawiki/extensions/Widgets.git
cd Widgets && sudo git submodule init && sudo git submodule update
WikimediaMessages no
GoogleLogin no
  • separate install, not in wmf extensions (/usr/share/mw-wmf-clone/extensions/)
# cd /usr/share/mediawiki26/extensions-simple-features
cd /usr/share/mediawiki26/extensions-rich-features
if [[ -d GoogleLogin ]];then sudo mv GoogleLogin GoogleLogin_backup`date +%Y%m%d%H%M`; fi
sudo git clone --depth 1 --branch REL1_26 https://gerrit.wikimedia.org/r/p/mediawiki/extensions/GoogleLogin.git
GoogleAPIClient no
  • separate install, not in wmf extensions (/usr/share/mw-wmf-clone/extensions/)
# cd /usr/share/mediawiki26/extensions-simple-features
cd /usr/share/mediawiki26/extensions-rich-features
if [[ -d GoogleAPIClient ]];then sudo mv GoogleAPIClient GoogleAPIClient_backup`date +%Y%m%d%H%M`; fi
sudo git clone --depth 1 --branch REL1_26 https://gerrit.wikimedia.org/r/p/mediawiki/extensions/GoogleAPIClient.git
NiceCategoryList2 no
  • separate install, not in wmf extensions (/usr/share/mw-wmf-clone/extensions/)
# cd /usr/share/mediawiki26/extensions-simple-features
cd /usr/share/mediawiki26/extensions-rich-features
if [[ -d NiceCategoryList2 ]];then sudo mv NiceCategoryList2 NiceCategoryList2_backup`date +%Y%m%d%H%M`; fi
  sudo mkdir NiceCategoryList2 && sudo chown www-data:www-data NiceCategoryList2 && cd NiceCategoryList2;
  wget "https://www.mediawiki.org/w/index.php?title=Extension:NiceCategoryList2/3.0&action=raw" --output-document="NiceCategoryList.php"
  sed --in-place '/<\/\?source/d; /<\/\?syntaxhighlight/d;' NiceCategoryList.php # delete lines containing </?source tag

// in LocalSettings.php
require_once ("$IP/extensions/NiceCategoryList2/NiceCategoryList.php");
Charinsert no Setup has changed works now in the Gadget-system (Extension:Gadgets, Read documentation of Extension:Charinsert)
ConfirmAccount yes The file /usr/share/mediawiki26/extensions-rich-features/ConfirmAccount/frontend/specialpages/actions/ConfirmAccount_body.php was modified to show for which wiki a user has requested an account and the comment from the administrator, if the request was put on hold or rejected. See File:ConfirmAccount body.patch for the changes.
Data Transfer yes The file /usr/share/mediawiki26/extensions-rich-features/DataTransfer/includes/DT_PageStructure.php has some loosely programmed regular expression replacements (https://phabricator.wikimedia.org/T135770) and was modified to replace non-template stuff recursively. See File:DT PageStructure recursive-escape parsePageContents.patch for the changes.
CiteThisPage no Requires an extra git clone as it is not part of the standard extensions wmf clone. Cloning a flat git: 
cd /usr/share/mediawiki26/extensions-simple-features
# cd /usr/share/mediawiki26/extensions-rich-features
if [[ -d CiteThisPage ]];then sudo mv CiteThisPage CiteThisPage_backup`date +%Y%m%d%H%M`; fi
sudo git clone --depth 1 --branch REL1_26 https://gerrit.wikimedia.org/r/p/mediawiki/extensions/CiteThisPage

Check citation setup at MediaWiki:Citethispage-content and set the empty MediaWiki:Sitesubtitle or remove it from the citation setup page.

PdfHandler no Requires extra pdf tools (pdfinfo etc.) 
sudo apt-get install poppler-utils

Did not work properly, PDFs of Floristische Rundbriefe are not created; no further debugging was done.

Translate no/yes
CommonsMediaCaching no custom written extension in the svn repository to cache common images.
  • set up cronjob for runJobs.php
  • set up logrotate for the log files
  • set up the template Template:Cached Commons Copy on OpenMedia

See also Extension:CommonsMediaCaching

Skin:Vector

Not sure if composer.local.json can also manage skins 

…
"require":{
  "mediawiki/vector-skin":"dev-wmf/1.27.0-wmf.9"
}
…

Collpaseable sidbar is not part of skin:Vector but now in extension:CollapsibleVector 

 cd /usr/share/mediawiki26/extensions-simple-features
 sudo git clone https://gerrit.wikimedia.org/r/p/mediawiki/extensions/CollapsibleVector.git
 cd /usr/share/mediawiki26/extensions-simple-features/CollapsibleVector
 sudo git checkout origin/REL1_26
 
 cd /usr/share/mediawiki26/extensions-rich-features
 sudo git clone https://gerrit.wikimedia.org/r/p/mediawiki/extensions/CollapsibleVector.git
 cd /usr/share/mediawiki26/extensions-simple-features/CollapsibleVector
 sudo git checkout origin/REL1_26

Skin:naturkunde

cd /usr/share/mediawiki26/skins
sudo git clone https://github.com/MfN-Berlin/mwSkinNaturkunde.git naturkunde
cd naturkunde/
sudo git submodule update --init

# load it in LocalSettigns.php
if (is_dir("$IP/skins/naturkunde")) {
    require_once("$IP/skins/naturkunde/Naturkunde.php");
    # wfLoadSkin('naturkunde'); // wfLoadSkin not yet applicable would need a skins/naturkunde/skin.json
}

CommonsMediaCaching

Biowikifarm extension CommonsMediaCaching is stored in svn, some files need to be at the right place:

  • at wiki-folder/maintenance and
  • in the cron job system (/etc/cron.d/)

Check that maintenance scripts are in place: 

cd /usr/share/mediawiki26/extensions-rich-features/CommonsMediaCaching
# do not overwrite blindly (--interactive)
# file should be owned by www-data (option --preserve)
sudo cp --interactive --preserve downImagesCommons.php /usr/share/mediawiki26/maintenance/
sudo cp --interactive --preserve importImagesCommons.inc /usr/share/mediawiki26/maintenance/
sudo cp --interactive --preserve importImagesCommons.php /usr/share/mediawiki26/maintenance/

Resolve perhaps issues:

  • check path settings in cron scripts /etc/cron.d/
  • check url of commons wiki (2015 changed from http to only working with https…)
  • …

See also Caching Wikimedia Commons file to local repository

Switching a Wiki Setup: Simple to Rich Features

(1) Change settings:

  • modify /var/www/MediaWikiCommonSettings_REL1_26.php
    • check the true case for $bwgSetUpAsRichWiki
    • modify perhaps in the semantics section enableSemantics('server-url.org');
    • modify for extension: GlobalUsage at the appropriate section

(2) change link setup accordingly:

  • all symbolic links should point afterwards to “…-rich-features-…” something:
cd /var/www/path/to/wiki 
 # do NOT REMOVE FOLDERS, just the setup of symbolic links!!!
 sudo rm --interactive vendor # the prompt should be like: « rm: remove symbolic link ‘vendor’? »
 sudo ln --symbolic /usr/share/mediawiki26/vendor-rich-features              vendor
 sudo rm --interactive extensions # the prompt should be like: « rm: remove symbolic link ‘extensions’? »
 sudo ln --symbolic /usr/share/mediawiki26/extensions-rich-features          extensions
# correct composer. NOTE: symbolic file links can be overridden forcefully, but not folders
 sudo ln --force -s /usr/share/mediawiki26/composer-rich-features.lock       composer.lock
 sudo ln --force -s /usr/share/mediawiki26/composer-rich-features.local.json composer.local.json
 sudo ln --force -s /usr/share/mediawiki26/composer-rich-features.json       composer.json
# check if there are still links to simple feature setups
 ls -l | grep simple

(3) run maintenance/update.php 

sudo -u www-data php ./maintenance/update.php --dbuser wikiadmin --quick --conf ./LocalSettings.php

Normally the wiki should be up again.

Retrieved from "https://biowikifarm.net/metawiki/index.php?title=Mediawiki_installation/REL1_26&oldid=6481"