Mediawiki installation/Version 1.23.8

MediaWiki version 1.23.8 is a long term support version.

Architecture

See also http://biowikifarm.net/meta/Mediawiki_installation#Cloning_a_new_wiki Approach:

• have a MediaWiki Git clone
• extract from the MediaWiki Git clone MW version, e.g. a Long Term Support (LTS)
• extract (correct versions of) extensions for the wiki version
• install a new WEB_WIKI using a Database template

Schema and file structure on the server:

 Schema                                              Path or Link:
 MW_WMF_CLONE                                        /usr/share/mw-wmf-clone
 ├ core (Git) ─────────git-archive-export───┐        /usr/share/mw-wmf-clone/core
 └ extensions (Git) ───git-archive-export─→ ) ─┐     /usr/share/mw-wmf-clone/extensions
                                            │  │
   USR_SHARE_WIKI  (e.g. REL1_23)           │  │
   ┌ core-files (local git archive-export) ←┘  │     /usr/share/mediawiki23
   ├ extensions (mixed:                        │     /usr/share/mediawiki23/extensions
   │   svn biowikifarm, (repo: as svn-post-commit)
   │   git-archive-export)  ←──────────────────┘
   ↓   
   │ WEB_WIKI (as LTS web Wiki)
   ├─→ core-files (linked)                           /var/www/v-xxx/d/
   └─→ extensions (linked)                           /var/www/v-xxx/d/extensions
         composer-extensions must be dealt with      /var/www/v-xxx/d/composer.json
         on level WEB_WIKI individually

Git Short Help

 #↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓
 git checkout <branch>
 git checkout <tag-name>
 git checkout -b <create-new-branch>
 git branch -r|--remotes       # list all branches
 git tag -l|--list             # list all tags
 git tag -l|--list <pattern>   # list tags matching the pattern
 git archive <tag-or-branch-or-tree-ish>   # extract an archive
 #↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑

Script(ed) Installation

The following scripts are not intended to be run at once but step by step or just partially!!

• most scripts that follow depend on settings in section of Global Variables

Try to install it from scratch

• use the architecture of the provided MW version directly to install a WEB_WIKI instead of a directory template, because each MW version might have a new architecture

Tips for an Upgrade

This documentation is just for the first install not for an upgrade, but for an upgrade the following approach can be a good strategy:

• do not upgrade the old WEB_WIKI directly but
• install a new WEB_WIKI version that will replace later on the old production WEB_WIKI
• copy the old LocalSettings.php to your new WEB_WIKI and adjust it
• if you think you can upgrade:
  1. rename the folders: old production WEB_WIKI becomes a backup WEB_WIKI, the new-WEB_WIKI becomes the new production WEB_WIKI
  2. run the update.php script in maintenance folder for the new production WEB_WIKI

This upgrade strategy ensures, that the new production WEB_WIKI has a clean architecture and for any problems you can roll back rather easily.

“Global Variables”

##↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓
# set global variables
do_not_install_but_show_dry_run="" # "" or "anything-but-non-empty"
if [ "$do_not_install_but_show_dry_run" != "" ]; then 
  install_notice="DRY RUN (nothing is done)";
else
  install_notice="INSTALL";
fi

git_mw_basedir="/usr/share/mw-wmf-clone"
git_mw_core_source_dir="${git_mw_basedir}/core"
git_mw_extension_source_dir="${git_mw_basedir}/extensions"
git_branch="origin/REL1_23"
git_tag="1.23.8"

# naming convention
#  path: is the absolute path, has NO trailing slash
#  directory: a *single* directory
# web_wiki_config_folder="/mw-config" # default /mw-config must e removed for security reasons
vhost_path="/var/www/v-infoflora"
vhost_wiki_path="${vhost_path}/d"

usr_share_wiki_path="/usr/share/mediawiki23"
usr_share_extension_path="${usr_share_wiki_path}/extensions"
# echo "$usr_share_extension_path"

printf "\e[47mBase directory of the Wiki will be: \e[42m'%s'\e[0m\n" "${vhost_wiki_path}"
echo -e "\e[31m$install_notice\e[0m"
# end of global variables
##↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑

MW_WMF_CLONE Upgrade

Possible approaches after upgrading the MW_WMF_CLONE

• checkout a LTS-branch and copy these sources to USR_SHARE_WIKI
• export by git archive <tag-or-branch-or-tree-ish> to USR_SHARE_WIKI

Upgrade wmf-git-clone manually

##↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓
cd "${git_mw_core_source_dir}"; 
  sudo git pull;
  git branch --remotes | sort --version-sort # list branches
  git tag --list | sort --version-sort # list tags
cd "${git_mw_extension_source_dir}";
  sudo git pull; sudo git submodule update --init --recursive;
  git branch --remotes | sort --version-sort # list branches
  git tag --list | sort --version-sort # list tags
##↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑

Checkout a branch can be skipped: this makes only sense, if you want to use just one MW version as source base. The subsequent approach instead uses an export of a specific branch or tag from the MW_WMF_CLONE.

##↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓
# STEP checkout git clone to git_branch for the subsequent export?
#
cd "${git_mw_core_source_dir}"
  # git branch -r | sort --version-sort # list branches
  # git tag -l | sort --version-sort # list tags
  if [ "$do_not_install_but_show_dry_run" == "" ]; then
    sudo -u root -g mwadmin git checkout -b $git_branch 
    # sudo -u root -g mwadmin git checkout $git_tag
  fi
  printf "\e[47mgit clone in '\e[42m%s\e[47m' is checkout to branch: '%s' tag: '%s'\e[0m\n"  "${git_mw_core_source_dir}" "$git_branch" "$git_tag"
cd "${git_mw_extension_source_dir}" 
  # git branch -r | sort --version-sort # list branches
  # git tag -l | sort --version-sort # list tags
  # extensions seem to have no tags
  if [ "$do_not_install_but_show_dry_run" == "" ]; then
    sudo -u root -g mwadmin git checkout -b "$git_branch"
  fi
  printf "\e[47mgit clone in '\e[42m%s\e[47m' is checkout to branch: '%s' (extensions seem to have no global tags)\e[0m\n" "${git_mw_extension_source_dir}" "$git_branch"
# end checkout of git clone in git_mw_basedir
##↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑

MW_WMF_CLONE → USR_SHARE_WIKI

• Create a MediaWiki versioned source (the source of a LTS-version)
• Create extension sources, extract Git extensions to USR_SHARE_WIKI
  • biowikifarm extensions
  • no further developed extension still necessary fro biowikifarm
  • Git extensions from MW_WMF_CLONE

USR_SHARE_WIKI Core Files

##↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓
# create MediaWiki versioned source (the source of a LTS-version)
if ! [ -d "${usr_share_wiki_path}" ]; then 
  if [ "$do_not_install_but_show_dry_run" == "" ]; then
    sudo -u root -g mwadmin mkdir --parent "${usr_share_wiki_path}"
  fi
  echo -e "\e[47mCreate wiki source directory ($git_branch): \e[42m'${usr_share_wiki_path}'\e[0m"
else
  echo -e "\e[47mWiki wiki source directory ($git_branch) exits already: \e[42m'${usr_share_wiki_path}'\e[0m"
fi

echo -e "\e[47mCheck also release notes, e.g. \e[42m'${usr_share_wiki_path}/RELEASE-NOTES-X.XX'\e[0m"
  
# cd "${git_mw_core_source_dir}"
# sudo -u root -g mwadmin git checkout -b origin/REL1_23 1.23.8 only from repo clone directly

###
# MediaWiki core files and directories
if [ "$do_not_install_but_show_dry_run" == "" ]; then
  # EXAMPLE: sudo git archive origin/REL1_23 | sudo -u root -g mwadmin tar --extract --overwrite --directory=/usr/share/mediawiki23
  cd "${git_mw_core_source_dir}" && sudo git archive "$git_tag" | sudo -u root -g mwadmin tar --extract --overwrite --directory="${usr_share_wiki_path}"
  if [ -d "${usr_share_extension_path}" ]; then
    sudo chown root:www-data -R "${usr_share_extension_path}"
  fi
else
  echo "cd \"${git_mw_core_source_dir}\" && sudo git archive \"$git_tag\" | sudo -u root -g mwadmin tar --extract --overwrite --directory=\"${usr_share_wiki_path}\""
fi
# cd "${git_mw_extension_source_dir}"
# ls -l
##↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑

USR_SHARE_WIKI Extensions

Check status of extensions, either git tags or git branch numbers needed later for installation of a LTS-version

#######################################
##↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓
# List, and write to a file, all extensions with their tags and branches. Write into the user's home folder
# DEPENDENCIES: global variables

sudo printf "\n== Tags ==\n"  > ~/"ext_tags_and_branches.txt"; 

IFS=$'\n'; # set Internal Field Separator in for loop to newline '\n' only! Deafult $IFS is ' \t\n': «printf %q "$IFS"»
for ext in `find "${git_mw_extension_source_dir}" -maxdepth 1 -type d -printf "%f\n" | sort`; do 
  printf "%s" "$ext" >> ~/"ext_tags_and_branches.txt";
  cd "${git_mw_extension_source_dir}/$ext";
  tags=""; tags=`git tag -l | sed ':anchor;N;$!banchor;s@\n@, @g; s@ \{2,\}@ @g;'` ; 
  if [ "$tags" != "" ]; then 
    printf "\n  tags: %s\n" "$tags" >> ~/"ext_tags_and_branches.txt"; 
  else 
    printf "\n" >> ~/"ext_tags_and_branches.txt";
  fi
done;

# check extensions: find all extensions and print branches
sudo printf "\n== Branches ==\n"  >> ~/"ext_tags_and_branches.txt";
for ext in `find "${git_mw_extension_source_dir}" -maxdepth 1 -type d -printf "%f\n" | sort`; do 
  printf "%s" "$ext" >> ~/"ext_tags_and_branches.txt"; 
  cd "${git_mw_extension_source_dir}/$ext";
  branches=""; branches=`git branch --remotes | sed ':anchor;N;$!banchor;s@\n@, @g; s@ \{2,\}@ @g;'`
  if [ "$branches" != "" ]; then 
    printf "\n  branches: %s\n" "$branches" >> ~/"ext_tags_and_branches.txt"; 
  else 
    printf "\n" >> ~/"ext_tags_and_branches.txt";
  fi
done;
cd "${git_mw_extension_source_dir}"
IFS=$' \t\n';
# end check extensions' branches
##↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑
#######################################

Extract the right branch or tag of an extension from MW_WMF_CLONE for this USR_SHARE_WIKI. Note, the list of EXTENSIONS may not be complete but it seems unnecessary to extract all 663 extensions shipped with the wmf git clone.

#######################################
##↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓
# create extensions sources for LTS-MediaWiki
# DEPENDENCIES: global variables
# IMPORTANT: extensions managed by PHP script composer.phar
#   must be dealt with individually on the level WEB_WIKI
# NOTE: this script part might be run better from a script file directly
#   because paste and copy can cause troubles: the terminal seams limited 
#   to a certain string length
# NOTE: basic extensions from a v1.23.8 tar.gz download are:
#   Cite, ConfirmEdit, Gadgets, ImageMap, InputBox, Interwiki, LocalisationUpdate, 
#   Nuke, ParserFunctions, Poem, Renameuser, SpamBlacklist, SyntaxHighlight_GeSHi, 
#   TitleBlacklist, WikiEditor
EXTENSIONS="
Arrays
BetaFeatures
CategoryTree
CharInsert
Cite
Collection
ConfirmAccount
ConfirmEdit
ContactPage
DataTransfer, 0.6
DismissableSiteNotice
ExternalData, 1.8
Gadgets
HeaderTabs, 1.0.1
ImageMap
InputBox
Interwiki
LiquidThreads
LocalisationUpdate
Lockdown
MassEditRegex
Math
MediaFunctions
MwEmbedSupport
MultimediaViewer
Nuke
OpenID
PageTools, REL_2_0
ParserFunctions
Poem
PdfExport
PdfHandler
Quiz
Renameuser
ReplaceText
SemanticCompoundQueries
SemanticDrilldown
SemanticForms
SemanticFormsInputs
SemanticInternalObjects, 0.8.1
SpamBlacklist
SyntaxHighlight_GeSHi
TimedMediaHandler
TitleBlacklist
TitleKey
UploadWizard
Variables
Widgets
WikiEditor
WikimediaMessages
"

# --------------------------------------
# split extension on ", ". If extension sting contains only one value
# * export $git_branch, 
# * else export the specified string above as $this_clone_version 
#   (the so-called tree-ish string, usually a tag or branch or anything the *developers* had defined)
# Example:
#   SemanticMediaWiki, 1.8
#      ↓                   ↓
#   $extension, $this_clone_version
#   will export 1.8 (which the developers have defined as a tag)
# NOTE: check the line (similar to ternary operator: condition ? do-this : do-other-things)
#   $( … ) substitutes/returns the command's output
#   → if ext-check-with-comma == ext then (&&) "HEAD" else (||) version-string-from-above-list-of-EXTENSIONS
#   ${ext%,*}    from back  of $ext → delete shortest (%) or longest (%%) match
#   ${ext#*,* }  from front of $ext → delete shortest (#) or longest (##) match
# --------------------------------------

this_ext_update_status=""
IFS=$'\n'; # set Internal Field Separator in for loop to newline '\n' only! Deafult $IFS is ' \t\n': «printf %q "$IFS"»
for ext in $EXTENSIONS; do
  if [ "${ext}" ];then
    this_extension=$([ "${ext%,*}" == "$ext" ] && echo "$ext" || echo "${ext%%,*}");
    this_clone_version=$([ "${ext%,*}" == "$ext" ] && echo "$git_branch" || echo "${ext#*, }");
    if ! [ -d "${git_mw_extension_source_dir}/${this_extension}" ]; then
      echo "${git_mw_extension_source_dir}/${this_extension} DOES NOT EXIST AS GIT SOURCE, check the extension's name or path!! (extension was skipped)";
    else
      if [ ! -d "${usr_share_extension_path}/${this_extension}" ]; then
        this_ext_update_status="+";
        if [ "$do_not_install_but_show_dry_run" == "" ]; then
          sudo mkdir --parents "${usr_share_extension_path}/${this_extension}";
          sudo chown www-data:www-data "${usr_share_extension_path}/${this_extension}";
        else
          echo "sudo mkdir --parents \"${usr_share_extension_path}/${this_extension}\"; # add $this_ext_update_status;"
          echo "sudo chown www-data:www-data \"${usr_share_extension_path}/${this_extension}\";"
        fi;
      else
        this_ext_update_status="u";
      fi;
      printf "git archive export: %1s %-30s %s\n" $this_ext_update_status $this_extension $this_clone_version;
      if [ "$do_not_install_but_show_dry_run" == "" ]; then
        if [ -d "${git_mw_extension_source_dir}/${this_extension}" ];then
          cd "${git_mw_extension_source_dir}/${this_extension}"
          AVAILABLE_CLONE_VERSIONS=`git branch -r | sort --version-sort`
          if [ "`echo "$AVAILABLE_CLONE_VERSIONS" | grep --count "$this_clone_version" `" == "0" ]; then
            echo "Clone version $this_clone_version not available!! Check these (git branch -r  | sort --version-sort):";
            echo "$AVAILABLE_CLONE_VERSIONS";
          else
            echo "No clone version available";
          fi;
          # EXAMPLE: sudo git archive --prefix=AdminLinks/ HEAD | sudo -u www-data tar --extract --overwrite --directory=/usr/share/mediawiki23/extensions
          sudo git archive --prefix="${this_extension}/" "$this_clone_version" | sudo -u www-data tar --extract --overwrite --directory="${usr_share_extension_path}";
        else
          echo  "Sorce not found: \"${git_mw_extension_source_dir}/${this_extension}\" "
        fi;
      else
        echo "cd \"${git_mw_extension_source_dir}/${this_extension}\" && sudo git archive --prefix=\"${this_extension}/\" \"$this_clone_version\" | sudo -u www-data tar --extract --overwrite --directory=\"${usr_share_extension_path}\";"
      fi;
    fi;
  fi;
done;
IFS=$' \t\n';
##↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑
#######################################

Unsupported extensions (svn)

https://www.mediawiki.org/wiki/Extension:StringFunctions is no more supported see Installation Issues, Problems, Transition form Older Wikis (StringFunctions).

Biowikifarm extensions (svn)

#######################################
##↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓
# subversion STEP 1: biowikifarm extensions to ${usr_share_extension_path} 
cd "${usr_share_extension_path}";
sudo -u www-data -g www-data svn checkout file:///var/lib/svn/LocalSVNextensions "${usr_share_extension_path}";

# subversion STEP 2
# create post hook commit to the MediaWiki LTS source
# open file 
# sudo nano /var/lib/svn/hooks/post-commit
#!/bin/bash
if [ -d "/usr/share/mediawiki23/extensions" ]; then
  echo -en "\n\npost-commit mediawiki23/extensions (r${REV})\n" >> "${LOGFILE}";
  cd /usr/share/mediawiki23/extensions;
  echo "try to get /usr/bin/svn update >> \"${LOGFILE}\"" >> "${LOGFILE}";
  /usr/bin/svn update >> "${LOGFILE}";
  echo -ne " (done)\n" >> "${LOGFILE}";
fi
##↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑
#######################################

USR_SHARE_WIKI → WEB_WIKI

##↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓
# STEP create web wiki directory
if ! [ -d "${vhost_wiki_path}" ]; then 
  echo -e "\e[47mCreate wiki web base directory (sudo): \e[42m'${vhost_wiki_path}'\e[0m"
  if [ "$do_not_install_but_show_dry_run" == "" ]; then
    sudo -u root -g mwadmin mkdir --parent "${vhost_wiki_path}"
    # sudo chown root:mwadmin "${vhost_wiki_path}" -R
  else
    echo "sudo -u root -g mwadmin mkdir --parent \"${vhost_wiki_path}\""
  fi
else
  echo -e "\e[47mWiki web base directory exits already: \e[42m'${vhost_wiki_path}'\e[0m"
fi
##↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑

OpenMedia, SpecialMedia Symlink

This may be skipped, because the NGINX configuration can catch URLs and redirect directly to the OpenMedia directory without a symlink but due to the Same-Origin-Policy (SOP) it is done here. Extension:MultimediaViewer complains about Cross domoain origins of media.

##↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓
# STEP create symlink to openmedia
if ! [ -d "${vhost_path}/o" ]; then 
  echo -e "\e[47mCreate Openmedia-symlink \e[42m'${vhost_path}/o'\e[47m → \e[42m'../v-species/o'\e[0m"
  if [ "$do_not_install_but_show_dry_run" == "" ]; then
    sudo ln -s "../v-species/o" "${vhost_path}/o"
    sudo mkdir "${vhost_path}/o = Openmedia-symlink"
    # sudo chown root:mwadmin "${vhost_wiki_path}" -R
  else
    echo "sudo ln -s \"../v-species/o\" \"${vhost_path}/o\";"
  fi
else
  echo -e "\e[47mOpenmedia-symlink exits already \e[42m'${vhost_path}/o'\e[0m"
fi
if ! [ -d "${vhost_path}/s" ]; then 
  echo -e "\e[47mCreate Specialmedia-symlink \e[42m'${vhost_path}/s'\e[47m → \e[42m'../v-species/s'\e[0m"
  if [ "$do_not_install_but_show_dry_run" == "" ]; then
    sudo ln -s "../v-species/s" "${vhost_path}/s"
    sudo mkdir "${vhost_path}/s = Specialmedia-symlink"
    # sudo chown root:mwadmin "${vhost_wiki_path}" -R
  else
    echo "sudo ln -s \"../v-species/s\" \"${vhost_path}/s\";"
  fi
else
  echo -e "\e[47mSpecialmedia-symlink exits already \e[42m'${vhost_path}/s'\e[0m"
fi
##↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑

Link Folders and Files from USR_SHARE_WIKI to WEB_WIKI

##↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓
# symbolic links of folders of LTS version
# for $vhost_wiki_path to $usr_share_wiki_path
cd "${usr_share_wiki_path}"
# \( ! -name '.*' \) avoids finding hidden directories
for this_dir in `find . -maxdepth 1 \( ! -name '.*' \) -type d -printf "%f\n"`; do
  if ! [ -e "${vhost_wiki_path}/${this_dir}" ]; then
    echo "symbolic link to ${vhost_wiki_path}/${this_dir}"
    sudo ln -s "${usr_share_wiki_path}/${this_dir}" "${vhost_wiki_path}/${this_dir}"
  fi
done
# symbolic links of files of LTS version
cd "${usr_share_wiki_path}"
# \( ! -name '.*' \) avoids finding hidden directories
for this_file in `find . -maxdepth 1 \( ! -name '.*' \) -type f -printf "%f\n"`; do
  if ! [ -e "${vhost_wiki_path}/${this_file}" ]; then
    echo "symbolic link to ${vhost_wiki_path}/${this_file}"
    sudo ln -s "${usr_share_wiki_path}/${this_file}" "${vhost_wiki_path}/${this_file}"
  fi
done
##↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑

Fix Links of Folders (Cache, Media, Extensions …)

Cache & media is on a separate partition. Folder extension cannot be a symbolic link because of composer’s management on level WEB_WIKI.

NOTE: for linking extensions there are two approches:

1. linking the entire extension directory and let composer override some extensions folders managed by composer
2. do not link the entire extension directory, take care manually for each extension directory wether it is managed by composer or comes from a USR_SHARE_WIKI-extension (i.e.: do no override USR_SHARE_WIKI-extensions. If composer manages extensions, then you must rename the symlink manually)

Both approaches seems be valid. The second approach was done here.

##↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓
# extension: remove symbolic link to folder extensions
  # no symbolic link to usr_share_extension_path
  # because composer.phar handles extension on level WEB_WIKI
  sudo rm -i "${vhost_wiki_path}/extensions" # remove interactively
  echo "create ${vhost_wiki_path}/extensions"
  sudo -u root -g www-data mkdir "${vhost_wiki_path}/extensions"
  
# STEP 1 cache: reroute cache folder to a dump partition (specific to biowikifarm)
  dump_path_for_webwiki="/mnt/dump/var/www/v-infoflora-d-cache";
  vhost_wiki_path="/var/www/v-infoflora/d";
  echo "create ${dump_path_for_webwiki}";
  sudo mkdir --parent "${dump_path_for_webwiki}"; 
  sudo chown www-data:www-data "${dump_path_for_webwiki}";
  sudo chmod 775 -R "${dump_path_for_webwiki}";
  echo "remove current ${vhost_wiki_path}/cache";
  sudo rm "${vhost_wiki_path}/cache" -r;
  echo "make symbolic link ${vhost_wiki_path}/cache → ${dump_path_for_webwiki}";
  sudo ln -s "${dump_path_for_webwiki}" "${vhost_wiki_path}/cache";
  
# STEP 2 images: use media instead, reroute to dump partition (specific to biowikifarm)
  dump_path_for_webwiki="/mnt/storage/var-www-v-inflora-d-media";
  vhost_wiki_path="/var/www/v-infoflora/d";
  echo "remove current ${vhost_wiki_path}/images";
  sudo rm -i "${vhost_wiki_path}/images"  # remove it interactively
  echo "create ${dump_path_for_webwiki}";
  sudo mkdir --parent "${dump_path_for_webwiki}"; 
  sudo chown www-data:www-data "${dump_path_for_webwiki}"; 
  sudo chmod 775 -R "${dump_path_for_webwiki}";
  echo "make symbolic link ${vhost_wiki_path}/media → ${dump_path_for_webwiki}";
  sudo ln -s "${dump_path_for_webwiki}" "${vhost_wiki_path}/media";

# STEP 3: remove mw-config symlink of web installation folder
  sudo rm -i "/var/www/v-infoflora/d/mw-config";
##↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑

Link Extensions from USR_SHARE_WIKI to WEB_WIKI

##↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓
# create symlinks to extensions individually
# reason: composer-extensions must be dealt with
# on level WEB_WIKI individually
# @dependency $EXTENSIONS
# @dependency global variables

SVN_BIOWIKIFARM_EXTENSIONS="
CommonsMediaCaching
ContributorList
IdentificationTool
ListAllExternalLinks
MobileKeyV1
MobileKeyV2
TemplateParameterIndex
"
LTS_EXTENSIONS=`echo "${EXTENSIONS}\n${SVN_BIOWIKIFARM_EXTENSIONS}" | sed 's@ *, .*@@g;/^$/d;' | sort --unique`

 this_ext_update_status=""
 IFS=$'\n'; # set Internal Field Separator in for loop to newline '\n' only! Deafult $IFS is ' \t\n': «printf %q "$IFS"»
 for ext in $LTS_EXTENSIONS; do
  if [ "${ext}" ];then
  # ${ext%,*}    from back  of $ext → delete shortest (%) or (%%) longest match 
  # ${ext#*,* }  from front of $ext → delete shortest (#) or (##) longest match
  this_extension=$([ "${ext%,*}" == "$ext" ] && echo "$ext" || echo "${ext%*,* *}");
  if ! [ -d "${usr_share_extension_path}/${this_extension}" ]; then
    echo "${usr_share_extension_path}/${this_extension} DOES NOT EXIST as LTS source, check '${usr_share_extension_path}' (extension was skipped)";
  else
    if ! [ -e "${vhost_wiki_path}/extensions/${this_extension}" ]; then
      this_ext_update_status="+";
      if [ "$do_not_install_but_show_dry_run" == "" ]; then
        sudo ln -s "${usr_share_extension_path}/${this_extension}" "${vhost_wiki_path}/extensions/${this_extension}";
      else
        echo "sudo ln -s \"${usr_share_extension_path}/${this_extension}\" \"${vhost_wiki_path}/extensions/${this_extension}\";";
      fi
    else
      this_ext_update_status=".";
    fi;
    printf "symlink: %1s %-30s\n" $this_ext_update_status $this_extension;
  fi;
  fi;
 done;
 IFS=$' \t\n';
##↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑

Install composer extensions

These extensions must be dealt with at level WEB_WIKI. Documentation

• https://www.mediawiki.org/wiki/Composer
• https://getcomposer.org/doc/

Extensions you want to remove you must do it with composer

• https://getcomposer.org/doc/03-cli.md#remove

##↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓
# get, install PHP composer tool
cd ~ # go to personal home directory
sudo curl -sS https://getcomposer.org/installer | php
sudo chown root:mwadmin composer.phar
sudo mv composer.phar /usr/local/bin/

# create ${vhost_wiki_path}/composer.json
# see ${vhost_wiki_path}/composer-example.json
sudo cp "${vhost_wiki_path}/composer-example.json" "${vhost_wiki_path}/composer.json"
sudo chown root:mwadmin  "${vhost_wiki_path}/composer.json"
##↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑

(Re)Move old Git extensions

The problem is, that extensions managed by git must be removed in favour of composer management. Composer installs also dependencies so take care also of dependent extensions documented by extension developers, e.g. SemanticMaps in composer will also install extension Maps. The following script just renames old extension folders to make room for the composer managed extensions. In case of SemanticMaps it’s only necessary to do …

sudo php /usr/local/bin/composer.phar require mediawiki/semantic-maps '3.1.3'

… but you must take care of the extension folder Maps (if present) manually.

##↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓
# rename e.g. extension SemanticForms folder → SemanticForms_2015-01-15
# create a file containing all composer commands to be installed manually later on
cd "${vhost_wiki_path}"
composer_command_file="composer_extensions_to_be_manually_installed.sh"
# ExtensionName, composer-require-command
COMPOSER_EXTENSIONS="
GraphViz, require mediawiki/graph-viz '*'
ImageMap, require mediawiki/image-map '@dev'
SemanticMediaWiki, require mediawiki/semantic-media-wiki '~2.0'
SemanticForms, require mediawiki/semantic-forms '3.1'
SemanticResultFormats, require mediawiki/semantic-result-formats '2.0'
SemanticMaps, require mediawiki/semantic-maps '3.1.3'
PHPExcel, require phpoffice/phpexcel '1.8.0'
"
echo "Install extensions manually, see '${vhost_wiki_path}/$composer_command_file'";
echo "You may check also dependencies manually and rename those extension folders manually too, e.g. SemanticMaps is dependent on Maps";
if [ "$do_not_install_but_show_dry_run" == "" ]; then
  sudo /bin/su -c "echo '' > \"${vhost_wiki_path}/$composer_command_file\""
else
  echo "sudo /bin/su -c \"echo '' > \\\"${vhost_wiki_path}/$composer_command_file\\\"\" "
fi
IFS=$'\n'; # set Internal Field Separator in for loop to newline '\n' only! Deafult $IFS is ' \t\n': «printf %q "$IFS"»
 for ext in $COMPOSER_EXTENSIONS; do
  if [ "${ext}" ];then
  # ${ext%,*}    from back  of $ext → delete shortest (%) or (%%) longest match 
  # ${ext#*,* }  from front of $ext → delete shortest (#) or (##) longest match
  this_composer_command=$([ "${ext%,*}" == "$ext" ] && echo "$ext" || echo "${ext#*,* }");
  this_extension=$([ "${ext%,*}" == "$ext" ] && echo "$ext" || echo "${ext%*,*}");
  if [ -h "${vhost_wiki_path}/extensions/${this_extension}" ]; then
    old_extension_date=`stat --format="%y" "${vhost_wiki_path}/extensions/${this_extension}" | awk  '{ print $1 }'` # e.g 2015-01-15
    echo "There is already an extension: rename '${this_extension}' to '${this_extension}_${old_extension_date}' "
    if [ "$do_not_install_but_show_dry_run" == "" ]; then
      sudo mv "${vhost_wiki_path}/extensions/${this_extension}" "${vhost_wiki_path}/extensions/${this_extension}_${old_extension_date}"
    else
      echo "sudo mv \"${vhost_wiki_path}/extensions/${this_extension}\" \"${vhost_wiki_path}/extensions/${this_extension}_${old_extension_date}\""
    fi
  fi
  printf "\e[0mFor \e[47m$this_extension\e[0m you may need to remove in \e[47mLocalSettings.php\e[0m the entry \e[42m%s\e[0m\n" "require_once '\$IP/extensions/$this_extension/$this_extension.php'; "
  printf "Append composer command to \e[42m%s\e[0m\n" "'${vhost_wiki_path}/$composer_command_file'"
  if [ "$do_not_install_but_show_dry_run" == "" ]; then
    sudo /bin/su -c "echo \"sudo php /usr/local/bin/composer.phar $this_composer_command\" >> \"${vhost_wiki_path}/$composer_command_file\""
  else
    echo "sudo /bin/su -c \"echo \\\"sudo php /usr/local/bin/composer.phar $this_composer_command\\\" >> \\\"${vhost_wiki_path}/$composer_command_file\\\"\""
  fi;
  fi;
 done
IFS=$' \t\n';
##↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑

##↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓
# commands from manually created file in '${vhost_wiki_path}/$composer_command_file'
cd "${vhost_wiki_path}"
sudo php /usr/local/bin/composer.phar require mediawiki/graph-viz '*'
sudo php /usr/local/bin/composer.phar require mediawiki/image-map '@dev'
sudo php /usr/local/bin/composer.phar require mediawiki/semantic-media-wiki '~2.0'
sudo php /usr/local/bin/composer.phar require mediawiki/semantic-forms '3.1'
sudo php /usr/local/bin/composer.phar require mediawiki/semantic-result-formats '2.0'
sudo php /usr/local/bin/composer.phar require mediawiki/semantic-maps '3.1.3'
sudo php /usr/local/bin/composer.phar require phpoffice/phpexcel '1.8.0'
##↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑
#######################################

Settings for LocalSettings.php

Set correct rights

sudo chmod 664 "${vhost_wiki_path}/LocalSettings.php" # rw-rw-r--
sudo chown root:mwadmin "${vhost_wiki_path}/LocalSettings.php"

There are alternative approaches:

1. use web installer of the new Wiki
   • NGINX configuration must be done
   • correct rights for writing of the www-data user
   • see Manual:Shared database → Upgrading
2. use command line installer in maintenance/install.php
   •  ?does it try to override a LocalSettings.php?
   • it seems to need an Admin User to be defined (problem with shared user tables?)
   • see Manual:Shared database → Upgrading. ?make use of command line option --doshared?
3. copy old database template (database z_en_template in https://biowikifarm.net/phpmyadmin_v3-3-7/) and adjust it as well as settings for LocalSettings.php
   • see in MediaWikiCommonSettings.php (some settings there might interfere with MW 1.23 due to software changes)
   • see other files of LocalSettings.php

Nginx Configuration

See also Nginx Installation and Configuration. Tips:

• location in nginx path must match $wgScriptPath $wgArticlePath
• http://blog.bigdinosaur.org/mediawiki-on-nginx/

General architecture

/etc/nginx/nginx.conf - basic settings
└→ include /etc/nginx/sites-enabled/* - actual sites configured ready for production
   * default file - for common settings of that server the no-URL-resolving redirects back to
┌→ * configuration files for each server
│ (linked)
/etc/nginx/sites-available/* - potentially available sites files for each server

Installation Issues, Problems, Transition form Older Wikis

• RELEASE-NOTES: read/check file mw-source/RELEASE-NOTES-X.XX, changes are described there, class changes, core changes and deprecated functions
• composer management: problem is that composer extensions are handled on the level WEB_WIKI by composer not by require_once("…")
• Extension:Cite becomes extension CiteThisPage (MW 1.24+)
• Extension:ContacPage has special installation (MW1.23+) https://www.mediawiki.org/wiki/Extension:ContactPage#Installation
• Extension:ExpandTemplates is now in MW1.23+ core, separate installation is no longer necessary (require_once("…"))
• Extension:GlobalUsage has no more class names with underscore. Adjust configuration of $wgLBFactoryConf in LocalSettings.php see also http://noc.wikimedia.org/conf/highlight.php?file=db.php
• Extension:Vector is in core MW 1.22+, separate installation is no longer necessary (require_once("…"))

Dependencies:

• Extension:TimedMediaHandler dependent on Extension:MwEmbedSupport
• biowikifarm Extension:MobileKeyV1, Extension:MobileKeyV2 are dependent on Extension:TemplateParameterIndex
• Extension:BetaFeatures is a light dependency of Extension:MultimediaViewer

Composer extensions see above

No further development of StringFunctions

• problem support https://www.mediawiki.org/wiki/Extension:StringFunctionsis terminated. Fix it by set $wgPFEnableStringFunctions = true; in LocalSettings.php.

Notes on Future Versions of MW

MediaWiki 1.24 and later don't include skins in the Git download.

cd "$git_mw_basedir"
git clone https://gerrit.wikimedia.org/r/p/mediawiki/skins.git
# Switch to the skins folder, that was just cloned. To get all the skins use:
git submodule update --init --recursive

Installation Problems

• A user_password_expires column has been added to the user table

/usr/share/mediawiki23/maintenance/archives/patch-user_password_expire.sql
# MYSQL-patch see release notes: ALTER TABLE user ADD COLUMN user_password_expires varbinary(14) DEFAULT NULL; -- done so far for biowikifarm

• location in nginx path must match $wgScriptPath $wgArticlePath
• load.php gave internal server error 500

  debug on: nothing conclusive

  nginx logs: PHP message: PHP Fatal error: Cannot override final method SMW\ResultPrinter::getResult() in /var/www/v-infoflora/d/extensions/SemanticResultFormats/formats/Exhibit/SRF_Exhibit.php on line 468

  $srfgFormats[] = 'exhibit'. in SemanticResultFormats v2.0 causes fatal error, see https://phabricator.wikimedia.org/T59760

• Google Maps API Key neccessary?
• MW 1.22+ <gallery> needs a new setting in $wgGalleryOptions for 'mode' otherwise MWException

  'mode' => 'traditional', // One of "traditional", "nolines", "packed", "packed-hover", "packed-overlay"

• if Extension:TitleKey is active you should run for the first time

sudo -u www-data php /var/www/v-mywiki/w/extensions/TitleKey/rebuildTitleKeys.php --conf LocalSettings.php

Installation Issues: Semantic MediaWiki

• Extension:SemanticForms 3.0+ (edit with form) the implementation of forms has changed to {{#default_form:form}}
• #set_internal saves differently on a property page "Property:property-name" not just "property-name", has consequences for querying. You could use a count check for the query to see if a property prefix is neccessary:

{{#ifexr: {{#ask: [[internal object::{{{property|}}}]]|format=count}}|<!-- no prefix: older SMW -->PAGENAME|<!-- prefix property -->FULLPAGENAME}}

• Extension:Semantic Drilldown 2.0+ sets up drill down by #drilldowninfo and #drilldownlink. Deprecated things:
  • namespace Filter
  • Constants for special properties