Difference between revisions of "Configuring the biowikifarm subversion repository"

From Biowikifarm Metawiki
Jump to: navigation, search
(SVN-Repository and post-commit-hook)
 
(10 intermediate revisions by 3 users not shown)
Line 20: Line 20:
  
 
<source lang="bash">
 
<source lang="bash">
  sudo htpasswd -c /etc/apache2/dav_svn.passwd user ## only for first user
+
  sudo htpasswd -c /etc/apache2/dav_svn_git.passwd user ## only for first user
  sudo htpasswd /etc/apache2/dav_svn.passwd user1
+
  sudo htpasswd /etc/apache2/dav_svn_git.passwd username1
  sudo htpasswd /etc/apache2/dav_svn.passwd user2 ## etc.
+
  sudo htpasswd /etc/apache2/dav_svn_git.passwd username2 ## etc.
 
</source>
 
</source>
 +
 +
(Password will be requested by htpasswd)
  
 
Create access control file and reload:
 
Create access control file and reload:
Line 40: Line 42:
 
</source>
 
</source>
  
== SVN-Repository and post-commit-hook ==
+
== SVN Web Access ==
  
 
Creating SVN-Repository, access only to Webserver:
 
Creating SVN-Repository, access only to Webserver:
Line 53: Line 55:
 
  svn co https://biowikifarm.net/svn/
 
  svn co https://biowikifarm.net/svn/
 
</source>
 
</source>
 +
 +
 +
== SVN post-commit-hook ==
  
 
To increase comfort we can ensure that committing to the webserver the most recent version is updated also in server’s repositories (i.e. with every commit from a normal subversion user, the webserver-local checkout is updated by a post-commit later on):
 
To increase comfort we can ensure that committing to the webserver the most recent version is updated also in server’s repositories (i.e. with every commit from a normal subversion user, the webserver-local checkout is updated by a post-commit later on):
Line 59: Line 64:
 
  sudo htpasswd /etc/apache2/dav_svn.passwd www-data  
 
  sudo htpasswd /etc/apache2/dav_svn.passwd www-data  
 
  <PASSWORD>
 
  <PASSWORD>
  sudo -s #  -s (shell)  
+
  # sudo -s #  -s (shell)  
  # chown -R www-data.www-data /var/www/v-artenquiz
+
  # sudo chown -R www-data.www-data /var/www/v-artenquiz
  # www-data needs a folder to store svn-configuration (esp. a plaintext password needs to be set and then stored):
+
  # Note: www-data needs a folder to store svn-configuration (esp. a plaintext password needs to be set and then stored):
 
  sudo mkdir /var/www/.subversion
 
  sudo mkdir /var/www/.subversion
 
  sudo chown www-data.www-data /var/www/.subversion/
 
  sudo chown www-data.www-data /var/www/.subversion/
  
  sudo mv /var/www/v-artenquiz /var/www/v-artenquiz-DELETE-ME-LATER
+
# Perhaps, if existing, safety first:
 +
# sudo mv /var/www/v-artenquiz /var/www/v-artenquiz-DELETE-ME-LATER
 
  sudo mkdir /var/www/v-artenquiz; cd /var/www/v-artenquiz
 
  sudo mkdir /var/www/v-artenquiz; cd /var/www/v-artenquiz
 +
# svn checkout URL local-target
 
  sudo svn checkout file:///var/lib/svn/artenquiz /var/www/v-artenquiz
 
  sudo svn checkout file:///var/lib/svn/artenquiz /var/www/v-artenquiz
 
  sudo chown -R  www-data.www-data /var/www/v-artenquiz
 
  sudo chown -R  www-data.www-data /var/www/v-artenquiz
 
+
  ##NOTE: BELOW WAS OBSOLETE CODE THAT REFERRED TO OLD SVN mediawiki checkout structure. Removed!
  sudo mv    /usr/share/mediawiki/ext-LOCAL-svn /usr/share/mediawiki/ext-LOCAL-svn-DELETE-ME-LATER
+
sudo mkdir /usr/share/mediawiki/ext-LOCAL-svn; cd /usr/share/mediawiki/ext-LOCAL-svn;
+
sudo svn checkout file:///var/lib/svn/LocalSVNextensions /usr/share/mediawiki/ext-LOCAL-svn
+
sudo chown -R  www-data.www-data /usr/share/mediawiki/ext-LOCAL-svn
+
 
+
sudo mv    /usr/share/mediawikistaging/ext-LOCAL-svn /usr/share/mediawikistaging/ext-LOCAL-svn-DELETE-ME-LATER
+
sudo mkdir /usr/share/mediawikistaging/ext-LOCAL-svn; cd /usr/share/mediawikistaging/ext-LOCAL-svn;
+
sudo svn checkout file:///var/lib/svn/LocalSVNextensions /usr/share/mediawikistaging/ext-LOCAL-svn
+
sudo chown -R  www-data.www-data /usr/share/mediawikistaging/ext-LOCAL-svn
+
 
+
# new for the 1.20 git version with new folder arrangement (this may be temporary, check later)
+
# first check out into separate folder, then change owner, then move to final extensions folder shared with the git-clone
+
cd /usr/share/mediawiki20/phase3/
+
sudo mkdir ext-LOCAL-svn; cd ext-LOCAL-svn;
+
sudo svn checkout file:///var/lib/svn/LocalSVNextensions /usr/share/mediawiki20/phase3/ext-LOCAL-svn
+
sudo chown -R  www-data.www-data /usr/share/mediawiki20/phase3/ext-LOCAL-svn
+
sudo mv /usr/share/mediawiki20/phase3/ext-LOCAL-svn/* /usr/share/mediawiki20/phase3/extensions
+
sudo mv /usr/share/mediawiki20/phase3/ext-LOCAL-svn/.svn /usr/share/mediawiki20/phase3/extensions
+
sudo rm /usr/share/mediawiki20/phase3/ext-LOCAL-svn -r
+
 
</source>
 
</source>
  
Line 95: Line 83:
 
  sudo nano /var/lib/svn/hooks/post-commit
 
  sudo nano /var/lib/svn/hooks/post-commit
 
</source>
 
</source>
with:
+
 
 
<source lang="bash">
 
<source lang="bash">
#!/bin/sh
+
##(no longer direct code in wiki, refer to backup of current file)##
REPOS="$1"
+
REV="$2"
+
HOME=/var/www # define HOME for settings in $HOME/.subversion
+
cd /var/www/v-artenquiz;                      /usr/bin/svn update
+
cd /usr/share/mediawiki/ext-LOCAL-svn;        /usr/bin/svn update
+
cd /usr/share/mediawikistaging/ext-LOCAL-svn; /usr/bin/svn update
+
 
</source>
 
</source>
  
Line 112: Line 94:
  
 
By means of this post-commit-hook, whenever a commit to SVN occurs, www/v-artenquiz is automatically updated as well!
 
By means of this post-commit-hook, whenever a commit to SVN occurs, www/v-artenquiz is automatically updated as well!
 
==SVN repo on apache behind nginx==
 
 
Changes made to dav_svn.conf: add right before "<Location /svn>"
 
## NECESSARY MODIFICATION FOR APACHE BEHIND NGINX (rev. proxy):
 
RequestHeader edit Destination ^https http early
 
Note: Normal commits would work without this, but renames of folders caused "Server sent unexpected return value (502 Bad Gateway) in response to COPY". Also needs headers enabled:
 
a2enmod headers
 
 
 
There was nothing about SVN in our sites-available/default anyways. Setting in nginx /etc/nginx/sites-available/default:
 
  location /svn {
 
      proxy_set_header X-Real-IP      $remote_addr;
 
      proxy_set_header X-Forwarded-For $remote_addr;
 
      proxy_set_header Host            $host;
 
      proxy_pass http://127.0.0.1:8880; # = APACHE2 Prefork
 
  }
 
 
Initially there was a problem with the existing checkouts on the server itself, this could be solved by checking out again, using file: instead of https: (e.g. svn checkout file:///var/lib/svn/LocalSVNextensions).
 
  
 
== Adding new directories ==
 
== Adding new directories ==
  
Adding new directories to the repository, the easiest way is to use a GUI like TortoiseSVN (Windows) or kdesvn (Linux KDE).
+
The easiest way to add new directories or files to the repository is to use a GUI like TortoiseSVN (Windows) or kdesvn (Linux KDE).
# check out the repository
+
# Check out the repository
# add or copy your folders you wish to add to the repository and add them to subversion control ''first'' (now subversion add them to version control)
+
# Add or copy the folders you wish to add to the repository and add them to subversion control locally using the local GUI
# commit those new files  or folders (needs a [[#addSubversionUsers|user account]] in dav)
+
# Commit your changes to the main repository (needs a [[#addSubversionUsers|user account]] in dav)  
  
An alternative is to use svn import in command line with
+
An alternative is to use svn import on command line with
 
<source lang="bash">
 
<source lang="bash">
  sudo svn import LocalSVNextensions file:///var/lib/svn/LocalSVNextensions -m 'import of mediawiki LocalSVNextensions'
+
sudo svn import LocalSVNextensions file:///var/lib/svn/LocalSVNextensions -m 'import of mediawiki LocalSVNextensions'
 
</source>
 
</source>
  
 
+
== Adding a log-edit feature ==
 
+
== Adding log edit feature ==
+
  
 
<source lang="bash" >
 
<source lang="bash" >
Line 164: Line 125:
 
</source>
 
</source>
 
See also http://subversion.apache.org/faq.html#change-log-msg.
 
See also http://subversion.apache.org/faq.html#change-log-msg.
 +
 +
==SVN repo on apache behind nginx==
 +
 +
Changes made to dav_svn.conf: add right before "<Location /svn>"
 +
## NECESSARY MODIFICATION FOR APACHE BEHIND NGINX (rev. proxy):
 +
RequestHeader edit Destination ^https http early
 +
Note: Normal commits would work without this, but renames of folders caused "Server sent unexpected return value (502 Bad Gateway) in response to COPY". This requires the headers mod to be enabled:
 +
a2enmod headers
 +
 +
There was nothing about SVN in our sites-available/default anyways. Setting in nginx /etc/nginx/sites-available/default:
 +
 +
  # SVN/GIT need ^~ to have priority over regexp with *.php!
 +
  location ^~ /svn {
 +
      proxy_set_header X-Real-IP      $remote_addr;
 +
      proxy_set_header X-Forwarded-For $remote_addr;
 +
      proxy_set_header Host            $host;
 +
      proxy_pass http://127.0.0.1:8880; # = APACHE2 Prefork
 +
  }
 +
 +
Note: with "location /svn" instead of  "location ^~ /svn" it would work with most files, but not with *.php files. PHP files would be caught by a regexp location (not shown), which has priority over normal nginx starts-with directives. The error message was: "Commit failed (details follow): File '/home/...i18n.php' is out of date '/svn/!svn/bc/2423/.../i18n.php' path not found. You have to update your working copy first. (Or German: "Übertragen schlug fehl (Details folgen): Datei »/home/.../i18n.php« ist veraltet. »/svn/!svn/bc/2423/.../i18n.php«. Pfad nicht gefunden."). This would occur for commits immediately after successful updates. Using a priority-start-with (which has priority over regexp locations) solved the problem.
 +
 +
Initially there also was a problem with checkouts on the server itself, this could be solved by checking these out again, using file: instead of https: (e.g. svn checkout file:///var/lib/svn/LocalSVNextensions).
 +
 
[[Category: Subversion]]
 
[[Category: Subversion]]
 +
[[Category:User management]]

Latest revision as of 16:53, 28 January 2015

This topic is about creating and configuring the repository on the server itself. For help with using or managing subversion, see Help: Subversion.

Configuration

Configuring a subversion repository, with svn already installed through apt-get, on Debian 5. The procedure was contributed by an anonymous helper. First: in /etc/apache2/mods-enabled/dav_svn.conf enable several lines:

<Location /svn>
DAV svn
SVNPath /var/lib/svn
AuthType Basic
AuthName "Subversion Repository"
AuthUserFile /etc/apache2/dav_svn.passwd
Require valid-user
AuthzSVNAccessFile /etc/apache2/dav_svn.authz
</Location>

Password file and users

 sudo htpasswd -c /etc/apache2/dav_svn_git.passwd user ## only for first user
 sudo htpasswd /etc/apache2/dav_svn_git.passwd username1
 sudo htpasswd /etc/apache2/dav_svn_git.passwd username2 ## etc.

(Password will be requested by htpasswd)

Create access control file and reload:

 sudo nano /etc/apache2/dav_svn.authz
 # Read/write permission for all SVN users:
 [/]
 * = rw
 sudo /etc/init.d/apache2 reload

SVN Web Access

Creating SVN-Repository, access only to Webserver:

 sudo mkdir /var/lib/svn
 sudo svnadmin create /var/lib/svn/
 sudo chown -R www-data.www-data /var/lib/svn/

Now one can check versions directly inside the browser using https://biowikifarm.net/svn/ or check locally out using a svn client:

 svn co https://biowikifarm.net/svn/


SVN post-commit-hook

To increase comfort we can ensure that committing to the webserver the most recent version is updated also in server’s repositories (i.e. with every commit from a normal subversion user, the webserver-local checkout is updated by a post-commit later on):

 # set password for www-data, use a random password, it will be stored plain text:
 sudo htpasswd /etc/apache2/dav_svn.passwd www-data 
 <PASSWORD>
 # sudo -s #  -s (shell) 
 # sudo chown -R www-data.www-data /var/www/v-artenquiz
 # Note: www-data needs a folder to store svn-configuration (esp. a plaintext password needs to be set and then stored):
 sudo mkdir /var/www/.subversion
 sudo chown www-data.www-data /var/www/.subversion/

 # Perhaps, if existing, safety first: 
 #  sudo mv /var/www/v-artenquiz /var/www/v-artenquiz-DELETE-ME-LATER
 sudo mkdir /var/www/v-artenquiz; cd /var/www/v-artenquiz
 # svn checkout URL local-target
 sudo svn checkout file:///var/lib/svn/artenquiz /var/www/v-artenquiz
 sudo chown -R  www-data.www-data /var/www/v-artenquiz
 ##NOTE: BELOW WAS OBSOLETE CODE THAT REFERRED TO OLD SVN mediawiki checkout structure. Removed!

Continuing as normal user, edit/create file

 sudo nano /var/lib/svn/hooks/post-commit
 ##(no longer direct code in wiki, refer to backup of current file)##

… and set this file as executable:

 sudo chmod +x /var/lib/svn/hooks/post-commit

By means of this post-commit-hook, whenever a commit to SVN occurs, www/v-artenquiz is automatically updated as well!

Adding new directories

The easiest way to add new directories or files to the repository is to use a GUI like TortoiseSVN (Windows) or kdesvn (Linux KDE).

  1. Check out the repository
  2. Add or copy the folders you wish to add to the repository and add them to subversion control locally using the local GUI
  3. Commit your changes to the main repository (needs a user account in dav)

An alternative is to use svn import on command line with

 sudo svn import LocalSVNextensions file:///var/lib/svn/LocalSVNextensions -m 'import of mediawiki LocalSVNextensions'

Adding a log-edit feature

cd /var/lib/svn/hooks/
# use templates provided by subversion
sudo cp  pre-revprop-change.tmpl pre-revprop-change 
# make it executable
sudo chmod +x pre-revprop-change

Now you are able to edit/or set log entries from a specific subversion revision number -r by:

1 cd /path/to/versioned/directory/
2 # edit log entry with editor
3 sudo svn propedit -r123 --revprop svn:log
4 # set log entry
5 sudo svn propset -r123 --revprop svn:log "new log message"

See also http://subversion.apache.org/faq.html#change-log-msg.

SVN repo on apache behind nginx

Changes made to dav_svn.conf: add right before "<Location /svn>"

## NECESSARY MODIFICATION FOR APACHE BEHIND NGINX (rev. proxy):
RequestHeader edit Destination ^https http early 

Note: Normal commits would work without this, but renames of folders caused "Server sent unexpected return value (502 Bad Gateway) in response to COPY". This requires the headers mod to be enabled:

a2enmod headers

There was nothing about SVN in our sites-available/default anyways. Setting in nginx /etc/nginx/sites-available/default:

 # SVN/GIT need ^~ to have priority over regexp with *.php!
 location ^~ /svn {
     proxy_set_header X-Real-IP       $remote_addr;
     proxy_set_header X-Forwarded-For $remote_addr;
     proxy_set_header Host            $host;
     proxy_pass http://127.0.0.1:8880; # = APACHE2 Prefork
 }

Note: with "location /svn" instead of "location ^~ /svn" it would work with most files, but not with *.php files. PHP files would be caught by a regexp location (not shown), which has priority over normal nginx starts-with directives. The error message was: "Commit failed (details follow): File '/home/...i18n.php' is out of date '/svn/!svn/bc/2423/.../i18n.php' path not found. You have to update your working copy first. (Or German: "Übertragen schlug fehl (Details folgen): Datei »/home/.../i18n.php« ist veraltet. »/svn/!svn/bc/2423/.../i18n.php«. Pfad nicht gefunden."). This would occur for commits immediately after successful updates. Using a priority-start-with (which has priority over regexp locations) solved the problem.

Initially there also was a problem with checkouts on the server itself, this could be solved by checking these out again, using file: instead of https: (e.g. svn checkout file:///var/lib/svn/LocalSVNextensions).