Difference between revisions of "Debian installation"
(→Tips for simple access of Debian file from remote Windows) |
(→General installations) |
||
(98 intermediate revisions by 5 users not shown) | |||
Line 1: | Line 1: | ||
− | + | (Part of the [[:Category:Software documentation|Software documentation]] of biowikifarm.) | |
− | + | This is a record of the installation and later updates of a Debian system (originally in version 4, later upgraded, see "[[Debian upgrade 4 to 5]]"). The page is intended to help running or repeating the server setup. In parts it may be outdated by subsequent configuration changes and software updates. Some notes may be updated, others not. However, much of the fundamental setup will remain evident from this document. | |
− | + | We use XenServer Express (see [[Server virtualization]]) and originally installed the Xen template for Debian 4.0 (kernel modified for paravirtualization) with the standard setup. This is a documentation of the steps performed to get the Debian server working. The notes have been updated since, now reflecting a debian 6 setup. | |
− | + | ||
− | + | ||
+ | See also: [[Biowikifarm server hardware]]. | ||
== Basics == | == Basics == | ||
− | The server can be reached at ''' | + | The server can be reached at '''212.201.100.117''' with ssh enabled. Settings for ssh: |
* /etc/ssh/ssh_config default settings for ssh when used as client | * /etc/ssh/ssh_config default settings for ssh when used as client | ||
* /etc/ssh/sshd_config default settings for ssh as server | * /etc/ssh/sshd_config default settings for ssh as server | ||
+ | ** use <code>nano /etc/ssh/sshd_config</code> to set<br>PermitRootLogin no<br>PasswordAuthentication no<br>UsePAM no<br>= disallowing root access through ssh and limiting access to ssh pubkey infrastructure. | ||
+ | ** use <code>nano /etc/pam.d/common-password</code> to increase minimum password length:<br>"password [success=1 default=ignore] pam_unix.so obscure sha512 min=11 max=30 nullok retry=3" | ||
+ | |||
+ | ** check <code>lastlog</code> to see last login of users | ||
Current port and firewall situation: See [[Server ports and shorewall installation]]. | Current port and firewall situation: See [[Server ports and shorewall installation]]. | ||
Line 20: | Line 23: | ||
=== Partitioning and formatting additional storage === | === Partitioning and formatting additional storage === | ||
+ | '''NOTE: updated information for debian6 on xen6. Swap is here in main.''' | ||
− | Xen will partition and format the first | + | Xen will partition and format the first hard drive (main = system and swap), but any additional, medium needs to be partitioned manually. The name of the 2./3 virtual hard disks under XEN is xvdb/xvdc. Run: |
− | fdisk /dev/xvdc | + | sudo fdisk /dev/xvdb |
− | # | + | sudo fdisk /dev/xvdc |
+ | # type n for new partition, p for primary, 1 for first partition, | ||
# accept defaults for size, type w to write out new partition. | # accept defaults for size, type w to write out new partition. | ||
− | # To format with | + | # To format with ext4 filesystem: |
− | mkfs. | + | sudo mkfs.ext4 /dev/xvdb1 |
− | + | sudo mkfs.ext4 /dev/xvdc1 | |
+ | sudo mkfs.ext4 /dev/xvde1 | ||
# To label partitions: | # To label partitions: | ||
− | e2label /dev/xvda1 root | + | sudo e2label /dev/xvda1 root |
− | e2label /dev/ | + | sudo e2label /dev/xvdb1 storage |
− | e2label /dev/ | + | sudo e2label /dev/xvdc1 dump |
+ | sudo e2label /dev/xvde1 BIG | ||
− | Note that the | + | Note that xvdd1 is the CD/DVD-ROM! It is under '''/media/cdrom0''', not under '''/mnt/cdrom0'''! |
− | Correct function of partitioning can be tested, e.g. using webmin (Hardware, Partitions on Local disks). To permanently mount, | + | Correct function of partitioning can be tested, e.g. using webmin (Hardware, Partitions on Local disks). To permanently mount, do |
− | /dev/ | + | sudo nano /etc/fstab |
− | /dev/ | + | #then edit: |
− | # to mount temporarily use something like: "mount /dev/ | + | /dev/xvdb1 /mnt/storage ext3 rw,noatime 0 0 |
+ | /dev/xvdc1 /mnt/dump ext3 rw,noatime 0 0 | ||
+ | # to mount temporarily use something like: "sudo mount /dev/xvde1 /mnt/temp" | ||
+ | # to mount BIG: "sudo mount /dev/xvde1 /mnt/BIG" | ||
− | + | Check swap with | |
− | + | ||
− | + | ||
− | + | ||
− | + | ||
cat /proc/swaps | cat /proc/swaps | ||
+ | free | ||
+ | (Result: /dev/xvda5!) | ||
+ | |||
+ | Note: in 2014 filesystem was enlarged and changed to ext4. | ||
=== Resizing virtual disks === | === Resizing virtual disks === | ||
Line 51: | Line 61: | ||
# The disk needs to be deactivated in XenCenter before resizing. | # The disk needs to be deactivated in XenCenter before resizing. | ||
# Resizing the virtual storage unit does not change the size of the partition under Linux! | # Resizing the virtual storage unit does not change the size of the partition under Linux! | ||
− | # Method 1 (manual): if the disk contains only relatively easily migrated data (no dev/ports, etc.), copy all content to another disk using <code>cp -pr</code> (preserving owner, permission, recursive) or <code>tar -p -s</code>. | + | # Method 1 (manual): if the disk contains only relatively easily migrated data (no dev/ports, etc.), create a new disk, partition and format as above, temporarily mount: sudo mount /dev/xvde1 /mnt/temp and copy all content to another disk using <code>sudo cp -pr</code> (preserving owner, permission, recursive) or <code>tar -p -s</code>. In XenCenter stop VM, remove previous disk (first detach, delete only later), change order of volumes, then restart. |
+ | #* NOTE: Vol. swapping when VM is down; use the upper disk number in XenCenter, set to the same number as the lower number equivalent disk, warning will occur and offer to swap volumes. Trying to avoid this manually results in multiple reboot cycles, because the disk number is no longer available! | ||
+ | # after: /dev/xvdX1 /mnt/storage2; sudo mount /dev/xvdX1 /mnt/dump2 | ||
+ | sudo rm /mnt/storage2/* -r; sudo rm /mnt/dump2/* -r; | ||
+ | sudo cp -pr /mnt/storage/* /mnt/storage2; | ||
+ | # cannot put the following into single line, sudo will have timed out! | ||
+ | # perhaps remove the DAILY backup from dump, which reduces copy time, then copy: | ||
+ | sudo cp -pr /mnt/dump/* /mnt/dump2; | ||
+ | |||
# Method 2 (better, tip Manol, not yet tested): | # Method 2 (better, tip Manol, not yet tested): | ||
#* install parted: <code>apt-get install parted</code> or ''gparted'' (graphical frontend) | #* install parted: <code>apt-get install parted</code> or ''gparted'' (graphical frontend) | ||
#* unmount partition and check with parted | #* unmount partition and check with parted | ||
− | #* OR: install <code>apt-get install dump</code> (dump - 4.4bsd dump and restore for ext2 filesystems) and use this to store | + | #* OR: install <code>apt-get install dump</code> (dump - 4.4bsd dump and restore for ext2 filesystems) and use this to store (somthing is missing here, not tested). |
=== Network, Hostname and IP address === | === Network, Hostname and IP address === | ||
Line 61: | Line 79: | ||
==== First installation ==== | ==== First installation ==== | ||
− | + | Hostnames are in several places, especially important when renaming a server: | |
− | + | sudo nano /etc/hostname # entered: biowikifarm (old) biowiki (new) | |
− | + | sudo nano /etc/mailname # entered: biowikifarm.net | |
− | + | sudo nano /etc/hosts # 212.201.100.117 biowikifarm.net biowikifarm | |
− | 160.45.63. | + | sudo nano /etc/hosts # 160.45.63.58 biowikifarm.net biowiki |
Open "nano /etc/network/interfaces" and enter: | Open "nano /etc/network/interfaces" and enter: | ||
− | # | + | NEW: |
− | # | + | # This file describes the network interfaces available on your system |
+ | # and how to activate them. For more information, see interfaces(5). | ||
+ | |||
# The loopback network interface | # The loopback network interface | ||
auto lo | auto lo | ||
Line 75: | Line 95: | ||
# The primary network interface | # The primary network interface | ||
− | + | allow-hotplug eth0 | |
iface eth0 inet static | iface eth0 inet static | ||
− | + | address 192.168.101.117 | |
− | + | netmask 255.255.255.0 | |
− | + | network 192.168.101.0 | |
− | + | #broadcast 192.168.101.255 | |
− | + | gateway 192.168.101.1 | |
− | + | # dns-* options are implemented by the resolvconf package, if installed | |
− | + | dns-nameservers 192.168.201.16 192.168.201.11 192.168.201.12 | |
+ | up ip addr add 212.201.100.117/32 dev $IFACE label $IFACE:0 | ||
+ | down ip addr del 212.201.100.117/32 dev $IFACE label $IFACE:0 | ||
+ | |||
# The following lines are desirable for IPv6 capable hosts | # The following lines are desirable for IPv6 capable hosts | ||
::1 ip6-localhost ip6-loopback | ::1 ip6-localhost ip6-loopback | ||
Line 92: | Line 115: | ||
ff02::3 ip6-allhosts | ff02::3 ip6-allhosts | ||
− | + | ||
+ | Note: for future IP changes, also check: | ||
+ | * /etc/resolv.conf | ||
+ | * /etc/apache2/sites-available/default | ||
+ | * /etc/hosts | ||
+ | * /etc/networks | ||
+ | * also often the mail configuration in the password file has to be updated (path here not shown by purpose) | ||
+ | |||
Reboot now (first command) and run the following commands (the two hostname commands verify that the hostname setting is ok): | Reboot now (first command) and run the following commands (the two hostname commands verify that the hostname setting is ok): | ||
− | reboot | + | sudo reboot |
− | hostname | + | hostname # verify |
− | hostname -f | + | hostname -f # verify |
− | apt-get update | + | sudo apt-get update; sudo apt-get upgrade; |
− | + | sudo apt-get dist-upgrade | |
− | apt-get dist-upgrade | + | sudo apt-get install ssh sudo |
It may be good to repeat this, some updates came only in second round. Finally, we may want to do some cleanup: | It may be good to repeat this, some updates came only in second round. Finally, we may want to do some cleanup: | ||
− | apt-get autoclean | + | sudo apt-get autoclean |
After doing this, the ssh fingerprints may have to be changed on all machines trying to access the new Debian by ssh (security issues fixed). Remove the keys for the previously opened IPs in the following file: | After doing this, the ssh fingerprints may have to be changed on all machines trying to access the new Debian by ssh (security issues fixed). Remove the keys for the previously opened IPs in the following file: | ||
Line 110: | Line 140: | ||
==== Related: Reconfiguring a copied virtual machine ==== | ==== Related: Reconfiguring a copied virtual machine ==== | ||
− | Before starting a copied VM check the MAC adress in XEN to avoid running accidentially two servers under the same IP. A copied VM may use other interfaces instead of eth0, e.g., eth2. Thus first use ifconfig -a to find the name of the interface, then change interface name AND IP-number in network AND shorewall interface files. Also edit hostname and hosts, and restart shorewall: | + | Before starting a copied VM check the MAC adress in XEN to avoid running accidentially two servers under the same IP. A copied VM may or may not use other interfaces instead of eth0, e.g., eth2. Thus first use ifconfig -a to find the name of the interface, then change interface name AND IP-number in network AND shorewall interface files. Use the interfaces shown there. Also edit hostname and hosts, and restart shorewall. (Note: .57 is good, .58 bad...). |
− | ifconfig -a | + | sudo ifconfig -a | more |
− | nano /etc/network/interfaces | + | sudo nano /etc/network/interfaces # interface and IP |
− | nano /etc/shorewall/interfaces # ( | + | sudo nano /etc/shorewall/interfaces # (interface at the bottom, check, update if necessary) |
− | nano /etc/hostname | + | sudo nano /etc/hostname |
− | nano /etc/hosts | + | sudo nano /etc/hosts |
− | /etc/init.d/shorewall restart | + | sudo /etc/init.d/shorewall restart |
− | + | ||
− | + | ||
+ | (Note: if due to a misconfiguration the shorewall continuously displays blocking messages on the console, try Alt-F2, user name and password. Type blindly, despite repeated shorewall messages. "shorewall stop" will stop shorewall (type blindly).) | ||
=== sudoers === | === sudoers === | ||
− | + | sudo nano /etc/sudoers | |
− | + | # then check wether (this is the default of debian6 or xen template): | |
%sudo ALL=(ALL) ALL | %sudo ALL=(ALL) ALL | ||
This lets all users of group ''sudo'' run any command as ''root'' without knowing the root password. | This lets all users of group ''sudo'' run any command as ''root'' without knowing the root password. | ||
Line 134: | Line 163: | ||
deb http://ftp.debian.org/debian/ etch main non-free contrib | deb http://ftp.debian.org/debian/ etch main non-free contrib | ||
deb http://www.backports.org/debian etch-backports main | deb http://www.backports.org/debian etch-backports main | ||
− | + | # Necessary for webmin (sarge is still the correct version for squeeze!!): | |
− | + | ||
− | + | ||
− | # Necessary for webmin: | + | |
deb http://download.webmin.com/download/repository sarge contrib | deb http://download.webmin.com/download/repository sarge contrib | ||
+ | # added for oggvideotools (June 13 2010) | ||
+ | deb http://www.debian-multimedia.org lenny main | ||
− | The last command adds a gpg security key for the backport repository. <small>(Documentation of further testing: We also tested the Google repository as an example. Using root console, we first had to get a pgp key (note: the original google instructions requested using https, but only http worked): <code>wget -q http://dl-ssl.google.com/linux/linux_signing_key.pub -O- | apt-key add -</code>, and <code>add deb http://dl.google.com/linux/deb/ stable non-free</code>.)</small> | + | LONGER WORKING: To install keys: wget -O - http://backports.org/debian/archive.key | apt-key add - ; sudo apt-get install debian-backports-keyring; --- The last command adds a gpg security key for the backport repository. <small>(Documentation of further testing: We also tested the Google repository as an example. Using root console, we first had to get a pgp key (note: the original google instructions requested using https, but only http worked): <code>wget -q http://dl-ssl.google.com/linux/linux_signing_key.pub -O- | apt-key add -</code>, and <code>add deb http://dl.google.com/linux/deb/ stable non-free</code>.)</small> |
+ | |||
+ | # more on clamav and rkhunter below, but install early: | ||
+ | sudo apt-get install clamav rkhunter | ||
===JAVA=== | ===JAVA=== | ||
Debian does not normally install Sun Java, but a slower "free Java". It is possible to install sun software after modifying the repository sources to include non-free software and backports (see here for more information: http://www.debian.org/doc/manuals/debian-java-faq/ch6.html): | Debian does not normally install Sun Java, but a slower "free Java". It is possible to install sun software after modifying the repository sources to include non-free software and backports (see here for more information: http://www.debian.org/doc/manuals/debian-java-faq/ch6.html): | ||
− | apt-get install sun-java6-jre | + | sudo apt-get install sun-java6-jre |
The install location is /usr/lib/jvm/java-6-sun/. Sun recommends to update the Debian "alternatives system" to have Sun's tools as the default: | The install location is /usr/lib/jvm/java-6-sun/. Sun recommends to update the Debian "alternatives system" to have Sun's tools as the default: | ||
− | update-java-alternatives -s java-6-sun | + | sudo update-java-alternatives -s java-6-sun |
− | Then edit /etc/profile | + | Then edit |
− | export JAVA_HOME=/usr/lib/jvm/java-6-sun | + | sudo nano /etc/profile |
+ | |||
+ | # and add at the top, before first "if": | ||
+ | export JAVA_HOME=/usr/lib/jvm/java-6-sun | ||
export JRE_HOME=/usr/lib/jvm/java-6-sun/jre | export JRE_HOME=/usr/lib/jvm/java-6-sun/jre | ||
− | PATH=$JAVA_HOME/bin:$PATH | + | export CATALINA_HOME=/usr/share/fedora/tomcat |
+ | export FEDORA_HOME=/usr/share/fedora | ||
+ | |||
+ | # and add further down, after last "fi": | ||
+ | PATH=$JAVA_HOME/bin:$FEDORA_HOME/server/bin:$FEDORA_HOME/client/bin:$PATH | ||
+ | export PATH | ||
+ | umask 022 | ||
+ | HISTFILE=~/.bash_history | ||
+ | HISTSIZE=10000 | ||
+ | HISTFILESIZE=999999 | ||
+ | readonly HISTFILE | ||
+ | readonly HISTSIZE | ||
+ | readonly HISTFILESIZE | ||
+ | export HISTFILE HISTSIZE HISTFILESIZE | ||
+ | |||
(The last line before "export PATH"; this export is probably only necessary (e.g. for Tomcat, see below) if an earlier version of Java is still running and is located in a path that is looked up first by the system.) | (The last line before "export PATH"; this export is probably only necessary (e.g. for Tomcat, see below) if an earlier version of Java is still running and is located in a path that is looked up first by the system.) | ||
Line 160: | Line 208: | ||
echo $JRE_HOME | echo $JRE_HOME | ||
echo $PATH | echo $PATH | ||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
---- | ---- | ||
− | |||
− | |||
==MySQL, Apache PHP, etc.== | ==MySQL, Apache PHP, etc.== | ||
− | ===General installations=== | + | ===General installations (2017) === |
− | apt-get install gcc cpp libtool binutils make autoconf automake1.9 flex | + | sudo apt-get install gcc cpp libtool binutils make autoconf automake1.9 flex tree locate |
− | apt-get install zip unzip rar unrar p7zip p7zip-full | + | sudo apt-get install zip unzip rar unrar p7zip p7zip-full |
− | + | ||
− | + | ||
− | + | ||
To use MediaWiki with high traffic, a memcached memory cache should be installed prior to installing MediaWiki: | To use MediaWiki with high traffic, a memcached memory cache should be installed prior to installing MediaWiki: | ||
− | |||
apt-get install memcached php5-memcache | apt-get install memcached php5-memcache | ||
− | Without further configuration, a single memcached | + | Without further configuration, a single memcached will be active on default port 11211, limited to localhost (test from outside using: telnet 212.201.100.117 11211), using up to 64 MB of your RAM. |
− | This can ''not'' be changed by editing <code>/etc/init.d/memcached</code>, but <code>/etc/memcached.config</code> must be modified. We increased size: | + | This can ''not'' be changed by editing <code>/etc/init.d/memcached</code>, but <code>/etc/memcached.config</code> must be modified. We increased size: -m 512 = MByte, -l 127.0.0.1 = listen only on local loopback interface. |
− | + | Stats on memcached use, using netcat (nc, see https://lzone.de/cheat-sheet/memcached): | |
− | + | echo stats | nc 127.0.0.1 11211 | |
− | + | echo stats slabs | nc 127.0.0.1 11211 | |
+ | # just for number of items, shown at end: echo stats items | nc 127.0.0.1 11211 | ||
===MySQL=== | ===MySQL=== | ||
− | apt-get install mysql-server mysql-client libmysqlclient15-dev | + | '''Installation''' |
− | nano /etc/mysql/my.cnf | + | <blockquote> |
− | # | + | <source lang="bash"> |
− | + | # debian5: sudo apt-get install mysql-server mysql-client libmysqlclient15-dev | |
+ | # on debian6 we used: | ||
+ | sudo apt-get install mysql-server mysql-client libmysqlclient-dev | ||
+ | |||
+ | sudo nano /etc/mysql/my.cnf | ||
+ | # making many changes, use a merge with a current config file | ||
/etc/init.d/mysql restart | /etc/init.d/mysql restart | ||
+ | </source> | ||
+ | </blockquote> | ||
+ | |||
+ | <small>Under Debian 6, We experienced a broken apt-get after an "apt-get remove mysql-server mysql-client libmysqlclient-dev" and then "apt-get install --reinstall mysql-server mysql-client libmysqlclient-dev" and removing the content of /var/lib/mysql (to start fresh over, after the first dump import failed). In the end, removing further parts or mysql AND manually installing mysql-server-core-5.1 fixed the broken installation situation!</small> | ||
'''Console commands''' | '''Console commands''' | ||
Line 208: | Line 250: | ||
Test with netstat (debian seems to report 8088 as omniorb) and set a password for root (otherwise anybody can access your MySQL database!): | Test with netstat (debian seems to report 8088 as omniorb) and set a password for root (otherwise anybody can access your MySQL database!): | ||
+ | <blockquote> | ||
+ | <source lang="bash"> | ||
netstat -tap | netstat -tap | ||
# -u = login as this user | # -u = login as this user | ||
mysqladmin -u root password (ThePassword) | mysqladmin -u root password (ThePassword) | ||
− | + | </source> | |
− | + | </blockquote> | |
New users can be created and granted privileges inside the mysql console: | New users can be created and granted privileges inside the mysql console: | ||
+ | <blockquote> | ||
+ | <source lang="bash"> | ||
# enter the mysql console (asks for passwort): | # enter the mysql console (asks for passwort): | ||
mysql -u root -p | mysql -u root -p | ||
− | + | </source> | |
+ | <source lang="mysql"> | ||
+ | -- within the console (mysql> ): | ||
grant all privileges on *.* to 'username'@'localhost' identified by 'PASSWORD' with grant option; | grant all privileges on *.* to 'username'@'localhost' identified by 'PASSWORD' with grant option; | ||
grant all privileges on *.* to 'username'@'%' identified by 'PASSWORD' with grant option; | grant all privileges on *.* to 'username'@'%' identified by 'PASSWORD' with grant option; | ||
− | + | -- to exit the mysql console type: | |
exit | exit | ||
− | + | </source> | |
+ | </blockquote> | ||
Notes: a) grant will create a user if it does not exist yet. b) username, hostname, PASSWORD must be quoted with '. c) the '%' allows the user to log in from any other host except localhost; an explicit grant for localhost is thus required in addition to the '%'. | Notes: a) grant will create a user if it does not exist yet. b) username, hostname, PASSWORD must be quoted with '. c) the '%' allows the user to log in from any other host except localhost; an explicit grant for localhost is thus required in addition to the '%'. | ||
− | |||
− | ''' | + | '''Changing password for existing users:''' |
+ | <blockquote> | ||
+ | <source lang="bash"> | ||
+ | # enter the mysql console (asks for passwort): | ||
+ | mysql -u root -p | ||
+ | </source> | ||
+ | <source lang="mysql"> | ||
+ | -- within the console (mysql> ): | ||
+ | set password = PASSWORD('NEW-PASSWORD-HERE'); | ||
+ | -- Alternative, for a different user: | ||
+ | set password FOR 'name'@'localhost' = PASSWORD('NEW-PASSWORD-HERE'); | ||
+ | </source> | ||
+ | </blockquote> | ||
+ | |||
+ | '''Some other mysql console commands are:''' | ||
+ | <blockquote> | ||
+ | <source lang="mysql"> | ||
show databases; | show databases; | ||
use mysql; Select * from user; | use mysql; Select * from user; | ||
drop user root@OudemansD; | drop user root@OudemansD; | ||
+ | </source> | ||
+ | </blockquote> | ||
''Important note:'' entering mysql command line with <code>"mysql -u wikiuser -p"</code> will ask for password; if you want to include the password (e.g. when using pipe into) ''do not add a blank between p and the password''. Also, when using the -h host option, only -hlocalhost, but not -hlocalhost:8088 will work! The host can usually be omitted. See the following examples: | ''Important note:'' entering mysql command line with <code>"mysql -u wikiuser -p"</code> will ask for password; if you want to include the password (e.g. when using pipe into) ''do not add a blank between p and the password''. Also, when using the -h host option, only -hlocalhost, but not -hlocalhost:8088 will work! The host can usually be omitted. See the following examples: | ||
+ | <blockquote> | ||
+ | <source lang="bash"> | ||
mysql -uwikiuser -pTHEPASSWORD | mysql -uwikiuser -pTHEPASSWORD | ||
mysql -hlocalhost -u wikiuser -pTHEPASSWORD < openid_table.sql | mysql -hlocalhost -u wikiuser -pTHEPASSWORD < openid_table.sql | ||
+ | </source> | ||
+ | </blockquote> | ||
− | + | '''Copying or importing a MySQL database''' | |
− | '''Copying or | + | |
Unlike MS SQL Server, you cannot reliably move a MySQL database by detaching, moving, and attaching binary files in the MySQL directory. You must dump the mysql database to a sql/text file using the mysqldump command. The SQL commands can then re-create the tables and contents. To use the file, start the mysql command interface, create and use the database and give the command "source your_filename" which reads and executes the commands from the dump file. | Unlike MS SQL Server, you cannot reliably move a MySQL database by detaching, moving, and attaching binary files in the MySQL directory. You must dump the mysql database to a sql/text file using the mysqldump command. The SQL commands can then re-create the tables and contents. To use the file, start the mysql command interface, create and use the database and give the command "source your_filename" which reads and executes the commands from the dump file. | ||
Line 247: | Line 316: | ||
To dump a database: | To dump a database: | ||
+ | <blockquote> | ||
+ | <source lang="bash"> | ||
mysqldump -u root -p wikidb > /var/backups/wikis/wikidb.sql | mysqldump -u root -p wikidb > /var/backups/wikis/wikidb.sql | ||
+ | </source> | ||
+ | </blockquote> | ||
+ | This writes the database into a file called sample.sql. To create or import it into a new database called xxx, creating tables with their inserts that were present in the former wikidb, use: | ||
+ | <blockquote> | ||
+ | <source lang="bash"> | ||
+ | mysql -u root -pMySecretPW | ||
+ | </source> | ||
+ | <source lang="mysql"> | ||
+ | -- DROP DATABASE IF EXIST `xxx`; | ||
+ | CREATE DATABASE `xxx`; | ||
+ | -- for a wiki: | ||
+ | CREATE DATABASE metawiki DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci; | ||
+ | USE `xxx`; | ||
+ | -- load from a file: | ||
+ | SOURCE /var/backups/wikis/wikidb.sql; | ||
+ | </source> | ||
+ | </blockquote> | ||
+ | You can also copy directly a database from one server to another or the same server. The second database must have been created however, | ||
− | + | <blockquote> | |
+ | <source lang="bash"> | ||
mysql -u root -p | mysql -u root -p | ||
− | + | </source> | |
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | <source lang="mysql"> | |
+ | CREATE DATABASE metawiki DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci; | ||
+ | exit; | ||
+ | </source> | ||
− | + | <source lang="bash"> | |
− | + | ||
− | + | ||
mysqldump -u root -p(ThePassword) --opt wikidb | mysql -u root -p(ThePassword) --host=localhost -C metawiki | mysqldump -u root -p(ThePassword) --opt wikidb | mysql -u root -p(ThePassword) --host=localhost -C metawiki | ||
− | + | </source> | |
+ | </blockquote> | ||
Here wikidb is copied to "localhost" (–C tells mysqldump to use data compression if both servers support it). "localhost" can be replaced by the IP address of any local or remote computer. | Here wikidb is copied to "localhost" (–C tells mysqldump to use data compression if both servers support it). "localhost" can be replaced by the IP address of any local or remote computer. | ||
Note: a wiki-backup script has been started in /var/backups/scripts | Note: a wiki-backup script has been started in /var/backups/scripts | ||
+ | |||
+ | '''Execute sql from command line:''' | ||
+ | |||
+ | <blockquote> | ||
+ | <source lang="bash"> | ||
+ | mysql -u root -p --database=orowiki < /var/www/v-orowiki/w/media/orowiki.sql | ||
+ | </source> | ||
+ | </blockquote> | ||
Line 273: | Line 368: | ||
In /etc/mysql/my.cnf we changed: | In /etc/mysql/my.cnf we changed: | ||
− | |||
key_buffer = 64M # was 16M | key_buffer = 64M # was 16M | ||
− | |||
− | |||
− | |||
# utf8 | # utf8 | ||
init-connect='SET NAMES utf8' | init-connect='SET NAMES utf8' | ||
Line 287: | Line 378: | ||
(Note: after changing this, run "REPAIR TABLE searchindex QUICK;" in mysql/myadmin on each affected database.) | (Note: after changing this, run "REPAIR TABLE searchindex QUICK;" in mysql/myadmin on each affected database.) | ||
+ | '''Updating user passwords''' | ||
+ | <source lang="bash"> | ||
+ | mysqladmin -u root -p'theOLDpassword' password 'theNEWpassword' | ||
+ | </source> | ||
+ | |||
+ | '''Version upgrading/moving all databases''': See [[MySQL Backup and Restore]] | ||
===Apache=== | ===Apache=== | ||
− | apt-get install apache2 apache2-doc apache2-mpm-prefork apache2-utils libexpat1 ssl-cert | + | sudo apt-get install apache2 apache2-doc apache2-mpm-prefork apache2-utils libexpat1 ssl-cert libapache2-mod-wsgi |
− | nano /etc/apache2/mods-available/dir.conf | + | sudo nano /etc/apache2/mods-available/dir.conf |
# add more values to DirectoryIndex: | # add more values to DirectoryIndex: | ||
# index.html index.htm index.shtml index.cgi index.php index.php3 index.pl index.xhtml | # index.html index.htm index.shtml index.cgi index.php index.php3 index.pl index.xhtml | ||
Line 300: | Line 397: | ||
Now we have to enable some Apache modules (SSL, rewrite, suexec, and include) and execute the last command restart: | Now we have to enable some Apache modules (SSL, rewrite, suexec, and include) and execute the last command restart: | ||
− | a2enmod ssl | + | sudo a2enmod ssl |
− | a2enmod rewrite | + | sudo a2enmod rewrite |
− | a2enmod suexec | + | sudo a2enmod suexec |
− | a2enmod include | + | sudo a2enmod include |
− | /etc/init.d/apache2 force-reload | + | sudo /etc/init.d/apache2 force-reload |
− | Apache config is in '''/etc/apache2/apache2.conf''', the virtual host configurations are included from: '''/etc/apache2/sites-enabled/''' | + | Apache config is in '''/etc/apache2/apache2.conf''', the virtual host configurations are included from: '''/etc/apache2/sites-enabled/''' which are links pointing to '''/etc/apache2/sites-available/'''. The command ''sudo a2ensite name'' enables an available site. |
By default, the www-root (DocumentRoot) is /var/www/. | By default, the www-root (DocumentRoot) is /var/www/. | ||
− | + | To test the configuration, use <code>sudo /usr/sbin/apache2 -t</code> | |
+ | : This used to work, but with a new apache2 version the envvars are no longer loaded. | ||
+ | : <code>sudo apache2ctl -M</code> | ||
+ | : <code>sudo apache2ctl -t</code> | ||
+ | : work. | ||
− | + | SSL: We use a self-signed certificate, using the OpenSSL command: | |
+ | sudo openssl req -new -x509 -days 2190 -nodes -out /etc/ssl/certs/ssl-cert-XXXX.pem -keyout /etc/ssl/certs/ssl-cert-XXXX.key | ||
+ | '''Note:''' Do not leave the "personal name" or "common name" empty. Apache will not complain, but subversion will complain about a missing commonname attribute ("Server certificate was missing commonName attribute in subject name"). | ||
− | + | To enable SSL, link in sites-enabled to /etc/apache2/sites-available/default-ssl, modify default-ssl by pointing to the new pem and key files and enable the directive "NameVirtualHost *:443". | |
+ | |||
+ | Restart:<br> | ||
+ | <code>sudo /etc/init.d/apache2 reload</code><br> | ||
+ | <small>(Note: forgetting the sudo, will result in cryptic messages like: "httpd not running, trying to start / (13)Permission denied: make_sock: could not bind to address [::]:80 / (13)Permission denied: make_sock: could not bind to address 0.0.0.0:80 / no listening sockets available, shutting down / Unable to open logs".)</small> | ||
+ | |||
+ | '''Logrotation: ''' sudo nano /etc/logrotate.d/apache2 - change any settings, e.g. daily or weekly, "create 640" to "create 660". logrotate has not problems with softlinks, but if moving the path, rights have to be observed. Test with | ||
+ | sudo logrotate --debug /etc/logrotate.d/apache2 &> logrotate_debug_log.txt since logrotate writes to standard error, not standard out! | ||
===Subversion=== | ===Subversion=== | ||
Line 328: | Line 438: | ||
svn revert filename # revert a modified file to its original state, so future updates will refresh the file again | svn revert filename # revert a modified file to its original state, so future updates will refresh the file again | ||
cd /usr/share/mediawiki; svn status phase3 # show locally modified files (potentially no longer updated) | cd /usr/share/mediawiki; svn status phase3 # show locally modified files (potentially no longer updated) | ||
+ | |||
+ | Later, a local repository was installed, see [[Configuring the subversion repository]] | ||
===PHP=== | ===PHP=== | ||
Line 333: | Line 445: | ||
Instructions for PHPMyAdmin: http://www.debianhelp.co.uk/phpmyadmin.htm, downloading not necessary, apt will do this. | Instructions for PHPMyAdmin: http://www.debianhelp.co.uk/phpmyadmin.htm, downloading not necessary, apt will do this. | ||
− | apt-get install libapache2-mod-php5 php5 php5-common php5-curl php5-dev php5-gd | + | sudo apt-get install libapache2-mod-php5 php5 php5-common php5-curl php5-dev php5-gd php-pear php-mail php-net-smtp php5-imagick php5-imap php5-json php5-mcrypt php5-mhash php5-ming php5-mysql php5-ps php5-pspell php5-recode php5-snmp php5-sqlite php5-tidy php5-xmlrpc php5-xsl php5-intl php5-gmp |
− | apt-get install phpmyadmin | + | sudo apt-get install phpmyadmin |
+ | |||
+ | <small>Note: Originally the installation included php5-idn as well. However, the two packages intl/idn seem to conflict. "apt-get install php5-intl php5-idn" results in: "The following packages have unmet dependencies: php5-intl : Conflicts: php5-idn". With only idn installed, we got the error "'/usr/lib/php5/20090626/intl.so' - /usr/lib/php5/20090626/intl.so: cannot open shared object file: No such file or directory". We have thus executed: "apt-get remove php5-idn; apt-get install php5-intl;"</small> | ||
+ | |||
Enable php after installing: | Enable php after installing: | ||
− | a2enmod php5 | + | sudo a2enmod php5 |
This creates symbolic links from /etc/apache2/mods-available/php5.conf and /etc/apache2/mods-available/php5.load to /etc/apache2/mods-enabled. | This creates symbolic links from /etc/apache2/mods-available/php5.conf and /etc/apache2/mods-available/php5.load to /etc/apache2/mods-enabled. | ||
Line 343: | Line 458: | ||
As long as all php runs on a single server, APC is better than memcached. Recommended reading: http://www.mediawiki.org/wiki/User:Robchurch/Performance_tuning it is recommended to use: | As long as all php runs on a single server, APC is better than memcached. Recommended reading: http://www.mediawiki.org/wiki/User:Robchurch/Performance_tuning it is recommended to use: | ||
− | apt-get install php-apc | + | sudo apt-get install php-apc |
Check phpinfo.php, section apc for success. Note: Initially we did not configure the APC user cache for the mediawiki object cache ($wgMainCacheType = CACHE_ACCEL), but use memcached for this ($wgMainCacheType = CACHE_MEMCACHED) to simplify transition to multiserver setup without APC. However, we have since moved the entire cache action to memcached. | Check phpinfo.php, section apc for success. Note: Initially we did not configure the APC user cache for the mediawiki object cache ($wgMainCacheType = CACHE_ACCEL), but use memcached for this ($wgMainCacheType = CACHE_MEMCACHED) to simplify transition to multiserver setup without APC. However, we have since moved the entire cache action to memcached. | ||
+ | |||
+ | ==== PECL extensions ==== | ||
+ | |||
+ | '''NOTE: The following did no longer work for the debian 6 installation. It was used on the earlier debian 5, but may be unnecessary, since we could install php5-intl on debian 6? Else reinvestigate!''' | ||
+ | |||
+ | See http://www.jejik.com/articles/2008/07/howto_build_and_install_the_intl_pecl_extension_for_php5_in_debian/ | ||
+ | # check whether gpg keys available: | ||
+ | sudo gpg --list-secret-keys | ||
+ | # if fail, then: sudo gpg --gen-key | ||
+ | ## libicu38 no longer works: sudo apt-get install php5-dev dh-make-php fakeroot libicu38 libicu-dev xsltproc | ||
+ | sudo apt-get install php5-dev dh-make-php fakeroot libicu-dev xsltproc | ||
+ | |||
+ | # download, extract, build: | ||
+ | cd ~/php-intl-1.1.2 | ||
+ | sudo dh-make-pecl --depends libicu38 --build-depends libicu-dev --only 5 --maintainer 'YOUR_GPGName_and_EMAIL' intl <!-- INTERNAL: Instead of YOUR_GPGName_and_EMAIL I used 'Gregor Hagedorn <g.m.hagedorn@gmail.com>' --> | ||
+ | cd php-intl-1.1.2 | ||
+ | dpkg-buildpackage -rfakeroot | ||
+ | |||
+ | (was build and installed by manol) | ||
====php.ini==== | ====php.ini==== | ||
− | + | The ini file for FPM is at <code>/etc/php5/fpm/php.ini</code>. The ini file for command-line scripts is at <code>/etc/php5/cli/php.ini</code>; this is separate from the one used for apache or nginx. | |
− | + | :: ''NOTE: The information below applies to the original setup under apache2. Since 2013 we use nginx + FPM + php. General information is available under [[Nginx Installation and Configuration]]. The information below may still be helpful with respect to some of the settings originally changed; however, additional changes may have occurred since.'' | |
+ | |||
+ | The php.ini file used by apache is at <code>/etc/php5/apache2/php.ini</code>. It already contains <code>file_uploads = On</code> and register_globals was already Off. However several settings were changed: | ||
+ | |||
+ | ; Maximum execution time of each script, in seconds | ||
+ | ; http://php.net/max-execution-time | ||
+ | ; DEFAULT: max_execution_time = 30 | ||
+ | ; Note: 240 still fails VERY large mediawiki identification keys | ||
+ | max_execution_time = 450 | ||
max_input_time = 90 ; Maximum amount of time in seconds each script may spend parsing request data | max_input_time = 90 ; Maximum amount of time in seconds each script may spend parsing request data | ||
− | |||
− | |||
− | |||
− | |||
− | + | ; Maximum amount of memory a script may consume (128MB) | |
+ | ; http://php.net/memory-limit | ||
+ | memory_limit = 260M | ||
+ | ; Default was 128M. Maximum amount of memory a script may consume. 2 MB long mediawiki pages may consume 40M | ||
+ | ; Before June 2012 we used: memory_limit = 300M - now slightly reduced: | ||
+ | ; NOTE GH: change suhosin.memory_limit in /etc/php5/conf.d/suhosin.in to same value! | ||
+ | |||
+ | upload_max_filesize = 50M | ||
+ | ; DEFAULT !!! was just 2M! | ||
+ | ; ALSO SET post_max_size = 52 ABOVE! | ||
+ | ; This affects both form data and file uploads. This value must be larger than upload_max_filesize! | ||
+ | |||
+ | post_max_size = 52M | ||
+ | ; was 5, affects file upload and must be larger fileupload! | ||
− | + | allow_call_time_pass_reference = Off | |
'''Syntax errors:''' The log files are relatively silence if a syntax error exists in php.ini. All values will be used with their defaults; thus this situation can be detected by checking [[Special:Upload]] whether it displays an upload limit of 2 MB. To manually check php.ini for syntax errors use: | '''Syntax errors:''' The log files are relatively silence if a syntax error exists in php.ini. All values will be used with their defaults; thus this situation can be detected by checking [[Special:Upload]] whether it displays an upload limit of 2 MB. To manually check php.ini for syntax errors use: | ||
Line 370: | Line 521: | ||
$conn = mysql_connect($MySQL_Host, $MySQL_User,$MySQL_Passwd); | $conn = mysql_connect($MySQL_Host, $MySQL_User,$MySQL_Passwd); | ||
mysql_query("SET NAMES utf8", $conn); | mysql_query("SET NAMES utf8", $conn); | ||
+ | |||
+ | ====suhosin.ini==== | ||
+ | |||
+ | suhosin is installed with: | ||
+ | apt-get install php5-suhosin | ||
+ | |||
+ | Change suhosin.memory_limit in /etc/php5/conf.d/suhosin.ini to same value! | ||
+ | ; DEFAULT WAS: suhosin.memory_limit = 0, now same value as in php.ini: | ||
+ | suhosin.memory_limit = 600 | ||
+ | Note: Unlike in php.ini there is No "M" after the number here! | ||
====apc.ini==== | ====apc.ini==== | ||
Line 383: | Line 544: | ||
# default: apc.shm_size=30 # i.e. 30 MB | # default: apc.shm_size=30 # i.e. 30 MB | ||
apc.shm_size=60 | apc.shm_size=60 | ||
− | |||
===PHPMyAdmin=== | ===PHPMyAdmin=== | ||
− | + | (Previous apache instructions now obsolete, we now use nginx.) | |
− | + | '''Note: the path to phpmyadmin has been changed!''' Please check the phpmyadmin* folder (symlink) inside /var/www for currently correct URL. The purpose of this change was: bots are constantly hitting /phpmyadmin in search of vulnerabilities. Although we keep updating the software regularly, it is safer if they don't know the location. Please do not document the location here! | |
− | + | ||
− | + | ===Django installation=== | |
− | + | See [[Django-Installation]]. For Artenquiz at least django Version 1.3 necessary! | |
− | |||
− | |||
− | |||
− | |||
− | |||
===Mediawiki=== | ===Mediawiki=== | ||
Line 406: | Line 560: | ||
Mediawiki installation information is available separately: [[Mediawiki_installation]] | Mediawiki installation information is available separately: [[Mediawiki_installation]] | ||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
===Tomcat=== | ===Tomcat=== | ||
Important: to get Tomcat5.5 running after installing, it is necessary that the environment variables JAVA_HOME and JRE_HOME are set correctly and that JDK 5 or higher is used. | Important: to get Tomcat5.5 running after installing, it is necessary that the environment variables JAVA_HOME and JRE_HOME are set correctly and that JDK 5 or higher is used. | ||
− | apt-get install tomcat5.5 tomcat5.5-admin tomcat5.5-webapps | + | # NOTE: native tomcat no longer used: |
+ | # apt-get install tomcat5.5 tomcat5.5-admin tomcat5.5-webapps | ||
apt-get install libapache2-mod-jk | apt-get install libapache2-mod-jk | ||
Line 455: | Line 571: | ||
Edit /etc/profile (as user root), add: | Edit /etc/profile (as user root), add: | ||
− | export CATALINA_HOME=/usr/share/tomcat5.5 | + | ## For debian standard: export CATALINA_HOME=/usr/share/tomcat5.5 |
+ | ## Instead, for fedora-tomcat: | ||
+ | export CATALINA_HOME=/usr/share/fedora-3.1/tomcat | ||
To make Tomcat work with Apache edit /etc/libapache2-mod-jk/workers.properties and set the two lines (using your correct paths): | To make Tomcat work with Apache edit /etc/libapache2-mod-jk/workers.properties and set the two lines (using your correct paths): | ||
− | workers.tomcat_home=/usr/share/ | + | workers.tomcat_home=/usr/share/fedora-3.1/tomcat |
workers.java_home=/usr/local/jdk | workers.java_home=/usr/local/jdk | ||
− | + | At the end of /etc/apache2/apache2.conf add: | |
− | At the end of /etc/apache2/apache2.conf add: | + | # Enable libapache2-mod-jk |
− | # Enable libapache2-mod-jk | + | |
Include /usr/share/doc/libapache2-mod-jk/httpd_example_apache2.conf | Include /usr/share/doc/libapache2-mod-jk/httpd_example_apache2.conf | ||
+ | |||
+ | --- | ||
+ | Presently, the tomcat used is the one supplied with Fedora, inside the Fedora folder. | ||
+ | The main tomcat access is fedora.keytonature.net. However, Lia Veja has installed a secondary point | ||
+ | http://species-id.net/services/ | ||
+ | L. Veja writes "it needs some edits in web.xml file for every application you will put there". | ||
Restart Apache and Tomcat. | Restart Apache and Tomcat. | ||
Line 481: | Line 604: | ||
rm -r /usr/share/fedora/server/logs | rm -r /usr/share/fedora/server/logs | ||
ln -s /mnt/dump/var/log/fedora /usr/share/fedora/server/logs | ln -s /mnt/dump/var/log/fedora /usr/share/fedora/server/logs | ||
− | /usr/share/fedora/tomcat/bin/startup.sh | + | # Old: /usr/share/fedora/tomcat/bin/startup.sh |
+ | # New, our own script: | ||
+ | /etc/init.d/fedora start | ||
Line 504: | Line 629: | ||
See also: [http://fedora-commons.org/confluence/display/FCR30/Command-Line+Utilities#Command-LineUtilities-rebuild Fedora Commons documentation]. | See also: [http://fedora-commons.org/confluence/display/FCR30/Command-Line+Utilities#Command-LineUtilities-rebuild Fedora Commons documentation]. | ||
− | |||
− | |||
− | |||
− | |||
==Webmin== | ==Webmin== | ||
Webmin is a web-based system configuration tool for Linux. With it you can configure many operating system internals, such as users, disk quotas, services, configuration files etc., as well as modify and control many open source apps, such as the Apache HTTP Server, PHP, MySQL etc. | Webmin is a web-based system configuration tool for Linux. With it you can configure many operating system internals, such as users, disk quotas, services, configuration files etc., as well as modify and control many open source apps, such as the Apache HTTP Server, PHP, MySQL etc. | ||
− | |||
− | |||
− | |||
− | |||
− | + | To install, first /etc/apt/sources.list needs to be updated (see above). Then: | |
− | + | sudo apt-get update; sudo apt-get install webmin | |
− | + | You will be asked if you wish to install the package without verification. Select yes. To configure a different port: | |
− | /etc/ | + | sudo nano /etc/webmin/miniserv.conf |
− | /etc/init.d/webmin | + | # change port AND listen options from 10000 to 18010 and restart: |
− | + | sudo /etc/init.d/webmin restart | |
− | + | ||
− | + | Only users with sudo rights can login with their usual password. Help for the standard modules used by Webmin can be found here: http://doxfer.com/Webmin/Modules. A wiki page for webmin can be found here: http://doxfer.com/Webmin | |
+ | |||
+ | To increase security read [http://www.linuxjunkies.org/adminstration%20Howto/webminguide/x625.htm this] and: | ||
+ | * in "Webmin: Webmin Configuration: Authentication" "set "Block hosts with more than 2 failed logins for 60 seconds; Block users with more than 6 failed logins for 12000 seconds" and select "Log blocked hosts, logins and authentication failures to syslog" and "Enable session authentication: Autologout after 10 Minutes" and disable "Offer to remember login permanently?". Rest remaining on defaults. | ||
+ | * Create new certificates, then in "Webmin: Webmin Configuration: SSL Encryption" set: "Private key file: .../ssl-cert-XXX-webmin.key" and "Certificate file: Separate file: .../ssl-cert-XXX-webmin.key" and "Redirect non-SSL requests to SSL mode? Yes". | ||
==mc (midnight commander) und jed (editor)== | ==mc (midnight commander) und jed (editor)== | ||
Midnight commander is the Linux equivalent of the Norton Commander. Anyone familiar with Norton Commander should feel right at home. Very useful for browsing through the file system. It has an integrated FTP client, editor and file viewer, and supports the use of a mouse inside an ssh window! Similarly, jed is an editor that supports the use of mouse in ssh. Install both with | Midnight commander is the Linux equivalent of the Norton Commander. Anyone familiar with Norton Commander should feel right at home. Very useful for browsing through the file system. It has an integrated FTP client, editor and file viewer, and supports the use of a mouse inside an ssh window! Similarly, jed is an editor that supports the use of mouse in ssh. Install both with | ||
− | apt-get install mc jed | + | sudo apt-get install mc jed |
No configuration necessary. Run with | No configuration necessary. Run with | ||
mc | mc | ||
jed | jed | ||
+ | |||
+ | ==cronjobs== | ||
+ | |||
+ | DEPRECATED ON BIOWIKIFARM: to install as root user-specific cronjob (copy see nano /var/spool/cron/crontabs/root, but not editable there!), use: | ||
+ | |||
+ | sudo crontab -u root -e # edit roots cron jobs | ||
+ | |||
+ | This uses jed as editor. Format at start of each line is: minutes, hour, day, etc. One can use * for every (hour, day, month, year). Output is normally sent by email. To suppress this redirect output to ">/dev/null 2&>1". Always add a comment after each line: # explain what this does and who installed it. | ||
+ | |||
+ | PREFERRED: | ||
+ | sudo nano /etc/crontab | ||
+ | This is directly editable, allows jobs for different users (www-data, root), and is more transparent to manage. | ||
+ | |||
+ | == Security, virus protection, rootkit hunter == | ||
+ | |||
+ | rootkit hunter: | ||
+ | # sudo apt-get install rkhunter | ||
+ | # OR, consider to install from backport, here squeeze backport a newer version: | ||
+ | sudo apt-get -t squeeze-backports install rkhunter | ||
+ | |||
+ | sudo rkhunter --update | ||
+ | sudo rkhunter --checkall | ||
+ | |||
+ | clamav: | ||
+ | sudo apt-get install clamav | ||
+ | /etc/init.d/clamav-freshclam start | ||
+ | # or: stop, restart, etc. | ||
+ | |||
+ | rest not sure. clamav, clamscan etc. don't work. Was not asking for configuration. -> ? | ||
+ | |||
+ | http://www.debianadmin.com/clamav-installation-and-configuration.html looks good, but did not help | ||
+ | |||
+ | General: use on files or folders to change group and rights: | ||
+ | sudo chgrp -R mw-extension-writers crossdomain*; sudo chmod -R 664 crossdomain*; | ||
+ | # or if containing folders: | ||
+ | sudo chgrp -R mw-extension-writers /var/www/tools/mediaIBIS; sudo chmod -R 775 /var/www/tools/mediaIBIS | ||
+ | |||
+ | ==Dropbox== | ||
+ | |||
+ | (updated 2013-03-06) | ||
+ | |||
+ | Dropbox is installed for a specific user and with manual daemon startup. We created a special user "dropbox" for this task. dropbox is member of www-data, and www-data member of dropbox group. Following: https://www.dropbox.com/install?os=lnx and http://linuxg.net/install-dropbox-on-linux-systems-ubuntu-debian-fedora-and-others/ | ||
+ | sudo su dropbox | ||
+ | cd ~ && wget -O - "https://www.dropbox.com/download?plat=lnx.x86_64" | tar xzf - | ||
+ | cd ~ && wget -O - http://www.dropbox.com/download?plat=lnx.x86_64 | tar xzf - | ||
+ | wget https://www.dropbox.com/download?dl=packages/dropbox.py | ||
+ | ~/.dropbox-dist/dropboxd | ||
+ | |||
+ | On first installation it will ask to visit a URL from your desktop-browser, to confirm linking the new dropbox with an existing account. In the dropbox web page, go on top to account, then below the tab "My Computers". The procedure is a bit tough, since you have to copy the long link without stopping the command on the command line. For putty use right-click mouse, not Ctrl-C. The mouse-click will copy and paste, looking wrong, but the commandline process keeps running. The process was successfull if the command line on the server issues "Client successfully linked, Welcome (YourName)". | ||
+ | |||
+ | We added a brief shell script "run_dropbox.sh" with: | ||
+ | #!/bin/bash | ||
+ | ~/.dropbox-dist/dropboxd& | ||
+ | ps | ||
+ | to help memorize Dropbox daemon startup. | ||
+ | |||
+ | Link the Dropbox folder inside dropbox user home into wiki for ease of importing: | ||
+ | cd /var/www/v-species/o; sudo ln -s /home/dropbox/Dropbox "Dropbox" | ||
==Tips for simple access of Debian file from remote Windows== | ==Tips for simple access of Debian file from remote Windows== | ||
Line 559: | Line 737: | ||
See [http://www.ntfs-3g.org/manual.html ntfs-3g manual] for further information. Some note mention that "To mount files with non-ASCII characters one may have to give the option -o locale=XXX to the mount options of ntfs-3g; see http://ntfs-3g.org/support.html#locale for further information." We did not do so far, using UTF-8 seems to work. | See [http://www.ntfs-3g.org/manual.html ntfs-3g manual] for further information. Some note mention that "To mount files with non-ASCII characters one may have to give the option -o locale=XXX to the mount options of ntfs-3g; see http://ntfs-3g.org/support.html#locale for further information." We did not do so far, using UTF-8 seems to work. | ||
+ | |||
+ | |||
+ | == Moving files to a new server directly == | ||
+ | |||
+ | Instead of backup/restore it is possible to go, on source machine, to the folder under which the subfolders shall be moved, then tar, ssh transfer, unpack. Caveat: on the destination machine the login MUST be root, else the user and group information will be lost. Temporarily change sudo nano /etc/ssh/sshd_config to permit root login! | ||
+ | |||
+ | cd / | ||
+ | # as root, being on old server: | ||
+ | sudo tar -czf - --numeric-owner root home var | ssh root@212.201.100.117 tar -xzf - --same-owner -C /mnt/dump/_oldroot_TEMP | ||
+ | sudo tar -czf - --numeric-owner var | ssh root@212.201.100.117 tar -xzf - --same-owner -C /mnt/storage/oldroot | ||
+ | cd /mnt/storage | ||
+ | sudo tar -czf - --numeric-owner * | ssh root@212.201.100.117 tar -xzf - --same-owner -C /mnt/storage | ||
+ | cd /mnt/dump | ||
+ | sudo tar -czf - --numeric-owner * | ssh root@212.201.100.117 tar -xzf - --same-owner -C /mnt/dump | ||
+ | cd /usr/share | ||
+ | sudo tar -czf - --numeric-owner mediaw* | ssh root@212.201.100.117 tar -xzf - --same-owner -C /mnt | ||
+ | sudo tar -czf - --numeric-owner fedora* Fedora* | ssh root@212.201.100.117 tar -xzf - --same-owner -C /mnt | ||
+ | |||
+ | Note: when moving, the user and group ID should be aligned, see: | ||
+ | http://it.toolbox.com/blogs/locutus/how-to-change-a-users-uid-and-gid-26368 | ||
[[Category: Software documentation]] | [[Category: Software documentation]] |
Latest revision as of 11:43, 25 December 2017
(Part of the Software documentation of biowikifarm.)
This is a record of the installation and later updates of a Debian system (originally in version 4, later upgraded, see "Debian upgrade 4 to 5"). The page is intended to help running or repeating the server setup. In parts it may be outdated by subsequent configuration changes and software updates. Some notes may be updated, others not. However, much of the fundamental setup will remain evident from this document.
We use XenServer Express (see Server virtualization) and originally installed the Xen template for Debian 4.0 (kernel modified for paravirtualization) with the standard setup. This is a documentation of the steps performed to get the Debian server working. The notes have been updated since, now reflecting a debian 6 setup.
See also: Biowikifarm server hardware.
Contents
Basics
The server can be reached at 212.201.100.117 with ssh enabled. Settings for ssh:
- /etc/ssh/ssh_config default settings for ssh when used as client
- /etc/ssh/sshd_config default settings for ssh as server
- use
nano /etc/ssh/sshd_config
to set
PermitRootLogin no
PasswordAuthentication no
UsePAM no
= disallowing root access through ssh and limiting access to ssh pubkey infrastructure. - use
nano /etc/pam.d/common-password
to increase minimum password length:
"password [success=1 default=ignore] pam_unix.so obscure sha512 min=11 max=30 nullok retry=3"
- use
- check
lastlog
to see last login of users
- check
Current port and firewall situation: See Server ports and shorewall installation.
Memo on other IPs: 10 = base, 11, 41 = xencenter, and 56 = dev
Partitioning and formatting additional storage
NOTE: updated information for debian6 on xen6. Swap is here in main.
Xen will partition and format the first hard drive (main = system and swap), but any additional, medium needs to be partitioned manually. The name of the 2./3 virtual hard disks under XEN is xvdb/xvdc. Run:
sudo fdisk /dev/xvdb sudo fdisk /dev/xvdc # type n for new partition, p for primary, 1 for first partition, # accept defaults for size, type w to write out new partition. # To format with ext4 filesystem: sudo mkfs.ext4 /dev/xvdb1 sudo mkfs.ext4 /dev/xvdc1 sudo mkfs.ext4 /dev/xvde1 # To label partitions: sudo e2label /dev/xvda1 root sudo e2label /dev/xvdb1 storage sudo e2label /dev/xvdc1 dump sudo e2label /dev/xvde1 BIG
Note that xvdd1 is the CD/DVD-ROM! It is under /media/cdrom0, not under /mnt/cdrom0!
Correct function of partitioning can be tested, e.g. using webmin (Hardware, Partitions on Local disks). To permanently mount, do
sudo nano /etc/fstab #then edit: /dev/xvdb1 /mnt/storage ext3 rw,noatime 0 0 /dev/xvdc1 /mnt/dump ext3 rw,noatime 0 0 # to mount temporarily use something like: "sudo mount /dev/xvde1 /mnt/temp" # to mount BIG: "sudo mount /dev/xvde1 /mnt/BIG"
Check swap with
cat /proc/swaps free
(Result: /dev/xvda5!)
Note: in 2014 filesystem was enlarged and changed to ext4.
Resizing virtual disks
- The disk needs to be deactivated in XenCenter before resizing.
- Resizing the virtual storage unit does not change the size of the partition under Linux!
- Method 1 (manual): if the disk contains only relatively easily migrated data (no dev/ports, etc.), create a new disk, partition and format as above, temporarily mount: sudo mount /dev/xvde1 /mnt/temp and copy all content to another disk using
sudo cp -pr
(preserving owner, permission, recursive) ortar -p -s
. In XenCenter stop VM, remove previous disk (first detach, delete only later), change order of volumes, then restart.- NOTE: Vol. swapping when VM is down; use the upper disk number in XenCenter, set to the same number as the lower number equivalent disk, warning will occur and offer to swap volumes. Trying to avoid this manually results in multiple reboot cycles, because the disk number is no longer available!
# after: /dev/xvdX1 /mnt/storage2; sudo mount /dev/xvdX1 /mnt/dump2 sudo rm /mnt/storage2/* -r; sudo rm /mnt/dump2/* -r; sudo cp -pr /mnt/storage/* /mnt/storage2; # cannot put the following into single line, sudo will have timed out! # perhaps remove the DAILY backup from dump, which reduces copy time, then copy: sudo cp -pr /mnt/dump/* /mnt/dump2;
- Method 2 (better, tip Manol, not yet tested):
- install parted:
apt-get install parted
or gparted (graphical frontend) - unmount partition and check with parted
- OR: install
apt-get install dump
(dump - 4.4bsd dump and restore for ext2 filesystems) and use this to store (somthing is missing here, not tested).
- install parted:
Network, Hostname and IP address
First installation
Hostnames are in several places, especially important when renaming a server:
sudo nano /etc/hostname # entered: biowikifarm (old) biowiki (new) sudo nano /etc/mailname # entered: biowikifarm.net sudo nano /etc/hosts # 212.201.100.117 biowikifarm.net biowikifarm sudo nano /etc/hosts # 160.45.63.58 biowikifarm.net biowiki
Open "nano /etc/network/interfaces" and enter: NEW:
# This file describes the network interfaces available on your system # and how to activate them. For more information, see interfaces(5). # The loopback network interface auto lo iface lo inet loopback # The primary network interface allow-hotplug eth0 iface eth0 inet static address 192.168.101.117 netmask 255.255.255.0 network 192.168.101.0 #broadcast 192.168.101.255 gateway 192.168.101.1 # dns-* options are implemented by the resolvconf package, if installed dns-nameservers 192.168.201.16 192.168.201.11 192.168.201.12 up ip addr add 212.201.100.117/32 dev $IFACE label $IFACE:0 down ip addr del 212.201.100.117/32 dev $IFACE label $IFACE:0
# The following lines are desirable for IPv6 capable hosts ::1 ip6-localhost ip6-loopback fe00::0 ip6-localnet ff00::0 ip6-mcastprefix ff02::1 ip6-allnodes ff02::2 ip6-allrouters ff02::3 ip6-allhosts
Note: for future IP changes, also check:
- /etc/resolv.conf
- /etc/apache2/sites-available/default
- /etc/hosts
- /etc/networks
- also often the mail configuration in the password file has to be updated (path here not shown by purpose)
Reboot now (first command) and run the following commands (the two hostname commands verify that the hostname setting is ok):
sudo reboot hostname # verify hostname -f # verify sudo apt-get update; sudo apt-get upgrade; sudo apt-get dist-upgrade sudo apt-get install ssh sudo
It may be good to repeat this, some updates came only in second round. Finally, we may want to do some cleanup:
sudo apt-get autoclean
After doing this, the ssh fingerprints may have to be changed on all machines trying to access the new Debian by ssh (security issues fixed). Remove the keys for the previously opened IPs in the following file:
nano /root/.ssh/known_hosts
Related: Reconfiguring a copied virtual machine
Before starting a copied VM check the MAC adress in XEN to avoid running accidentially two servers under the same IP. A copied VM may or may not use other interfaces instead of eth0, e.g., eth2. Thus first use ifconfig -a to find the name of the interface, then change interface name AND IP-number in network AND shorewall interface files. Use the interfaces shown there. Also edit hostname and hosts, and restart shorewall. (Note: .57 is good, .58 bad...).
sudo ifconfig -a | more sudo nano /etc/network/interfaces # interface and IP sudo nano /etc/shorewall/interfaces # (interface at the bottom, check, update if necessary) sudo nano /etc/hostname sudo nano /etc/hosts sudo /etc/init.d/shorewall restart
(Note: if due to a misconfiguration the shorewall continuously displays blocking messages on the console, try Alt-F2, user name and password. Type blindly, despite repeated shorewall messages. "shorewall stop" will stop shorewall (type blindly).)
sudoers
sudo nano /etc/sudoers # then check wether (this is the default of debian6 or xen template): %sudo ALL=(ALL) ALL
This lets all users of group sudo run any command as root without knowing the root password.
Adding repositories to package manager
It is possible to add further repositories to the package manager by editing the repository file: "nano /etc/apt/sources.list". One important addition is the backports (needed for OpenID further down), and we also expanded the scope from "main" to "main non-free contrib":
deb http://ftp.debian.org/debian/ etch main non-free contrib deb http://www.backports.org/debian etch-backports main # Necessary for webmin (sarge is still the correct version for squeeze!!): deb http://download.webmin.com/download/repository sarge contrib # added for oggvideotools (June 13 2010) deb http://www.debian-multimedia.org lenny main
LONGER WORKING: To install keys: wget -O - http://backports.org/debian/archive.key | apt-key add - ; sudo apt-get install debian-backports-keyring; --- The last command adds a gpg security key for the backport repository. (Documentation of further testing: We also tested the Google repository as an example. Using root console, we first had to get a pgp key (note: the original google instructions requested using https, but only http worked): wget -q http://dl-ssl.google.com/linux/linux_signing_key.pub -O- | apt-key add -
, and add deb http://dl.google.com/linux/deb/ stable non-free
.)
# more on clamav and rkhunter below, but install early: sudo apt-get install clamav rkhunter
JAVA
Debian does not normally install Sun Java, but a slower "free Java". It is possible to install sun software after modifying the repository sources to include non-free software and backports (see here for more information: http://www.debian.org/doc/manuals/debian-java-faq/ch6.html):
sudo apt-get install sun-java6-jre
The install location is /usr/lib/jvm/java-6-sun/. Sun recommends to update the Debian "alternatives system" to have Sun's tools as the default:
sudo update-java-alternatives -s java-6-sun
Then edit
sudo nano /etc/profile # and add at the top, before first "if": export JAVA_HOME=/usr/lib/jvm/java-6-sun export JRE_HOME=/usr/lib/jvm/java-6-sun/jre export CATALINA_HOME=/usr/share/fedora/tomcat export FEDORA_HOME=/usr/share/fedora # and add further down, after last "fi": PATH=$JAVA_HOME/bin:$FEDORA_HOME/server/bin:$FEDORA_HOME/client/bin:$PATH export PATH umask 022 HISTFILE=~/.bash_history HISTSIZE=10000 HISTFILESIZE=999999 readonly HISTFILE readonly HISTSIZE readonly HISTFILESIZE export HISTFILE HISTSIZE HISTFILESIZE
(The last line before "export PATH"; this export is probably only necessary (e.g. for Tomcat, see below) if an earlier version of Java is still running and is located in a path that is looked up first by the system.)
Log out and log back in again (this may also be another user) and then confirm that these settings are in effect by:
echo $JAVA_HOME echo $JRE_HOME echo $PATH
MySQL, Apache PHP, etc.
General installations (2017)
sudo apt-get install gcc cpp libtool binutils make autoconf automake1.9 flex tree locate sudo apt-get install zip unzip rar unrar p7zip p7zip-full
To use MediaWiki with high traffic, a memcached memory cache should be installed prior to installing MediaWiki:
apt-get install memcached php5-memcache
Without further configuration, a single memcached will be active on default port 11211, limited to localhost (test from outside using: telnet 212.201.100.117 11211), using up to 64 MB of your RAM.
This can not be changed by editing /etc/init.d/memcached
, but /etc/memcached.config
must be modified. We increased size: -m 512 = MByte, -l 127.0.0.1 = listen only on local loopback interface.
Stats on memcached use, using netcat (nc, see https://lzone.de/cheat-sheet/memcached):
echo stats | nc 127.0.0.1 11211 echo stats slabs | nc 127.0.0.1 11211 # just for number of items, shown at end: echo stats items | nc 127.0.0.1 11211
MySQL
Installation
# debian5: sudo apt-get install mysql-server mysql-client libmysqlclient15-dev # on debian6 we used: sudo apt-get install mysql-server mysql-client libmysqlclient-dev sudo nano /etc/mysql/my.cnf # making many changes, use a merge with a current config file /etc/init.d/mysql restart
Under Debian 6, We experienced a broken apt-get after an "apt-get remove mysql-server mysql-client libmysqlclient-dev" and then "apt-get install --reinstall mysql-server mysql-client libmysqlclient-dev" and removing the content of /var/lib/mysql (to start fresh over, after the first dump import failed). In the end, removing further parts or mysql AND manually installing mysql-server-core-5.1 fixed the broken installation situation!
Console commands
Test with netstat (debian seems to report 8088 as omniorb) and set a password for root (otherwise anybody can access your MySQL database!):
netstat -tap # -u = login as this user mysqladmin -u root password (ThePassword)
New users can be created and granted privileges inside the mysql console:
# enter the mysql console (asks for passwort): mysql -u root -p
-- within the console (mysql> ): grant all privileges on *.* to 'username'@'localhost' identified by 'PASSWORD' with grant option; grant all privileges on *.* to 'username'@'%' identified by 'PASSWORD' with grant option; -- to exit the mysql console type: exit
Notes: a) grant will create a user if it does not exist yet. b) username, hostname, PASSWORD must be quoted with '. c) the '%' allows the user to log in from any other host except localhost; an explicit grant for localhost is thus required in addition to the '%'.
Changing password for existing users:
# enter the mysql console (asks for passwort): mysql -u root -p
-- within the console (mysql> ): set password = PASSWORD('NEW-PASSWORD-HERE'); -- Alternative, for a different user: set password FOR 'name'@'localhost' = PASSWORD('NEW-PASSWORD-HERE');
Some other mysql console commands are:
show databases; use mysql; Select * from user; drop user root@OudemansD;
Important note: entering mysql command line with "mysql -u wikiuser -p"
will ask for password; if you want to include the password (e.g. when using pipe into) do not add a blank between p and the password. Also, when using the -h host option, only -hlocalhost, but not -hlocalhost:8088 will work! The host can usually be omitted. See the following examples:
mysql -uwikiuser -pTHEPASSWORD mysql -hlocalhost -u wikiuser -pTHEPASSWORD < openid_table.sql
Copying or importing a MySQL database
Unlike MS SQL Server, you cannot reliably move a MySQL database by detaching, moving, and attaching binary files in the MySQL directory. You must dump the mysql database to a sql/text file using the mysqldump command. The SQL commands can then re-create the tables and contents. To use the file, start the mysql command interface, create and use the database and give the command "source your_filename" which reads and executes the commands from the dump file.
Over the web, myphpadmim offers backup (which works) and restore (which does not work for any reasonably sized database, despite uploading as zip file, we are getting out of memory errors). So we have to use command line programming. Note that the dump created by myphpadmin includes a database create statement; to copy to a different database name you need to modify this (extremely long lines may become corrupted in some editors!); the mysqldump does not contain database statements and is easier to reuse.
To dump a database:
mysqldump -u root -p wikidb > /var/backups/wikis/wikidb.sql
This writes the database into a file called sample.sql. To create or import it into a new database called xxx, creating tables with their inserts that were present in the former wikidb, use:
mysql -u root -pMySecretPW-- DROP DATABASE IF EXIST `xxx`; CREATE DATABASE `xxx`; -- for a wiki: CREATE DATABASE metawiki DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci; USE `xxx`; -- load from a file: SOURCE /var/backups/wikis/wikidb.sql;
You can also copy directly a database from one server to another or the same server. The second database must have been created however,
mysql -u root -pCREATE DATABASE metawiki DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci; exit;mysqldump -u root -p(ThePassword) --opt wikidb | mysql -u root -p(ThePassword) --host=localhost -C metawiki
Here wikidb is copied to "localhost" (–C tells mysqldump to use data compression if both servers support it). "localhost" can be replaced by the IP address of any local or remote computer.
Note: a wiki-backup script has been started in /var/backups/scripts
Execute sql from command line:
mysql -u root -p --database=orowiki < /var/www/v-orowiki/w/media/orowiki.sql
Performance tuning and unicode:
In /etc/mysql/my.cnf we changed:
key_buffer = 64M # was 16M # utf8 init-connect='SET NAMES utf8' character-set-server=utf8 collation-server=utf8_unicode_ci
We include three-letter words in the full text index:
ft_min_word_len=3
(Note: after changing this, run "REPAIR TABLE searchindex QUICK;" in mysql/myadmin on each affected database.)
Updating user passwords
mysqladmin -u root -p'theOLDpassword' password 'theNEWpassword'
Version upgrading/moving all databases: See MySQL Backup and Restore
Apache
sudo apt-get install apache2 apache2-doc apache2-mpm-prefork apache2-utils libexpat1 ssl-cert libapache2-mod-wsgi sudo nano /etc/apache2/mods-available/dir.conf # add more values to DirectoryIndex: # index.html index.htm index.shtml index.cgi index.php index.php3 index.pl index.xhtml nano /etc/apache2/ports.conf # add ssl port to the existing "Listen 80" on new line: Listen 443
Now we have to enable some Apache modules (SSL, rewrite, suexec, and include) and execute the last command restart:
sudo a2enmod ssl sudo a2enmod rewrite sudo a2enmod suexec sudo a2enmod include sudo /etc/init.d/apache2 force-reload
Apache config is in /etc/apache2/apache2.conf, the virtual host configurations are included from: /etc/apache2/sites-enabled/ which are links pointing to /etc/apache2/sites-available/. The command sudo a2ensite name enables an available site.
By default, the www-root (DocumentRoot) is /var/www/.
To test the configuration, use sudo /usr/sbin/apache2 -t
- This used to work, but with a new apache2 version the envvars are no longer loaded.
-
sudo apache2ctl -M
-
sudo apache2ctl -t
- work.
SSL: We use a self-signed certificate, using the OpenSSL command:
sudo openssl req -new -x509 -days 2190 -nodes -out /etc/ssl/certs/ssl-cert-XXXX.pem -keyout /etc/ssl/certs/ssl-cert-XXXX.key
Note: Do not leave the "personal name" or "common name" empty. Apache will not complain, but subversion will complain about a missing commonname attribute ("Server certificate was missing commonName attribute in subject name").
To enable SSL, link in sites-enabled to /etc/apache2/sites-available/default-ssl, modify default-ssl by pointing to the new pem and key files and enable the directive "NameVirtualHost *:443".
Restart:
sudo /etc/init.d/apache2 reload
(Note: forgetting the sudo, will result in cryptic messages like: "httpd not running, trying to start / (13)Permission denied: make_sock: could not bind to address [::]:80 / (13)Permission denied: make_sock: could not bind to address 0.0.0.0:80 / no listening sockets available, shutting down / Unable to open logs".)
Logrotation: sudo nano /etc/logrotate.d/apache2 - change any settings, e.g. daily or weekly, "create 640" to "create 660". logrotate has not problems with softlinks, but if moving the path, rights have to be observed. Test with sudo logrotate --debug /etc/logrotate.d/apache2 &> logrotate_debug_log.txt since logrotate writes to standard error, not standard out!
Subversion
apt-get install subversion libapache2-svn # (note: modules are automatically enabled, no a2enmod necessary)
- Tested with MediaWiki svn download (see below)
- Potential additional instructions (not used): Configuring Subversion
Special commands:
svn update -rXXXX # update to specific revision (XXXX being an integer revision number) svn revert filename # revert a modified file to its original state, so future updates will refresh the file again cd /usr/share/mediawiki; svn status phase3 # show locally modified files (potentially no longer updated)
Later, a local repository was installed, see Configuring the subversion repository
PHP
Instructions for PHPMyAdmin: http://www.debianhelp.co.uk/phpmyadmin.htm, downloading not necessary, apt will do this.
sudo apt-get install libapache2-mod-php5 php5 php5-common php5-curl php5-dev php5-gd php-pear php-mail php-net-smtp php5-imagick php5-imap php5-json php5-mcrypt php5-mhash php5-ming php5-mysql php5-ps php5-pspell php5-recode php5-snmp php5-sqlite php5-tidy php5-xmlrpc php5-xsl php5-intl php5-gmp sudo apt-get install phpmyadmin
Note: Originally the installation included php5-idn as well. However, the two packages intl/idn seem to conflict. "apt-get install php5-intl php5-idn" results in: "The following packages have unmet dependencies: php5-intl : Conflicts: php5-idn". With only idn installed, we got the error "'/usr/lib/php5/20090626/intl.so' - /usr/lib/php5/20090626/intl.so: cannot open shared object file: No such file or directory". We have thus executed: "apt-get remove php5-idn; apt-get install php5-intl;"
Enable php after installing:
sudo a2enmod php5
This creates symbolic links from /etc/apache2/mods-available/php5.conf and /etc/apache2/mods-available/php5.load to /etc/apache2/mods-enabled.
As long as all php runs on a single server, APC is better than memcached. Recommended reading: http://www.mediawiki.org/wiki/User:Robchurch/Performance_tuning it is recommended to use:
sudo apt-get install php-apc
Check phpinfo.php, section apc for success. Note: Initially we did not configure the APC user cache for the mediawiki object cache ($wgMainCacheType = CACHE_ACCEL), but use memcached for this ($wgMainCacheType = CACHE_MEMCACHED) to simplify transition to multiserver setup without APC. However, we have since moved the entire cache action to memcached.
PECL extensions
NOTE: The following did no longer work for the debian 6 installation. It was used on the earlier debian 5, but may be unnecessary, since we could install php5-intl on debian 6? Else reinvestigate!
# check whether gpg keys available: sudo gpg --list-secret-keys # if fail, then: sudo gpg --gen-key ## libicu38 no longer works: sudo apt-get install php5-dev dh-make-php fakeroot libicu38 libicu-dev xsltproc sudo apt-get install php5-dev dh-make-php fakeroot libicu-dev xsltproc
# download, extract, build: cd ~/php-intl-1.1.2 sudo dh-make-pecl --depends libicu38 --build-depends libicu-dev --only 5 --maintainer 'YOUR_GPGName_and_EMAIL' intl cd php-intl-1.1.2 dpkg-buildpackage -rfakeroot
(was build and installed by manol)
php.ini
The ini file for FPM is at /etc/php5/fpm/php.ini
. The ini file for command-line scripts is at /etc/php5/cli/php.ini
; this is separate from the one used for apache or nginx.
- NOTE: The information below applies to the original setup under apache2. Since 2013 we use nginx + FPM + php. General information is available under Nginx Installation and Configuration. The information below may still be helpful with respect to some of the settings originally changed; however, additional changes may have occurred since.
The php.ini file used by apache is at /etc/php5/apache2/php.ini
. It already contains file_uploads = On
and register_globals was already Off. However several settings were changed:
; Maximum execution time of each script, in seconds ; http://php.net/max-execution-time ; DEFAULT: max_execution_time = 30 ; Note: 240 still fails VERY large mediawiki identification keys max_execution_time = 450 max_input_time = 90 ; Maximum amount of time in seconds each script may spend parsing request data
; Maximum amount of memory a script may consume (128MB) ; http://php.net/memory-limit memory_limit = 260M ; Default was 128M. Maximum amount of memory a script may consume. 2 MB long mediawiki pages may consume 40M ; Before June 2012 we used: memory_limit = 300M - now slightly reduced: ; NOTE GH: change suhosin.memory_limit in /etc/php5/conf.d/suhosin.in to same value!
upload_max_filesize = 50M ; DEFAULT !!! was just 2M! ; ALSO SET post_max_size = 52 ABOVE! ; This affects both form data and file uploads. This value must be larger than upload_max_filesize!
post_max_size = 52M ; was 5, affects file upload and must be larger fileupload!
allow_call_time_pass_reference = Off
Syntax errors: The log files are relatively silence if a syntax error exists in php.ini. All values will be used with their defaults; thus this situation can be detected by checking Special:Upload whether it displays an upload limit of 2 MB. To manually check php.ini for syntax errors use:
php --info --php-ini /etc/php5/apache2/php.ini
Important for inserting utf-8 text into mysql:
Although a mysql table may be set to CHARACTER SET utf8
, the mysql client's standard charset may still be Latin1. So when inserting utf-8 data with PHP, the client's charset may have to be changed to utf-8 after creating the connection:
$conn = mysql_connect($MySQL_Host, $MySQL_User,$MySQL_Passwd); mysql_query("SET NAMES utf8", $conn);
suhosin.ini
suhosin is installed with:
apt-get install php5-suhosin
Change suhosin.memory_limit in /etc/php5/conf.d/suhosin.ini to same value!
; DEFAULT WAS: suhosin.memory_limit = 0, now same value as in php.ini: suhosin.memory_limit = 600
Note: Unlike in php.ini there is No "M" after the number here!
apc.ini
This is in /etc/php5/conf.d/apc.ini
The status can be viewed by looking at apc_1417869461139.php in the www root.
Our content is (size increased from 30 to 60, especially for mediawiki user-cache usage):
extension=apc.so # apc.shm_segments=3 apc.enabled=true # default: apc.shm_size=30 # i.e. 30 MB apc.shm_size=60
PHPMyAdmin
(Previous apache instructions now obsolete, we now use nginx.)
Note: the path to phpmyadmin has been changed! Please check the phpmyadmin* folder (symlink) inside /var/www for currently correct URL. The purpose of this change was: bots are constantly hitting /phpmyadmin in search of vulnerabilities. Although we keep updating the software regularly, it is safer if they don't know the location. Please do not document the location here!
Django installation
See Django-Installation. For Artenquiz at least django Version 1.3 necessary!
Mediawiki
Mediawiki installation information is available separately: Mediawiki_installation
Tomcat
Important: to get Tomcat5.5 running after installing, it is necessary that the environment variables JAVA_HOME and JRE_HOME are set correctly and that JDK 5 or higher is used.
# NOTE: native tomcat no longer used: # apt-get install tomcat5.5 tomcat5.5-admin tomcat5.5-webapps apt-get install libapache2-mod-jk
In /etc/default/tomcat5.5 remove comment markers from TOMCAT_USER=tomcat55 (default), or another user, and do the same for JAVA_HOME with the correct path (peferably a symbolic link to current version, e.g. JAVA_HOME=/usr/lib/jvm/java-6-sun). The other options only need to be uncommented if you want to override the defaults (which are set in /etc/init.d/tomcat5.5).
Edit /etc/profile (as user root), add:
## For debian standard: export CATALINA_HOME=/usr/share/tomcat5.5 ## Instead, for fedora-tomcat: export CATALINA_HOME=/usr/share/fedora-3.1/tomcat
To make Tomcat work with Apache edit /etc/libapache2-mod-jk/workers.properties and set the two lines (using your correct paths):
workers.tomcat_home=/usr/share/fedora-3.1/tomcat workers.java_home=/usr/local/jdk
At the end of /etc/apache2/apache2.conf add:
# Enable libapache2-mod-jk Include /usr/share/doc/libapache2-mod-jk/httpd_example_apache2.conf
--- Presently, the tomcat used is the one supplied with Fedora, inside the Fedora folder. The main tomcat access is fedora.keytonature.net. However, Lia Veja has installed a secondary point
http://species-id.net/services/
L. Veja writes "it needs some edits in web.xml file for every application you will put there".
Restart Apache and Tomcat.
/etc/init.d/apache2 restart /etc/init.d/tomcat5.5 restart
Fedora
See FEDORA Installation, FEDORA Batch Import
For manual Fedora 3.1 start method see section further below under "Stop services / demons"
Note: Fedora does not install the log folder in the log path of debian. GSearch logs become quickly very large (Gigabytes), so it is important to move them. The following code copies existing logs (assuming /var/log/fedora/ already created) and creates symbolic links. THIS NEEDS TO BE REPEATED WITH EVERY UPDATE OF FEDORA TO A NEW VERSION!
cp -pr /usr/share/fedora/server/logs/* /var/log/fedora/ # destination may be /mnt/dump/var/log/fedora /usr/share/fedora/tomcat/bin/shutdown.sh rm -r /usr/share/fedora/server/logs ln -s /mnt/dump/var/log/fedora /usr/share/fedora/server/logs # Old: /usr/share/fedora/tomcat/bin/startup.sh # New, our own script: /etc/init.d/fedora start
Note 2009-09:
http://fedora.keytonature.net/x/x.jpg
links to /var/lib/tomcat5.5/webapps/x/x.jpg with /usr/share/tomcat5.5/webapps/ being a link to /var/lib/tomcat5.5/webapps/. However the root of http://fedora.keytonature.net/ links to webapps/ROOT, but on 2009-09 was misconfigured to link to fedora-3.1/tomcat/webapps... folder.
Fedora Rebuild Index
In several situations it may be necessary to rebuild the Fedora indices:
- Recovering from inconsistencies and/or corruption of the indexes.
- Upgrading from a previous version of Fedora when the SQL database or Resource Index changed significantly between releases
- Migrating from one SQL database product to another in an existing Fedora installation. Such migration can be done by modifying /usr/share/fedora-3.XXX/server/config/fedora.fcfg to point to a properly-configured <datastore..>
The batch service fedora-rebuild.sh must be started as follows:
* Stop the Fedora server (if using Fedora-Tomcat, this can be done with the /usr/share/fedora/tomcat/bin ./shutdown.sh command) * Run /usr/share/fedora/server/bin ./fedora-rebuild.sh * Select option "1" in order to rebuild indexes * Confirm this procedure with "1" * Restart the Fedora server (if using the Fedora-Tomcat, this can be done with the /usr/share/fedora/tomcat/bin ./startup.sh command)
See also: Fedora Commons documentation.
Webmin
Webmin is a web-based system configuration tool for Linux. With it you can configure many operating system internals, such as users, disk quotas, services, configuration files etc., as well as modify and control many open source apps, such as the Apache HTTP Server, PHP, MySQL etc.
To install, first /etc/apt/sources.list needs to be updated (see above). Then:
sudo apt-get update; sudo apt-get install webmin
You will be asked if you wish to install the package without verification. Select yes. To configure a different port:
sudo nano /etc/webmin/miniserv.conf # change port AND listen options from 10000 to 18010 and restart: sudo /etc/init.d/webmin restart
Only users with sudo rights can login with their usual password. Help for the standard modules used by Webmin can be found here: http://doxfer.com/Webmin/Modules. A wiki page for webmin can be found here: http://doxfer.com/Webmin
To increase security read this and:
- in "Webmin: Webmin Configuration: Authentication" "set "Block hosts with more than 2 failed logins for 60 seconds; Block users with more than 6 failed logins for 12000 seconds" and select "Log blocked hosts, logins and authentication failures to syslog" and "Enable session authentication: Autologout after 10 Minutes" and disable "Offer to remember login permanently?". Rest remaining on defaults.
- Create new certificates, then in "Webmin: Webmin Configuration: SSL Encryption" set: "Private key file: .../ssl-cert-XXX-webmin.key" and "Certificate file: Separate file: .../ssl-cert-XXX-webmin.key" and "Redirect non-SSL requests to SSL mode? Yes".
mc (midnight commander) und jed (editor)
Midnight commander is the Linux equivalent of the Norton Commander. Anyone familiar with Norton Commander should feel right at home. Very useful for browsing through the file system. It has an integrated FTP client, editor and file viewer, and supports the use of a mouse inside an ssh window! Similarly, jed is an editor that supports the use of mouse in ssh. Install both with
sudo apt-get install mc jed
No configuration necessary. Run with
mc jed
cronjobs
DEPRECATED ON BIOWIKIFARM: to install as root user-specific cronjob (copy see nano /var/spool/cron/crontabs/root, but not editable there!), use:
sudo crontab -u root -e # edit roots cron jobs
This uses jed as editor. Format at start of each line is: minutes, hour, day, etc. One can use * for every (hour, day, month, year). Output is normally sent by email. To suppress this redirect output to ">/dev/null 2&>1". Always add a comment after each line: # explain what this does and who installed it.
PREFERRED:
sudo nano /etc/crontab
This is directly editable, allows jobs for different users (www-data, root), and is more transparent to manage.
Security, virus protection, rootkit hunter
rootkit hunter:
# sudo apt-get install rkhunter # OR, consider to install from backport, here squeeze backport a newer version: sudo apt-get -t squeeze-backports install rkhunter
sudo rkhunter --update sudo rkhunter --checkall
clamav:
sudo apt-get install clamav /etc/init.d/clamav-freshclam start # or: stop, restart, etc.
rest not sure. clamav, clamscan etc. don't work. Was not asking for configuration. -> ?
http://www.debianadmin.com/clamav-installation-and-configuration.html looks good, but did not help
General: use on files or folders to change group and rights:
sudo chgrp -R mw-extension-writers crossdomain*; sudo chmod -R 664 crossdomain*; # or if containing folders: sudo chgrp -R mw-extension-writers /var/www/tools/mediaIBIS; sudo chmod -R 775 /var/www/tools/mediaIBIS
Dropbox
(updated 2013-03-06)
Dropbox is installed for a specific user and with manual daemon startup. We created a special user "dropbox" for this task. dropbox is member of www-data, and www-data member of dropbox group. Following: https://www.dropbox.com/install?os=lnx and http://linuxg.net/install-dropbox-on-linux-systems-ubuntu-debian-fedora-and-others/
sudo su dropbox cd ~ && wget -O - "https://www.dropbox.com/download?plat=lnx.x86_64" | tar xzf - cd ~ && wget -O - http://www.dropbox.com/download?plat=lnx.x86_64 | tar xzf - wget https://www.dropbox.com/download?dl=packages/dropbox.py ~/.dropbox-dist/dropboxd
On first installation it will ask to visit a URL from your desktop-browser, to confirm linking the new dropbox with an existing account. In the dropbox web page, go on top to account, then below the tab "My Computers". The procedure is a bit tough, since you have to copy the long link without stopping the command on the command line. For putty use right-click mouse, not Ctrl-C. The mouse-click will copy and paste, looking wrong, but the commandline process keeps running. The process was successfull if the command line on the server issues "Client successfully linked, Welcome (YourName)".
We added a brief shell script "run_dropbox.sh" with:
#!/bin/bash ~/.dropbox-dist/dropboxd& ps
to help memorize Dropbox daemon startup.
Link the Dropbox folder inside dropbox user home into wiki for ease of importing:
cd /var/www/v-species/o; sudo ln -s /home/dropbox/Dropbox "Dropbox"
Tips for simple access of Debian file from remote Windows
Install WinSCP, which will allow to browse the Debian server almost like a Windows explorer (or a graphical Midnight commander). It allows to easily and securely (using sftp, we do not use ftp) create folders, copy files within the server, or upload/download files to your machine.
WinSCP also very nicely integrates with PuTTY, a ssh client. To make this as simple as possible, do the following:
- You can either
- install PuTTY (copy all files from the zip with all executables and help) into the Putty subfolder of WinSCP
- Or (perhaps already) have installed it anywhere else
- In any event go in WinSCP to the menu: Options/Preferences, then in the Dialog box: integration/applications and check the path. It should be something like "%PROGRAMFILES%\PuTTY\putty.exe" if putty has been installed in that folder.
- In the same dialog, turn on Remember... and Automatically...
- Now whenever you open WinSCP, a second window with ssh will automatically be opened.
- To change the display options in PuTTY, go to the Window menu (top left), Change settings (e. g. Window, Color, Use System colors will use black-on-white text if you have set you Windows command window that way). Under Window increase Columns (e.g. to 120), Rows (e.g. to 63), and scrollback (e.g. to 900) to suit your taste. On "Session" itself select "Default" and click "Save" to preserve your changes.
Reversely, adding NTFS support to Debian: In our case this was desirable to add read/write support for NTFS-Formatted USB-disks. Provided backports are installed for apt-get (as instructed above), one can install:
aptitude install ntfs-3g
To temporarily mount from command line use (assuming the usb-disk is xvdf, check in Xen Center)
mount -t ntfs-3g /dev/xvdf1 /mnt/usb-disk # (if disk is fat32, use: mount -t auto /dev/xvdf1 /mnt/usb-disk)
To mount the NTFS during boot add the following to the end of the /etc/fstab file:
/dev/xvdf1 /mnt/usb-disk ntfs-3g defaults 0 0
See ntfs-3g manual for further information. Some note mention that "To mount files with non-ASCII characters one may have to give the option -o locale=XXX to the mount options of ntfs-3g; see http://ntfs-3g.org/support.html#locale for further information." We did not do so far, using UTF-8 seems to work.
Moving files to a new server directly
Instead of backup/restore it is possible to go, on source machine, to the folder under which the subfolders shall be moved, then tar, ssh transfer, unpack. Caveat: on the destination machine the login MUST be root, else the user and group information will be lost. Temporarily change sudo nano /etc/ssh/sshd_config to permit root login!
cd / # as root, being on old server: sudo tar -czf - --numeric-owner root home var | ssh root@212.201.100.117 tar -xzf - --same-owner -C /mnt/dump/_oldroot_TEMP sudo tar -czf - --numeric-owner var | ssh root@212.201.100.117 tar -xzf - --same-owner -C /mnt/storage/oldroot cd /mnt/storage sudo tar -czf - --numeric-owner * | ssh root@212.201.100.117 tar -xzf - --same-owner -C /mnt/storage cd /mnt/dump sudo tar -czf - --numeric-owner * | ssh root@212.201.100.117 tar -xzf - --same-owner -C /mnt/dump cd /usr/share sudo tar -czf - --numeric-owner mediaw* | ssh root@212.201.100.117 tar -xzf - --same-owner -C /mnt sudo tar -czf - --numeric-owner fedora* Fedora* | ssh root@212.201.100.117 tar -xzf - --same-owner -C /mnt
Note: when moving, the user and group ID should be aligned, see: http://it.toolbox.com/blogs/locutus/how-to-change-a-users-uid-and-gid-26368