Difference between revisions of "Mediawiki XML page importing"
m (→Create a page (first time): added hints) |
m (→Create a page (first time): hint to run maintenance) |
||
Line 75: | Line 75: | ||
# <text xml:space="preserve">Wiki text</text> | # <text xml:space="preserve">Wiki text</text> | ||
Then put it together with the exported {{abbr|XML}} document header: | Then put it together with the exported {{abbr|XML}} document header: | ||
− | <blockquote>< | + | <blockquote><syntaxhighlight lang="xml"> |
<mediawiki xmlns="http://www.mediawiki.org/xml/export-0.4/" | <mediawiki xmlns="http://www.mediawiki.org/xml/export-0.4/" | ||
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" | xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" | ||
Line 98: | Line 98: | ||
</page> | </page> | ||
</mediawiki> | </mediawiki> | ||
− | </ | + | </syntaxhighlight></blockquote> |
This will create the page “GBIF:cultivar”. | This will create the page “GBIF:cultivar”. | ||
Line 120: | Line 120: | ||
Before you use the import page '''Special:Import''', make sure you have valid XML. Use a XML checker on your computer first if possible. | Before you use the import page '''Special:Import''', make sure you have valid XML. Use a XML checker on your computer first if possible. | ||
+ | |||
+ | After a large import of many pages it is recommended to run maintenance scripts from the command line to update category data or data for semantic media wiki: | ||
+ | <blockquote><syntaxhighlight lang="bash"> | ||
+ | cd /var/www/v-xxxwiki/w; sudo php ./maintenance/rebuildall.php --conf ./LocalSettings.php | ||
+ | cd /var/www/v-xxxwiki/w; sudo php ./maintenance/runJobs.php --conf ./LocalSettings.php --procs=3 | ||
+ | </syntaxhighlight></blockquote> | ||
==Creating exports from a database== | ==Creating exports from a database== |
Revision as of 12:57, 8 November 2013
This is about importing the text of wiki pages using an xml format. For images or other files see: Batch importing files into MediaWiki.
Contents
Creating an export to be then reimported
Use Command Line (shell access to server required!):
cd /var/www/testwiki; php ./maintenance/dumpBackup.php --full --conf ./LocalSettings.php > ./testwikiexport.xml
(See also DumpBackup.php)
or the special page Special:Export. Highly interesting: Parameters_to_Special:Export
Importing Data from Command Line Interface
This is the preferred method, as it does not create additional versions (compare next section).
Import works directly with 7z (or zip, bzip) compressed xml files! HOWEVER, PRESENTLY .7z DOES NOT WORKTransfer the xml file to the server, and execute (example):
cd /var/www/v-xxx/w; sudo php ./maintenance/importDump.php /var/www/v-xxx/w/import.xml --conf ./LocalSettings.php
cd /var/www/v-xxx/w; sudo php ./maintenance/rebuildall.php --conf ./LocalSettings.php
cd /var/www/v-xxx/w; sudo php ./maintenance/runJobs.php --conf ./LocalSettings.php --procs=3
# e.g. FOR OPENMEDIA:
cd /var/www/v-species/o; sudo php ./maintenance/importDump.php ./atmp/import.xml --conf ./LocalSettings.php
cd /var/www/v-species/o; sudo php ./maintenance/rebuildall.php --conf ./LocalSettings.php
cd /var/www/v-species/o; sudo php ./maintenance/runJobs.php --conf ./LocalSettings.php --procs=3
# e.g. FOR Naturführer:
cd /var/www/v-on/w; sudo php ./maintenance/importDump.php ./atmp/import.xml --conf ./LocalSettings.php
cd /var/www/v-on/w; sudo php ./maintenance/rebuildall.php --conf ./LocalSettings.php
cd /var/www/v-on/w; sudo php ./maintenance/runJobs.php --conf ./LocalSettings.php --procs=3
(Rebuilding internal indices is necessary after import; rebuildall may be slow and can be replaced with
cd /var/www/v-on/w; sudo php ./maintenance/rebuildrecentchanges.php --conf ./LocalSettings.php
if necessary. RunJobs: if import contains complex template relations or when updating template relations, data entries in templates, manually emptying the job queue may be necessary, check Special:Statistics in the wiki. Note: "--procs=3" will run three jobs in parallel, if the server has the necessary number of processor cores.)
Important: for all batchimporting, revisiondate must be set to something newer than all old revisions; else mediawiki will sort the imported revision behind existing revisions. The id in the imported xml is not necessary, however.
Batch deleting pages
Example:
sudo php ./maintenance/deleteBatch.php \
--conf ./LocalSettings.php \
-r "remove wrong resolution" ./maintenance/deleteBatch.txt
deleteBatch.txt contains only the filenames.
Note that for File:x.jpg pages, this will delete the file itself AND the page itself, but the file will still seem to exist when called. Only manually clicking "delete all" in file history will finish this. This is probably a bug, the php code attempts to handle file deletions.
Importing Data through Special Pages Web Interface
The Web interface under Special:Import will create extra revisions (in addition to those imported) designating the importing user. If you don't want to document who did a transfer, it may therefore be desirable to use the command-line version (see below). For the web import it may be desirable to create a special "Import-User" so that the name better documents authorship than using a normal username during upload of the xml file. Important creates two revisions for each page: Revision 1 is the imported revision, Revision 2 is the revision documenting the import process. If the imported data alone document this (e.g. when they already are using Import-User and an appropriate comment), it is possible to delete the second revisions in the database (assuming Import-User has ID=4):
Delete FROM PREFIX_revision WHERE PREFIX_revision.rev_user=4 AND PREFIX_revision.rev_minor_edit=1; -- Then need to fix the latest revision stored in page: UPDATE PREFIX_revision AS R2 INNER JOIN (PREFIX_page LEFT JOIN PREFIX_revision AS R1 ON PREFIX_page.page_latest=R1.rev_id) ON R2.rev_page=PREFIX_page.page_id SET page_latest=R2.rev_id WHERE R1.rev_id Is Null
Create a page (first time)
To do this, export a single page to get the XML-header and footer you need as a template later on. For creating a page, the important things are:
- <contributor>your user name</contributor> and your ID <id>123</id> (Preferences-tab User profile see Username: + <uid>)
- <title>new title</title> and the
- <text xml:space="preserve">Wiki text</text>
Then put it together with the exported XML document header:
<mediawiki xmlns="http://www.mediawiki.org/xml/export-0.4/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.mediawiki.org/xml/export-0.4/ http://www.mediawiki.org/xml/export-0.4.xsd" version="0.4" xml:lang="en"> <siteinfo> <!-- … the XML header from an arbitrary wiki page export --> </siteinfo> <page> <title>GBIF:cultivar</title> <revision> <contributor><username>User name</username><id>123</id></contributor> <text xml:space="preserve">{{Term | collection = GBIF | short URI = cultivar | full URI = http://vocabularies.gbif.org/services/gbif/taxon_rank/cultivar | label = cultivar | code = cultivar | see also = http://rs.gbif.org/vocabulary/gbif/rank.xml }}</text> </revision> </page> </mediawiki>
This will create the page “GBIF:cultivar”.
Note it is:
<text xml:space="preserve">{{Term
...
}}</text>
| … and not with line break after < xml:space="preserve"> … |
<text xml:space="preserve">
{{Term
...
}}
</text>
|