MediaWiki based online Tasks management for ingest tool
Contents
Generalities
The issue of MediaWiki view of Tasks management for Ingest Tool naturally close a framework that begin with a medatata submision in MediaWiki and ends with metadata ingestion in Fedora Commons repository. Ingest Tool uses a transactional Tasks Management for all the operation. See also: [[1]]
During the ingestion, the work flow is driven by a MySQL database, fedoralogs. There are 5 tables which contains:
- tasks;
- successful jobs;
- waiting jobs;
- failure jobs;
- last harvesting time.
Technically, the Tasks management online view has two components:
- MediaWiki Extension JSHTMLWidget
- Java based xml exporting component which export periodically work database for ingest tool.
Java based xml exporting component
A simple java based application which read periodically fedoralogs database and export all 5th tables as xml. This application can be manually started or can be included scheduled into crontab.
XML file structure
The xml file has a fixed name, "time.xml", and will be exported while k2n.xml.transmit=true. It will be stored in a place established by k2n.properties file as follows:
# export working database fedoralogs k2n.xml.transmit=true # storage for xml file generated from fedoralogs k2n.xml.files=/var/www/v-k2n/w/media/K2N-files/
time.xml file structure:
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<ROOT>
<Times>
<row>
<field name="Last_Time">2009-12-28 22:03:38.0</field>
<field name="IDTask">K2N_28-12-09T22-03-34</field>
</row>
</Times>
<Running>
<row>
<field name="E_Date_O">2009-09-25 17:32:29.0</field>
<field name="Name">Interactive_Flora_of_the_British_Isles_(ETI)</field>
<field name="HarvestedSource">http://160.45.63.55/testwiki/</field>
<field name="Base">http://www.keytonature.eu/wiki/</field>
<field name="IDTasks">K2N_25-09-09T17-49-20</field>
<field name="Modified">2009-08-18 14:23:47.0</field>
<field name="Number">12265</field>
<field name="Running">6</field>
<field name="Images">1</field>
<field name="type">3</field>
<field name="B_Date_P">2009-09-25 15:06:17.0</field>
</row>
...................
</Running>
<Waiting>
<row>
<field name="Name">Database_of_Invertebrate_Pictures_(PMSL)</field>
<field name="HarvestedSource">http://160.45.63.55/testwiki/</field>
<field name="Base">http://www.keytonature.eu/wiki/</field>
<field name="IDTasks">K2N_28-12-09T22-03-34</field>
<field name="Modified">2009-12-23 23:02:31.0</field>
<field name="Images">0</field>
<field name="B_Date_P">2009-12-28 22:03:38.0</field>
</row>
....................
</Waiting>
<Failure>
<row>
<field name="E_Date_O">2000-01-01 00:00:00.0</field>
<field name="Name">Butterflies_and_Moths_of_the_World_(NHM</field>
<field name="HarvestedSource">http://160.45.63.55/testwiki/</field>
<field name="Base">http://www.keytonature.eu/wiki/</field>
<field name="E_Date_P">2009-11-10 09:03:25.0</field>
<field name="IDTasks">K2N_10-11-09T09-03-23</field>
<field name="Modified">2009-11-03 00:06:56.0</field>
<field name="Images">0</field>
<field name="type">1</field>
<field name="B_Date_P">2009-11-10 09:03:24.0</field>
</row>
</Failure>
<Success>
<row>
<field name="E_Date_O">2009-12-13 16:40:09.0</field>
<field name="Name">Postcode_Plants_Database_(NHM)</field>
<field name="HarvestedSource">http://160.45.63.55/testwiki/</field>
<field name="Base">http://www.keytonature.eu/wiki/</field>
<field name="E_Date_P">2009-12-13 16:40:36.0</field>
<field name="IDTasks">K2N_13-12-09T15-47-19</field>
<field name="Modified">2009-12-10 15:06:06.0</field>
<field name="Number">5215</field>
<field name="Images">1</field>
<field name="type">1</field>
<field name="B_Date_P">2009-12-13 15:47:41.0</field>
</row>
...................
</Success>
</ROOT>
JSHTMLWidget Extension
JSHTMLWidget is an extension that allows the instantiation of a html widget and inclusion of dependent js/css files. Due to security reasons there are special directories where the html files are placed. Also the js and css files that are referenced from the resulting page need to reside in a special location (in order to prevent cross site scripting attacks).
Use of JSHTMLWidget
<JSHTMLWidget htmlbody="widget.html" jsincludes="file1.js|file2.js|..|filen.js" cssincludes="file1.css|file2.css|...|filen.css"> </JSHTMLWidget>
The above markup will be expanded in with the content of the “widget.html” file. Also the jsincludes files and cssinclude files will be referenced in the <head> section of the rendered page.
- Requirements
In order to be able to include the specified js and css files the $wgAllowUserJs and $wgAllowUserCss variables need to be set to the value “true” in the LocalSettings.php (or whatever configuration file is used).
TaskList Widget
The TaskList widget uses the JSHTMLWidget extension. The widget uses the following files:
- TaskMonitor/body.html – the html body of the widget.
- TaskMonitor/refresh.js – the js file that contains the js code that send AJAX requests to the server in order to update the status of the tasks;
- TaskMonitor/taskmonitor.css – contains css markup used by the html elements in the widget’s body;
- It also uses the JQuery library for DOM manipulation and JTemplates for manipulation of html templates in javascript.
TaskList widget periodically sends an AJAX request to the server to update the status of the tasks displayed in the page.
Use of TaskList widget
<JSHTMLWidget htmlbody="TaskMonitor/body.html" jsincludes="lib/jquery/jquery-1.3.1.js|lib/jTemplates/jquery-jtemplates_uncompressed.js|TaskMonitor/refresh.js" cssincludes="TaskMonitor/taskmonitor.css"></JSHTMLWidget>
Notice: Special thanks to my friend Cristian Botau who helping me in this extension development. --LiaVeja 12:05, 6 January 2010 (CET)
