OAI-PMH service for K2N Fedora Commons based repository

The current document should be considered work-in-progress. Insofar as we achieve advanced knowledge about OAI Provider Service for Fedora Commons based repository, we will complete the present document. Any contribution are welcome!

1 Generalities regarding OAI-PMH protocol
- 1.1 Protocol Requests and Responses
- 1.2 OAI Provider for Fedora Commons Repository
2 OAI Provider Service Installation
- 2.1 Installation
- 2.2 Demos ingestion
3 How to configure OAI Provider Service
4 First successful tests
- 4.1 Digital objects used for test purpose
  - 4.1.1 ListRecords verb test results
  - 4.1.2 GetRecord verb test results
5 Further thoughts and plans
6 Conclusions: Is this service reliable for our purpose or not?
7 K2N-DCMI-Europeana metadata elements mapping proposal
8 References

Generalities regarding OAI-PMH protocol

The Open Archives Initiative Protocol for Metadata Harvesting (referred to as the OAI-PMH in the remainder of this document) provides an application-independent interoperability framework based on metadata harvesting. There are two classes of participants in the OAI-PMH framework:

Data Providers administer systems that support the OAI-PMH as a means of exposing metadata; and
Service Providers use metadata harvested via the OAI-PMH as a basis for building value-added services.

For the moment, we are interested in exposing our K2N metadata as Data Provider to harvesters (Service Providers) such Europeana. Fedora Commons repository has been provided with such a rough service since 2.2.1 version. Actually, since 3.0 version, with major changes in Fedora Commons Content Model approach, the OAI Provider service it seems to be very flexible and we believe, reliable. The current implementation is Fedora Commons 3.0-3.2 compatible.

Protocol Requests and Responses

1. GetRecord

Arguments:

- identifier a required argument that specifies the unique identifier of the item in the repository from which the record must be disseminated.
- metadataPrefix a required argument that specifies the metadataPrefix of the format that should be included in the metadata part of the returned record . A record should only be returned if the format specified by the metadataPrefix can be disseminated from the item identified by the value of the identifier argument.

2. Identify

3. ListIdentifiers

Arguments:

- from an optional argument with a UTCdatetime value, which specifies a lower bound for datestamp-based selective harvesting.
- until an optional argument with a UTCdatetime value, which specifies a upper bound for datestamp-based selective harvesting.
- metadataPrefix a required argument, which specifies that headers should be returned only if the metadata format matching the supplied metadataPrefix is available or, depending on the repository's support for deletions, has been deleted. The metadata formats supported by a repository and for a particular item can be retrieved using the ListMetadataFormats request.
- set an optional argument with a setSpec value , which specifies set criteria for selective harvesting.
- resumptionToken an exclusive argument with a value that is the flow control token returned by a previous ListIdentifiers request that issued an incomplete list.

4. ListMetadataFormats

Arguments:

- identifier an optional argument that specifies the unique identifier of the item for which available metadata formats are being requested.

5. ListRecords

Arguments:

- from an optional argument with a UTCdatetime value, which specifies a lower bound for datestamp-based selective harvesting.
- until an optional argument with a UTCdatetime value, which specifies a upper bound for datestamp-based selective harvesting.
- metadataPrefix a required argument, which specifies that headers should be returned only if the metadata format matching the supplied metadataPrefix is available or, depending on the repository's support for deletions, has been deleted. The metadata formats supported by a repository and for a particular item can be retrieved using the ListMetadataFormats request.
- set an optional argument with a setSpec value , which specifies set criteria for selective harvesting.
- resumptionToken an exclusive argument with a value that is the flow control token returned by a previous ListIdentifiers request that issued an incomplete list.

6. ListSets

Arguments:

- resumptionToken an exclusive argument with a value that is the flow control token returned by a previous ListSets request that issued an incomplete list.

For details, see also OAI-PMH Protocol Requests and Responses

OAI Provider for Fedora Commons Repository

The new provider is based on PROAI, an open source caching, polling OAI provider. It has the following features:

Supports any metadata format available through your Fedora Repository via a Datastream or dissemination service
Supports sets that are expressed as RDF relationships in your digital objects' RELS-EXT Datastreams as exposed via the Resource Index
Runs as a Web application in any servlet container, acting as a Web service client to the Fedora Repository
Caches the content of Fedora Repository disseminations and Datastreams intended to be exposed as OAI records allowing for fast response times and ensuring that the OAI provider can continue to run even when the Fedora Repository is temporarily stopped.

OAI Provider Service Installation

Download from Fedora Commons Services oaiprovider distribution under Fedora Commons 3.0 version. The .src distribution contains under "src/demo/" directory very useful .foxml demo files. See oaiprovider-1.2-src.zip For advanced users, accustomed already with oaiprovider service, the simple oaiprovider-1.2.zip it's enough.

Installation

To install the service:

1. Make sure you have a suitable database installed (MySQL, PostgreSQL, Oracle, or McKoi) and a database user account that can create tables in the database. We used the same user that works with fedora32 MySQL database.
2. Make sure your Fedora Repository is running with the Resource Index turned ON. This is necessary because the OAI provider periodically queries the resource index to discover which records of interest have changed.
3. Deploy the oaiprovider.war file into your servlet container (Tomcat 5.5).
4. Configure the OAI Provider as described in the Configuration section below by editing proai.properties file .
5. Re-start the Web application (this is often done by restarting the servlet container itself).

Demos ingestion

Complete installation steps 1-4 above. Start with the default values in the proai.properties configuration file and ensure the following properties are set according to your own Fedora Repository installation:
- driver.fedora.baseURL
- driver.fedora.user
- driver.fedora.pass
Make sure your Fedora Repository installation is configured to retain PIDs of objects in the "demo" PID namespace on ingest. You can check this in your fedora.fcfg file: If one of the values of "retainPIDs" is "demo" or "*" (asterisk), your repository is configured correctly. Otherwise, you should add this value and re-start it.
Use the fedora-admin GUI or fedora-ingest command-line utility to ingest all demonstration objects in the src/test/foxml directory of the Fedora OAI Provider service source distribution.
Start the Web application.

How to configure OAI Provider Service

The configuration file, proai.properties file contains 10 sections, as follows:

1. Front-End OAI-PMH Behavior - this specifies the maximum time to wait between requests before cleaning up the session
2. Back-End Update Behavior - sets how often Proai should poll the driver for updates
3. File Storage Locations - the directory where cache files should be stored
4. Primary Database Configuration - database used for the cache
5. Advanced Database Configuration - sets connections from the pool 
6. Fedora Driver: General Settings - settings need for proai.driver.OAIDriver 
7. Fedora Driver: Metadata Format Configuration - metadata formats to make available - very important!
8. Fedora Driver: Set Membership Configuration - the name of the RELS-EXT property that indicates the unique setSpec 
string of a Fedora object representing an OAI set.
9. Fedora Driver: MPTStore Configuration - if Fedora's Resource Index is backed by MPTStore
10. Fedora Driver: Advanced Configuration - RELS-EXT predicate settings that indicates the state of all records within the object.

Briefing, we made the following settings (only sections that need changes):

Back-End Update Behavior

proai.validateUpdates = false
We don't have a functional schema for our k2n RELS-EXT datastream yet.

Primary Database Configuration

proai.db.url = jdbc:mysql://localhost/proai?useUnicode=true&characterEncoding=UTF-8&autoReconnect=true
proai.db.driverClassName = com.mysql.jdbc.Driver
proai.db.username        = fedoraAdmin
proai.db.password        = fedoraPassword

Fedora Driver: General Settings

driver.fedora.baseURL = http://localhost:8080/fedora/
driver.fedora.user = fedoraAdmin
driver.fedora.pass = fedoraPassword

Fedora Driver: Metadata Format Configuration

driver.fedora.md.formats = oai_dc test_format formatX formatY k2n
driver.fedora.md.format.k2n.loc =  http://example.org/k2n.xsd 
** Notice: we keep http://example.org/ for our tests.
driver.fedora.md.format.k2n.dissType = info:fedora/*/RELS-EXT

First successful tests

First tests have been performed on a local installed Fedora Commons 3.2. Some elements of the proai.properties file was set as follows:

proai.validateUpdates = false

Digital objects used for test purpose

demo digital objects ingested via Fedora Commons Admin Client GUI interface;
some digital objects from Algae (UNITS) collection used only for test purpose and transformed in order to be OAI Provider compliant.

Notice: It's need to perform more tests here.

After firsts test, step-by-step modification both in K2N Fedora Commons repository and proai.properties file, finally, some results it seems to come into sight.

ListRecords verb test results

At the request:

http://localhost:8080/oaiprovider/?verb=ListRecords&metadataPrefix=k2n

The response for new ingested digital object RELS-EXT datastream is:

<?xml version="1.0" encoding="UTF-8"?>
 <OAI-PMH xmlns="http://www.openarchives.org/OAI/2.0/"
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:schemaLocation="http://www.openarchives.org/OAI/2.0/
                            http://www.openarchives.org/OAI/2.0/OAI-PMH.xsd">
  <responseDate>2009-11-12T21:38:41Z</responseDate>
  <request verb="ListRecords" metadataPrefix="k2n">http://localhost:8080/oaiprovider/</request>
<ListRecords>
<record xmlns="http://www.openarchives.org/OAI/2.0/">
 <header>
   <identifier>oai:example.org:item22</identifier>
   <datestamp>2009-11-12T21:33:39Z</datestamp>
 </header>
 <metadata>
<rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:rdfs="http://www.w3.org/2000/01/rdf-schema#"
xmlns:fedora="info:fedora/fedora-system:def/relations-external#" xmlns:k2n="http://example.org/k2n/" xmlns:fedora- 
model="info:fedora/fedora-system:def/model#" xmlns="http://www.fedora.info/definitions/" xmlns:oai="http://www.openarchives.org 
/OAI/2.0/">
 <rdf:Description rdf:about="info:fedora/K2N:SI_8fdaa4be929fad338e7442f58c5c13">
      <k2n:Normal_Preview_Availability>online (free)</k2n:Normal_Preview_Availability>
      <k2n:Creation_Date>2008</k2n:Creation_Date>
      <k2n:Scientific_Names>Halimeda tuna (J. Ellis et Solander) J.V.Lamouroux</k2n:Scientific_Names>
      <k2n:Format>jpg</k2n:Format>
      <k2n:Country_Names>Italy</k2n:Country_Names>
      <k2n:Creators>Diego Poloniato</k2n:Creators>
      <k2n:Best_Quality_URI>http://dbiodbs.units.it/quint/al/foto/AL000501.jpg.jpg</k2n:Best_Quality_URI>
      <k2n:Metadata_Creator>Annalisa Falace</k2n:Metadata_Creator>
      <k2n:Best_Quality_Availability>online (free)</k2n:Best_Quality_Availability>
      <k2n:Copyright_Statement>Copyright of the author</k2n:Copyright_Statement>
      <k2n:Normal_Preview_URI>http://dbiodbs.units.it/quint/al/foto/pics/AL000501.jpg.jpg</k2n:Normal_Preview_URI>
      <k2n:License_Statement>To be discussed with the author</k2n:License_Statement>
      <k2n:Collection_By_Resource_ID>Algae_(UNITS)</k2n:Collection_By_Resource_ID>
      <k2n:Resource_ID>http://dbiodbs.units.it/quint/al/foto/AL000501.jpg</k2n:Resource_ID>
      <k2n:Metadata_Language>en</k2n:Metadata_Language>
      <k2n:Taxon_Category>Algae</k2n:Taxon_Category>
      <oai:itemID>oai:example.org:item22</oai:itemID>
     <fedora:isMemberOf rdf:resource="info:fedora/K2N:Collection_1e52cf53dfa639a4c94dc1396f37aa"></fedora:isMemberOf>
     <fedora:serviceProvidedBy rdf:resource="info:fedora/K2N:Provider_4f2e3b2b2b85ac2b1f9638b08b87a8"</fedora:serviceProvidedBy>
   </rdf:Description>
  </rdf:RDF>
 </metadata>
</record>
<record xmlns="http://www.openarchives.org/OAI/2.0/">
 <header>
   <identifier>"""oai:example.org:item22"""</identifier>
   <datestamp>2009-11-12T20:09:51Z</datestamp>
 </header>
 <metadata>
<rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
 <rdf:Description rdf:about="info:fedora/demo:SI_ex1">
   <itemID xmlns="http://www.openarchives.org/OAI/2.0/">"oai:example.org:item22"</itemID>
   <isMemberOf xmlns="info:fedora/fedora-system:def/relations-external#" rdf:resource="info:fedora/demo:SetPrime"></isMemberOf>
   <serviceProvidedBy xmlns="info:fedora/fedora-system:def/relations-external#" rdf:resource="info:fedora/demo:SetPrime">
</serviceProvidedBy>
   <Scientific_Names xmlns="http://example.org/k2n/">"Halimeda tuna (J. Ellis et Solander) J.V.Lamouroux"</Scientific_Names>
 </rdf:Description>
</rdf:RDF>
 </metadata>
</record>
<record xmlns="http://www.openarchives.org/OAI/2.0/">
 <header>
   <identifier>"""\\""oai:example.org:item22\\"""""</identifier>
   <datestamp>2009-11-12T20:11:54Z</datestamp>
 </header>
 <metadata>
<rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
 <rdf:Description rdf:about="info:fedora/demo:SI_ex1">
   <itemID xmlns="http://www.openarchives.org/OAI/2.0/">"\"oai:example.org:item22\""</itemID>
   <isMemberOf xmlns="info:fedora/fedora-system:def/relations-external#" rdf:resource="info:fedora/demo:SetPrime"></isMemberOf>
   <serviceProvidedBy xmlns="info:fedora/fedora-system:def/relations-external#" rdf:resource="info:fedora/demo:SetPrime"> 
</serviceProvidedBy>
   <Scientific_Names xmlns="http://example.org/k2n/">Halimeda tuna (J. Ellis et Solander) J.V.Lamouroux</Scientific_Names>
 </rdf:Description>
</rdf:RDF>
 </metadata>
</record>
<record xmlns="http://www.openarchives.org/OAI/2.0/">
 <header>
   <identifier>oai:example.org:item23</identifier>
   <datestamp>2009-11-12T21:13:26Z</datestamp>
 </header>
 <metadata>
<rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:rdfs="http://www.w3.org/2000/01/rdf-schema#"  
xmlns:fedora="info:fedora/fedora-system:def/relations-external#" xmlns:k2n="http://example.org/k2n/"  
xmlns:oai="http://www.openarchives.org/OAI/2.0/" xmlns:fedora-model="info:fedora/fedora-system:def/model#" 
xmlns="http://www.fedora.info/definitions/">
              <rdf:Description rdf:about="info:fedora/demo:SI_ex2">
                 <k2n:Normal_Preview_Availability>online (free)</k2n:Normal_Preview_Availability>
                 <k2n:Creation_Date>2008</k2n:Creation_Date>
                 <k2n:Scientific_Names>Halimeda tuna (J. Ellis et Solander) J.V.Lamouroux</k2n:Scientific_Names>
                 <k2n:Format>jpg</k2n:Format>
                 <k2n:Country_Names>Italy</k2n:Country_Names>
                 <k2n:Creators>Diego Poloniato</k2n:Creators>
                 <k2n:Best_Quality_URI>http://dbiodbs.units.it/quint/al/foto/AL000500.jpg.jpg</k2n:Best_Quality_URI>
                 <k2n:Metadata_Creator>Annalisa Falace</k2n:Metadata_Creator>
                 <k2n:Best_Quality_Availability>online (free)</k2n:Best_Quality_Availability>
                 <k2n:Copyright_Statement>Copyright of the author</k2n:Copyright_Statement>
                 <k2n:Normal_Preview_URI>http://dbiodbs.units.it/quint/al/foto/pics/AL000500.jpg.jpg</k2n:Normal_Preview_URI>
                 <k2n:License_Statement>To be discussed with the author</k2n:License_Statement>
                 <k2n:Collection_By_Resource_ID>Algae_(UNITS)</k2n:Collection_By_Resource_ID>
                 <k2n:Resource_ID>http://dbiodbs.units.it/quint/al/foto/AL000500.jpg</k2n:Resource_ID>
                 <k2n:Metadata_Language>en</k2n:Metadata_Language>
                 <k2n:Taxon_Category>Algae</k2n:Taxon_Category>
                 <oai:itemID>oai:example.org:item23</oai:itemID>
                 <fedora:isMemberOf rdf:resource="info:fedora/K2N:Collection_1e52cf53dfa639a4c94dc1396f37aa"></fedora:isMemberOf>
                 <fedora:serviceProvidedBy rdf:resource="info:fedora/K2N:Provider_4f2e3b2b2b85ac2b1f9638b08b87a8"> 
</fedora:serviceProvidedBy>
              </rdf:Description>
           </rdf:RDF>
 </metadata>
</record>

For updated RELS-EXT datastream, the response is:

<record xmlns="http://www.openarchives.org/OAI/2.0/">
 <header>
   <identifier>K2N:SI_8fdaa4be929fad338e7442f58c5c13</identifier>
   <datestamp>2009-11-12T21:37:42Z</datestamp>
 </header>
<metadata>
 <rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
  <rdf:Description rdf:about="info:fedora/K2N:SI_8fdaa4be929fad338e7442f58c5c13">
   <Normal_Preview_Availability xmlns="http://example.org/k2n/">"online (free)"</Normal_Preview_Availability>
   <Creation_Date xmlns="http://example.org/k2n/">"2008"</Creation_Date>
   <Scientific_Names xmlns="http://example.org/k2n/">"Halimeda tuna (J. Ellis et Solander)  
J.V.Lamouroux&quot</Scientific_Names>
   <Format xmlns="http://example.org/k2n/">"jpg"</Format>
   <Country_Names xmlns="http://example.org/k2n/">"Italy"</Country_Names>
   <Creators xmlns="http://example.org/k2n/">"Diego Poloniato"</Creators>
   <Best_Quality_URI xmlns="http://example.org/k2n/">http://dbiodbs.units.it/quint/al/foto/AL000501.jpg.jpg</Best_Quality_URI>
   <Metadata_Creator xmlns="http://example.org/k2n/">Annalisa Falace</Metadata_Creator>
   <Best_Quality_Availability xmlns="http://example.org/k2n/">online (free)</Best_Quality_Availability>
   <Copyright_Statement xmlns="http://example.org/k2n/">"Copyright of the author"</Copyright_Statement>
   <Normal_Preview_URI xmlns="http://example.org/k2n/">http://dbiodbs.units.it/quint/al/foto  
/pics/AL000501.jpg.jpg</Normal_Preview_URI>
   <License_Statement xmlns="http://example.org/k2n/">"To be discussed with the author"</License_Statement>
   <Collection_By_Resource_ID xmlns="http://example.org/k2n/">"Algae_(UNITS)"</Collection_By_Resource_ID>
   <Resource_ID xmlns="http://example.org/k2n/">"http://dbiodbs.units.it/quint/al/foto/AL000501.jpg%22</Resource_ID>
   <Metadata_Language xmlns="http://example.org/k2n/">"en"</Metadata_Language>
   <Taxon_Category xmlns="http://example.org/k2n/">"Algae"</Taxon_Category>
   <itemID xmlns="http://www.openarchives.org/OAI/2.0/">K2N:SI_8fdaa4be929fad338e7442f58c5c13</itemID>
   <isMemberOf xmlns="info:fedora/fedora-system:def/relations-external#"
rdf:resource="info:fedora/K2N:Collection_1e52cf53dfa639a4c94dc1396f37aa"></isMemberOf>
   <serviceProvidedBy xmlns="info:fedora/fedora-system:def/relations-external#" 
rdf:resource="info:fedora/K2N:Provider_4f2e3b2b2b85ac2b1f9638b08b87a8"></serviceProvidedBy>
 </rdf:Description>
</rdf:RDF>
 </metadata>
</record>
</ListRecords>
</OAI-PMH>

At the request:

http://localhost:8080/oaiprovider/?verb=ListRecords&metadataPrefix=oai_dc

The response is:

<?xml version="1.0" encoding="UTF-8"?>
 <OAI-PMH xmlns="http://www.openarchives.org/OAI/2.0/"
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:schemaLocation="http://www.openarchives.org/OAI/2.0/ http://www.openarchives.org/OAI/2.0/OAI-PMH.xsd">
  <responseDate>2009-11-17T10:47:17Z</responseDate>
   <request verb="ListRecords" metadataPrefix="oai_dc">http://localhost:8080/oaiprovider/</request>
      <ListRecords>
        <record xmlns="http://www.openarchives.org/OAI/2.0/">
          <header>
            <identifier>oai:example.org:item1</identifier>
            <datestamp>2009-11-09T20:26:16Z</datestamp>
          </header>
          <metadata>
           <oai_dc:dc xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:oai_dc="http://www.openarchives.org/OAI/2.0/oai_dc/"  
             xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.openarchives.org/OAI/2.0/oai_dc/
                http://www.openarchives.org/OAI/2.0/oai_dc.xsd">
                  <dc:identifier>urn:example:resource1</dc:identifier>
           </oai_dc:dc>
          </metadata>
          <about>
           <provenance xmlns="http://www.openarchives.org/OAI/2.0/provenance" xmlns:xsi="http://www.w3.org/2001/XMLSchema-
               instance"  xsi:schemaLocation="http://www.openarchives.org/OAI/2.0/provenance
                                  http://www.openarchives.org/OAI/2.0/provenance.xsd">
               <originDescription altered="true" harvestDate="2002-02-02T14:10:02Z">
                <baseURL>http://the.oa.org</baseURL>
                <identifier>oai:r2:klik001</identifier>
                <datestamp>2002-01-01</datestamp>
                <metadataNamespace>http://www.openarchives.org/OAI/2.0/oai_dc/</metadataNamespace>
               </originDescription>
            </provenance>
         </about>
         <about>
           <sample xmlns="http://example.org/sample/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
              xsi:schemaLocation="http://example.org/sample/ http://example.org/sample.xsd">
             <sample-element>text</sample-element>
           </sample>
         </about>
       </record>
       <record xmlns="http://www.openarchives.org/OAI/2.0/">
        <header>
               <identifier>oai:example.org:item2</identifier>
               <datestamp>2009-11-09T20:26:16Z</datestamp>
               <setSpec>prime</setSpec>
        </header>
       <metadata>
        <oai_dc:dc xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:oai_dc="http://www.openarchives.org/OAI/2.0/oai_dc/" 
          xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.openarchives.org/OAI/2.0
       /oai_dc/ http://www.openarchives.org/OAI/2.0/oai_dc.xsd">
           <dc:identifier>urn:example:resource2</dc:identifier>
       </oai_dc:dc>
      </metadata>
     </record>
     <record xmlns="http://www.openarchives.org/OAI/2.0/">
      <header>
        <identifier>oai:example.org:item3</identifier>
        <datestamp>2009-11-09T20:26:16Z</datestamp>
        <setSpec>abovetwo:odd</setSpec>
        <setSpec>prime</setSpec>
     </header>
     <metadata>
       <oai_dc:dc xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:oai_dc="http://www.openarchives.org/OAI/2.0/oai_dc/"  
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.openarchives.org/OAI/2.0/oai_dc/
           http://www.openarchives.org/OAI/2.0/oai_dc.xsd">
           <dc:identifier>urn:example:resource3</dc:identifier>
       </oai_dc:dc>
     </metadata>
     <about>
       <sample xmlns="http://example.org/sample/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
          xsi:schemaLocation="http://example.org/sample/ http://example.org/sample.xsd">
         <sample-element>text</sample-element>
       </sample>
     </about>
   </record>
   <record xmlns="http://www.openarchives.org/OAI/2.0/">
    <header>
      <identifier>oai:example.org:item4</identifier>
      <datestamp>2009-11-09T20:26:16Z</datestamp>
      <setSpec>abovetwo:even</setSpec>
    </header>
    <metadata>
     <oai_dc:dc xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:oai_dc="http://www.openarchives.org/OAI/2.0/oai_dc/" 
      xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.openarchives.org/OAI/2.0/oai_dc/  
      http://www.openarchives.org/OAI/2.0/oai_dc.xsd">
         <dc:identifier>urn:example:resource4</dc:identifier>
     </oai_dc:dc>
    </metadata>
   </record>
   <record xmlns="http://www.openarchives.org/OAI/2.0/">
    <header>
      <identifier>oai:example.org:item22</identifier>
      <datestamp>2009-11-12T21:33:39Z</datestamp>
    </header>
    <metadata>
      <oai_dc:dc xmlns:oai_dc="http://www.openarchives.org/OAI/2.0/oai_dc/" xmlns="http://www.fedora.info/definitions/" 
       xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:dc="http://purl.org/dc/elements/1.1/">
              <dc:type>StillImage</dc:type>
              <dc:title>Halimeda tuna (J. Ellis et Solander) J.V.Lamouroux</dc:title>
              <dc:language>en</dc:language>
              <dc:identifier>K2N:SI_8fdaa4be929fad338e7442f58c5c13</dc:identifier>
       </oai_dc:dc>
     </metadata>
    </record>
    <record xmlns="http://www.openarchives.org/OAI/2.0/">
     <header>
       <identifier>"""oai:example.org:item22"""</identifier>
       <datestamp>2009-11-12T20:09:51Z</datestamp>
     </header>
     <metadata>
      <oai_dc:dc xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:oai_dc="http://www.openarchives.org/OAI/2.0/oai_dc/" 
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.openarchives.org/OAI/2.0 
        /oai_dc/ http://www.openarchives.org/OAI/2.0/oai_dc.xsd">
             <dc:identifier>urn:example:resource22</dc:identifier>
      </oai_dc:dc>
     </metadata>
    </record>
    <record xmlns="http://www.openarchives.org/OAI/2.0/">
      <header>
             <identifier>"""\\""oai:example.org:item22\\"""""</identifier>
             <datestamp>2009-11-12T20:11:54Z</datestamp>
      </header>
      <metadata>
       <oai_dc:dc xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:oai_dc="http://www.openarchives.org/OAI/2.0/oai_dc/" 
           xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.openarchives.org/OAI/2.0/oai_dc/
             http://www.openarchives.org/OAI/2.0/oai_dc.xsd">
               <dc:identifier>urn:example:resource22</dc:identifier>
       </oai_dc:dc>
      </metadata>
     </record>
     <record xmlns="http://www.openarchives.org/OAI/2.0/">
      <header>
        <identifier>K2N:SI_8fdaa4be929fad338e7442f58c5c13</identifier>
        <datestamp>2009-11-12T21:37:42Z</datestamp>
      </header>
     <metadata>
       <oai_dc:dc xmlns:oai_dc="http://www.openarchives.org/OAI/2.0/oai_dc/" xmlns="http://www.fedora.info/definitions/" 
         xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:dc="http://purl.org/dc/elements/1.1/">
              <dc:type>StillImage</dc:type>
              <dc:title>Halimeda tuna (J. Ellis et Solander) J.V.Lamouroux</dc:title>
              <dc:language>en</dc:language>
              <dc:identifier>K2N:SI_8fdaa4be929fad338e7442f58c5c13</dc:identifier>
       </oai_dc:dc>
     </metadata>
   </record>
 </ListRecords>
</OAI-PMH>

GetRecord verb test results

At the request:

http://localhost:8080/oaiprovider/?verb=GetRecord&identifier=K2N:SI_03e5e3df5818c0d7e25c1bfc672e24&metadataPrefix=k2n

The response is:

<OAI-PMH xsi:schemaLocation="http://www.openarchives.org/OAI/2.0/ http://www.openarchives.org/OAI/2.0/OAI-PMH.xsd">
 <responseDate>2009-11-17T17:51:38Z</responseDate>
  <request verb="GetRecord" identifier="K2N:SI_03e5e3df5818c0d7e25c1bfc672e24" 
             metadataPrefix="k2n">http://localhost:8080/oaiprovider/</request>
   <GetRecord>
    <record>
      <header>
        <identifier>K2N:SI_03e5e3df5818c0d7e25c1bfc672e24</identifier>
        <datestamp>2009-11-17T17:02:11Z</datestamp>
      </header>
      <metadata>
       <rdf:RDF>
        <rdf:Description rdf:about="info:fedora/K2N:SI_03e5e3df5818c0d7e25c1bfc672e24">
           <k2n:Country_Codes>marine</k2n:Country_Codes>
           <k2n:Country_Codes>jp</k2n:Country_Codes>
           <k2n:Resource_ID>calappa_japonica.jpg</k2n:Resource_ID>
           <k2n:Metadata_Language>en</k2n:Metadata_Language>
           <k2n:Collection_By_Resource_ID>Crabs_of_Japan_(ETI)</k2n:Collection_By_Resource_ID>
           <k2n:Scientific_Names>Calappa japonica</k2n:Scientific_Names>
           <k2n:Subject_Category>Malacotraca</k2n:Subject_Category>
           <k2n:Best_Quality_URI>
               http://www.species-identification.org/image_window.php?url=BIS/crabs_of_japan/pictures/calappa_japonica.jpg
           </k2n:Best_Quality_URI>
           <k2n:Copyright_Statement>(c) ETI</k2n:Copyright_Statement>
           <k2n:Best_Quality_Availability>online (free)</k2n:Best_Quality_Availability>
           <k2n:Page_Context_URI>
                  http://species-identification.org/species.php?species_group=crabs_of_japan&menuentry=soorten&id=700&tab=multimedia
           </k2n:Page_Context_URI>
           <k2n:License_URL>http://species-identification.org/about.php</k2n:License_URL>
           <k2n:License_Statement>
              This website offers free access to collaborative work of the taxonomic community and ETI under a non-commercial 
              e-publishing model designed to promote information sharing and capacity building. Please note that text and 
              multimedia materials on this website can not be copied or re-used without consent of ETI and the authors and 
              contributors. Contact ETI for information.
           </k2n:License_Statement>
           <oai:itemID>K2N:SI_03e5e3df5818c0d7e25c1bfc672e24</oai:itemID>
           <fedora:isMemberOf rdf:resource="info:fedora/K2N:Collection_c57fca9a585357696d7fb539eae3c8"/>
           <fedora:serviceProvidedBy rdf:resource="info:fedora/K2N:Provider_35d13727e732d93446c7f88f698d08f"/>
       </rdf:Description>
      </rdf:RDF>
     </metadata>
    </record>
   </GetRecord>
 </OAI-PMH>

Further thoughts and plans

to ingest more real digital objects;
to see how can use an xslt transformation in order to translate our metadata format in Europeana compliant format;
to see how we can express sets with OAI Provider Service;
to test all OAI-PMH verbs in order to have a real view about oaiprovider service;
to see how can attach a schema to RELS-EXT datastream;
to see how Metadata Space Manager (REPOX) works? This framework is used in Europeana too. See also References section.
to ask our Europeana's contact if our OAI Provider implementation meets Europeana's needs? See also Europeana Semantic Elements specifications

Conclusions: Is this service reliable for our purpose or not?

It seems that RELS-EXT datastream are treated in a dual manner: as special RELS-EXT datastream for relationships expressions and normal datastream (and we should take advantages from this).
A special element is necessary on RELS-EXT datastream, in order to identify the record id for OAI-PMH:

<oai:itemID>K2N:SI_8fdaa4be929fad338e7442f58c5c13</oai:itemID>

We need the answer from Europeana library: if this format is desirable for them, being in OAI-PMH format it should be. But we still need a schema.
Other solution: to build a wrapper in order to expose our K2N metadata to harvesters. This solution might present some other implications:
- It would be necessary to build a wrapper for every harvester - is not a reliable solution;
- to adopt a standard format as EAD or MODS in order to expose our metadata;
- to build a schema for our metadata inside of RELS-EXT datastream and to use OAI Provider Fedora Commons Service in order to expose our metadata. Maybe a wrapper application between our OAI Provider and harvesters would be necessary.

Discussions are welcome.

K2N-DCMI-Europeana metadata elements mapping proposal

See K2N-DCMI-Europeana_Metadata_Elements_Mapping

References

OAI-PMH service for K2N Fedora Commons based repository

Contents

Generalities regarding OAI-PMH protocol

Protocol Requests and Responses

OAI Provider for Fedora Commons Repository

OAI Provider Service Installation

Installation

Demos ingestion

How to configure OAI Provider Service

First successful tests

Digital objects used for test purpose

ListRecords verb test results

GetRecord verb test results

Further thoughts and plans

Conclusions: Is this service reliable for our purpose or not?

K2N-DCMI-Europeana metadata elements mapping proposal

References

Navigation menu

Personal tools

Namespaces

Variants

Views

More

Search

Navigation

Tools