DiGIR Schema Readme

Explanation, Usage, and Tips

DiGIR request and response formats are specified via a number of XML Schema documents. The primary structure of a request or response is specified in the protocol XML Schema (digir.xsd). For the latest version(s) see:

http://digir.sourceforge.net/prot

The available data elements, and their types, for searching and returning in a request or response are specified in a federation (or content) schema. A current federation schema and excellent example is that representing the Darwin Core V2 dataset (darwin2.xsd). For the latest version(s) see:

Additionally, predefined record structures for responses can be defined and some examples of such (brief and full) exist in the above location.

Understanding the DiGIR Schema

• In the schema you will see a number of abstract elements for various kinds of data. For example:
  • searchableData
  • alphaSearchableData
  • numericSearchableData
  • returnableData
  • alphaReturnableData
  • numericReturnableData
  • searchableReturnableData
  • alphaSearchableReturnableData
  • numericSearchableReturnableData

  As abstract elements, one cannot create concrete instances of them. The elements are provided to be used as substitutionGroups when content data elements are defined in federation schema. They act as placeholders in the protocol schema within the filter (particularly within COPs) structure.

  In terms of data defined within a federation schema, it can be either searchable, returnable or both. Because XML Schema does not allow the use of multiple substitutionGroups per single element, each combination had to be created as its own element. One might consider why we did not create types and a complex inheritance tree to achieve the sort of typing we desired. The answer is two fold: 1) instance documents would not be as clean without the use of substitutionGroups and 2) XML Schema does not support multiple inheritance.

  Examining how substitutionGroups were used here, one can think of them as acting like tag interfaces in java. In specific, in java there is an interface called Serializable and objects can implement this interface when they are serializable. Although there is no contract about supported implementation here, we have, in effect, tagged a data element as searchable, returnable, or both by specifying its substitutionGroup as one of the above. This is an important tehnique which one might find used elsewhere within the protocol schema.

  One other note regarding these substituionGroups is the alpha and numeric versions of each. Clearly this is not an exhaustive type set, so I believe these will be somewhat shortlived. The idea behind having "typed" substitutionGroups is to allow comparison operators (COPs) to only deal with data of the sort valid for that particular operator. Consider the LIKE operator which can only accep alphanumeric data. If typing was not done in any manner here, the advantage of parser validation on an instance document would be lost and would have to be done at the provider itself. Its a nice idea but shortlived because as the datasets become more complex and as providers support more (or less) diverse sets of operators for each data element, these two generalized types (alpha and numeric) will not buy much. Eventually, each provider will specify in its own metadata the data elements and operators supported for each element. At that point, the use of the alpha and numeric can be removed and one can just deal with data in a general sense.

• Similar to the Data elements above, we've defined the comparison operators (COPs) and made each of them part of substitutionGroups within the larger filter structure (LOP). However, with the COPs, each does extend from a COP (or multiCOP) type and this was important to ensure common and similar structure for each one (i.e. contined only one element and also the appropiate attribute).

• Also, a few other place holder elements are defined, to be substituted or used in a request. These include:
  • record: to be used as the container in custom (user defined) record structures
  • requiredList: to be used as a flag (via substitutionGroup) for an element within a federation schema that is simply a sequence of the data elements that are required for each record within a result set

• For the purposes of the multiCOP a listType (with alpha and numeric) versions are defined. This is necesary to limit the elements within a multiCOP (such as IN) to be N of any 1 element.

Using the DiGIR Schema to create a Federation (content) schema

• All data elements must be defined with the nillable attribute set to true (i.e. nillable="true"). In result data, if a column value is null the record should be returned with xsi:nil="true" and the element should be left empty. Also, as seen above, if an element is not supported in results, in order for the element to be left empty, it must be defined as null. For example, here is how one would specify a nillable data element:

  <xsd:element name="Collector" type="digir:stringData" substitutionGroup="digir:alphaSearchableReturnableData" nillable="true"/>

  And how the element looks as null in the results:

  <darwin:Collector xsi:nil="true"/>

• All data elements much define a substitutionGroup that tags them as searchable, returnable, or both. For example:

  <xsd:element name="Collector" type="digir:stringData" substitutionGroup="digir:alphaSearchableReturnableData" nillable="true"/>

• All data elements that can be instantiated must not be abstract. Any abstract element should not be used as a searchable concept.
• Its highly recommended that for each data element an annotation is provided describing such things as, data types, possible ranges, domains, etc. This will encourage more valid instance documents.
• Noting required elements (a returnable data element that is required in all result sets) is done through the creation of an abstract element with the substitutionGroup of digir:requiredList. The element is abstract, and therefore can never be instantiated and we flag it as the requiredList via the substituionGroup. An example is:

  
     <xsd:element name="requiredList" substitutionGroup="digir:requiredList" abstract="true">
        <xsd:complexType>
           <xsd:sequence>
              <xsd:element ref="DateLastModified "/>
              <xsd:element ref="InstitutionCode"/>
              <xsd:element ref="CollectionCode"/>
              <xsd:element ref="CatalogNumber"/>
              <xsd:element ref="ScientificName"/>
           </xsd:sequence>
        </xsd:complexType>	
     </xsd:element>
     

  This should be done in all federation schema but the list (sequence) could contain no elements.

Schema Versioning Process

1. Use the DiGIR CVS repository on the SourceForge site. The schema documents, for both the protocal and federation schemas, as well as some examples, are in the xml module in CVS. All edits to a schema should be done via a checked out version of the document and commits should be done regularly. It is not recommended that you keep a document out with changes for an extended period of time as that can introduce conflicts in a multi-developer environment.
2. Once you are satisfied with you changes and they have been tested, you are ready to "release" the schema. After the schema(s) are committed to CVS, you should tag (via cvs tag) the schema, as well as any dependant schemas with the release name (e.g. beta1, release1.0, etc.). All dependant schemas should use the same tag.
3. After tagging the files, create a directory with the same name as the tags used (e.g. beta1, release1.0, etc.) in the htdocs (webspace) area of the DiGIR SourceForge site under the appropriate directory (prot - for protocol schema and fed - for federation schema).
4. Then scp (secure copy) the same versions of the schema(s) to this(these) new directory(ies). Here, snapshots of all released version will reside. This location acts as, and should be used in other schema or instance documents, as the schemaLocation. You should not remove previous versions of the schemas as other people may still be referencing them.