Australian Biodiversity Information Standard

The Australian Biodiversity Information Standard (ABIS) is a multi-part standard and this document is its normative specification.

The other parts of ABIS are:

Namespaces are used within the identifiers for model elements to ensure that they are globally unique. For example, the TERN Ontology's Sample class is identified by the IRI https://w3id.org/tern/ontologies/tern/Sample which distinguishes it from the SOSA Sample class, identified with http://www.w3.org/ns/sosa/Sample that it is based on but extends.

Namespaces are used in shortened form in documents and data by assigning them a prefix and the prefixes used in this document are given in the table below.

Using the table above, the TERN Ontology's Sample class would be identified as tern:Sample and the SOSA Sample class as sosa:Sample.

The following terms & definitions are used throughout this document.

In this document, figures showing model elements use the following key:

Where examples of ABIS data are given in this document, RDF data serialised in the Turtle format is used. For example:

The above example data, while invalid according to the ABIS Validator, provides a simple example of a dataset and a sample and a relationship between them, encoded in Turtle.

If prefixes - ex:, schema: and tern: in the example above - are not declared within the example, as they are here - lines starting PREFIX - then they will be found in the Namespaces table above.

ABIS is centred on the TERN Ontology - see Figure 1 - which is a data model that represents ecological survey information. The TERN Ontology is designed in accordance with the principles of the Observations & Measurement standard ISO19156 that separates acts of observation from the results of those observations and ever-present details of the things observed.

The TERN Ontology itself inherits most of its modelling from several general-purpose and well-known Semantic Web models (ontologies), in particular:

Each of these "background models" are shown in Figure 1 and listed in the Background Models section below.

In addition to the TERN Ontology and the background models it uses, ABIS also use other "background models":

ABIS also includes additional "component models" that extend the range of things ABIS can cover:

Details of all these models are given in the Models section.

ABIS was established to enable a national Biodiversity Data Repository BDR in Australia that absorbs data from a range of sources.

These data supplier sources - Australian State & Territories and other organisations such as museums, herbaria, research organisations and companies - have been collecting biodiversity data for over a century in Australia and have been doing so independently, so there is a wide range of information collected relating to where, when and how species are observed.

Since the BDR aims for a comprehensive and up-to-date data holding, it needs to be able to absorb large volumes of biodiversity data and process them quickly. Given that existing source data collections contain millions of observations, this is only achievable through automatic data processing.

To enable effective automatic processing of data - both in terms of effort and quality - common models and formats must be implemented by data suppliers who map their data holdings to a form that can be automatically assessed (validated) and processed. ABIS is that form.

The TERN Ontology was created in 2020 for TERN to use for their own observations Knowledge Graph.

In 2021, the BDR adopted the TERN Ontology as its base.

In 2022, the first version of ABIS was created, building on top of the TERN Ontology to cater for messaging methods and some aspects of data management, such as the withholding of data publication due to data embargoes. This version of ABIS was used to for the data acquired in November 2022 through the scan of State & Territory biodiversity observation data holdings.

At the end of 2023, version 2.0 of ABIS was created which removed all representation of messaging protocols, requirements and rules. Version 2.0 focuses on building on top of the TERN Ontology data elements used to manage and provide access to expected BDR data holdings.

ABIS is managed day-to-day by the Department of Climate Change, Energy, the Environment and Water (DCCEEW) however many parts of ABIS are maintained externally, such as the TERN Ontology which is maintained by TERN.

To contact the ABIS managers, please see use the following details:

BDR Team
Biodiversity Information Section
Department of Climate Change, Energy, the Environment and Water
https://www.dcceew.gov.au
bdr@dcceew.gov.au

Governance of ABIS is undertaken by AusBIGG, the Australian Biodiversity Information Governance Group, which comprises DCCEEW, State & Territory biodiversity data holders and other organisations in the sector, such as TERN.

As of 2023, the processes for maintaining and extending ABIS are not fully established with AusBIGG still coming to understand ABIS' uses and its place in the landscape of international biodiversity observation data models, such as GBIF’s New Data Model.

An Issue Tracker for this document and for any other aspects of ABIS is public and online at: https://github.com/AusBIGG/abis/issues. Anyone with an interest in ABIS is invited to submit Issues there to be considered by ABIS management & governance parties.

ABIS comprises multiple models and some of them inherit from other models and standards. See the Models section for details.

Additionally, ABIS exists within a domain that already contains many international and national standards and well-known models. The Mappings section described how ABIS relates to many of them.

This specification aims to cover all the information needed to understand ABIS and use it. The following is a list of the major sections in this document and their purposes.

Two additional models - extensions to ABIS - are defined in this document:

Extended examples TERN Ontology data, in use within ABIS, are given in Annex C.

The most basic pattern used by this model and all the OWL-based models it profiles is that of Set Theory modelling, that is, modelling according to the mathematical notion of sets.

The basic principles are that things - all things - can be modelled as atomic objects and groups of objects known as sets. The basic object/set relations (membership of an object within a set) and set/set relations (union, intersection, disjoint etc.) are likely familiar to all Australian high school graduates.

In OWL modelling, objects are usually called Named Individuals or Instances and sets are called Classes.

The set of all people, the class Person is defined like this:

Nicholas Car is an object in the set of Person so, in OWL modelling, Nicholas Car is a Named Individual of type Person:

Subsetting is very important in OWL: Nicholas Car might also be an Engineer where Engineer is defined as a subset of Person. This means every Engineer is also a Person but not all Person objects are Engineers:

In OWL modelling, classes may be seen by grouping instances with similar properties. Nicholas Car might be understood to be an Australian by virtue of the predicate citizenship being given for him indicating Australian:

The figure below shows Nicholas Car, the individual, declared as a Person, and Engineer and understood to be an Australian.

In the domain of ABIS, all animals living in the sea constitute the class marine animals and there are intersecting classes of terrestrial animals for things like crabs and penguins that live both in the sea and on land.

All complex objects in OWL modelling - classes, predicates and instances of classes - are identified either with an IRI or a Blank Node. Classes and predicates defined in ABIS or inherited from models it profiles use the identifiers for them given in this document. Instances of classes, for example a particular sample, Sample Y of the class tern:Sample, are identified by IRIs assigned to them often deriving from the IRI of the dataset in which they are first presented. If the instance is referred to again later - perhaps further observations were made on the sample - then the original identifier for the object is still used to allow linking of information. An example for Sample Y in Dataset X which also contains Observation Z:

In the above example data, Dataset X - http://example.com/dataset/x - contains Sample Y - http://example.com/dataset/x/sample/y - and Observation Z - http://example.com/dataset/x/obs/z - was on the sample. The demonstration IRIs clearly all build on Dataset X’s.

The result of the observation - the value 42 parts per million uses a Blank Node, not an IRI, for identity which is essentially an unknown ID. This is because there’s no point in referring to the Result other than via the Observation that recorded it, so no IRI is ever needed to directly refer to it from elsewhere. The Blank Node is seen here in the Turtle syntax of RDF with information given between [ & ].

IRI identifiers for datasets take the form https://{IRI-STEM}/{DATASET-ID} and act as a unique namespace for objects within it. If Dataset abc-123-def-456 contained Sample Y, we may have the following identifiers:

It is likely that initiatives will be created to manage data for Sites, Samples or other classes of object that ABIS knows about. If so, these initiatives might issue identifiers for those things and, if they do, those identifiers should be used. See the next section for how.

Many objects represented using ABIS will usefully have external identifiers recorded, for example, samples with museum IDs or catalogue numbers. All forms of such identifiers SHOULD be recorded and how they are recorded and used depends on their type.

If an object already has an IRI identifier, and that identifier responds to Linked Data operations, it SHOULD be used as the primary identifier of the object.

If an object has a Linked Data IRI assigned to it AND another assigned to it within an ABIS data generation process, perhaps automatically, the two IRIs should be linked like this:

Here the OWL predicate owl:sameAs indicates the two IRIs identify the same thing.

If an object has an IRI assigned to it that does not link to RDF data, it should be recorded in the following manner:

Here the {ORIGINAL-IRI}, since it does not act as a Linked Data IRI, is indicated as being a literal of a specialised data type - {CUSTOM-DATATYPE}.

If the datatype of the {ORIGINAL-IRI} is of a known form, such as a DOI or IGSN, then that type might be found in the BDR Datatypes vocabulary at https://vocabs.bdr.gov.au, and it should be used. If its type is not known or is a generic URL, the type xsd:anyURI should be used like this:

All special IRI types, such as DOI, should be recorded in the BDR Datatypes vocabulary

Alternate identifiers for objects that are not IRIs/URLs MUST have their identifier regime indicated. For example, if Museum X issues identifiers for samples and Sample Y has an issued identifier of SAM-Y1234, then this should be given like this:

…​where {SAMPLE-IRI} is an IRI assigned to the sample and the predicate schema:identifier is used to give the literal identifier value of SAM-Y1234 which has the datatype ex:museum-x-id indicated.

Multiple alternate identifiers may be given, as long as their datatypes are unique:

ABIS provides representations of chunks of data for management - cataloguing, data governance and so on. It does this by implementing a very simple, and common, catalogue model which consists of catalogues (or catalogs, for Americans) that contain datasets or other kinds of resources such as vocabularies. ABIS then insists that datasets must, in turn, contain biodiversity records for records of such as what ABIS is all about.

This models is as per Figure 4, below, which is taken from the Biodiversity Record Model, detailed in full in Annex A.

Since this model is used for data management, it places requirements on ABIS datasets, above and beyond those imposed by the models it profiles, such as the TERN Ontology, to ensure all ABIS data has a minimum level of metadata present. See the BDR Profile of ABIS' Model Element Mandates for specifics.

implemented by schema.org and the Data Catalog Vocabulary (DCAT) which consists of

ABIS is fundamentally about records of the occurrence of biodiversity. For this reason, ABIS contains representations of chunks of data, as per the Data Cataloguing pattern above, a representation of an occurrence and a mechanism to link them.

The ABIS class BiodiversityRecord represents a single recording of an occurrence or recording of the results of a survey. This is usually a single row in a spreadsheet of biodiversity observations, or a single point of a map of occurrences. It is linked to the representation of actual occurrence itself, represented by the Darwin Core Terms's Occurrence class.

The reason for this distinction between representations of the record of the occurrence and the occurrence itself is because some data models record metadata about the recording itself - who did it, where it is stored and managed, what ID the recording has etc. - and some don’t, choosing to focus only on the where/what/when of the occurrence.

The relationship between a Biodiversity Record and the Occurrence it is about is given with the schema:about predicate, like this:

ABIS inherits its spatial modelling from GeoSPARQL, as does the TERN Ontology.

Patterns:

ABIS inherits its temporal modelling from the Time Ontology. The TERN Ontology uses the Time Ontology in places and used direct time representations elsewhere. This may be harmonised in the future.

ABIS uses a feature-centric approach for temporality, just as it does with spatiality. Just as per <GSP, GeoSPARQL>> where spatial objects are conceptual things with associated geometries, following OWL TIME, temporal objects are conceptual things with associated temporal "geometries" or temporal footprint.

Where for spatial objects we link a Feature to a Geometry which in turn links to a literal representation of the spatial footprint, say a Well-Known Text polygon, for temporal objects we link the temporal feature to a time:TemporalEntity, the equivalent of a temporal geometry, which is either a time:Instant or a time:Interval` which then contains dates, date/times etc.

The following code shows several example Survey instances, which are temporal features, linked to different temporal entities. The selection of particular temporal entities will come down to what is known about the Survey.

OWL TIME allows a number of data and time literal representations for date/time instants and any may be used: use the one that best corresponds with the reality of the thing you are modelling. The code example above shows three different ones in use: xsd:date, xsd:gYearMonth and xsd:dateTime.

The following table gives a list of date/time data types and the OWL TIME predicate used to indicate them. Format values in the table below use the codes from https://strftime.org/, for example %Y is "Year with century as a decimal number" e.g. 2013, not 13.

Note that when you use the predicates listed in the table above, ABIS will infer the datatype of the object indicated by the predicate and give an error if the data cannot be parsed as that datatype. You do not need to specify the datatype but you must get the format correct.

How things derive from other things, when and where this occurs and who may be responsible for actions is the domain of the Provenance Ontology (PROV) which is one of ABIS’s Background Models.

PROV’s basic classes and the predicates that relate them to one another are given below.

Many of ABIS' models follow on from the pattern in the figure above and many figures further down in this document are coloured according to PROV’s basic classes. For example, the TERN Ontology's Sampling class is a subclass of PROV’s Activity class and instances of it may have used an instance of a Site, which is a subclass of PROV’s Entity, to have generated an instance of the class Sample which is another subclass of Entity.

The figures in the Observations & Results and the Feature of Interest patterns use this colouring.

PROV’s provenance reasoning is also applicable to parts of ABIS. For example, the Projects Model indicates that instances of its Project class, which is a subclass of PROV’s Activity, can have generated instances of the TERN Ontology's tern:Dataset class, which is a subclass of PROV’s Entity, and may have been associated with an Agent - an Organisation or Person. If so, then the resulting tern:Dataset instances will be able to have an attributional relationship to the Agent instance calculated. This is shown in the figure below.

Things with agency to do work such as Organisations, People, Groups (of orgs and people) and perhaps software systems - are modelled as Agents in many Semantic Web and Knowledge Graph systems. In ABIS, we follow the general pattern for Agents outlined in the Provenance Ontology (PROV), see the section above.

In addition to the simple and direct kind of relationship between Agents and data (Entities) show above in Figure 9 where an Entity can be attributed to an Agent, we use a nuanced version of that was attributed to relationship, qualified attribution, which is also defined in PROV. qualified attribution is a type of qualified relation which means it relates two things together with qualifying facts. While these qualifying facts can be anything, in ABIS, the expected qualifier is a role: a role that the Agent plays with respect to the Entity. For example, for a dataset, one Agent might be the author of the data, another the publisher. This could be modelled as in Figure 10, below.

The roles that can be used for the qualification of attribution in ABIS are open-ended - new ones can be defined - but they are controlled - they must be proposed and accepted into a vocab to be legitimate. The vocab ABIS uses for these roles is the IDN Role Codes vocabulary - see the Vocabularies. It contains concepts such as author, publisher, collaborator and funder.

This pattern is inherited from the TERN Ontology, which, in turn, inherits it from SOSA.

The result of any observation in ABIS is a numerical or classification value for an observed property of a feature of interest (see next pattern).

The observation act is a temporal activity which, recorded or not, follows some procedure.

The value may be qualified with units of measure and uncertainty.

Using this pattern, the thing that ties the result value to the property of the thing observed is the observation.

A feature of interest is the object of some activity’s focus. This is a relative term: when an observation considers the property of something - perhaps the colour of a part of a leaf - then the part leaf only - a sample of it - is the feature of interest. If the observation were to consider the colour of the whole leaf, then the leaf is the feature of interest. If the colour of the whole bush from which the leaf came was considered, then the whole bush is the feature of interest.

If the average colour of a whole area of bushland was considered, the feature of interest would be the whole area, not any single bush or plant within it.

A site is a special kind of feature of interest with a particular definition in the domain of biodiversity science. As per the TERN Ontology, a Site in ABIS is:

"An ecological monitoring site where observations and samplings occur"

So when we see a site in ABIS, we know that it is the focus of the activity of ecological monitoring and not some other form of "site", for example an administrative region.

Additionally, a TERN/ABIS site is also a form of sample in that the activity at the ecological monitoring is designed to characterise biodiversity at the site where it is representative of an ultimate feature of interest - perhaps a certain kind of habitat. The site might be the entire ultimate feature of interest too in cases where a site is the entire area of concern such as a proposed mine location.

ABIS site s MUST have spatiality and temporality indicated as per the Spatially & Temporality patterns above.

ABIS site s MAY have additional information about them (such as sub areas, transects, subplots, methods used etc.) recorded for them according to particular protocols, such as the Ecological Monitoring System Australia (EMSA) protocols. Adherence to such protocols MAY result in addition RDF data being linked to an ANSI site or additional textual or other literal information but it MUST NOT conflict with the ABIS site information requirements.

Detailed site modelling according to EMSA or similar protocols should result in extension models to ABIS, just as the Projects Model is an extension.

The roles played by the various models within ABIS are:

The dependency hierarchy indicates which models profile other models and thus are dependent on them. The mechanism of profiling are as per the Profiles Vocabulary but can easily be summarised as:

In practice this means that any model profiling any other may use elements from that profiled model but must not break any rules imposed on that element’s use by the profiled model. These rules usually take the forms of required predicates and values for instances of particular classes, categorisation of object with terms from vocabularies and so on.

The following figure depicts the profiling relationships between the various models within ABIS.

The ABIS Foreground Model is the union of the Component Models listed in the next section. The dependencies of these models are shown in Figure 13, above.

When modelling biodiversity observations - ABIS' main concern - ABIS uses the TERN Ontology for the scientific domain information. When modelling taxonomic name objects, ABIS uses the NSL Model. When data packaging is necissary, the Biodiversity Record Model is used. Project information and data release details are modelled using the ABIS Projects & ABIS Data Release respectively.

The Component Models of ABIS are:

The TERN Ontology and the NSL Model are defined externally to ABIS whereas the other 3 are defined here, within the ABIS Specification. A short overview of each of these models follows.

The Background Models within ABIS are all those profiled by the Component Models. They are shown visually in the Model Dependency Hierarchy, above.

The main Background Models for ABIS are:

Of these models, all provide Semantic Web rules that can be used for data validation except for Darwin Core. Validators for each of these models, other than Darwin Core, are given in the Validation Section. These validators may be used individually or combined, within the ABIS Validator.

These models in turn profile several fundamental Semantic Web models:

Neither these models nor ABIS provide validators, however syntactic and some semantic data validation for RDF, RDFS & OWL data is built in to many Semantic Web / Linked Data tooling and, for example, syntactically invalid RDF data will not be able to be processed by ABIS other validators.

Additional Background Models - the Profiles Vocabulary & Olis - are used to describe the relationships between ABIS models and between units of ABIS data within datasets, respectively, and do not need to be directly considered by users of ABIS: their impact is felt within the descriptions of this specification document itself.

Specific details of all these Background Models are not directly given here, other than certain patterns they impose and these are presented in the Patterns Section.

ABIS is a profile of many other specifications, as per the Model Dependency Hierarchy.

ABIS is made of multiple parts, as per the Section Parts of ABIS.

Further profiles of ABIS may be made to constrain and/or extend its use.

Profile makers are able to define extended constraints on the use of ABIS to meet particular business needs. This can include constraining value ranges, vocabularies used and so on. Profile rules are entirely a matter for profile defines however data validated according to a profile of a standard, must always be valid according to the standard itself.

Profiles of ABIS may be defined anywhere however profile creators are encouraged to list their profiles here by contacting ABIS' maintainers.

The following profiles of ABIS are known to exist:

This profile defines the requirements for data to be absorbed into the Biodiversity Data Repository.

Just as the ABIS model is a collection of models and also inherits model elements from other models (see Multiple Models), so too is the ABIS validator a collection of ABIS-created validators and inherited validation rules. For data to validly conform ABIS, it must also be valid according all these validators. Part use of ABIS may choose to be valid according to only some ABIS validators.

The following table lists all the validators relevant to ABIS and describes their roles.

The preferred way to validate RDF data using the validators above is to use the ABIS Portal which contains all the validators pre-loaded for use.

However, any one of a number of tools designed for SHACL validation may be used and some may be more useful than the ABIS Portal for certain scenarios. Some such tools are:

ABIS background models such as Sensor, Observations, Sampling & Actuation (SOSA) ontology and GeoSPARQL are mapped to from ABIS foreground data in a simple manner: since all TERN Ontology/ABIS class objects are specialised versions of the background model’s classes, if a background model export is needed, ABIS data can either be sent as-is and the receiver can then infer generic objects from specialised ones, or we could perform the inference and export objects according to that model only.

The following methodology for export of ABIS data according to a SOSA will work with all background models.

Coming in July, 2024

On receipt of ABIS data, ABIS-compliant systems MAY make available data that has been expended, based on the rules in the Rule List below. How a system does this is a matter for the system but the general approaches are:

Forwards chaining involves expanding received data in accordance with the rules and materialising all the extension elements. Backwards chaining involves applying the reasoning rules to queries of the data such that the query can be answered as if the data had been expanded.

The rules are identified below, described in English and then defined in SPARQL queries. Implementing systems do not have to expand the data using SPARQL queries - they may use any system they like, such as application code - but the SPARQL queries serve as the canonical definition of the rule.

This rule allows for the creation of Record instances from TERN Ontology data that doesn’t directly indicate such as class of object.

The following spatial reasoning rules will be applied to ABIS data.

TODO: complete this section

Classes defined here:

Classes defined elsewhere:

Predicates defined here:

Predicates defined elsewhere:

The following SHACL shapes are graph patterns mandated by this model.

INCOMPLETE

#TODO: add details of the EMSA Protocol Model(s) from https://emsa.tern.org.au/

Classes defined here:

Classes defined elsewhere:

This model defines only one predicate - purpose - but also requires the use of others defined elsewhere. Definitions for all predicates are copied from source and given here.

Predicates defined here:

Predicates defined elsewhere:

TODO: list and define validators fro this model

INCOMPLETE

INCOMPLETE

This model defines none of its own classes and only indicates the TERN Ontology's tern:Dataset class for use with the predicates it does define.

TODO: list and define validators fro this model

These examples use TERN Ontology elements only.