Skip to content

DataFAQs Core Services

Jump to bottom
timrdf edited this page Sep 1, 2012 · 125 revisions

What is first

  • Getting started lets you find the right series of documentation pages based on what you're looking to get out of DataFAQs.
  • DataFAQs Core Services are SADI services, based on the SADI Semantic Web Services framework.
  • DataFAQs Core Services are used to determine the FAqT Services to apply during an evaluation.
  • DataFAQs Core Services are used to create FAqT Bricks.

What we will cover

This page provides details for each of the DataFAQs Core Services, which are used to determine [which evaluation services to apply](Selecting the evaluation services to apply) and [which datasets to analyze](Selecting the datasets to analyze) during a given evaluation epoch. Five different Dataset Selectors can provide "lists" of datasets that should be evaluated during a given epoch, four FAqT Service Selectors can provide "lists" of evaluation services that can be applied to a dataset during a given epoch, X Dataset Referencers can provide pointers to other URLs that describe the dataset, and Y Dataset Augmenters provide supplemental dataset descriptions that can be used during evaluation.

For each of the Core Services (FAqT Service Selector, Dataset Selector, Dataset Referencer, an Dataset Augmenter) we'll show the showing:

  • what it does,
  • how to use it (with an example),
  • where its source code lives,
  • where it is deployed (so you can call it right now, if you like)

So,

Let's get to it!

DataFAQs Core Services are [SADI](SADI Semantic Web Services framework) services, so to understand what they do we need to understand the RDF instance data that they accept and return. This page outlines each of the twelve DataFAQs Core Services, links to any deployed instances of the service, cites the input and output classes, and provides sample input and output instance data.

All twelve DataFAQs Core Services described on this page are listed in the SADI registry and available for SPARQL query from http://biordf.net/sparql's named graph http://sadiframework.org/registry/. services/sadi/core holds the implementations for the DataFAQs Core Services. http://aquarius.tw.rpi.edu/projects/datafaqs/configure-epoch is an initial interface to configure an epoch slice by selecting services to list FAqT evaluation services and datasets to analyze.

The following namespaces are used throughout this page:

@prefix datafaqs: <http://purl.org/twc/vocab/datafaqs#> .
@prefix dcat:     <http://www.w3.org/ns/dcat#> .
@prefix void:     <http://rdfs.org/ns/void#> .
@prefix dcterms:  <http://purl.org/dc/terms/> .
@prefix rdfs:     <http://www.w3.org/2000/01/rdf-schema#> .
@prefix rdf:      <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .

select-faqts (4)

A select-faqts service returns FAqT Services that will evaluate the dcat:Datasets during the given evaluation epoch.

select-faqts/identity (1/4)

services/sadi/core/select-faqts/identity.rpy is deployed at http://aquarius.tw.rpi.edu/projects/datafaqs/services/datafaqs/core/select-faqts/identity and allows you to list the FAqT services directly.

  • input class: datafaqs:FAqTService
  • output class: datafaqs:FAqTServiceCollection

Sample input 1 (max-1-topic-tag.ttl):

...
<http://aquarius.tw.rpi.edu/projects/datafaqs/services/sadi/faqt/lodcloud/max-1-topic-tag> a datafaqs:FAqTService .

Sample output 1:

<http://aquarius.tw.rpi.edu/projects/datafaqs/services/sadi/faqt/lodcloud/max-1-topic-tag> 
   a datafaqs:FAqTServiceCollection;
   dcterms:hasPart <http://aquarius.tw.rpi.edu/projects/datafaqs/services/sadi/faqt/lodcloud/max-1-topic-tag> .

select-faqts/via-sparql-query (2/4)

services/sadi/core/select-faqts/via-sparql-query is deployed at http://aquarius.tw.rpi.edu/projects/datafaqs/services/sadi/core/select-faqts/via-sparql-query and accepts an RDF description of a SPARQL query to apply, and where to apply it.

  • input class: datafaqs:QueryToApply
  • output class: datafaqs:FAqTServiceCollection

Sample input 1 (from-official-sadi-registry.ttl):

...
   a datafaqs:SPARQLQuery;
   rdfs:comment "One could resolve the URI for this query, or use the given rdf:value";
   rdf:value """
...
select distinct ?service
where {
   graph <http://sadiframework.org/registry/> {
      ?service
         moby:hasOperation [
            a moby:operation;
            moby:inputParameter [
               moby:objectType void:Dataset;
            ];
            moby:outputParameter [
               moby:objectType datafaqs:EvaluatedDataset;
...

select-faqts/towards/ckan-group (3/4)

Inspired by the datahub's 'lod' group that intends to become the 'lodcloud' group after official approval.

select-faqts/visko-planner (4/4)

Inspired by Nick Del Rio's visualization planner, which returns a plan of services that can be composed to obtain a visualization suiting the requested characteristics. This will lead to DataFAQs composing services instead of performing the evaluations in parallel.

select-datasets (5)

select-datasets/identity (1/5)

services/sadi/core/select-datasets/identity is deployed at http://aquarius.tw.rpi.edu/projects/datafaqs/services/sadi/core/select-datasets/identity and returns the list of datasets that it is given. The example below receives the datasets and returns the same instances.

  • input class: dcat:Dataset
  • output class: dcat:Dataset

Sample input 1 (drug-molecules.ttl):

...
<http://atlas.bio2rdf.org/sparql>
    void:sparqlEndpoint <http://atlas.bio2rdf.org/sparql> ;
    a void:Dataset, dcat:Dataset .

<http://bind.bio2rdf.org/sparql>
    void:sparqlEndpoint <http://bind.bio2rdf.org/sparql> ;
    a void:Dataset, dcat:Dataset .
...

Sample output:

@prefix dcat: <http://www.w3.org/ns/dcat#> .
@prefix void: <http://rdfs.org/ns/void#> .

<http://atlas.bio2rdf.org/sparql> a void:Dataset,
        dcat:Dataset .

<http://bind.bio2rdf.org/sparql> a void:Dataset,
        dcat:Dataset .
...

The void:Datasets to evaluate are provided by a select-datasets service.

select-datasets/via-sparql-query (2/5)

services/sadi/core/select-datasets/via-sparql-query.py is deployed at http://aquarius.tw.rpi.edu/projects/datafaqs/services/sadi/core/select-datasets/via-sparql-query and does TODO.

  • input class: datafaqs:QueryToApply
  • output class: datafaqs:DatasetCollection

Sample input 1 (logd-converted-datasets-with-samples.ttl):

todo

Sample output 1:

todo

select-datasets/by-ckan-group (3/5)

  • input class: datafaqs:CKANGroup
  • output class: datafaqs:DatasetCollection

Sample input 1 (thedatahub-datafaqs.ttl):

<http://ckan.net/group/datafaqs> a <http://purl.org/twc/vocab/datafaqs#CKANGroup> .

services/sadi/core/select-datasets/by-ckan-group is deployed at http://aquarius.tw.rpi.edu/projects/datafaqs/services/sadi/core/select-datasets/by-ckan-group and returns the list of datasets that are in a CKAN group at thedatahub.org. The first example below receives the datasets in the "datafaqs" group on ckan.org, while the second receives the datasets in the "lodcloud" group (the one that produces the Linked Data diagram).

curl -H "Content-type: text/turtle" -d ' <http://ckan.net/group/datafaqs> a <http://purl.org/twc/vocab/datafaqs#CKANGroup> .' http://aquarius.tw.rpi.edu/projects/datafaqs/services/sadi/core/select-datasets/by-ckan-group gives:

Sample output 1:

<http://ckan.net/group/datafaqs> 
   a datafaqs:DatasetCollection;
   dcterms:hasPart 
        <http://thedatahub.org/dataset/congresspeople>,
        <http://thedatahub.org/dataset/farmers-markets-geographic-data-united-states>,
        <http://thedatahub.org/dataset/white-house-visitor-access-records> .

<http://thedatahub.org/dataset/congresspeople> 
   a datafaqs:CKANDataset .
<http://thedatahub.org/dataset/farmers-markets-geographic-data-united-states> 
   a datafaqs:CKANDataset .
<http://thedatahub.org/dataset/white-house-visitor-access-records> 
   a datafaqs:CKANDataset .

Sample input 2 (thedatahub-lodcloud.ttl):

<http://ckan.net/group/lodcloud> a <http://purl.org/twc/vocab/datafaqs#CKANGroup> .

Sample output 2:

<http://ckan.net/group/lodcloud> 
   a datafaqs:Composite;
   dcterms:hasPart 
      <http://thedatahub.org/dataset/2000-us-census-rdf>,
      <http://thedatahub.org/dataset/aemet>,
      <http://thedatahub.org/dataset/agrovoc-skos>,
      <http://thedatahub.org/dataset/amsterdam-museum-as-edm-lod>,
      ...
.
<http://thedatahub.org/dataset/2000-us-census-rdf> 
   a datafaqs:CKANDataset .
<http://thedatahub.org/dataset/aemet> 
   a datafaqs:CKANDataset .
...

select-datasets/by-ckan-tag (4/5)

Sample input 1 (ckan-lod.ttl):

@prefix moat: <http://moat-project.org/ns#> .

<http://ckan.net/tag/lod>
    moat:name "lod" ;
    a moat:Tag .
  • input class: moat:Tag
  • output class: moat:Tag

services/sadi/core/select-datasets/by-ckan-tag is deployed at http://aquarius.tw.rpi.edu/projects/datafaqs/services/sadi/core/select-datasets/by-ckan-tag and returns the list of datasets in thedatahub.org that are tagged with the given moat:Tag. The first example below receives the datasets with the "lod" tag on ckan.org, this tag is used for datasets that are on their way to being the "lodcloud" group (the one that produces the Linked Data diagram) after they meet certain criteria.

curl -H "Content-type: text/turtle" @ckan-lod.ttl http://aquarius.tw.rpi.edu/projects/datafaqs/services/sadi/core/select-datasets/by-ckan-tag

Sample output:

@prefix dcterms: <http://purl.org/dc/terms/> .

<http://ckan.net/tag/lod> a <http://moat-project.org/ns#Tag>;
    dcterms:hasPart 
        <http://thedatahub.org/dataset/2000-us-census-rdf>,
        <http://thedatahub.org/dataset/addgene>,
        <http://thedatahub.org/dataset/aemet>,
        <http://thedatahub.org/dataset/agrovoc-skos>,
...
<http://thedatahub.org/dataset/2000-us-census-rdf> 
   a <http://purl.org/twc/vocab/datafaqs#CKANDataset>,
     <http://www.w3.org/ns/dcat#Dataset> .

<http://thedatahub.org/dataset/addgene> 
   a <http://purl.org/twc/vocab/datafaqs#CKANDataset>,
     <http://www.w3.org/ns/dcat#Dataset> .

<http://thedatahub.org/dataset/aemet> 
   a <http://purl.org/twc/vocab/datafaqs#CKANDataset>,
     <http://www.w3.org/ns/dcat#Dataset> .

<http://thedatahub.org/dataset/agrovoc-skos>
   a <http://purl.org/twc/vocab/datafaqs#CKANDataset>,
     <http://www.w3.org/ns/dcat#Dataset> .

Sample input 2: get the datasets in thedatahub.org with the "lifesciences" tag. This example gives the POST content directly in the curl call (be sure to include a space before the @ symbol).

curl 
  -H "Content-type: text/turtle" 
  -d ' @prefix moat: <http://moat-project.org/ns#> . <http://ckan.net/tag/lifesciences> moat:name "lifesciences" ; a moat:Tag .' http://aquarius.tw.rpi.edu/projects/datafaqs/services/sadi/core/select-datasets

select-faqts/towards/by-ckan-tag (5/5)

TODO

augment-datasets (1)

augment-datasets accept void:Datasets and return references to other locations that provide descriptions of the dataset. To get a larger description of the dataset, these references should be obtained.

augment-datasets/with-preferred-uri-and-ckan-meta-void (1/1)

services/sadi/core/augment-datasets/with-preferred-uri-and-ckan-meta-void is deployed at http://aquarius.tw.rpi.edu/projects/datafaqs/services/sadi/core/augment-datasets/with-preferred-uri-and-ckan-meta-void and includes two references if available. The first is the con:preferredURI of the dataset and the second is the VoID file. Both of these are drawn from the original CKAN description.

  • input class: dcat:Dataset
  • output class: datafaqs:WithReferences

Sample input 1 (datafaqs-3.ttl):

...
<http://ckan.net/group/datafaqs> a <http://purl.org/twc/vocab/datafaqs#Composite>;
    dcterms:hasPart 
        <http://thedatahub.org/dataset/congresspeople>,
        <http://thedatahub.org/dataset/farmers-markets-geographic-data-united-states>,
        <http://thedatahub.org/dataset/white-house-visitor-access-records> .

<http://thedatahub.org/dataset/congresspeople>                                
   a <http://purl.org/twc/vocab/datafaqs#CKANDataset> .
<http://thedatahub.org/dataset/farmers-markets-geographic-data-united-states> 
   a <http://purl.org/twc/vocab/datafaqs#CKANDataset> .
<http://thedatahub.org/dataset/white-house-visitor-access-records>            
   a <http://purl.org/twc/vocab/datafaqs#CKANDataset> .

Sample output 1:

<http://thedatahub.org/dataset/congresspeople> 
    a datafaqs:WithReferences;
    rdfs:seeAlso 
    <http://logd.tw.rpi.edu/source/contactingthecongress/dataset/directory-for-the-112th-congress> 
.

<http://thedatahub.org/dataset/farmers-markets-geographic-data-united-states> 
    a datafaqs:WithReferences;
    rdfs:seeAlso 
    <http://logd.tw.rpi.edu/source/data-gov/dataset/4383/version/2011-Nov-29>,
    <http://logd.tw.rpi.edu/source/data-gov/file/4383/version/2011-Nov-29/conversion/data-gov-4383-2011-Nov-29.void.ttl> 
.

<http://thedatahub.org/dataset/white-house-visitor-access-records> 
    a datafaqs:WithReferences .

What's next

  • FAqT Brick, the accumulation of evaluations that is configured by invoking DataFAQs Core Services.
  • DataFAQs Core Services are invoked to determine which components should be included in the dataset and faqt dimensions of a FAqT Brick's epoch slice.

Clone this wiki locally