1. 2014-03-18 Konstantin Baierer: Omnom & Pubby 1/36 DM2E Interoperability platform Omnom & Pubby Konstantin Baierer konstantin.baierer@ibi.hu-berlin.de

2. 2014-03-18 Konstantin Baierer: Omnom & Pubby 2/36 Components of the DM2E Software Stack What is Omnom? Modelling workflows HTTP Protocol Provenance What is Pubby Example Pubby setup Omnom/Pubby Demo Agenda

3. 2014-03-18 Konstantin Baierer: Omnom & Pubby 3/36 Omnom and Pubby in DM2E

4. 2014-03-18 Konstantin Baierer: Omnom & Pubby 4/36 Omnom and Pubby in DM2E Omnom

5. 2014-03-18 Konstantin Baierer: Omnom & Pubby 5/36 Omnom and Pubby in DM2E PubbyOmnom

6. 2014-03-18 Konstantin Baierer: Omnom & Pubby 6/36 dm2e-ws Workflow Engine Web Services dm2e-gui Graphical User Interface (Omnom / dm2e-gui) dm2e-silk Linking dm2e-pubby RDF Browser Components of WP2 Interop Platform dm2e-ontologies Data models/ Validation dm2e-direct Command line ingestion dm2e-josso Single Sign On

7. 2014-03-18 Konstantin Baierer: Omnom & Pubby 7/36 dm2e-ws Workflow Engine Web Services dm2e-gui Graphical User Interface (Omnom / dm2e-gui) dm2e-silk Linking dm2e-pubby RDF Browser Components of WP2 Interop Platform Omn m dm2e-ontologies Data models/ Validation dm2e-direct Command line ingestion dm2e-josso Single Sign On

8. 2014-03-18 Konstantin Baierer: Omnom & Pubby 8/36 dm2e-ws Workflow Engine Web Services dm2e-gui Graphical User Interface (Omnom / dm2e-gui) dm2e-silk Linking dm2e-pubby RDF Browser Components of WP2 Interop Platform Omn m dm2e-ontologies Data models/ Validation dm2e-direct Command line ingestion dm2e-josso Single Sign On

9. 2014-03-18 Konstantin Baierer: Omnom & Pubby 9/36 Omnom: Workflows and Configurations Workflow Abstract description of algorithm Blueprint or Data Wiring Less than a dozen for DM2E Configuration Instantiation of a workflow with concrete values One for every ingestion

10. 2014-03-18 Konstantin Baierer: Omnom & Pubby 10/36 Omnom: What's in a Workflow? Typed Input and Output Parameters Positions wrapping Web Services Input and Output Parameters Iterative positions for repeatable output Connectors wiring the parameters => Define data flow

11. 2014-03-18 Konstantin Baierer: Omnom & Pubby 11/36 Simple XSLT Publish Workflow

12. 2014-03-18 Konstantin Baierer: Omnom & Pubby 12/36 Simple XSLT Publish Workflow Workflow Input Workflow Output

13. 2014-03-18 Konstantin Baierer: Omnom & Pubby 13/36 Simple XSLT Publish Workflow Workflow Input Workflow Output Input Web Service Output Position 1 Input Web Service Output Position 2

14. 2014-03-18 Konstantin Baierer: Omnom & Pubby 14/36 Simple XSLT Publish Workflow Workflow Input Workflow Output Input Web Service Output Position 1 Input Web Service Output Position 2 Connectors

15. 2014-03-18 Konstantin Baierer: Omnom & Pubby 15/36 Omnom: What's in a configuration? Parameter Values Resolvable URLs Strings Examples URL of result RDF/XML URL of input XSLT Name of a dataset for URI generation Name-Value pair for XSLT parameters

16. 2014-03-18 Konstantin Baierer: Omnom & Pubby 16/36 Web services: Protocol WS describe themselves in RDF: Input / Output parameters Human readable description / label WS expect configuration in RDF: Concrete values for input parameters Either resolvable URLs or Literals WS immediately return a Job in RDF: Asynchronously work in the background Write (intermediate) results to the Job resource

17. 2014-03-18 Konstantin Baierer: Omnom & Pubby 17/36 Webservice: Execution

18. 2014-03-18 Konstantin Baierer: Omnom & Pubby 18/36 Convention for description/thing Accept conflicts for RDF Inf. Res. RDF/XML at http://mydomain/file/123 Document: Output of a transformation Metadata: File Size, Creator, MD5 ... HTTP Accept: application/rdf+xml Return the document ?! Return the metadata ?! Our solution: No Accept / application/octet-stream => Data Accept: RDF/JSON => Metadata

19. 2014-03-18 Konstantin Baierer: Omnom & Pubby 19/36 Types of Services Backend Services Workflow Service Configuration Service Job Service File Services DM2E File Service MINT Web Services to be used in workflows

20. 2014-03-18 Konstantin Baierer: Omnom & Pubby 20/36 Web Services for Workflows Transformation Transform XML with XSLT to RDF/XML Transform RDF with SPARQL to RDF Publishing With provenance to Triplestore Iteration Feed every contained resource to another web service ZIP of XML files, OAI-PMH harvesting Execute a workflow Nested Workflows

21. 2014-03-18 Konstantin Baierer: Omnom & Pubby 21/36 More complex: ZIP XSLT-ZIP Publish

22. 2014-03-18 Konstantin Baierer: Omnom & Pubby 22/36 Even more complex: Nested Workflows

23. 2014-03-18 Konstantin Baierer: Omnom & Pubby 23/36 Provenance in DM2E All Omnom resources are described in RDF All results are accesible on the Web Either Directly in RDF Or as Web Documents, with RDF metadata All data is in versioned datasets [ Dm2e-pubby offers version-aware API ] Consumers should "Follow their nose"

24. 2014-03-18 Konstantin Baierer: Omnom & Pubby 24/36 Provenance: PROV Omnom

25. 2014-03-18 Konstantin Baierer: Omnom & Pubby 25/36 Omnom Web User Interface Single-Page Application BackboneJS MVC framework Client/Server: Same data representation: JSON-LD (ish, kind of) Bootstrap Looks like Twitter Sane Defaults

26. 2014-03-18 Konstantin Baierer: Omnom & Pubby 26/36 Pubby Frontend for Linked Data Make triplestores web-accesible Dereferenceable URI 303 Redirects Content-Negotiation HTML interface

27. 2014-03-18 Konstantin Baierer: Omnom & Pubby 27/36 DM2E Extensions to Pubby CONSTRUCT instead of DESCRIBE Support Named Graphs Configuration defines the relevant subgraph that describes a URI Multiple Resources per output graph More flexible URIs 'data' and 'page' configurable Dataset-specific URIs Regex Matching in URIs Generate thumbnails

28. 2014-03-18 Konstantin Baierer: Omnom & Pubby 28/36 Example Pubby Use Case Want to expose library data Book data from triplestore 1 Subject data from triplestore 2 Using the same consistent interface Full example here http://data.dm2e.eu/pubby-example

29. 2014-03-18 Konstantin Baierer: Omnom & Pubby 29/36 Omnom: Workflow Edit

30. 2014-03-18 Konstantin Baierer: Omnom & Pubby 30/36 Configuration Edit

31. 2014-03-18 Konstantin Baierer: Omnom & Pubby 31/36 Job Page

32. 2014-03-18 Konstantin Baierer: Omnom & Pubby 32/36 Pubby View of the result

33. 2014-03-18 Konstantin Baierer: Omnom & Pubby 33/36 Provenance: Follow your nose Resource Versioned Dataset Versioned Dataset Dataset Versioned Dataset Publish Job Publish Job Publish Job Configuration Publish Job Configuration Workflow Job Workflow Job Workflow Workflow User Let's start here

34. 2014-03-18 Konstantin Baierer: Omnom & Pubby 34/36 Omnom: Next steps Performance UI/UX improvements Easier deployment

35. 2014-03-18 Konstantin Baierer: Omnom & Pubby 35/36 Pubby: Next steps Handle weird URIs more gracefully Properly handle blank nodes Collection Ontology Lists Caching (hard) Re-Merge with upstream Pubby

36. 2014-03-18 Konstantin Baierer: Omnom & Pubby 36/36 Thank You! Demos: http://omnom.dm2e.eu http://data.dm2e.eu/data/ Source Code / Issue Tracker http://github.com/DM2E/dm2e-ws http://github.com/DM2E/dm2e-gui http://github.com/DM2E/pubby Get in touch: konstantin.baierer@ibi.hu-berlin.de http://dm2e.eu, @DM2Europeana