Service Registries Blog

Register Locally - Discover Globally

Ann Apps — Thu, 03 Apr 2008 14:20:00 +0000

On Monday 31 March a second meeting of the Global Registries Initiative took place at the University of Southampton, UK. The initial meeting was held in Washington DC, USA on December 10 2007.

With the slogan "Register Locally - Discover Globally", the initiative aims to enable sharing of collection descriptions and serivce details across registries, thus making them discoverable globally.

The three registries that have begun this initiative are the Australian ORCA registry of repository collections, the US OCKHAM registry of the National Science Digital Library, and the UK JISC Information Environment Service Registry (IESR) of collection descriptions and services that provide access to them.

Currently this initiative is at a discussion stage, with an international workshop planned for later this year. I expect that this workshop will include the development of use cases.

The architecture for the initiative still needs to be discussed, but it will certainly be standards-based. All three of the registries currently involved describe resources using the IESR metadata schema, although with some differences, e.g. with nationally relevant property values. The collection description part of the IESR metadata schema is based on the Dublin Core Collections Application Profile, which is now endorsed by the Dublin Core Metadata Initiative. To cope with the differences in use of the IESR schema in the implementation of the three registries, an interchange format may be developed.

The devlopment of the ORCA registry was guided ISO2146 "Registry Services for Libraries and Related Organisations".

IESR already has experience of harvesting resource descriptions from other catalogues using OAI-PMH. Thus I would expect the first prototype experiments of the Global Registries Initiative will involve sharing records by OAI-PMH.

New Data in IESR

Ann Apps — Thu, 03 Apr 2008 14:15:00 +0000

I'm pleased to report that data records created by DRAI (JISC Digital Repositories and Archives Inventory) (http://www.jisc.ac.uk/whatwedo/programmes/digitalrepositories2007/project_inventory.aspx) have been imported into IESR (JISC Information Environment Service Registry) (http://iesr.ac.uk).

Also IESR now includes details of ESDS International (http://esds.ac.uk/) Macro and Micro Economic datasets. This information is harvested regularly from the UK Data Archive.

Atom-based service registries

Ann Apps — Fri, 11 Jan 2008 14:21:00 +0000

There is an interesting article in the September/October issue of IEEE Internet Computing (ISSN 1089-7801), pp 68-71: Active Web Service Registries by Martin Trieber and Schahram Dustdar of the Vienna University of Technology. They are investigating building service registries that disseminate their records using RSS (currently but intending to upgrade to Atom). Each service provider sets up their own RSS feed describing their services at a known location, ie their own local service registry, aka active web service registry. Then there are wider service registries that are built as aggregators of the distributed local registries by whatever selection. One could envisage building higher-level, even global registries by aggregating the distributed aggregators.

They are discussing registries of Web Services (ie. SOAP) only - partly it is seen as a solution for the demise of public UDDI registries. So they have a narrower remit than IESR's of supporting many service protocols and also descibing collections. Also I get the impression that they see users of this information being people, application developers, who look up details of Web Services to plug them in to applications. There is no mention of machine-to-machine use, apart from the aggregation.

But using Atom, and aggregating local services registries, seems a neat idea. It would certainly lend itself to investigation if new instantiations of service registries were to be designed. It may provide a paradigm for building global registries.

One particular point they make struck me as being interesting to consider. They recommend including examples of use with each registered Web Service, for example a concrete SOAP message that will invoke the Web Service, to help application developers using the information, and so to better advertise and assist take-up of the Web Service.

Another interesting article in the same journal issue is pp 17-25: Requirements and Services for Metadata Management by Paolo Missier, Pinar Alper, Oscar Corcho, Ian Dunlop, and Carole Goble of The University of Manchester. This talks about capturing annotations about data resources, here in a bioinformatics context. The mass of such annotations is in fact a valuable resource in its own right, which it would be useful to capture and associate with the data resource. Their solution is very semantic web / RDF based. But the ideas are interesting, especially in this age of social bookmarking. But maybe capturing annotations about the collections within a registry is still in the 'nice to have, but in the future when we have resource to implement it' category.

Usage Guides for Collection Description

Ann Apps — Thu, 04 Oct 2007 09:10:00 +0000

The Dublin Core Metadata Initiative (DCMI) Collection Description Community is gathering information about usage guidelines for, and examples of, collection description. A webpage that assembles the information already contributed is at:

http://epub.mimas.ac.uk/DC/dc-collections/usage-guides.html

Further contributions are welcome. Please send them to me (ann.apps@manchester.ac.uk) or to the DCMI Collection Description Community email listserv (dc-collections@jiscmail.ac.uk) (you need to be subscribed to the listserv to post to it). Following is the original request sent to the DCMI Collection Description Community for information about usage guidelines.

----------------------------------------

Please would anyone who has a collection-based project / service / application, and who is willing to share details of their usage guidelines, or examples of collection description, send them to this group (or to me). I will create a web page of links to produce an accumulated set of knowledge and expertise in this area.

We learnt at the DC2007 conference that a Dublin Core (DC) Application Profile (actually a DC Abstract Model-conformant Application Profile) is now a package, known as the Singapore Framework, which comprises:

Functional requirements (recommended)

Domain model (mandatory)

Description Set Profile (DSP) (mandatory)

Usage Guidelines (optional)

Encoding syntax guidelines (optional)

It has been suggested that we should create usage guidelines for collection description. It seems to me that it would be a significant job. And maybe there would need to be guidelines on different levels, eg some for library collections with strict cataloguing rules, and different guidelines for collections in other domains.

A first step would be to assess existing usage guidelines from applications, projects, services that are collection-based. Hence my request at the start of this message.

Thanks in advance for any contributions

Ideas about Registries and IESR

Ann Apps — Mon, 02 Jul 2007 12:57:00 +0000

Last week I attended two workshops held at the JISC offices in London: the Strategic e-Content Alliance Registries Workshop; and the Shared Infrastructure Services Workshop. These are various thoughts and ideas about IESR triggered by discussions at these workshops.

There was some attempt at the Registries Workshop to define a Registry. My offering, that was picked over by the meeting was: "A collection of official metadata records that describe, and assist access to, resources". On reflection, 'official' is not the right word, and suggestions included: governed, authorititive, etc. There was discussion about whether a registry differed from a catalogue or an inventory, but the workshop didn't come up with any significant differences.

A Registry for End Users. IESR's original remit was to be a middleware registry providing machine-readable (XML) resource descriptions to be used by other services. There was an initial requirement that IESR should not have a human-readable web interface at all. In fact, IESR does have a web interface, if only so that its content can be viewed by IESR staff, but it provides just a tabular view of the XML data. It is fairly apparent that so far IESR has no significant use of these machine readable descriptions. Maybe using a middleware registry in a dynamic way is too high a barrier for application developers, or maybe they prefer to choose resources to plug into their applications. Certainly IESR's lack of use is mirrored by the sparsity of machine interfaces into resources. It now occurs to me that it may be more sensible to change tack. IESR could be a registry for human readers that also has machine interfaces. This would correspond with other similar registries, which see their primary interface as being the web interface for human users. In the JISC IE technical architecture, 'service registries' are shown as components of the 'shared infrastructure'. But it doesn't seem unreasonable to me for IESR to also provide a service within the 'presentation layer' that is a web interface into the registry.

Surfacing hidden content. This should be one of the purposes of a registry. Surfacing more obscure and less popular content fits with the 'long tail' idea of Web 2.0. Clearly such resources need to be registered in IESR, but once there they will be discoverable. This does imply use of IESR either by people for general resource discovery, or by applications in a dynamic way. This purpose could be thwarted if IESR were used by application developers to cherry-pick resources to plug-in to their applications.

Advertising and promotion of resources. This is another purpose of a registry, which relates to the point above. I heard a presentation at the ELPUB2007 conference in Vienna about how content created by projects subsequently gets little, or sometimes no, use because no-one knows about it. Registering such resources in IESR would advertise their existence.

Collection descriptions are boring. They need to be embedded in other services for display to the end user. This would seem to present a challenge to IESR if we decide to provide a user facing web interface. Maybe we need two views, one for general resource discovery, and a second for application developer users. It would at least seem sensible to move technical details onto a subsidiary web page. Displaying transactional services would be a further challenge.

Funder stakeholder requirements. It is probable that funders as stakeholders will have some requirements on data descriptions and functionality in a registry. Currently the IESR data contribution model allows a Contributor to edit only those records that they have provided. But funder requirements may imply populating data fields across Contributors' records. In particular:

There is likely to be a requirement to discover all resources funded by funding body. This implies the need for a new data field isFundedBy. This would be repeatable (there may be more than one funder) and expected to be provided where appropriate.

There have been requests for a data field isEndorsedBy, which would allow a governing body to indicate which resources within the registry it endorses for use by organisations under its remit. This seems to be functionality on top of IESR. Possibly it could be an application run by the governing body. Alternatively IESR could consider providing this as an additional service on top of the basic registry, essentially an annotation layer.

Google indexing of registry content. There was some discussion at both workshops about whether registry content should be exposed to Google et al. Does this work against the resource in terms of Google ranking because its description is available both from the resource itself and from the registry? This problem would be compounded if records are shared between registries, when there would be even more descriptions of the same content indexed by Google. However the purpose of 'surfacing hidden content' would be aided by exposing it to Google by the registry. Possibly the answer is to expose primary registry content only to Google, i.e. resource description contributed directly to IESR, but not harvested records.

Perpetual beta. This is another tenet of Web 2.0, recognising that the online world is continually developing and changing. This seems to conflict with objectives to create official services with service level agreements and (often articifial) performance indicators. I guess the problem is that a service has to be paid for in some way and funding for a 'perpetual beta' is unlikely to be forthcoming.

Farewell, and Bon Voyage

Leigh — Fri, 08 Jun 2007 10:02:00 +0000

Today we bid farewell to Amanda, the IESR Project Manager. Amanda has taken the bold decision to emigrate to Canada, and we wish her and her family all the best.

Dublin Core Collections Application Profile is Conformant

Ann Apps — Tue, 03 Apr 2007 09:07:00 +0000

I am very pleased to report that the Dublin Core Collections Application Profile is now 'conformant'.

The DCMI Usage Board, at their recent meeting, reviewed the updated version of the Dublin Core Collections Application Profile (DCCAP) and agreed that it conforms with the Dublin Core Abstract Model, is internally consistent, and is documented according to guidelines.

The revision of the DCCAP to produce this conformant version was undertaken by the DCMI Collections Application Profile Task Group. So I wish to acknowledge this group for all their hard work, in particular the Task Group leaders Sarah Shreeves and Muriel Foulonneau. Credit should also be given to Pete Johnston, the previous chair of the DCMI Collection Description Working Group, whose hard work over quite some time resulted in the previous, substantial versions of the DCCAP.

The conformant DCCAP is at:
http://www.dublincore.org/groups/collections/collection-application-profile/

There is a summary at:
http://www.dublincore.org/groups/collections/collection-ap-summary/

Several associated documents are:
Collections Type Vocabulary: http://www.dublincore.org/groups/collections/colldesc-type/

Collections Terms: http://www.dublincore.org/groups/collections/collection-terms/

Accrual Policy terms: http://www.dublincore.org/groups/collections/accrual-policy/

Accrual Method terms: http://www.dublincore.org/groups/collections/accrual-method/

Frequency Vocabulary: http://www.dublincore.org/groups/collections/frequency/

e-Framework Services Knowledgebase

Ann Apps — Wed, 14 Feb 2007 13:46:00 +0000

On Monday 12 February I attended the JISC e-Framework Modelling Workshop. From one of the presentations, about 'Using and Contributing to the e-Framework' (http://www.e-framework.org/) I learnt that the e-Framework activities include developing a knowledge base of Services. The e-Framework is based on a service oriented architecture (SOA), advocating this approach to save development costs by promoting re-use of existing solutions. This does not mean a restriction to SOAP Web Services, many other service protocols that are designed to share data being recognised such as RSS and Z39.50.

Projects are encouraged to register their services in the knowledgebase, both non technical information about what a service does and technical information about how to write such a service. Benefits of this knowledgebase are that developers and institutions can see what others have already done and can possibly re-use contributed software and intelligence, though there is no exclusivity about a particular solution. This knowledgebase should be of value to developers, both as an information source and as a channel through which to distribute their work, if it contains: information on coding with interoperable standards, such as what others have done and specfics such as message sizes; information on data semantics; and links to actual projects and services.

Service details are submitted via a template - a Word document (maybe they need some SOA here...) - and there is a QA process before publication. Services are classified by a genre taken from a vocabulary defined by the e-Framework. The genre essentially describes broadly what the service does, eg Search. Also captured is a 'service expression'. This is a specialisation of a service genre, binding to a particular service, ie. a specified way of doing something. These expressions are standards-based but with more detail for an implementer than the standard may provide, and possibly covering more than one standard (eg. Z39.50 and CQL). The template asks for details about implementation, eg. toolkit used, and also service instances (concrete usable services) and interfaces (eg. WSDL).

The knowledgebase also registers Service Usage Models (SUMs), which are composite services, composed of more basic services. The description of these will include details of process modelling, choreography and workflows, possibly using a business process modelling language, or possibly by UML diagrams. Registration of these SUMs within the e-Framework shoud assist in identifying commonly recurring processes that could be re-used, known as Core SUMs.

I came away from this event wondering how this relates to IESR and service registries generally. They seem to be registering services as details of their software components, rather than how to call and use them. And what is recorded is documentation not actual software for downloading. But the registration of service interfaces and of service instances, even if these are just examples, seems to overlap somewhat with IESR's remit and content. It seems to me that there should be at least some conversation to discuss commonality. Maybe there could be some data sharing to some extent. Of course a big difference is that the e-Framework knowledgebase appears to be aimed at human readers - those who want to develop a service, or those looking for a service to re-use. Whereas the primary purpose of IESR is to provide details to machines as a middleware service.

I considered the e-Framework service genre list when thinking about such a list for IESR a couple of weeks ago. But it seemed to me to be too fine-grained in places and very e-learning biased. In fact some of the finer-grained terms do not seem to fit with the aim of the service genre being the 'big picture' of a service's functionality.

At the start of the workshop we were treated to a showing of an animation that has been developed to demonstrate the benefits of the SOA approach.
See http://www.jisc.ac.uk/whatwedo/programmes/programme_eframework/soa. Enjoy...

IESR Metadata Review and Service Types

Ann Apps — Wed, 31 Jan 2007 15:37:00 +0000

IESR is currently undertaking a review of its metadata schema. Details are at http://iesr.ac.uk/metadata/reviews/review-200702.html. Comments are welcome, either on this Blog or to IESR personnel.

One particular area that may be of interest is developing a list of Service Types (item 2.3 on the review list). I suggest rechristening these `Service Functions' and partly using the registry services functions list proposed for ISO 2146 (There doesn't appear to be a standard to reference yet, but they are listed in http://www.nla.gov.au/nla/staffpaper/2005/pearce1.html). This list is:

Common Services: Authenticate; Authorise; Pay
Metadata Services: Contribute; Save; Alert; Harvest
Discovery Services: Find; Locate; Request
Delivery Services: Resolve; Supply; Lend; Reserve
User Services: Register; Ask; Personalise; Monitor

There is also a much longer, finer-grained (in places) service genre list within the e-Framework, but many items are very e-learning specific (and the length would probably confuse our contributors and users): http://www.e-framework.org/Services/Genres/ServiceGenreRegistry/tabid/655/Default.aspx

However we may need some additions. Terminology is on our current list - e-Framework has `Map terms', the generalistion of which would be Map. IESR Use Case step 6.1a1 / scenario is looking for a `format validation' service, of which the generalisation would be Validate (the Use Case could find `format' in the description). e-Framework has `Validate Courses' but that is too specific. Some others from e-Framework which seem significant: Annotate; Archive; Rate (which would include: assess, classify, recommend); Translate.

This makes a suggested list (which would be annotated with provenance in documentation):

Alert

Annotate

Archive

Authenticate

Authorise

Contribute

Find

Harvest

Lend

Locate

Monitor

Pay (would Purchase be better?)

Personalise

Rate

Request

Reserve

Resolve

Save

Supply

Translate

Validate

Although it would be preferable to use an existing standard list, there does not seem to be a suitable one. The advantage of maintaining our own IESR list is that it gives us flexibility for future extensibility.

Some service types could be filled automatically by data creation software for particular service protocols:

ftp: Request; Contribute
ldap: Locate
oai-pmh: Harvest; Request
opengis: Find
opensearch: Find
openurl: Locate
rss: Alert
rsync: Request
soap: Request
sru: Find
srw: Find
webcgi: Find
z3950: Find

Correlation with other digital library work in this area:

Obtain; Get: Request
Put: Contribute
Search: Find, and in some cases also Request (eg. Z39.50 Search/Retrieve)
Publish-Subscribe: Alert

Service Registries at the DNS Level

Ann Apps — Tue, 30 Jan 2007 11:25:00 +0000

During the OCKHAM Registry Experiment meeting held in Seattle, USA, on 16-17 January 2007, there was some discussion about the possibility and practicalities of implementing a service registry at the DNS level.

This could be based on the technology used by Zeroconf/Bonjour (http://en.wikipedia.org/wiki/Zeroconf). If this is enabled with a service discovery mechanism, it allows for dynamic connection to services. Zeroconf is utilised by iTunes to automate music sharing by users who are on the same sub-net. RFC 2782 (http://tools.ietf.org/html/rfc2782) specifies fields in DNS records, which if set can enable automatic discovery of suitable services of a particular type.

At the meeting Dan Chudnov was quickly able to set up a demo of this in action, using some service data records he'd gathered from IESR via OAI-PMH.

Clearly developments at the DNS level are complementary to initiatives like IESR and OCKHAM. It allows for discovery of services that implement a particular protocol or serve a particular function. IESR's `transactional services' probably include those that would lend themselves to `low level' service discovery. But it doesn't seem to me possible to use this method to implement use scenarios that are based on discovery via IESR's and OCKHAM's rich collection descriptions, i.e. discovery of `informational services'.

Experiments of low level service discovery are outside of IESR's scope, but it seems to me they are worth pursuing. Maybe this area would make a good student project for someone somehwere.

Demonstrating use of registries

Amanda — Fri, 19 Jan 2007 16:15:00 +0000

The OCKHAM team organised a meeting in Seattle this week to bring together registry providers and some potential registry users. The potential users were some of the projects funded by the National Science Digital Library (NSDL) and included the New Jersey Institute of Technology's the General Recommendation Engine, Utah State University's Services to Link Opencourseware Repositories and the NSDL (one of these services is the wonderfully-named Scrumdidilyumptious) and the Pathways resources of the NSDL itself.

The meeting was held in the Seattle Public Library's Central Library, an innovative building which was opened in 2004. One of its features is a 'Book Spiral' which connects four floors of books and is decorated by Dewey Decimal numbers set into the floor, to help library users find their way around.

The Midwinter Meeting of the American Library Association is taking place in Seattle. This is a huge affair, with 15,000 librarians attending, according to the Seattle Times which published an editorial on the importance of librarians on 19 January. The generally positive buzz about libraries and librarians in Seattle was refreshing.

DCMI Collection Description Community Group

Ann Apps — Mon, 18 Dec 2006 14:28:00 +0000

From today, 18 December 2006, the Dublin Core Metadata Initiative (DCMI) Collection Description Community Group is available as a forum for discussion about collection-level description. The charter of the group is:

---
The DCMI Collection Description Community is a forum for individuals and organisations with an interest in collection-level description and in the use of Dublin Core metadata for that purpose.
More specifically, it also provides a forum for discussion of issues related to the implementation of the DCMI Collection Description Application Profile.
---

The group operates as a jiscmail listserv (http://www.jiscmail.ac.uk/lists/DC-COLLECTIONS.html) open to everyone, although you do have to join the list to be able to post to it. The group also has a website at http://www.dublincore.org/groups/collections/

There is also an associated Task Group that is responsible for continuing the development of the DCMI Collection Description Application Profile. Progress is available from the Wiki:
http://dublincore.org/collectionwiki/

[The listserv was previously the DCMI Collection Description Working Group. But the structure for DCMI technical work has now been changed. DCMI Working Groups have been restructured as Community Groups that serve as open forums for discussion on particular topics or domains, and Task Groups that develop specific deliverables.]

Collection Description is an important component of a Service Registry. The purpose of very many services is to provide access to a collection of data / information. In IESR terminology these are known as 'informational' services, whereas 'stand-alone' services are known as 'transactional'. The collection description part of the IESR metadata is based on the DCMI Collection Description Application Profile (which also informs the NISO Metasearch Initiative Collection Description Schema), although there are some IESR-specific extensions. The use of standard application profiles is important for interoperability. IESR is an example of using the DCMI Collection Description Application Profile in practice.

There have been suggestions that a service registry such as IESR should describe services as primary resources, with any data collections they support as secondary. But it seems to me that such a model is not so clean as the IESR model. Because most services provide access to data, that data collection needs some description, and where several services provide access to a single collection there would be multiple descirptions of it.

There have also been suggestions that there should be separate registries for collections and for services. However IESR's experience has shown that describing both types of resource in a single registry fits together well. It would be possible to provide effectively separate registries by appropriate views on IESR if there were a requirement.

Use Cases for Service Registries

Ann Apps — Thu, 14 Dec 2006 14:19:00 +0000

The IESR project has developed some Use Cases and Scenarios describing ways in which we envisage a Service Registry may be used (http://iesr.ac.uk/use/use-cases/). We'd be pleased to hear comments about these Use Cases and suggestions for further possible uses of a Service Registry.

First post

Amanda — Fri, 08 Dec 2006 10:08:00 +0000

We've set up this blog to act as a source of information about Service Registries.

In the UK, we're working on the JISC Information Environment Service Registry (IESR), while there are parallel initiatives going on in the USA (the OCKHAM registry and a registry at the Los Alamos National Laboratory) and in Australia (the Australian Partnership for Sustainable Repositories).

The aim of Service Registries is to help users get access to useful electronic resources. Service Registries provide a mechanism for advertising such resources on the network so that other applications, such as portals, can find and connect to them.

The metadata schema devised for IESR (and now being adapted for use in other projects) allows the recording of information about the electronic resources themselves, technical details about how to access the resources, and contact details for the resource providers. Jeremy Frumkin of the OCKHAM project identified three primary functions of such registries:

discovery - allowing a user or a machine to discover available, relevant services
resolution - providing the ability for a person or machine to locate, or resolve to, a service; and
configuration - provide information necessary for a client to access a particular service

(In 'The need for a digital library service registry', OCLC Systems & Services: International digital library perspectives Vol. 22 No. 1, 2006, pp.23-25)