HTTP SPARQL server and client for twisted and rdflib

This library is for programs that use twisted for networking, use rdflib for RDF, and want to do sparql over http. It contains a client and a server, but you should be able to mix and match with other sparql clients and servers.

There is also a LocalGraph class, which presents exactly the same API as RemoteGraph, but it talks to an in-process rdflib graph. This is convenient for testing your async RemoteGraph code without bothering to setup the http connection.

# this code will work whether graph is a RemoteGraph or a LocalGraph

    def method(self):
        d = graph.remoteLabel(subj)
        d.addCallback(self.withLabel)

    def withLabel(self, label):
        print label

Download

Darcs source code repository

Requirements

• elementtree
• rdflib 2.3.3 +
• nevow (twisted, zope.interface)

Future plans

• Remove the ad-hoc update protocol and share sesame's (does anyone want sparql 1.1 update yet?)
• Complete the support for reading and writing all the sparql results formats (xml, json, sesame binary), and try to share this code with other projects like sparql-wrapper
• Clean up the RemoteGraph factory code
• Incorporate the sync version of Graph2 that I have, and organize all the local/remote sync/async versions of everything

Demo

1. Start a server, creating a new berkeleydb instance in ./db/

   ./sparqlserve --home db

   2007/01/22 00:58 -0700 [-] Log opened.
   2007/01/22 00:58 -0700 [-] twisted.web.server.Site starting on 9991
   2007/01/22 00:58 -0700 [-] Starting factory <twisted.web.server.Site instance at 0xb7646aac>
   

2. In another shell, parse some NT and POST it to the server. The post will be to "/add?context=http://example.org#context" with data consisting of NT statements. That part of my protocol is not part of any standard.

   ./graphimport hello.nt

   Server says:

   2007/01/22 01:14 -0700 [HTTPChannel,0,127.0.0.1] 127.0.0.1 - - [22/Jan/2007:08:14:08 +0000] "POST /add?context=http%3A//example.org/%23context HTTP/1.0" 200 2 "-" "Twisted PageGetter"

3. GET the query "SELECT ?s ?p ?o WHERE { ?s ?p ?o }" from the server. This uses the standard SPARQL query and results formats.

   ./dumpgraph

   Server says:

   2007/01/22 01:15 -0700 [HTTPChannel,1,127.0.0.1] 127.0.0.1 - - [22/Jan/2007:08:15:32 +0000] "GET /?query=SELECT%20%3Fs%20%3Fp%20%3Fo%20WHERE%20%7B%20%3Fs%20%3Fp%20%3Fo%20%7D HTTP/1.0" 200 617 "-" "Twisted PageGetter"

   Client says:

   [(u'http://example.org/hello',
     u'http://example.org/to',
     rdflib.Literal('world',language=None,datatype=None)),
    (u'http://example.org/hello',
     u'http://www.w3.org/2000/01/rdf-schema#label',
     rdflib.Literal('Hello',language=None,datatype=None))]
   

4. Do an arbitrary query on the server.

   ./query "SELECT ?label WHERE { ?s <http://www.w3.org/2000/01/rdf-schema#label> ?label }"

   [{'label': rdflib.Literal('Hello',language=None,datatype=None)}]
   

5. Check out the server diagnostics at http://localhost:9991/

   I deliberately made an error request to cause the the traceback below. The table at the bottom is really nice for large programs, since it shows what lines in your code made the various queries.

   sparqlhttp server status

   • 2 queries
   • Last error:
     query:

     SELECT ?broken { ?typo here }

     error:

     Traceback (most recent call last):
       File "sparqlhttp-1.0/sparqlhttp/serve.py", line 48, in getQuery
         results = self.graph.queryd(query)
       File "sparqlhttp-1.0/sparqlhttp/dictquery.py", line 80, in queryd
         initNs=self.initNs)
       File "/usr/lib/python2.4/site-packages/rdflib-2.3.3-py2.4-linux-i686.egg/rdflib/Graph.py", line 644, in query
         return plugin.get('SPARQLQueryResult',QueryResult)(p.query(strOrQuery, initBindings, initNs, DEBUG))
       File "/usr/lib/python2.4/site-packages/rdflib-2.3.3-py2.4-linux-i686.egg/rdflib/sparql/bison/Processor.py", line 26, in query
         strOrQuery = Parse(strOrQuery, DEBUG)
       File "/usr/lib/python2.4/site-packages/rdflib-2.3.3-py2.4-linux-i686.egg/rdflib/sparql/bison/Processor.py", line 16, in Parse
         return p.parse(unicode(query,'utf-8'))
     SyntaxError: lexical error at line 1, column 30: no action found for 'here }'
     

   • Last query:

     SELECT ?broken { ?typo here }

     Again, as one line:
     SELECT ?broken { ?typo here }
   • All query types, by total time spent

     total secs (per)	count	query (with sample initBindings)	bound	avg rows	sources
     0.01 (0.0110)	1	SELECT ?s ?p ?o WHERE { ?s ?p ?o }	()	2.0	dumpgraph:9

6. Try running the tests. They use twisted trial and they run from the test directory:

   cd test
   trial test_dictquery.py test_remotegraph.py    # all these should work
   trial test_syncimport.py   # this one might not do as well