In which we achieve a small simple step away from Enterprise space and towards the big belly.
James Bennett very neatly pinned the web services vs. REST debate with his wry 'defense' of the enterprise architect's XML stacks:
“Real” data formats have specifications and schemas and type checking and gobs and gobs and gobs of metadata. Otherwise, how could they ever work?
At iMatix we use a lot of XML, but as a description language, not a stack. We don't use XML schemas. When we use metadata, it's always at a cost to accessibility. The model-oriented metaprogramming we do - which lets me build a web server and Zyre in a few months rather than a few years - also excludes most average developers.
I've never liked the way XML - as an essentially simplifying concept - was turned into fat books and fat technologies that made poor products. So Bennett's post was fun to read. And then I read a comment from Scott Raymond, who wrote:
Great post. I find that JSON makes a great representation format for use in REST-style services, too — especially if those services might be used by browser-based (i.e. Ajax) applications.
Aha! RestMS is meant for the big belly, while AMQP is firmly in 'enterprise' space. I have to admit, iMatix mostly works in that space too, we used to make web applications but messaging is more fun. So I've never looked at JSON before today. Still, how hard can it be? The specification fits on a small part of a single page.
… five hours later …
- Figured out what JSON is and how to map to/from plain old XML in a clean way.
- Updated the RestMS specs to define how to get JSON from the server, using the "Accept: application/json" header. While at it, also defined the proper header for XML responses.
- Added JSON support to the iMatix base2/ipr_xml_tree module. Also fixed the XML formatting to be pretty. Might as well make it nice.
- Added Accept: processing to the base2/http server. If the client specified XML, give it XML. If the client specified JSON, give it JSON.
Rebuilt Zyre, no changes, and wrote a small test program:
#!/usr/bin/perl
use LWP::UserAgent;
$ua = new LWP::UserAgent;
$ua->credentials ($hostname, "restms", "guest", "guest");
$request = HTTP::Request->new (GET => "http://localhost:8080/restms/");
$request->header ("Accept" => "application/json");
$response = $ua->request ($request);
$response->code == 200 || die $response->status_line;
print $response->content."\n";
and got this:
ph@nb200803:~/work/trunk/openamq/zyre$ perl testit.pl localhost:8080
{
"restms":{ "version":"1.0", "status":"ok",
"pipe_class":[ { "name":"pipe", "uri":"http://localhost:8080/pipe" } ],
"feed_class":[ { "name":"fanout", "uri":"http://localhost:8080/fanout",
"feed":[ { "name":"amq.fanout", "uri":"http://localhost:8080/fanout/amq.fanout" } ]
} ],
"feed_class":[ { "name":"direct", "uri":"http://localhost:8080/direct",
"feed":[ { "name":"amq.direct", "uri":"http://localhost:8080/direct/amq.direct" } ],
"feed":[ { "name":"amq.service", "uri":"http://localhost:8080/direct/amq.service" } ],
"feed":[ { "name":"amq.status", "uri":"http://localhost:8080/direct/amq.status" } ]
} ],
"feed_class":[ { "name":"topic", "uri":"http://localhost:8080/topic",
"feed":[ { "name":"amq.topic", "uri":"http://localhost:8080/topic/amq.topic" } ],
"feed":[ { "name":"amq.notify", "uri":"http://localhost:8080/topic/amq.notify" } ],
"feed":[ { "name":"amq.data", "uri":"http://localhost:8080/topic/amq.data" } ]
} ],
"feed_class":[ { "name":"headers", "uri":"http://localhost:8080/headers",
"feed":[ { "name":"amq.headers", "uri":"http://localhost:8080/headers/amq.headers" } ],
"feed":[ { "name":"amq.dataex", "uri":"http://localhost:8080/headers/amq.dataex" } ]
} ],
"feed_class":[ { "name":"system", "uri":"http://localhost:8080/system",
"feed":[ { "name":"amq.system", "uri":"http://localhost:8080/system/amq.system" } ]
} ],
"feed_class":[ { "name":"service", "uri":"http://localhost:8080/service" } ],
"feed_class":[ { "name":"rotator", "uri":"http://localhost:8080/rotator" } ]
}
}
Ran that through JSONLint which said "Valid JSON". High five! Now, in theory this should be instantly loadable and usable by a JavaScript app.
Hi,
Do you have any thoughts on how to handle browser based authentication and access control?
Using client-side certificates is great for some deployments, impossible for others. If you added digest auth or something else, how will Zyre validate credentials?
Naturally we'll want LDAP and 'custom' plugins for authentication, and then there's access control policies that could limit which feeds/pipes can be used, etc.
This quickly gets beyond the fun performance stuff to boring drudge work, sorry but I had to ask. In my use-case, I need SSL, digest auth to LDAP and some kind of policy mechanism.
Maybe it'd be simplest to allow zyre to send out a message to get credentials validated (allowing caching of digest or basic auth results for some period of time) by some amqp endpoint, same for determining if joins are authorized when they are set up.
Then if you added an option to list specific IP address/ports that should not be authenticated or policy-controlled, we could implement the authenticator/policy service using RestMS as well (if not using the amqp api directly)
This way end-users can create whatever authenticator/policy systems they need and Zyre doesn't need any custom plugins or ldap code to support authentication and access control.
thanks
Since RestMS runs over straight HTTP, we can choose whatever mechanisms we like from what HTTP offers. Right now Zyre implements basic authentication (simple & stupid), but probably we'd want to use digest auth and SSL for anything serious.
Looking at what we did in Xitami:
I like the idea of custom plugins for LDAP et al. One of the reasons I like using HTTP is that we can reuse lots of existing drudge work.
I'll add an issue/request tracker to this site, then you can write up what you'd like to see.
Portfolio
Brad, I've added issue tracking to the site (Wikidot magic) and written up an issue. Using AMQP as the authentication plug-in transport is sneaky and I like it. Maybe as a first step, and then look at doing LDAP directly in the http server. (It's not that difficult, especially compared to SSL.)
Portfolio
Hi, Thanks for all your efforts on RestMS and Zyre - very interesting. I'm just starting to look at REST architecture… and was interested to see the RestMS spec explicitly defines resource content formats, e.g. xml, json. I con't comment authoritatively on how this reconciles with REST's HATEOS idea, but I had thought RESTful was implicitly if not explicitly content agnostic. Nontheles, you may be interested in extprot:
http://eigenclass.org/R2/writings/extprot-extensible-protocols-intro
HTH?
REST is content agnostic, yes, but it's not a formal spec, rather an architecture pattern. Any real life protocol - like AtomPub, or RestMS - needs to explicitly define its resources.
Extprot looks very interesting. In RestMS I deliberately kept it agnostic wrt the encoding of resources, which means we could slot-in Extprot, and thus solve some upcoming challenges with performance. It's fine to use XML or JSON for the low-volume parts of the work, e.g. creating feeds, pipes, and joins. It's not fine for message envelopes, where every byte counts.
We have the choice of defining a fixed wire encoding for envelopes (like [http://wiki.amqp.org/spec:6 DMP)) or using an abstraction like Extprot. Using Extprot would let us use XML envelopes when debugging, and binary envelopes in real use. Nice!
So I'd really like to have Extprot support in RestMS and Zyre. This would require C support.
Any ideas on how to get Extprot working in C? (Writing C is not a problem, iMatix can do this.) Is there a formal spec for Extprot somewhere?
Portfolio
REST as an arch pattern but it seems to be 'defined' - I've heard one dev indicate 'it is what Roy says it is, andif you deviate describe your app using some term other than REST'. I've also seen a Roy ask the same. For better or worse ;)
The only extprot material I'm aware of is at the eigienclass site and the github repo:
http://github.com/mfp/extprot/tree/master
There is a c decoder under './ruby/ext', the rest seems to be Ocaml and Ruby, so AFAICT no C encoder. Not sure if this suffices as a spec, see encoding .md here:
http://github.com/mfp/extprot/tree/cdf9e7d1ad564e8001cab1602612d69459c3882a/doc
HTH
Mark
I'm pretty happy that RestMS fits well within what Roy would consider RESTful. I've asked him to critique, but he's a busy guy.
It seems that extprot will be useful in some scenarios, so at some point we'll probably make a C codec. Thanks for that link.
Portfolio