In which we try to use Zyre for something real and discover it's full of bugs.
This morning, after I fixed the download packages, Ganesh Prasad downloaded Zyre, built and tested it, and got it to crash immediately. Maybe releasing the code for Xmas was too optimistic. Even running the Hello World test in the RestMS spec caused Zyre to assert. I looked outside at the blue sky and freezing temperature and balanced my options. Go skating with my daughter, or fix the code.
Not a tough choice. The young thing got put in front of her new Asus Eee 900, and I decided to make a real application using RestMS. Incidentally, I'm also working on an Eee, a USD 350 1000HD, which has a nasty slow Dothan CPU and poor 2-hour battery life but is addictive for its small size which begs "hold me, pick me up, carry me everywhere". It's like going back to 2002, when I used a supermodel-thin X505, only a lot cheaper. Since the shiny black casing seemed designed to smear, I spray-painted the lid bright red. This has nothing to do with Zyre except that now you know the code is being built on a cheap, slow, cheerfully black & red Asus Eee 1000HD.
Zyre is of course made for web applications. There is a web application I've been wanting to write for ages, namely a web-based console for OpenAMQ. Yes, OpenAMQ has a perfectly good text mode console, but a nice clickable web interface is highly desirable, because it makes the stodgy command-line application spring to life. Logos! Stylesheets! Themes! (Oh, yes, and easier to learn and use than a text-based UI.)
There are several ways to construct a web interface for OpenAMQ. Perhaps optimal: integrate a web server in the broker, and plug a URI handler directly into the OpenAMQ console agent. We do have a web server in our tool stack, the multi-threaded extensible http core that runs Zyre. But I don't like designing the optimal solution immediately. It usually turns out to be a trap.
Always start with something simpler and stupider. Ideally, something that can be done in one day… so how about accessing CML over HTTP? This would let us manage OpenAMQ servers (or any AMQP server that implements CML) from a web application. That starts to be fun.
CML is, incidentally, XML. This means that an application can trivially read and process it, if it can send and receive CML messages Up to now, sending or receiving CML depended on having an AMQP client stack. Kind of a pain if you want to use a language that is not supported. Like Perl, Lua, Lisp, or (vitally for web apps) JavaScript. Problem solved with Zyre.
I whipped up a quick and dirty Perl script to do a 'schema request', which is the CML function that gets the schema from the server. This shows how simple RestMS is (the HTTP work is done by the LWP class):
#!/usr/bin/perl
use LWP::UserAgent;
use HTTP::Request::Common;
$ua = new LWP::UserAgent;
$ua->credentials ($hostname, "restms", "super", "super");
$baseuri = "http://localhost:8080/restms";
# Create a named pipe and join on amq.direct to get console replies
$response = $ua->request (PUT "$baseuri/pipe/my.pipe/my.pipe\@amq.direct");
$response->code == 200 || die $response->status_line;
# Send schema-request to console, as a sanity assertion
$request = HTTP::Request->new (POST => "$baseuri/amq.console\@amq.system");
$request->content ('<cml version="1.0"><schema-request/></cml>');
$request->header ("RestMS-reply-to" => "my.pipe");
$response = $ua->request ($request);
$response = $ua->request (GET "$baseuri/pipe/my.pipe/");
$response->code == 200 || die $response->status_line;
print $response->content."\n";
We're really getting close to that "trivial AMQP client" I boasted about being able to write earlier this year on openamq-dev.
This morning, RestMS (and Zyre) did not know anything about message properties. Now the spec explains how to set and get the reply-to and message-id properties. There are a bunch of properties on AMQP messages but most were inherited from JMS and don't have compelling utility. That is, until someone can show me a valid use case, they won't go into RestMS. Design by removal.
There were more gotchas, and I found that the Zyre regression test was failing when run twice. Create a feed, create a join on the feed, destroy the feed, and recreate the feed. Is the join still present, or not? Correctly, no. If you delete a feed, any joins that refer to it must also disappear.
OK, by this evening, as I write this blog, the regression test works properly, and CML access over RestMS works. Tomorrow, a new release of OpenAMQ/1.4 so that Ganesh and others can get working code.
I'll take the young thing skating on Sunday. Some Sunday. Definitely. But first I have a Thinkpad X40 to paint red. I'm working up to redecorating the MacBook Air.
I wrote a more refined version of the CML-over-RestMS script that shows how to do further CML requests. The script is more realistic. It inspects the broker, and returns this XML data:
<?xml version="1.0"?>
<cml version="1.0" xmlns="http://www.openamq.org/schema/cml">
<inspect-reply class="broker" object="0" status="ok">
<field name="name">OpenAMQ 1.4a0</field>
<field name="started">2008-12-26T21:12+01:00</field>
<field name="locked">0</field>
<field name="datamem">8817K</field>
<field name="bucketmem">0K</field>
<field name="messages">2</field>
<field name="consumers">2</field>
<field name="bindings">2</field>
<field name="exchange">2</field>
<field name="exchange">3</field>
<field name="exchange">4</field>
<field name="exchange">5</field>
<field name="exchange">6</field>
<field name="exchange">7</field>
<field name="exchange">8</field>
<field name="exchange">9</field>
<field name="exchange">10</field>
<field name="exchange">11</field>
<field name="exchange">12</field>
<field name="connection">13</field>
<field name="config">1</field>
</inspect-reply>
</cml>
Here is the full Perl code:
#!/usr/bin/perl
# Zyre example showing how console access to an OpenAMQ server using
# CML over RestMS.
#
# Copyright (c) 1996-2007 iMatix Corporation
#
# This file is licensed under the GPL as follows:
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or (at
# your option) any later version.
#
# This program is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
# General Public License for more details.
#
# For information on alternative licensing for OEMs, please contact
# iMatix Corporation.
use LWP::UserAgent;
use HTTP::Request::Common;
$verbose = 1; # Display contents from Zyre
$hostname = $ARGV [0];
$hostname = "localhost" unless $hostname;
$baseuri = "http://$hostname/restms";
$ua = new LWP::UserAgent;
$ua->agent ('WebCML');
$ua->credentials ($hostname, "restms", "super", "super");
$ua->timeout (1); # Keep things impatient
# Create auto-named pipe and get pipe name from response
restms (PUT => "$baseuri/pipe");
if ($response->content =~ /name\s*=\s*"([^"]+)"/) {
$pipe = $1;
}
else {
die "Failed: malformed response for pipe\n";
}
# Create a join on amq.direct to get console replies
restms (PUT => "$baseuri/pipe/$pipe/$pipe\@amq.direct");
# Send schema-request to console, as a sanity assertion
cml_request ('<schema-request/>');
# Inspect object zero, which is the OpenAMQ broker
cml_request ('<inspect-request object = "0" />');
#############################################################################
sub restms {
my ($method, $uri) = @_;
if (verbose) {
print "$method => $uri\n";
}
$request = HTTP::Request->new ($method => $uri);
$response = $ua->request ($request);
check_ok ($response);
}
sub check_ok {
my ($response) = @_;
if (verbose) {
carp ("Response=" . $response->code . " Content-type=" . $response->content_type);
carp ($response->content) if $response->content_length;
}
if ($response->code != 200) {
carp ("E: " . $response->status_line);
carp ($response->content);
exit (1);
}
}
sub carp {
my ($string) = @_;
# Prepare date and time variables
($sec, $min, $hour, $day, $month, $year) = localtime;
$date = sprintf ("%04d-%02d-%02d", $year + 1900, $month + 1, $day);
$time = sprintf ("%2d:%02d:%02d", $hour, $min, $sec);
print "$date $time $string\n";
}
sub cml_request {
($_) = @_;
# Send CML formatted request to amq.console@amq.system
$request = HTTP::Request->new (POST => "$baseuri/amq.console\@amq.system");
$request->content ('<cml version="1.0">'.$_.'</cml>');
$request->header ("RestMS-reply-to" => $pipe);
$request->content_type ('text/xml');
$response = $ua->request ($request);
check_ok ($response);
# Read CML response
restms (GET => "$baseuri/pipe/$pipe/");
}
If you read the CML spec you can see that it looks like a kind of RESTful API over AMQP. Kind of. I think it will be possible to do a proper RESTful console interface, by defining URIs for all OpenAMQ resources and then processing them properly, with no side effects for GETs, and idempotently for PUTs and DELETEs.
So in the long run it looks like the best architecture will be:
- Native HTTP support in the OpenAMQ broker;
- An HTTP API based on mapping the CML resources to RESTful HTTP;
- An Ajax application that can use this API and present the user interface.
But first, more stupid and simple things that get results fast.