Not guts, but 6: part 5

It’s time for me to start building a simple Stomp::Server class, test-driven. I’ll need to extend my Test::IO::Socket::Async to make this possible, as it currently doesn’t handle listening sockets.

The simplest start

I’ll start out by stubbing an almost empty Stomp::Server class, which goes in lib/Stomp/Server.pm6:

class Stomp::Server {
    method socket-provider() {
        IO::Socket::Async
    }
}

Once again, I’ll give it a socket-provider method so I can inject a socket test double. Then, it’s time for a new test file, server.t:

use Test;
use Test::IO::Socket::Async;
use Stomp::Server;

constant $test-socket = Test::IO::Socket::Async.new;
my \TestableServer = Stomp::Server but role {
    method socket-provider() {
        $test-socket
    }
}

So, where to begin? To set me off in a consistent direction, I take a look at Stomp::Client and notice it expects to be constructed with a host and port. That seems like a good starting point. So, some tests:

constant $test-host = 'localhost';
constant $test-port = 1234;
dies-ok { TestableServer.new }, "Must provide host and port to new (1)";
dies-ok { TestableServer.new(host => $test-host) }, "Must provide host and port to new (2)";
dies-ok { TestableServer.new(port => $test-port) }, "Must provide host and port to new (3)";

These are easily passed, by adding to Stomp::Server:

has Str $.host is required;
has Int $.port is required;

A typical pattern for asynchronous server-like things in Perl 6 is to expose a supply of incoming connections. I may as well call that listen. Here’s the simplest test I can write for that:

my $test-server = TestableServer.new(host => $test-host, port => $test-port);
my $listen-supply = $test-server.listen();
isa-ok $listen-supply, Supply, "Stomp::Server listen method returns a Supply";

It explodes since there’s no listen method. Stubbing one in that contains a supply block gets me a pass:

method listen() {
    supply {
    }
}

So far, so easy.

The audience is listening

Now for something a little more involved. I want to make sure that a listening socket is only opened once I tap the supply that comes back from Stomp::Server’s listen method:

my $socket-listening = $test-socket.start-listening;
nok $socket-listening, "Not listening before supply is tapped";
my $listen-tap = $listen-supply.tap(-> $incoming { });
my $socket-listener = await $socket-listening;
ok $socket-listener, "Listening once supply is tapped";

I’d also like to make sure it listens on the correct host/port:

is $socket-listener.host, $test-host, "Listening on correct host";
is $socket-listener.port, $test-port, "Listening on correct port";

And that closing the tap on the supply Stomp::Server gives me back will also close the listen supply from the socket:

$listen-tap.close;
ok (await $socket-listener.is-closed), "Closing supply tap also closes socket";

So, that’s the tests, but Test::IO::Socket::Async isn’t up to the job yet – so that’s first in line. I’ll want an object that represents a listening socket, and I can see that at the very least it’ll need a host and a port. I’ll also need to deal with the same race between tests and code under test that I had with connect, meaning that Listener itself should hold the Supply that I will use to simulate incoming connections. Finally, I need to provide a way to test that at some point it stops listening, exposing a Promise that is kept when that happens. That’s quite a few things that need wiring together. Happily, it falls out really quite easily, by wiring things up at construction time:

class Listener {
    has $.host;
    has $.port;
    has $.is-closed = Promise.new;
    has $!is-closed-vow = $!is-closed.vow;
    has $!connection-supplier = Supplier.new;
    has $.connection-supply = $!connection-supplier
        .Supply
        .on-close({ $!is-closed-vow.keep(True) });
}

That’s really quite pretty. Perl 6 promises that attribute initializers run in order, so I can safely rely on $!is-closed containing the Promise I next take a vow from – and also keep that vow private. I also keep the ability to inject new connections private, and then tweak the Supply with on-close, which lets me run some logic to keep the is-closed promise when a tap on the supply is closed. Since everything exposed is either immutable or concurrent, there’s no need for this to be a monitor rather than a class.

That just leaves me to write a couple of methods in Test::IO::Socket::Async. One is listen, which should match the IO::Socket::Async method. The other is start-listening, which returns a Promise that tests can await to get the Listener instance. As with testing incoming connections, I’ll need a couple of attributes too.

has @!waiting-listens;
has @!waiting-start-listening-vows;

method listen(Str() $host, Int() $port) {
    my $listener = Listener.new(:$host, :$port);
    with @!waiting-start-listening-vows.shift {
        .keep($listener);
    }
    else {
        @!waiting-listens.push($listener);
    }
    $listener.connection-supply
}

method start-listening() {
    my $p = Promise.new;
    with @!waiting-listens.shift {
        $p.keep($_);
    }
    else {
        @!waiting-start-listening-vows.push($p.vow);
    }
    $p
}

Recall that Test::IO::Socket::Async is a monitor, meaning there are no data races here. Installing these updates, and running my tests, things get a bit further, then hang here:

my $socket-listener = await $socket-listening;

That’s not surprising, because the code under test never actually starts to listen. Let me make that happen with the simplest possible addition to Stomp::Server’s listen method:

method listen() {
    supply {
        whenever self.socket-provider.listen($!host, $!port) {

        }
    }
}

With that, all the tests pass. But wait…how did the socket closed test pass? That’s thanks to supply blocks being smart enough to keep track of all the things tapped by a whenever inside of them, and closing them automatically when the corresponding tap on the supply block itself is closed. Resource management is one of the things that supply blocks quietly take care of, avoiding all kinds of potential resource leaks.

Incoming connections

Next, I want to test and implement the server side of the incoming connection handshake. This will need me to finish up Test::IO::Socket::Async. As usual, I’ll start by writing the tests I want to have:

constant $test-login = 'user';
constant $test-password = 'correcthorsebatterystaple';

my $test-server = TestableServer.new(host => $test-host, port => $test-port);
my $listen-tap = $test-server.listen().tap(-> $conn { });
my $socket-listener = await $test-socket.start-listening;
my $test-conn = $socket-listener.incoming-connection;

$test-conn.receive-data: Stomp::Message.new(
    command => 'CONNECT',
    headers => (
        login => $test-login,
        passcode => $test-password,
        accept-version => '1.2'
    ));

my $message-text = await $test-conn.sent-data;
my $parsed-message = Stomp::Parser.parse($message-text);
ok $parsed-message, "Server responded to CONNECT with valid message";
my $message = $parsed-message.made;
is $message.command, "CONNECTED", "Server sent CONNECTED command";
ok $message.headers<accept-version>:exists, "Server sent accept-version header";
is $message.body, "", "Server sent no message body";

The key new piece is the incoming-connection method. The API for simulating received data and obtaining sent data works just like in the client sockets testing – suggesting they’ll want to share a lot of the same code. But can they share all of the same code?

Glancing at the Connection monitor I already wrote, it seems the answer is no. The first four attributes:

has $.host;
has $.port;
has $.connection-promise = Promise.new;
has $!connection-vow = $!connection-promise.vow;

Aren’t really interesting for an incoming connection. The rest of the code, which deals purely with sending and receiving data, seems relevant, however. So, I’ll rename Connection to ClientConnection. I’ll then add a role called Connection, and factor the common bits out there. This gives me:

role Connection {
    has @!sent;
    has @!waiting-sent-vows;
    has $!received = Supplier.new;

    # print, write, sent-data, Supply, receive-data...
    ...
}

monitor ClientConnection does Connection {
    has $.host;
    has $.port;
    has $.connection-promise = Promise.new;
    has $!connection-vow = $!connection-promise.vow;

    method accept-connection() {
        $!connection-vow.keep(self);
    }

    method deny-connection($exception = "Connection refused") {
        $!connection-vow.break($exception);
    }
}

I’ll then define a ServerConnection monitor, which for now simply composes the Connection role:

monitor ServerConnection does Connection {
}

I note how nice it is that code factored out to a role can happily be composed into classes and monitors, and will automatically get the desired mutual exclusion behaviour when composed into a monitor. Now, I can implement the incoming-connection method in Listener:

method incoming-connection() {
    my $conn = ServerConnection.new;
    $!connection-supplier.emit($conn);
    $conn
}

And I’m done extending Test::IO::Socket::Async to support testing server sockets. Committed! And yes, I owe that module some tests and a README some time soon…maybe on the plane tomorrow. For now, the STOMP must go on!

So, how can I make my test pass? By doing the absolute easiest thing possible, of course. Here it is:

method listen() {
    supply {
        whenever self.socket-provider.listen($!host, $!port) -> $conn {
            whenever $conn {
                await $conn.print: Stomp::Message.new:
                    command => 'CONNECTED',
                    headers => ( accept-version => '1.2' );
            }
        }
    }
}

This is a fairly epic cheat. It just responds with a CONNECTED message when it receives anything on the socket! It passes the test, though. I’ve learned – mostly when doing ping-pong pair programming – that being willing to “cheat” my way to passing simple tests is actually a good thing. It makes it clear what tests I need to write next.

Sharing the parsing

To check I do treat CONNECT messages correctly, I’ll now write a test case where I sent complete junk to the server:

my $test-server = TestableServer.new(host => $test-host, port => $test-port);
my $listen-tap = $test-server.listen().tap(-> $conn { });
my $socket-listener = await $test-socket.start-listening;
my $test-conn = $socket-listener.incoming-connection;

$test-conn.receive-data: "EPIC FAIL!";

my $message-text = await $test-conn.sent-data;
my $parsed-message = Stomp::Parser.parse($message-text);
ok $parsed-message, "Server responded to invalid message with valid message";
is $parsed-message.made.command, "ERROR", "Server sent ERROR command";

Obviously, it fails, sending back a CONNECTED message instead of an ERROR. So what to do about it? Clearly, I’ll need to start paying a bit more attention to the messages coming it. That is, I need to turn a sequence of packets into a sequence of Stomp::Message objects. Hmm, that sounds familiar! Glancing in Stomp::Client, I see this:

method !process-messages($incoming) {
    supply {
        my $buffer = '';
        whenever $incoming -> $data {
            $buffer ~= $data;
            while Stomp::Parser::ServerCommands.subparse($buffer) -> $/ {
                given $/.made -> $message {
                    die $message.body if $message.command eq 'ERROR';
                    emit $message;
                }
                $buffer .= substr($/.chars);
            }
        }
    }
}

Well, that’s precisely what I want – except it’s parsing server commands, and I need to parse client commands. Factoring a method out feels like a job for a role – and needing to make the factored out code be parametric on something makes a parametric role just the ticket. So, I’ll add a Stomp::MessageStream role:

role Stomp::MessageStream[::MessageGrammar] {
    method !process-messages($incoming) {
        supply {
            my $buffer = '';
            whenever $incoming -> $data {
                $buffer ~= $data;
                while MessageGrammar.subparse($buffer) -> $/ {
                    given $/.made -> $message {
                        die $message.body if $message.command eq 'ERROR';
                        emit $message;
                    }
                    $buffer .= substr($/.chars);
                }
            }
        }
    }
}

And use it in Stomp::Client:

class Stomp::Client does Stomp::MessageStream[Stomp::Parser::ServerCommands] {
    ...
}

The client.t tests still happily pass, so it goes in as a commit. Now I can also compose the role into my Stomp::Server and use it:

class Stomp::Server does Stomp::MessageStream[Stomp::Parser::ClientCommands] {
    has Str $.host is required;
    has Int $.port is required;

    method listen() {
        supply {
            whenever self.socket-provider.listen($!host, $!port) -> $conn {
                whenever self!process-messages($conn) {
                    await $conn.print: Stomp::Message.new:
                        command => 'CONNECTED',
                        headers => ( accept-version => '1.2' );
                }
            }
        }
    }

    method socket-provider() {
        IO::Socket::Async
    }
}

That turns my test fail into…a hang. Why? Because the process-messages private method simply assumes that if it didn’t manage to parse a message, the reason must be that it’s incomplete – not that it’s broken. It will therefore just accumulate broken data in its buffer. Clearly, the parser needs to fail more violently if the incoming message is utterly bogus.

So, over in lib/Stomp/Parser.pm6, I’ll add an exception:

class X::Stomp::MalformedMessage is Exception {
    has $.reason;
    method message() {
        "Malformed STOMP message: $!reason"
    }
}

Then, I’ll tweak the TOP token to:

token TOP {
    [ <command> \n || <.maybe-command> ]
    [<header> \n]*
    \n
    <body>
    \n*
}

The addition here is the sequential alternation, calling maybe-command (the dot indicates to not capture the result). The idea of maybe-command is to check if the data so far might viably parse as a command if some more data were to arrive. It can look like this:

token maybe-command {
    <[A..Z]>**0..15 $ || <.malformed('invalid command')>
}
method malformed($reason) {
    die X::Stomp::MalformedMessage.new(:$reason);
}

With that, I’ve got from a hanging test to an exploding test. Happily, all of the client.t tests that also use the parser still work, though, so it can go in as a commit of its own.

It’s at this point that I finally ran into my first Rakudo bug of this series. The code I want to now write looks like this:

method listen() {
    supply {
        whenever self.socket-provider.listen($!host, $!port) -> $conn {
            whenever self!process-messages($conn) {
                await $conn.print: Stomp::Message.new:
                    command => 'CONNECTED',
                    headers => ( accept-version => '1.2' );

                QUIT {
                    when X::Stomp::MalformedMessage {
                        await $conn.print: Stomp::Message.new:
                            command => 'ERROR',
                            body => .message;
                    }
                }
            }
        }
    }
}

Unfortunately, QUIT phasers suffer a scoping bug if there is an exception before any other messages have been received. Thankfully, it’s easy enough to drop the whenever sugar in this case and fall back on the tap method:

method listen() {
    supply {
        whenever self.socket-provider.listen($!host, $!port) -> $conn {
            self!process-messages($conn).tap:
                {
                    await $conn.print: Stomp::Message.new:
                        command => 'CONNECTED',
                        headers => ( accept-version => '1.2' );
                },
                quit => {
                    when X::Stomp::MalformedMessage {
                        await $conn.print: Stomp::Message.new:
                            command => 'ERROR',
                            body => .message;
                    }
                };
        }
    }
}

And with that, the test passes.

And that’s Stomp::Server’s first steps

There’s still a bit more to do to ensure the first message really is a CONNECT, and to provide a hook for authentication. And then I’ll need to work out how the API for incoming messages is going to look, along with subscription/unsubscription requests. But that’ll be for next time. In the meantime, here’s a commit of the server work so far.

This entry was posted in Uncategorized. Bookmark the permalink.

3 Responses to Not guts, but 6: part 5

  1. Thanks for your posts, this is actually the reason I am trying to give P6 a chance. One question, what does the “my \TestableServer” do? In Perl5 \ is often used to create a reference but I don’t fully get what its doing here?

    • pau4o says:

      It’s a variable name without sigil if I remember corectly

    • The my \foo = … syntax is a name binding, introducing a lexically scoped symbol foo. Since they work as bindings rather than assignment into a container, you know what they’re bound to won’t change (which gives the reader one less piece of state to keep track of). Some folks call them sigilless variables, though I don’t really like that because the point is that they don’t vary! :-) They have various uses; here I’d just like TestableServer to look type-ish, but they’re also useful when constructing data processing pipelines and want to give names to the bits, without forcing scalar/array context on them.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s