Messaging pattern with (statically) enforced order?

I'm trying to devise an API for a set of messages/events that get passed in a streaming manner.

Specifically, I'm build a test-framework, and I want to separate the back-end listeners (such as XML reporter, console reporter, code coverage reporter, etc.) such that these implement an interface (or a set of interfaces) that enables the test-framework to send a fixed/known set of message types, which occur in a defined valid order.

For example:

• Begin Test Suite A
• Begin Test Case 1
• Log Assertion Result X
• Log Assertion Result Y
• End Test Case 1
• Begin Test Case 2
• Log Assertion Result Z
• End Test Case 2
• End Test Suite A
• Begin Test Suite B
• ...

In other words, there's an required order for these messages to make sense: A Test Case can't start before a Test Suite has been started, each Test Case much end before starting a new Test Case, Assertion Results must be logged while there's an active Test Case, and so on.

My first attempt was a set of interfaces, and my idea here was you wouldn't be able to start a Test Case until you've started a Test Suite, because that's how you get your Test Case listener, and you wouldn't be able to submit an Assertion Result until you've started a Test Case, and so on.

This approach only enforces one rule though, ensuring that you can't post a certain type of message before posting another given type of message - there's nothing that guarantees that client code doesn't just keep a reference to a previous Test Case, for example, and submit Assertion Results out of order. There's also nothing that guarantees you call the end() methods to close a Test Suite or Case. So this doesn't really work.

The only alternative I've been able to think of, is a more traditional event stream - where each message type is an interface that extends a marker interface, and all of the messages are submitted via the same method to a bus that distributes the messages to the listeners.

This is probably easier to implement and understand, and avoids the issue with client code keeping a reference to a previous listener, since there's just one way to submit a message. I can write a standard implementation of such a listener that simply validates message order, and the framework can internally put an instance of this before the user-supplied list of listeners.

That might be simpler and cleaner, but you of course still submit messages in the wrong order - the message order still needs to be validated at run-time.

Is there any sort of messaging pattern that statically enforces message order?

Comments

[Kasey Speakman]

I am not aware of a statically enforceable message order. Typically, you would not have listeners check the ordering since they probably cannot fix it. They would just respond to what they were sent even if incorrect. Sending the correct ordering would be the responsibility of the test processing logic. To verify that was correct before-hand, you would need some sort of formal verification tool. Typically though, most of us settle for unit testing of logic to verify behavior. In any case, you probably want to think about how you would handle an incorrect ordering of events in case there was a bug. One common solution is to send a corrective event after the fact.

As far the client API, you could try to design it such that doing the right things is easy and/or automatic. You might use a fluent API which returns a new type of object that only allows certain actions. For example, the only way to get a TestCase object is to call TestSuite.Start() or TestSuite.NextCase(). Or a usage pattern that automatically calls End events when the method goes out of scope. For example, C# has IDisposable and using for this purpose.

[Rasmus Schultz]

Thanks for all the ideas :-)

Typically, you would not have listeners check the ordering since they probably cannot fix it

This is actually my favorite point! Sending messages in the correct order is of course the framework's concern - it seems I've been overthinking it somewhat.

You might use a fluent API which returns a new type of object that only allows certain actions

That's basically what I've been doing - but it seems like a dead end, as explained, since the fluent API only guarantees "A before B", which is only a small part of the problem.

Or a usage pattern that automatically calls End events when the method goes out of scope.

I thought about this, but it won't really work - or at least won't always work correctly. For some listeners, it's important to know when the stream of events ends - for example, a profiler needs to know precisely when a test-case ends, an XML reporter needs to write after the last Test Suite, and so on.

Anyways, thank you for reminding me to focus on the right problems here :-)

[Kasey Speakman]

Oh, I see. I was thinking about the framework process (calling with fluent API or self-closing usage), not the listener. You are right, these methods won't work for listener code. Happy to participate. :)

[Rasmus Schultz]

Yeah, the idea is to create an API for listeners that is largely framework-neutral. I've simplified it quite a bit tonight :-)

[Betsy Haibel]

This doesn't belong in the message listeners. Depending on language, you might be able to constrain the message SENDER such that some kinds of messages are only sendable in particular states, and instructions to send messages trigger state transitions. But I'm a little leery of that, because the order problem ultimately would have to do with other code rather than the message sender. The only context in which I could see this constraint being useful would be setting things up to crash fast if events were happening outside of the proper order.

[Rasmus Schultz]

I'm honestly not worried about it - there are much fewer test-frameworks than there are listeners. The test-framework should have a test-suite that checks the message order, and the listeners shouldn't need to worry about it at all - they should simply assume the message order is sane. It's not on the interfaces to provide this guarantee, it's on the test-framework.