Peer-to-peer is very important, but it is also abstract in ways that make it hard to understand for many people. It seems hard for many developers to think of their apps as peers, it seems to be far easier to think in terms of client-server. This may change over time. The broker acts as a concrete object that helps them conceptualize and build sensible architectures.

RestMS's semantics are still evolving. I think the orthogonality will emerge over time, since the semantics are pluggable, and can thus be improved. It seems to me, today, that feed types are about persistence, that join types are about routing & filtering, and that pipe types are about persistence and delivery.

Xump is meant to act as an engine to develop and refine these notions. It's important for me that the resources actually exposed to the developer are high level, and not simply a network of queues. Thus, feeds and pipes, which may be internally similar but externally serve different goals.

So, as you say, we build constrained and high-level patterns from more generic low level primitives.

I like the RestMS decomposition for the most part but I don't like the fact that feeds, joins and pipes are not completely orthogonal. It's a bit unexpected that certain join types only make sense when combined with certain feed types. Why do we need feed types anyways? Why not envision feeds only as message entry points and associate them loosely with pipes by way of an intermediary join graph of orthogonal filter / distribution rules?

I like that you mentioned peer to peer. I'm in a situation right now where I am writing an application that initially uses a little local message broker for internal communication. Then it forks off other processes with their own message brokers and joins them to the message bus. On the side, it might also connect to well-known message brokers elsewhere on the network and join them to the bus also. To improve fault tolerance, message brokers could send presence advertisements when they join and leave the bus (or when their clients join and leave or when other resources are added or removed). I have not decided how to handle scoping yet since different participants should perhaps see different views of the feeds based on certain visibility rules.

I am a little disheartened by the fact that none of the extant general purpose messaging protocols that I can find are optimized for an ad-hoc peer to peer message bus. Most are designed for client / server around a central cluster of shared messaging resources. I believe peer to peer messaging and presence will be very important for pervasive message in the long term.