Selectors
The slow consumer mitigation mechanisms, shard splitting and adaptive conflation, work independently of your application code. If you are migrating from MetaFluent Enterprise Server (v5), they are also fully backwards compatible. Your existing application code connects and benefits without changes.
One interesting new feature supported by Elastic MDS is the use of the JMS selector construct. The selector mechanism is standard JMS. The idea is to control and filter the data returned to your subscriptions. Selectors are an optional parameter on your subscribe calls. Selector usage is a trivial code change but they are worth understanding both as a convenience and as another way to reduce the load on your application before it has a chance to become a slow consumer in the first place.
What a selector is
A selector is a string parameter added to a subscription call. In its simplest form it is a comma-separated list of the fields you actually need:
BID, BID_SIZE, ASK, ASK_SIZE, TRDPRC_1, TRDVOL_1That is all. The platform uses this list to filter what it sends you. Initial images contain only those fields. Updates contain only those fields. Updates that touch none of them are not sent at all.
You are telling the infrastructure to stop sending data you did not want in the first place.
Why it matters
With a selector, only the fields you asked for make the trip. Less bandwidth consumed. Less CPU spent on both ends. And depending on how chatty the underlying data is, the use of selectors could cause a meaningful reduction in update volume.
Video: A.4 — Field Filtering. Shows the bandwidth difference in real time using network monitoring.
Beyond field filtering
Elastic MDS interprets the selector string as SQL. In its simplest form the selector is treated as the SELECT clause of an SQL expression. So “BID, ASK” is like “SELECT BID, ASK”. Because Elastic MDS understands SQL, this means you can also rename fields to match your application’s conventions:
BID, ASK, TRDPRC_1 AS LASTIn both images and updates, your application receives LAST instead of TRDPRC_1. No translation layer in your code, no mapping tables to maintain. The infrastructure handles it.
And more interestingly, this means you can inject computations directly into your data stream. A simple example:
BID, ASK, ASK-BID AS SPREADYour application receives a SPREAD field alongside BID and ASK, computed by the platform on every update. You did not write the computation. You do not maintain it. It arrives as a named field, the same as any other. If you only needed BID and ASK to compute the spread, you could just select the computation. Then the BID and ASK data doesn’t get delivered to your application at all.
Where is this going?
The computation example above is, of course, trivial, but it points the way to some of the benefits of upstream computations. The principle is that Elastic MDS can customize and augment your data model including injected computations as part of the subscription. Besides the bandwidth reduction, upstream computations benefit from the resilience and scalability of the Elastic MDS infrastructure. More on this in a subsequent post.
The more work done upstream, the less work required in your application, and the less data that has to move across your network. Selectors are the first example of this principle. They are not the last.
See it in action
Related
About this blog
Elastic MDS is an autoscaling, real-time market data distribution platform developed by MetaFluent. This blog is about Elastic MDS infrastructure deployment and application development.
Further reading
