A customer shared a great example use case of Databento recently. This showcases several interesting pieces of market microstructure at once:
- Sweeps of two related instruments. This happens much faster than private fill ack (difference between red and blue vertical lines).
- New level formation. There’s a wide spectrum of response latency here, but notably multiple orders are added to the level before the trade is published on the public feed (close to the green vertical line).
“Faster-than-light” orders
There’re many explanations for how many participants were able to respond faster than the trade was published on the public feed. Here’s a non-exhaustive list, roughly in descending order from fastest to slowest:
- Residual from initial taker. The initial taker may have residual that gets posted as the first bid on the level.
- Preemptive books in same firm as initial taker. The initial taker may be a large shop that has a consolidated book that published the outgoing marketable order first to their internal teams, so other teams were able to update their preemptive books and react even before the exchange matched the trade.
- Multiple takers. Other takers may have picked up on the same signal, e.g. it could’ve been based on an index options move; they could’ve been hedging out on that move, or they could’ve received a private fill on another leg of the trade. But these takers were too late to lift the offer. They didn’t attach a FOK instruction because there’s still a good fill rate even if they miss and form a new level so early, so their thought-to-be-marketable limits ended up being posted as bids instead.
- Private fills or canary orders. The market makers on the offer side get a private fill ack faster than the trade summary is published by the exchange and decide to rejoin the level with bids.
Receive timestamp and importance of packet captures (pcaps)
In the past, the minimum time to reconstruct this study from scratch could be weeks or months.
Crucially, it’s very expensive to obtain the timestamp marked with the green vertical line — you can’t extract it from exchange-embedded timestamps that historical tick data providers usually carry.
This timestamp, taken at or close to the boundary switch where the exchange handoff terminates, allows you to simulate the theoretical minimum latency that you can react to a feed event, assuming fiber equalization and fair gateways. It’s why many firms choose to colocate and capture their own data or buy pcaps.
You had to talk to some salesperson on the phone, coordinate a transfer of large pcap files, write a parser or use an open source pcap dissector, and then find some way to query the data.
Using Databento for microstructure research
It’s easy to construct this type of plot with Databento because our API lets you do a few things in the manner of seconds:
- Look up any arbitrary time interval of interest to the nanosecond resolution, with amortized constant time on common intervals.
- Fetch any combination of instruments with extremely fast, server-side merge and sort.
- Access historical, normalized MBO (“L3”) data, even over a limited WiFi connection.
- Convert the data into a dataframe container like pandas.
- Extract multiple, high precision timestamps associated with each event. Including the critical receive timestamp (ts_recv).
About Databento
Databento provides APIs that make it simpler and faster to access market data. We also provide the same packet captures with high-precision timestamping that our normalized data is generated from.
Contact sales@databento.com for inquiries.
