REVIEW 4 minor 28 references
OpenMarket: A Synchronized Polymarket-Binance Dataset for High-Frequency Prediction-Market Research
T0 review · 0 major / 4 minor · reviewed 2026-08-01 · deepseek-v4-flash
Pith's one-line read This paper releases a first-of-its-kind millisecond-level paired Polymarket–Binance dataset and reports the null result that 43 microstructure features cannot out-forecast the probability already implied by Polymarket's own order book.
desk verdict A careful, honest data-and-methods release with a genuinely new ms-level paired corpus and a well-supported null result; worth referee time as a reproducibility artifact. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The argument rests on pairing and timestamp discipline. Every event carries two timestamps — source_ts_ms from the exchange and ingest_ts_ms from the collector — and lead–lag is defined as lead_lag_ms = polymarket_source_ts_ms − binance_source_ts_ms, computed by a nearest-neighbor match inside a 750 ms alignment window and stored as the lag_pairs_ms table with quality flags. That identity makes cross-venue timing measurable, while the parallel ingest timestamps make clock drift a measured object: per-day minimum transport-delay envelopes bound relative drift to ≤6 ms and expose a ±99 ms single-vantage constant-offset ambiguity the paper does not assume away. The second load-bearing piece is
What would settle it
Take a random sample of scored out-of-sample rows, recompute all 43 features from the raw Parquet tables, and verify that every input event timestamp is strictly earlier than the feature-cutoff time and that the label and naive mid use that same cutoff; then re-run the frozen trainer and check whether pooled OOS AUC reproduces 0.8377 against the naive prior's 0.8405. If any feature uses post-cutoff data, or the figures shift on reproduction, the null forecasting result is an artifact of the harness rather than a property of the market.
Extended reading notes
Core claim
On its own terms, the central discovery is negative: over 355,814 strictly out-of-sample rows across 559 walk-forward windows, a logistic regression on 43 microstructure features reaches AUC 0.8377, while the naive baseline — the midpoint probability implied by Polymarket's own order book at the identical feature-cutoff timestamp — reaches AUC 0.8405. The model slightly underperforms the market's own price, and simulated trading nets −0.116 normalized payoff units per trade under stated fees and slippage. Around that null, the corpus yields descriptive findings: 91.9% of top-of-book spreads are one tick wide; 2.9 million paired events show an apparent 16 ms source-clock lead-lag median with
Load-bearing premise
The load-bearing premise is that the walk-forward pipeline leaks no future information: every feature row must be computed only from data available at the feature-cutoff timestamp, with the label and the naive mid-price baseline sampled at that identical instant. The paper itself flags that mishandled settlement labels can leak information and that step3 export covered only 2,251 of 4,450 markets, so the null result stands or falls on timestamp discipline that only running th
Editorial extensions
If this is right
- New forecasting models for BTC 15-minute prediction markets now face a concrete, reproducible bar: beat the naive Polymarket mid's pooled out-of-sample AUC of 0.8405 on the released 357,390-row step3 set.
- Cross-venue timing questions become answerable at scale: the apparent 16 ms median lead-lag with 5th/95th percentiles of −186/316 ms, and a stable 16–19 ms band across disagreement quintiles, gives future work a distribution rather than a single number.
- The one-tick spread finding implies mid-price backtests systematically overstate executable edge, since fees and slippage turn apparent positive-EV signals into −0.116 per attempted trade.
- The synchronization-free event study provides a venue-clock-independent timing anchor: Polymarket quotes move after a median 347 ms following large Binance moves, unaffected by the ±99 ms offset ambiguity.
- The ~21% row collapse when merging overlapping archival snapshots shows that recorder-archive merges require explicit table-specific deduplication keys to avoid double-counting events.
Reading between the lines
- If the null result survives leakage scrutiny, it suggests BTC 15-minute Polymarket binaries are close to informationally efficient at the horizon these features can see: the order book already absorbs the public flow information the features encode, and any real edge is more likely to live in latency, queue position, or execution than in statistical forecasting.
- The unresolved ±99 ms constant-offset ambiguity means true cross-venue ordering is not yet settled: the apparent 16 ms Binance lead could be anywhere from roughly −83 ms to +115 ms, so a multi-vantage collector deployment would be the decisive test of whether Binance actually leads at all.
- The flat 16–19 ms median across disagreement regimes hints the lag is mechanical — quote propagation or relay latency — rather than information-driven; conditioning the released paired table on market and venue activity could test that directly.
- A direct extension the paper leaves open is whether model class explains the null: tree ensembles, neural sequence models, or full order-book state reconstruction might beat the mid prior where logistic failed, which would bracket how much of the result is information content versus model capacity.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper presents OpenMarket, a frozen open-source Rust pipeline and Hugging Face dataset that pairs millisecond-level Polymarket BTC 15-minute order-book event data with Binance BTC/USDT trades. It claims, to its knowledge, the first public paired corpus of this type with explicit pairing metadata: 727,098,247 deduplicated rows across 202 archival snapshots and 2,936,031 explicit lead–lag pairs. The central empirical claim is a null out-of-sample forecasting result: a walk-forward logistic model over 43 features does not beat, and slightly underperforms, the Polymarket mid prior sampled at the same feature-cutoff timestamp (pooled OOS AUC 0.8377 vs 0.8405; Brier 0.165 vs 0.163), and simulated positive-EV trading nets -0.116 normalized payoff units per attempted trade under stated fees and slippage. The paper also reports descriptive microstructure findings: one-tick top-of-book spreads, an apparent 16 ms median source-clock lead-lag with bounded drift and a roughly ±99 ms constant-offset ambiguity, and a synchronization-free median 347 ms Polymarket quote-response lag after large Binance moves.
Significance. If the released artifacts match their documentation, this is a valuable public-infrastructure contribution. The corpus is concrete and independently checkable; the walk-forward design is clearly described; the naive mid prior is sampled at the identical feature-cutoff timestamp as the model inputs; and the paper explicitly separates pooled out-of-sample rows from diagnostic full-timeline rows. The clock-offset analysis is careful and honest, and the synchronization-free event study provides a venue-clock-independent check on the lead-lag ordering. The paper also ships reproducible commands, pinned versions, validation scripts, and a data-availability statement, which are genuine strengths for a data-and-methods release. The central null result is falsifiable and reported with appropriate humility.
minor comments (4)
- [Table 2] The primary pooled OOS comparison is reported only as point estimates. The paper reports a paired market-block bootstrap CI for the diagnostic full-timeline comparison but not for the primary out-of-sample comparison. Please add a confidence interval or test for the pooled OOS AUC/Brier difference (e.g., the same paired bootstrap over market blocks) so that 'slightly underperforms' is not read as an unquantified point estimate.
- [Section 6] The phrase 'walk-forward logistic regression by market' is ambiguous: it could mean one model per market or a pooled model evaluated market-by-market. Please state explicitly whether the 559 walk-forward windows pool all markets or are estimated separately, since this affects how the OOS rows are generated.
- [Section 5.2] The statement 'relative clock drift bounded to ≤6 ms' is an inference from observed stability of the per-day min-delay envelopes. The text explains this, but the wording risks overclaiming a formal bound. Consider phrasing such as 'the observed envelope variation implies an empirical bound of at most 6 ms on relative drift across the archive.'
- [Table 2] The 'µs/row' column is empty for all naive-prior rows. Clarify whether the prior was not passed through the Rust scoring pipeline or whether the entry is intentionally omitted; a sentence in the caption would remove ambiguity.
Circularity Check
No circularity: empirical data release with null benchmark; no load-bearing self-citations or by-construction reductions.
full rationale
The paper's central claims are empirical rather than derivational: the corpus exists at stated scale, the synchronization diagnostics measure observed timestamp differences, and the forecasting benchmark compares a walk-forward logistic model against the Polymarket mid prior on strictly out-of-sample rows. The mid prior is explicitly sampled at the same feature-cutoff timestamp as the model inputs, so the comparison is not constructed to favor either side. The null result (AUC 0.8377 vs 0.8405) is a measured outcome, not a fitted parameter renamed as a prediction. The clock-offset analysis is also empirical: the 16 ms median is labeled 'apparent', the ±99 ms constant-offset ambiguity is disclosed, and the 347 ms quote-response result is measured on the collector clock without any venue-timestamp comparison. There are no self-citations at all, and the related-work comparisons are to external datasets. Residual concerns such as possible lookahead leakage would, if anything, inflate the model's OOS AUC and make underperformance harder to observe, so they do not indicate circularity. The reproducibility commands and frozen version pins make the claims independently checkable. No load-bearing step reduces by definition or self-citation to its own inputs.
Assumptions & free parameters
free parameters (4)
- Alignment window W =
750 ms
- Large-move threshold =
>= 5 bps within 1 s
- Response matching window =
2 s
- Quality flag thresholds =
100 ms / 300 ms
assumptions (5)
- domain assumption Binance public WebSocket trade timestamps are accurate to within a millisecond of actual execution.
- domain assumption Polymarket order-book update messages reflect the complete set of top-of-book changes.
- domain assumption The collector clock is stable and monotonic over the archive.
- domain assumption Settlement labels (binary outcome) are correctly derived from BTC close prices.
- domain assumption Public-feed trade direction inference has roughly 59% reliability (per Dubach [6]).
Cite this review
Pith. "Pith review of OpenMarket: A Synchronized Polymarket-Binance Dataset for High-Frequency Prediction-Market Research." pith.science (2026). https://pith.science/paper/WRO46S46
@misc{pith2026260726245,
author = {Pith},
title = {Pith review of: OpenMarket: A Synchronized Polymarket-Binance Dataset for High-Frequency Prediction-Market Research},
year = {2026},
howpublished = {\url{https://pith.science/paper/WRO46S46}},
note = {Machine review of arXiv:2607.26245}
}
read the original abstract
OpenMarket began as an attempt to trade Polymarket's BTC 15-minute binary markets against Binance BTC/USDT order flow. The attempt did not produce a tradable edge: out-of-sample, a walk-forward logistic model over 43 microstructure features does not beat, and slightly underperforms, the probability already implied by Polymarket's own order book, and simulated trading nets -0.116 normalized payoff units per attempted trade under stated fee and slippage assumptions. We release the synchronized corpus and infrastructure that attempt produced and, to our knowledge, the first public millisecond-level Polymarket BTC / Binance BTC-USDT paired corpus with explicit pairing metadata. The frozen archive (tag v0.5.2) contains 727,098,247 deduplicated rows across 202 archival snapshots, with event data on 54 observed Polymarket days (57 Binance days) between 2026-02-12 and 2026-05-15, including 2,936,031 explicit lead-lag pairs, alongside a reproducible Rust pipeline for collection, millisecond pairing, Parquet export, and walk-forward calibration. Initial analyses establish Polymarket stylized facts (one-tick top-of-book spreads) and characterize cross-venue timing: an apparent 16 ms median lag on venue source clocks, with relative clock drift bounded to at most 6 ms over the archive but a remaining single-vantage constant-offset ambiguity of approximately plus or minus 99 ms. A synchronization-free event study, measured only on the collector clock, independently shows that Polymarket quotes respond to large Binance moves after a median 347 ms. We position this work as a data-and-methods release whose central empirical result is a null out-of-sample forecasting result.
Figures
Figures from the paper (4 more)
Reference graph
Works this paper leans on
-
[1]
Prediction markets as a research tool.The Economists’ Voice, 5(1):1–4, 2008
Joyce E Berg, Forrest D Nelson, and Thomas A Rietz. Prediction markets as a research tool.The Economists’ Voice, 5(1):1–4, 2008. doi: 10.2202/1553-3832.1277
arXiv 2008
-
[2]
Price discovery on Bitcoin exchanges.Journal of International Financial Markets, Institutions and Money, 36:18–35,
Morten Brandvold, Peter Molnár, Kristian Vagstad, and Ole Chris- tian Andreas Valstad. Price discovery on Bitcoin exchanges.Journal of International Financial Markets, Institutions and Money, 36:18–35,
-
[3]
Verification of forecasts expressed in terms of probability
Glenn W Brier. Verification of forecasts expressed in terms of probability. Monthly Weather Review, 78(1):1–3, 1950. doi: 10.1175/1520-0493(1950 )078<0001:VOFEIT>2.0.CO;2
-
[4]
PolyBench: Benchmarking LLM forecasting and trading capabilities on live prediction market data
Pu Cheng, Juncheng Liu, and Yunshen Long. PolyBench: Benchmarking LLM forecasting and trading capabilities on live prediction market data. arXiv preprint arXiv:2604.14199, 2026. doi: 10.48550/arXiv.2604.14199
-
[5]
The price impact of order book events.Journal of Financial Econometrics, 12(1):47–88,
Rama Cont, Arseniy Kukanov, and Sasha Stoikov. The price impact of order book events.Journal of Financial Econometrics, 12(1):47–88,
-
[6]
Philipp D Dubach. The anatomy of a decentralized prediction market: Microstructure evidence from the Polymarket order book.arXiv preprint arXiv:2604.24366, 2026. doi: 10.48550/arXiv.2604.24366
-
[7]
European Commission, 2016
Commission Delegated Regulation Supplementing Directive 2014/65/EU with Regard to Regulatory Technical Standards for the Level of Accuracy of Business Clocks. European Commission, 2016. RTS 25 annex
2014
-
[8]
Limit order books.Quantitative Finance, 13(11):1709–1742, 2013
Martin D Gould, Mason A Porter, Stacy Williams, Mark McDonald, Daniel J Fenn, and Sam D Howison. Limit order books.Quantitative Finance, 13(11):1709–1742, 2013. doi: 10.1080/14697688.2013.803148
arXiv 2013
Show all 28 references
-
[9]
On calibration of modern neural networks
Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. On calibration of modern neural networks. InProceedings of the 34th International Conference on Machine Learning, volume 70 ofProceedings of Machine Learning Research, pages 1321–1330, 2017
2017
-
[10]
Combinatorial information market design.Information Systems Frontiers, 5(1):107–119, 2003
Robin Hanson. Combinatorial information market design.Information Systems Frontiers, 5(1):107–119, 2003. doi: 10.1023/A:1022055808304
2003 doi
-
[11]
One security, many markets: Determining the contri- butions to price discovery.Journal of Finance, 50(4):1175–1199, 1995
Joel Hasbrouck. One security, many markets: Determining the contri- butions to price discovery.Journal of Finance, 50(4):1175–1199, 1995. doi: 10.1111/j.1540-6261.1995.tb04052.x
1995
-
[12]
Lobster: Limit order book recon- struction system
Ruihong Huang and Tomas Polak. Lobster: Limit order book recon- struction system. Technical report, Humboldt University of Berlin, 2011
2011
-
[13]
IEEE, 2019
IEEE Standard for a Precision Clock Synchronization Protocol for Net- worked Measurement and Control Systems. IEEE, 2019
2019
-
[14]
Unlocking the forecasting economy: A suite of datasets for the full lifecycle of prediction market: Experiments & analysis.arXiv preprint arXiv:2604.20421, 2026
Huaiyu Jia, Luofeng Zhou, Wentao Zhang, Lin William Cong, Siguang Li, and Shuo Sun. Unlocking the forecasting economy: A suite of datasets for the full lifecycle of prediction market: Experiments & analysis.arXiv preprint arXiv:2604.20421, 2026. doi: 10.48550/arXiv.2604.20421. 22
-
[15]
Market microstructure: A survey.Journal of Financial Markets, 3(3):205–258, 2000
Ananth Madhavan. Market microstructure: A survey.Journal of Financial Markets, 3(3):205–258, 2000. doi: 10.1016/S1386-4181(00)000 07-0
2000 doi
-
[16]
Trading and arbitrage in cryp- tocurrency markets.Journal of Financial Economics, 135(2):293–319,
Igor Makarov and Antoinette Schoar. Trading and arbitrage in cryp- tocurrency markets.Journal of Financial Economics, 135(2):293–319,
-
[17]
Network time protocol version 4: Protocol and algorithms specification
David L Mills, Jim Martin, Jack Burbank, and William Kasch. Network time protocol version 4: Protocol and algorithms specification. RFC 5905, Internet Engineering Task Force, 2010
2010
-
[18]
Benchmark dataset for mid-price fore- casting of limit order book data with machine learning methods.Journal of Forecasting, 37(8):852–866, 2018
Adamantios Ntakaris, Martin Magris, Juho Kanniainen, Moncef Gab- bouj, and Alexandros Iosifidis. Benchmark dataset for mid-price fore- casting of limit order book data with machine learning methods.Journal of Forecasting, 37(8):852–866, 2018. doi: 10.1002/for.2543
2018 doi
-
[19]
High frequency market microstructure.Journal of Financial Economics, 116(2):257–270, 2015
Maureen O’Hara. High frequency market microstructure.Journal of Financial Economics, 116(2):257–270, 2015. doi: 10.1016/j.jfineco.2014 .10.006
2015 doi
-
[20]
Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods
John C Platt. Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. InAdvances in Large Margin Classifiers, volume 10, pages 61–74. MIT Press, 1999
1999
- [21]
-
[22]
SoK: Market microstructure for decentralized prediction markets (DePMs).arXiv preprint arXiv:2510.15612, 2025
Nahid Rahman, Joseph Al-Chami, and Jeremy Clark. SoK: Market microstructure for decentralized prediction markets (DePMs).arXiv preprint arXiv:2510.15612, 2025. doi: 10.48550/arXiv.2510.15612
-
[23]
Unravelling the probabilistic forest: Arbitrage in prediction markets.arXiv preprint arXiv:2508.03474, 2025
Oriol Saguillo, Vahid Ghafouri, Lucianna Kiffer, and Guillermo Suarez- Tangil. Unravelling the probabilistic forest: Arbitrage in prediction markets.arXiv preprint arXiv:2508.03474, 2025. doi: 10.48550/arXiv.2 508.03474
-
[24]
Prediction markets.Journal of Economic Perspectives, 18(2):107–126, 2004
Justin Wolfers and Eric Zitzewitz. Prediction markets.Journal of Economic Perspectives, 18(2):107–126, 2004. doi: 10.1257/0895330041 371321
2004 doi
-
[25]
DeepLOB: Deep convolutional neural networks for limit order books.IEEE Transactions on Signal Processing, 67(11):3001–3012, 2019
Zihao Zhang, Stefan Zohren, and Stephen Roberts. DeepLOB: Deep convolutional neural networks for limit order books.IEEE Transactions on Signal Processing, 67(11):3001–3012, 2019. doi: 10.1109/TSP.2019.2 907260. 23
2019 doi
-
[2014]
doi: 10.1093/jjfinec/nbt003
-
[2015]
doi: 10.1016/j.intfin.2015.02.010. 21
2015 doi
-
[2020]
doi: 10.1016/j.jfineco.2019.07.001
2019 doi
Reviewed August 1, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.