Pith. sign in

REVIEW 4 minor 28 references

OpenMarket: A Synchronized Polymarket-Binance Dataset for High-Frequency Prediction-Market Research

T0 review · 0 major / 4 minor · reviewed 2026-08-01 · deepseek-v4-flash

Pith's one-line read This paper releases a first-of-its-kind millisecond-level paired Polymarket–Binance dataset and reports the null result that 43 microstructure features cannot out-forecast the probability already implied by Polymarket's own order book.

desk verdict A careful, honest data-and-methods release with a genuinely new ms-level paired corpus and a well-supported null result; worth referee time as a reproducibility artifact. read the letter →

arxiv 2607.26245 v1 pith:WRO46S46 submitted 2026-07-28 q-fin.TR

classification q-fin.TR
keywords predictionmarketsmarketmicrostructurehigh-frequencydatalead-laganalysisclocksynchronizationwalk-forwardvalidationPolymarketBinance
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish two linked claims: that a millisecond-synchronized public corpus pairing Polymarket's BTC 15-minute binary markets with Binance BTC/USDT order flow can be built honestly — storing both source and ingest timestamps, explicit lead–lag pairing metadata, and clock-drift diagnostics — and that on that corpus the attempt to forecast the binary outcome with 43 microstructure features fails out-of-sample. A walk-forward logistic model slightly underperforms the probability already implied by Polymarket's own order book (AUC 0.8377 vs 0.8405), and simulated trading loses −0.116 normalized payoff units per trade under stated fees and slippage. The paper also establishes stylized facts the corpus makes possible: top-of-book spreads are one tick wide in 91.9% of observations, the apparent source-clock lead–lag median is 16 ms with heavy tails, and Polymarket quotes follow large Binance moves after a median 347 ms on the collector clock. A sympathetic reader would read this as a data-and-methods release whose central empirical finding is a null forecasting result, published together with the data that produced it.

What carries the argument

The argument rests on pairing and timestamp discipline. Every event carries two timestamps — source_ts_ms from the exchange and ingest_ts_ms from the collector — and lead–lag is defined as lead_lag_ms = polymarket_source_ts_ms − binance_source_ts_ms, computed by a nearest-neighbor match inside a 750 ms alignment window and stored as the lag_pairs_ms table with quality flags. That identity makes cross-venue timing measurable, while the parallel ingest timestamps make clock drift a measured object: per-day minimum transport-delay envelopes bound relative drift to ≤6 ms and expose a ±99 ms single-vantage constant-offset ambiguity the paper does not assume away. The second load-bearing piece is

What would settle it

Take a random sample of scored out-of-sample rows, recompute all 43 features from the raw Parquet tables, and verify that every input event timestamp is strictly earlier than the feature-cutoff time and that the label and naive mid use that same cutoff; then re-run the frozen trainer and check whether pooled OOS AUC reproduces 0.8377 against the naive prior's 0.8405. If any feature uses post-cutoff data, or the figures shift on reproduction, the null forecasting result is an artifact of the harness rather than a property of the market.

Watch

Extended reading notes

Core claim

On its own terms, the central discovery is negative: over 355,814 strictly out-of-sample rows across 559 walk-forward windows, a logistic regression on 43 microstructure features reaches AUC 0.8377, while the naive baseline — the midpoint probability implied by Polymarket's own order book at the identical feature-cutoff timestamp — reaches AUC 0.8405. The model slightly underperforms the market's own price, and simulated trading nets −0.116 normalized payoff units per trade under stated fees and slippage. Around that null, the corpus yields descriptive findings: 91.9% of top-of-book spreads are one tick wide; 2.9 million paired events show an apparent 16 ms source-clock lead-lag median with

Load-bearing premise

The load-bearing premise is that the walk-forward pipeline leaks no future information: every feature row must be computed only from data available at the feature-cutoff timestamp, with the label and the naive mid-price baseline sampled at that identical instant. The paper itself flags that mishandled settlement labels can leak information and that step3 export covered only 2,251 of 4,450 markets, so the null result stands or falls on timestamp discipline that only running th

Editorial extensions

If this is right

  • New forecasting models for BTC 15-minute prediction markets now face a concrete, reproducible bar: beat the naive Polymarket mid's pooled out-of-sample AUC of 0.8405 on the released 357,390-row step3 set.
  • Cross-venue timing questions become answerable at scale: the apparent 16 ms median lead-lag with 5th/95th percentiles of −186/316 ms, and a stable 16–19 ms band across disagreement quintiles, gives future work a distribution rather than a single number.
  • The one-tick spread finding implies mid-price backtests systematically overstate executable edge, since fees and slippage turn apparent positive-EV signals into −0.116 per attempted trade.
  • The synchronization-free event study provides a venue-clock-independent timing anchor: Polymarket quotes move after a median 347 ms following large Binance moves, unaffected by the ±99 ms offset ambiguity.
  • The ~21% row collapse when merging overlapping archival snapshots shows that recorder-archive merges require explicit table-specific deduplication keys to avoid double-counting events.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the null result survives leakage scrutiny, it suggests BTC 15-minute Polymarket binaries are close to informationally efficient at the horizon these features can see: the order book already absorbs the public flow information the features encode, and any real edge is more likely to live in latency, queue position, or execution than in statistical forecasting.
  • The unresolved ±99 ms constant-offset ambiguity means true cross-venue ordering is not yet settled: the apparent 16 ms Binance lead could be anywhere from roughly −83 ms to +115 ms, so a multi-vantage collector deployment would be the decisive test of whether Binance actually leads at all.
  • The flat 16–19 ms median across disagreement regimes hints the lag is mechanical — quote propagation or relay latency — rather than information-driven; conditioning the released paired table on market and venue activity could test that directly.
  • A direct extension the paper leaves open is whether model class explains the null: tree ensembles, neural sequence models, or full order-book state reconstruction might beat the mid prior where logistic failed, which would bracket how much of the result is information content versus model capacity.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

0 major / 4 minor

Summary. The paper presents OpenMarket, a frozen open-source Rust pipeline and Hugging Face dataset that pairs millisecond-level Polymarket BTC 15-minute order-book event data with Binance BTC/USDT trades. It claims, to its knowledge, the first public paired corpus of this type with explicit pairing metadata: 727,098,247 deduplicated rows across 202 archival snapshots and 2,936,031 explicit lead–lag pairs. The central empirical claim is a null out-of-sample forecasting result: a walk-forward logistic model over 43 features does not beat, and slightly underperforms, the Polymarket mid prior sampled at the same feature-cutoff timestamp (pooled OOS AUC 0.8377 vs 0.8405; Brier 0.165 vs 0.163), and simulated positive-EV trading nets -0.116 normalized payoff units per attempted trade under stated fees and slippage. The paper also reports descriptive microstructure findings: one-tick top-of-book spreads, an apparent 16 ms median source-clock lead-lag with bounded drift and a roughly ±99 ms constant-offset ambiguity, and a synchronization-free median 347 ms Polymarket quote-response lag after large Binance moves.

Significance. If the released artifacts match their documentation, this is a valuable public-infrastructure contribution. The corpus is concrete and independently checkable; the walk-forward design is clearly described; the naive mid prior is sampled at the identical feature-cutoff timestamp as the model inputs; and the paper explicitly separates pooled out-of-sample rows from diagnostic full-timeline rows. The clock-offset analysis is careful and honest, and the synchronization-free event study provides a venue-clock-independent check on the lead-lag ordering. The paper also ships reproducible commands, pinned versions, validation scripts, and a data-availability statement, which are genuine strengths for a data-and-methods release. The central null result is falsifiable and reported with appropriate humility.

minor comments (4)
  1. [Table 2] The primary pooled OOS comparison is reported only as point estimates. The paper reports a paired market-block bootstrap CI for the diagnostic full-timeline comparison but not for the primary out-of-sample comparison. Please add a confidence interval or test for the pooled OOS AUC/Brier difference (e.g., the same paired bootstrap over market blocks) so that 'slightly underperforms' is not read as an unquantified point estimate.
  2. [Section 6] The phrase 'walk-forward logistic regression by market' is ambiguous: it could mean one model per market or a pooled model evaluated market-by-market. Please state explicitly whether the 559 walk-forward windows pool all markets or are estimated separately, since this affects how the OOS rows are generated.
  3. [Section 5.2] The statement 'relative clock drift bounded to ≤6 ms' is an inference from observed stability of the per-day min-delay envelopes. The text explains this, but the wording risks overclaiming a formal bound. Consider phrasing such as 'the observed envelope variation implies an empirical bound of at most 6 ms on relative drift across the archive.'
  4. [Table 2] The 'µs/row' column is empty for all naive-prior rows. Clarify whether the prior was not passed through the Rust scoring pipeline or whether the entry is intentionally omitted; a sentence in the caption would remove ambiguity.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: empirical data release with null benchmark; no load-bearing self-citations or by-construction reductions.

full rationale

The paper's central claims are empirical rather than derivational: the corpus exists at stated scale, the synchronization diagnostics measure observed timestamp differences, and the forecasting benchmark compares a walk-forward logistic model against the Polymarket mid prior on strictly out-of-sample rows. The mid prior is explicitly sampled at the same feature-cutoff timestamp as the model inputs, so the comparison is not constructed to favor either side. The null result (AUC 0.8377 vs 0.8405) is a measured outcome, not a fitted parameter renamed as a prediction. The clock-offset analysis is also empirical: the 16 ms median is labeled 'apparent', the ±99 ms constant-offset ambiguity is disclosed, and the 347 ms quote-response result is measured on the collector clock without any venue-timestamp comparison. There are no self-citations at all, and the related-work comparisons are to external datasets. Residual concerns such as possible lookahead leakage would, if anything, inflate the model's OOS AUC and make underperformance harder to observe, so they do not indicate circularity. The reproducibility commands and frozen version pins make the claims independently checkable. No load-bearing step reduces by definition or self-citation to its own inputs.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central claims are empirical measurements on a released dataset. No new entities are posited. The free parameters listed are analysis thresholds, not fitted to force a specific result. The axioms are standard domain assumptions about data quality and timestamp semantics, all disclosed or checked within the paper.

free parameters (4)
  • Alignment window W = 750 ms
    Chosen for pairing Polymarket and Binance events; affects lead-lag pairs and quality-flag distribution, but not the central null forecast result.
  • Large-move threshold = >= 5 bps within 1 s
    Defines the event-study trigger; affects which Binance moves are analyzed and therefore the 347 ms median response.
  • Response matching window = 2 s
    Limits the search for a Polymarket quote response; censors responses longer than 2 s and affects the median 347 ms estimate.
  • Quality flag thresholds = 100 ms / 300 ms
    Split lead-lag into tight/medium/wide bands; descriptive, not load-bearing for the null result.
assumptions (5)
  • domain assumption Binance public WebSocket trade timestamps are accurate to within a millisecond of actual execution.
    The lead-lag pairing and drift diagnostics assume source timestamps are meaningful at millisecond resolution; invoked in Section 4.1 and 5.
  • domain assumption Polymarket order-book update messages reflect the complete set of top-of-book changes.
    Spread and quote-response claims assume the collector captured every book update; stated as a quality metric in Section 5.1.
  • domain assumption The collector clock is stable and monotonic over the archive.
    Ingest timestamps are used for the synchronization-free event study and drift envelopes; the paper mentions monotonicity checks in Section 5.4 but relies on this throughout.
  • domain assumption Settlement labels (binary outcome) are correctly derived from BTC close prices.
    The model training and evaluation use labels; if labels were misaligned, the null result could change. The paper states one tied BTC close is dropped, implying correct handling.
  • domain assumption Public-feed trade direction inference has roughly 59% reliability (per Dubach [6]).
    The paper inherits this limitation for side labels and imbalance features, noting it in Section 10.4.

how reviews work

0 comments
Cite this review

Pith. "Pith review of OpenMarket: A Synchronized Polymarket-Binance Dataset for High-Frequency Prediction-Market Research." pith.science (2026). https://pith.science/paper/WRO46S46

@misc{pith2026260726245,
  author       = {Pith},
  title        = {Pith review of: OpenMarket: A Synchronized Polymarket-Binance Dataset for High-Frequency Prediction-Market Research},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WRO46S46}},
  note         = {Machine review of arXiv:2607.26245}
}
read the original abstract

OpenMarket began as an attempt to trade Polymarket's BTC 15-minute binary markets against Binance BTC/USDT order flow. The attempt did not produce a tradable edge: out-of-sample, a walk-forward logistic model over 43 microstructure features does not beat, and slightly underperforms, the probability already implied by Polymarket's own order book, and simulated trading nets -0.116 normalized payoff units per attempted trade under stated fee and slippage assumptions. We release the synchronized corpus and infrastructure that attempt produced and, to our knowledge, the first public millisecond-level Polymarket BTC / Binance BTC-USDT paired corpus with explicit pairing metadata. The frozen archive (tag v0.5.2) contains 727,098,247 deduplicated rows across 202 archival snapshots, with event data on 54 observed Polymarket days (57 Binance days) between 2026-02-12 and 2026-05-15, including 2,936,031 explicit lead-lag pairs, alongside a reproducible Rust pipeline for collection, millisecond pairing, Parquet export, and walk-forward calibration. Initial analyses establish Polymarket stylized facts (one-tick top-of-book spreads) and characterize cross-venue timing: an apparent 16 ms median lag on venue source clocks, with relative clock drift bounded to at most 6 ms over the archive but a remaining single-vantage constant-offset ambiguity of approximately plus or minus 99 ms. A synchronization-free event study, measured only on the collector clock, independently shows that Polymarket quotes respond to large Binance moves after a median 347 ms. We position this work as a data-and-methods release whose central empirical result is a null out-of-sample forecasting result.

Figures

Figures reproduced from arXiv: 2607.26245 by the authors.

Figure 1
Figure 1. OpenMarket pipeline. Binance and Polymarket WebSocket streams [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. Clock-offset validation. (a) Per-day minimum transport-delay en [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗
Figure 3
Figure 3. Lead–lag pairing timeline. Source timestamps determine Equa [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Unified split table row counts (log scale). [PITH_FULL_IMAGE:figures/full_fig_p014_4.png]
Figure 5
Figure 5. Figure 5: Empirical lead_lag_ms distribution from the unified lag_pairs_ms table (n = 2, 936, 031): the median is compact, but the distribution has heavy tails. 10.1 Scale and Coverage The unified archive contains 727,098,247 rows with event data on 54 observed Polymarket days b…
Figure 6
Figure 6. Figure 6: Median lead–lag by cross-venue disagreement. Across both [PITH_FULL_IMAGE:figures/full_fig_p017_6.png]
Figure 7
Figure 7. Figure 7: Forecast benchmark ROC AUC on step3 rows. Pooled out-of [PITH_FULL_IMAGE:figures/full_fig_p018_7.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

28 extracted references · 8 canonical work pages

  1. [1]

    Prediction markets as a research tool.The Economists’ Voice, 5(1):1–4, 2008

    Joyce E Berg, Forrest D Nelson, and Thomas A Rietz. Prediction markets as a research tool.The Economists’ Voice, 5(1):1–4, 2008. doi: 10.2202/1553-3832.1277

  2. [2]

    Price discovery on Bitcoin exchanges.Journal of International Financial Markets, Institutions and Money, 36:18–35,

    Morten Brandvold, Peter Molnár, Kristian Vagstad, and Ole Chris- tian Andreas Valstad. Price discovery on Bitcoin exchanges.Journal of International Financial Markets, Institutions and Money, 36:18–35,

  3. [3]

    Verification of forecasts expressed in terms of probability

    Glenn W Brier. Verification of forecasts expressed in terms of probability. Monthly Weather Review, 78(1):1–3, 1950. doi: 10.1175/1520-0493(1950 )078<0001:VOFEIT>2.0.CO;2

  4. [4]

    PolyBench: Benchmarking LLM forecasting and trading capabilities on live prediction market data

    Pu Cheng, Juncheng Liu, and Yunshen Long. PolyBench: Benchmarking LLM forecasting and trading capabilities on live prediction market data. arXiv preprint arXiv:2604.14199, 2026. doi: 10.48550/arXiv.2604.14199

  5. [5]

    The price impact of order book events.Journal of Financial Econometrics, 12(1):47–88,

    Rama Cont, Arseniy Kukanov, and Sasha Stoikov. The price impact of order book events.Journal of Financial Econometrics, 12(1):47–88,

  6. [6]

    The anatomy of a decentralized prediction market: Microstructure evidence from the Polymarket order book.arXiv preprint arXiv:2604.24366, 2026

    Philipp D Dubach. The anatomy of a decentralized prediction market: Microstructure evidence from the Polymarket order book.arXiv preprint arXiv:2604.24366, 2026. doi: 10.48550/arXiv.2604.24366

  7. [7]

    European Commission, 2016

    Commission Delegated Regulation Supplementing Directive 2014/65/EU with Regard to Regulatory Technical Standards for the Level of Accuracy of Business Clocks. European Commission, 2016. RTS 25 annex

  8. [8]

    Limit order books.Quantitative Finance, 13(11):1709–1742, 2013

    Martin D Gould, Mason A Porter, Stacy Williams, Mark McDonald, Daniel J Fenn, and Sam D Howison. Limit order books.Quantitative Finance, 13(11):1709–1742, 2013. doi: 10.1080/14697688.2013.803148

Show all 28 references
  1. [9]

    On calibration of modern neural networks

    Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. On calibration of modern neural networks. InProceedings of the 34th International Conference on Machine Learning, volume 70 ofProceedings of Machine Learning Research, pages 1321–1330, 2017

  2. [10]

    Combinatorial information market design.Information Systems Frontiers, 5(1):107–119, 2003

    Robin Hanson. Combinatorial information market design.Information Systems Frontiers, 5(1):107–119, 2003. doi: 10.1023/A:1022055808304

  3. [11]

    One security, many markets: Determining the contri- butions to price discovery.Journal of Finance, 50(4):1175–1199, 1995

    Joel Hasbrouck. One security, many markets: Determining the contri- butions to price discovery.Journal of Finance, 50(4):1175–1199, 1995. doi: 10.1111/j.1540-6261.1995.tb04052.x

  4. [12]

    Lobster: Limit order book recon- struction system

    Ruihong Huang and Tomas Polak. Lobster: Limit order book recon- struction system. Technical report, Humboldt University of Berlin, 2011

  5. [13]

    IEEE, 2019

    IEEE Standard for a Precision Clock Synchronization Protocol for Net- worked Measurement and Control Systems. IEEE, 2019

  6. [14]

    Unlocking the forecasting economy: A suite of datasets for the full lifecycle of prediction market: Experiments & analysis.arXiv preprint arXiv:2604.20421, 2026

    Huaiyu Jia, Luofeng Zhou, Wentao Zhang, Lin William Cong, Siguang Li, and Shuo Sun. Unlocking the forecasting economy: A suite of datasets for the full lifecycle of prediction market: Experiments & analysis.arXiv preprint arXiv:2604.20421, 2026. doi: 10.48550/arXiv.2604.20421. 22

  7. [15]

    Market microstructure: A survey.Journal of Financial Markets, 3(3):205–258, 2000

    Ananth Madhavan. Market microstructure: A survey.Journal of Financial Markets, 3(3):205–258, 2000. doi: 10.1016/S1386-4181(00)000 07-0

  8. [16]

    Trading and arbitrage in cryp- tocurrency markets.Journal of Financial Economics, 135(2):293–319,

    Igor Makarov and Antoinette Schoar. Trading and arbitrage in cryp- tocurrency markets.Journal of Financial Economics, 135(2):293–319,

  9. [17]

    Network time protocol version 4: Protocol and algorithms specification

    David L Mills, Jim Martin, Jack Burbank, and William Kasch. Network time protocol version 4: Protocol and algorithms specification. RFC 5905, Internet Engineering Task Force, 2010

  10. [18]

    Benchmark dataset for mid-price fore- casting of limit order book data with machine learning methods.Journal of Forecasting, 37(8):852–866, 2018

    Adamantios Ntakaris, Martin Magris, Juho Kanniainen, Moncef Gab- bouj, and Alexandros Iosifidis. Benchmark dataset for mid-price fore- casting of limit order book data with machine learning methods.Journal of Forecasting, 37(8):852–866, 2018. doi: 10.1002/for.2543

  11. [19]

    High frequency market microstructure.Journal of Financial Economics, 116(2):257–270, 2015

    Maureen O’Hara. High frequency market microstructure.Journal of Financial Economics, 116(2):257–270, 2015. doi: 10.1016/j.jfineco.2014 .10.006

  12. [20]

    Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods

    John C Platt. Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. InAdvances in Large Margin Classifiers, volume 10, pages 61–74. MIT Press, 1999

  13. [21]

    Polymarket-v1 database.arXiv preprint arXiv:2606.04217, 2026

    Boka Qin and Rui Yang. Polymarket-v1 database.arXiv preprint arXiv:2606.04217, 2026. doi: 10.48550/arXiv.2606.04217

  14. [22]

    SoK: Market microstructure for decentralized prediction markets (DePMs).arXiv preprint arXiv:2510.15612, 2025

    Nahid Rahman, Joseph Al-Chami, and Jeremy Clark. SoK: Market microstructure for decentralized prediction markets (DePMs).arXiv preprint arXiv:2510.15612, 2025. doi: 10.48550/arXiv.2510.15612

  15. [23]

    Unravelling the probabilistic forest: Arbitrage in prediction markets.arXiv preprint arXiv:2508.03474, 2025

    Oriol Saguillo, Vahid Ghafouri, Lucianna Kiffer, and Guillermo Suarez- Tangil. Unravelling the probabilistic forest: Arbitrage in prediction markets.arXiv preprint arXiv:2508.03474, 2025. doi: 10.48550/arXiv.2 508.03474

  16. [24]

    Prediction markets.Journal of Economic Perspectives, 18(2):107–126, 2004

    Justin Wolfers and Eric Zitzewitz. Prediction markets.Journal of Economic Perspectives, 18(2):107–126, 2004. doi: 10.1257/0895330041 371321

  17. [25]

    DeepLOB: Deep convolutional neural networks for limit order books.IEEE Transactions on Signal Processing, 67(11):3001–3012, 2019

    Zihao Zhang, Stefan Zohren, and Stephen Roberts. DeepLOB: Deep convolutional neural networks for limit order books.IEEE Transactions on Signal Processing, 67(11):3001–3012, 2019. doi: 10.1109/TSP.2019.2 907260. 23

  18. [2014]

    doi: 10.1093/jjfinec/nbt003

  19. [2015]

    doi: 10.1016/j.intfin.2015.02.010. 21

  20. [2020]

    doi: 10.1016/j.jfineco.2019.07.001

Pith tools

Reviewed August 1, 2026 · model on record in the stance chip above.