{"id":"dfd422b3-7852-4331-be41-90c56c211a22","arxiv_id":"2412.16174","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":5,"one_line_summary":"Machine learning with numeric, text, and OCR-extracted features predicts Indian IPO listing-day price direction and underpricing, and grey market premium is shown to be a good proxy for Main Board but not SME IPOs.","lead":"This paper trains machine learning and language models to predict whether Indian IPOs will list above their issue price, using prospectus text, financials, market data, and news. It also finds that the unofficial grey market premium is a reliable guide for Main Board IPOs but not for small-cap SME IPOs.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"In-sample probability stacking in Section 5.1 invalidates the multimodal claim; text-augmented gains are within noise and would vanish under out-of-fold re-evaluation.","rationale":"The reader's weakest assumption identifies exactly the same load-bearing concern: first-stage models are trained on the full training set and their in-sample predicted probabilities are appended as features for the second-stage models. This is not a valid stacking procedure and can encode the target through the fitted model, especially with high-capacity tree ensembles. I verified that the text features are used as probabilities from first-stage classifiers trained on the same training data, with no mention of cross-validated or out-of-fold predictions. In addition, the reported text-augmented improvements are tiny (F1(1) differences of 0.001–0.003), far below what could be distinguished from noise on a 57-sample Main Board test set, so even a correctly implemented stacking pipeline is unlikely to support the central claim. The GMP analysis is a separate empirical observation that does not depend on the flawed ML pipeline and may be a useful contribution, but it is not the paper's central claim. Therefore the reader's REJECT verdict is appropriate and should remain unchanged; a revision with out-of-fold stacking, proper baselines, and significance testing would be needed before any multimodal predictive claim could be accepted.","tokens_in":22954,"tokens_out":6373,"duration_ms":61203,"concrete_test":"Re-run the Section 5.1 stacking pipeline for the text-augmented configurations (N-C-Tn, N-C-Td) using 5-fold out-of-fold predicted probabilities from the first-stage models as inputs to the second-stage models, keeping the same 2009–2022/2023 split and the same H2O AutoML settings. If the test AUC/F1 values in Table 2 shift materially or the high-price text gains (MB F1(1) 0.935 vs 0.934; SME 0.942 vs 0.939) disappear, the reported multimodal improvement is an artifact of in-sample probability leakage.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 5.1 trains first-stage models (Nomic-embedding ML models and fine-tuned DeBERTa classifiers) on the full 2009–2022 training set, then appends the predicted positive-class probabilities of the training samples as features for second-stage models trained on that same training set. This is target leakage through the first-stage fit. Tree-based models (DRF, XGB, GBM) can memorize training labels, so in-sample probabilities encode the label directly; the second-stage model therefore learns a mapping that is part label, part signal. The text-augmented configurations (N-C-Tn, N-C-Td, N-C-Tn-Nw) in Tables 2 and 3 thus do not measure whether text features generalize; they measure the meta-model's ability to exploit the first-stage model's in-sample overconfidence. Standard stacking requires out-of-fold (cross-validated) probabilities, and the paper never states that such a procedure was used. Moreover, the claimed text advantage is negligible: for Main Board High price direction, the best F1(1) moves from 0.934 (N-C) to 0.935 (N-C-Td); for SME High, from 0.939 to 0.942. These differences are far below any significance threshold, especially on a 57/183-sample single-year test set. The load-bearing central claim—that multimodal fusion improves IPO success prediction—rests on this leaky stacking and on noise-level metric differences. The GMP finding in Section 5.3 is independent and plausible, but it does not rescue the predictive claim.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents two new datasets on Indian Main Board and SME IPOs (2009–2023 and 2017–2023, respectively), and proposes a multi-classifier system that fuses numeric, categorical, text (from prospectuses and news), and OCR-derived features to predict the direction and magnitude of price movements at the opening, high, and closing prices on the listing day. The authors also analyze the relation between Grey Market Premium (GMP) and listing-day returns, reporting that GMP aligns with listing-price/issue-price differences in 80.29% of Main Board cases but only 21.29% of SME cases. The main empirical claims are that text features improve prediction for the high-price direction, that the proposed models outperform zero-shot Gemini and Llama baselines, and that GMP is a useful proxy for Main Board IPOs but not SME IPOs.","tokens_in":23314,"tokens_out":4455,"duration_ms":44744,"significance":"If the predictive claims were sound, the paper would contribute useful resources and a practical decision system: the two datasets are a valuable addition, the RAG-based extraction of prospectus content is carefully designed, the authors provide hyperparameters and code links for reproducibility, and the GMP analysis is a concrete, falsifiable empirical finding. However, the central claim that multimodal fusion improves IPO success prediction is not supported by the experiments as reported, due to target leakage in the stacking procedure and the absence of trivial baselines and significance testing. The GMP finding is independent and plausible, but it does not rescue the predictive claim.","major_comments":[{"comment":"The text-feature stacking procedure leaks target information. The paper states: 'We selected the best model in each case and appended the probabilities of the positive class as features to the list of existing numeric and categorical features,' and the first-stage models (Nomic-embedding ML models and fine-tuned DeBERTa classifiers) are trained on the same 2009–2022 training set whose labels are then used to compute these probabilities for the second-stage models. No out-of-fold or nested cross-validation is described. Because tree-based second-stage models (DRF, XGB, GBM) can memorize in-sample first-stage predictions, the augmented configurations in Tables 2 and 3 do not measure whether text features generalize; they measure the second-stage model's ability to exploit the first-stage model's overfitting. The reported F1 and AUC values for all N-C-Tn and N-C-Td configurations are therefore not valid estimates of out-of-sample performance, and the central claim that text features improve prediction is unsupported.","section":"Section 5.1"},{"comment":"No majority-class or constant-predictor baseline is reported, and many configurations have F1(0)=0.000 (e.g., GBM O N-C-Tn, Ens O N-C-Tn, DL Cl N-C-Td), meaning the model never predicts the negative class. With the high positive-class share in the 2023 test set, an F1(1) around 0.90 is achievable by predicting the majority class for every instance. Moreover, the claimed text advantage is contradicted by the table's own numbers: for MB High direction, the best N-C model (XGB) has F1(1)=0.943 while the best N-C-Td model (GLM) has F1(1)=0.935; for SME High direction, the difference between N-C (0.939) and N-C-Tn-Nw (0.942) is 0.003. These differences are far below any reasonable significance threshold, especially given the small test set, so the conclusion in Section 5.1 that 'texts features do play a role' is not established.","section":"Tables 2 and 3"},{"comment":"The test set consists of a single year (2023) with only 57 Main Board and 183 SME IPOs. All model rankings in Tables 2 and 3 are based on this one-year, relatively small sample, with no confidence intervals, error bars, or significance tests. The performance gaps between the best and second-best configurations are often 0.001–0.005 in F1 or 0.001–0.01 in MAE, which is within the noise expected from such a sample. This is a load-bearing limitation for the predictive claim because the reported 'best' models may simply be the result of random variation on a particular test year.","section":"Table 1"}],"minor_comments":[{"comment":"There are multiple typos and grammatical errors: 'The reminder of this paper' should be 'The remainder'; 'we complied a list' should be 'we compiled a list'; and Section 5.1 contains 'DeBERRTa' instead of 'DeBERTa'.","section":"Section 1 and Section 4"},{"comment":"Questions 17 and 19 are identical ('Is the IPO price is reasonable and offers potential for growth?'); this appears to be a copy-paste error, and one of them should be replaced with the intended distinct question.","section":"Table A.5"},{"comment":"The paper claims to use 'images' as a modality, but the only image processing is OCR to extract text, which is then embedded and treated as text. The multimodal framing is therefore overstated; the actual inputs are numeric, categorical, and text features.","section":"Section 4 and Figure 5"},{"comment":"The Llama 3.2 3b under-pricing results are discussed in the text for Main Board IPOs, but Table 3 only reports Gemini for the under-pricing tasks. The reader cannot compare the LLM baselines directly, and the absence of Llama rows for SME under-pricing should be noted in the table or caption.","section":"Section 5.2 and Table 3"},{"comment":"The comparison with Gemini and Llama is not apples-to-apples: the LLMs are prompted zero-shot with JSON content, while the ML models are trained on the same features. The paper should acknowledge that this is an advantage for the trained models and not claim a fair 'outperform' without a matched setting.","section":"Table 2"}],"recommendation":"major_revision","confidential_remarks":"The paper has a useful dataset contribution and an interesting GMP analysis, but the central predictive claim is invalidated by the in-sample stacking leakage. This is fixable by re-running the experiments with out-of-fold probability generation and adding majority-class baselines; however, given the noise-level differences in the current tables, the text-modality claim may not survive such a re-analysis. The editor may wish to consider whether the paper would be better positioned as a data-and-analysis resource rather than a predictive-model paper."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The one thing to know: the GMP analysis is a genuinely interesting empirical result, but the headline claim about multimodal prediction doesn't survive contact with the methodology. The paper's own Section 5.1 describes training first-stage models on the full training set and then feeding their in-sample positive-class probabilities as features to second-stage models trained on the same data. That's target leakage through the fitted model, plain and simple. Standard stacking requires out-of-fold probabilities; the paper never reports them. So the text-augmented gains in Tables 2 and 3—most of which are noise-level, like 0.934 to 0.935 F1(1)—are not credible evidence that text helps.\n\nWhat is actually new and useful: the two datasets (418 Main Board, 681 SME IPOs) are a real contribution, and they're released. The finding that GMP aligns with listing direction in 80.29% of Main Board cases but only 21.29% of SME cases is simple, falsifiable, and consistent with the market chatter about SME manipulation. The RAG pipeline with 25 curated prospectus questions is a sensible way to extract text features, and the authors are transparent about their limitations and provide reproducibility details.\n\nThe soft spots are concentrated in the experimental protocol. Besides the stacking leak, there is no majority-class or mean-predictor baseline, and many configurations show F1(0)=0.000, meaning the model never predicts the negative class; high F1(1) is then trivial. The test set is a single year (57 MB, 183 SME), so configuration rankings are fragile. None of this kills the GMP result, which is independent of the ML pipeline.\n\nWho is this for? Someone working on Indian IPO data or investor decision tools might want the datasets and the GMP observation. The core predictive framework would need rework—out-of-fold stacking, proper baselines, pooled cross-validation—before its results mean anything.\n\nI'd send this to peer review, not desk reject, because the datasets and GMP finding deserve referee time and the main flaw is methodologically fixable. But the reviewers should be instructed to focus on the stacking issue; as written, the multimodal claim fails.","headline":"The GMP result is real and useful; the multimodal prediction claim collapses on in-sample stacking leakage.","tokens_in":23837,"tokens_out":2213,"would_cite":false,"duration_ms":20696,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Text and numbers together predict Indian IPO listing-day moves","keywords":["IPO success prediction","grey market premium","multimodal fusion","multi-classifier system","Indian stock market","natural language processing","retrieval-augmented generation","listing day returns"],"falsifier":"Re-run the two-stage pipeline using out-of-fold (cross-validated) first-stage probabilities instead of in-sample probabilities on the same 2023 test split; if the text-augmented models no longer beat numeric-categorical baselines on high-price direction, the claimed text benefit vanishes and the multimodal advantage is an artifact of target leakage.","tokens_in":22772,"feed_emoji":"📈","tokens_out":3776,"duration_ms":36140,"temperature":0.7,"pith_summary":"The paper asks whether an IPO's first-day price movements can be predicted before listing using everything available at subscription time: company financials, market and macroeconomic indicators, subscription rates, broker recommendations, and the prospectus text itself. It curates two new datasets covering Indian Main Board and SME IPOs, then runs a two-stage multi-classifier system that fuses numeric, categorical, text, and OCR-derived features, evaluating on 2023 listings. The central empirical claim is that these trained models beat zero-shot large language models, and that adding text features most helps prediction of whether the day's high price exceeds the issue price. A separate analysis argues that grey market premium (GMP) aligns with listing-day returns in about 80% of Main Board IPOs but only about 21% of SME IPOs, so GMP is trustworthy for Main Board offerings but not for SME offerings.","feed_headline":"Text and numbers together predict Indian IPO listing-day moves","feed_subtitle":"Fusing prospectus text, OCR, and numeric features beats zero-shot LLMs; grey market premium only signals Main Board IPOs.","key_machinery":"The central mechanism is the two-stage probability-augmentation pipeline. First-stage models (generalized linear models, distributed random forests, deep learning, XGBoost, gradient boosting machines, and their ensemble, trained with an AutoML library) produce positive-class probabilities from text representations: either Nomic embeddings fed to traditional ML models or fine-tuned DeBERTa classifiers. These probabilities are appended to numeric and categorical features, and a second-stage classifier makes the final prediction. For prospectus content, a retrieval-augmented generation step extracts 25 answer columns by retrieving pages with cosine similarity and BM25 and generating answers with Llama 3.2 3B. The GMP analysis is a separate descriptive mechanism: comparing the sign of GMP with the sign of (listing price minus issue price).","core_discovery":"The paper's central claim is that a multi-classifier decision system, which appends probabilities from first-stage text models to structured features and feeds them to a second-stage classifier, can predict the direction and magnitude of opening, high, and closing price movements on the listing day of Indian IPOs. The claim is conditional on IPO type: for opening and closing price direction, models trained on numeric and categorical features alone perform best, whereas for high price direction, adding text-derived probabilities improves F1 for the positive class. The paper further claims that grey market premium serves as a proxy for Main Board IPO success, aligning with the actual listing-versus-issue price relationship in 80.29% of cases, but fails for SME IPOs, with only 21.29% alignment. Across all tasks, the proposed models outperform zero-shot Gemini 1.5 flash and Llama 3.2 3b prompting.","pith_inferences":["If the leakage concern is resolved and text gains persist, the result suggests prospectus narratives contain incremental information about listing-day demand beyond structured signals; an ablation over the 25 RAG questions could identify which parts of the prospectus carry that signal.","The GMP asymmetry between Main Board and SME IPOs may reflect the regulatory cracks and manipulation concerns documented in the SME segment, implying that GMP is an institutional-sentiment measure rather than a universal demand gauge.","A testable extension is to apply the same two-stage probability-augmentation pipeline to IPO markets in other emerging economies, where prospectus documents and grey markets exist in similar forms.","The paper's own limitation list notes that social-media sentiment, expert analyst reviews, and major event shocks were not modeled, so a natural next step is to fold those modalities into the same fusion architecture."],"forward_implications":["If the claims hold, investors can use GMP as a sanity check for Main Board IPOs but should discount it for SME IPOs, where the paper finds it misaligns with listing-day outcomes in most cases.","Text-augmented features would give a practical edge specifically in predicting the listing-day high price direction, while opening and closing price direction are already well served by structured features alone.","Trained multi-classifier systems would be a more reliable decision-support tool for IPO subscription than zero-shot prompting of large language models.","The curated datasets provide a reusable resource for future studies of Indian IPO success, including the separate dynamics of Main Board and SME listings."],"supporting_citations":[{"why":"Establishes that media sentiment influences IPO underpricing, motivating the use of text features in the model.","marker":"Bajo and Raimondo (2017)"},{"why":"Prior work predicting IPO performance from prospectus text, providing the baseline the paper extends toward multimodal fusion.","marker":"Ly and Nguyen (2020)"},{"why":"Supplies the Nomic embed text encoder used to embed prospectus pages and curated questions for retrieval.","marker":"Nussbaum et al. (2024)"},{"why":"DeBERTa is the model family fine-tuned to produce positive-class probabilities from each text feature.","marker":"He et al. (2020)"},{"why":"XGBoost is one of the five model families used in the two-stage classifiers.","marker":"Chen and Guestrin (2016)"},{"why":"Gradient boosting machine is another of the five model families used in the classifiers.","marker":"Friedman (2001)"},{"why":"BM25s is the lexical retrieval method paired with cosine similarity in the RAG pipeline.","marker":"L`u (2024)"}],"fun_headline_variants":["Text+numbers beat LLMs on Indian IPO listing-day prices","Multi-modal fusion predicts Indian IPO opening and closing moves","Grey market premium signals Main Board, not SME IPOs","How text, images, and numbers forecast Indian IPO success","Fusing prospectus text with numeric data predicts IPO direction"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reported gains from text features depend on the practice of computing first-stage probabilities on the same training data used to fit the second-stage model; if the first-stage models overfit the training labels, the second stage is partly trained on the target itself.","fun_headline_variants_meta":{"raw":{"variants":["Text+numbers beat LLMs on Indian IPO listing-day prices","Multi-modal fusion predicts Indian IPO opening and closing moves","Grey market premium signals Main Board, not SME IPOs","How text, images, and numbers forecast Indian IPO success","Fusing prospectus text with numeric data predicts IPO direction"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000707,"raw_usage":{"total_tokens":3145,"prompt_tokens":865,"completion_tokens":2280,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":481,"completion_tokens_details":{"reasoning_tokens":2208}},"tokens_in":481,"tokens_out":2280,"duration_ms":17496,"temperature":1.0,"reasoning_tokens":2208,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T20:18:44.123466+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the two-stage pipeline using out-of-fold (cross-validated) first-stage probabilities instead of in-sample probabilities on the same 2023 test split; if the text-augmented models no longer beat numeric-categorical baselines on high-price direction, the claimed text benefit vanishes and the multimodal advantage is an artifact of target leakage.","supporting_citations":[{"cited_title":", author Raimondo, C","cited_arxiv_id":null,"evidence_quote":"Establishes that media sentiment influences IPO underpricing, motivating the use of text features in the model."},{"cited_title":", year 2001","cited_arxiv_id":null,"evidence_quote":"Gradient boosting machine is another of the five model families used in the classifiers."}],"review_version":1}