{"id":"1a6cc1e2-6182-42e9-aa42-d28d1b69735b","arxiv_id":"2606.31163","paper_version":2,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":5.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":1,"one_line_summary":"A classifier before any LLM inference routes PII queries to local endpoints and simple queries to small models, reporting 39% latency reduction and 33-52% cost savings on 600 queries with 99.2% classifier accuracy.","lead":"The paper proposes a pre-inference encoder classifier that routes LLM queries based on PII sensitivity and complexity to tiered models in compliant geographic locations. This architecture aims to make data residency violations impossible while reducing latency and cost compared to single large-model deployments.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.3","headline":"Near-perfect recall on 600 queries does not establish that PII routing violations are structurally impossible","rationale":"The reader's weakest assumption directly names the classifier's performance on unseen queries as the load-bearing point. The abstract-only review correctly flags the absence of supporting evidence for generalization. The full manuscript would need to demonstrate training details, error analysis, and robustness tests; absent those, the strong claim exceeds the reported evidence. No other internal inconsistency appears in the given summary.","tokens_in":1753,"tokens_out":323,"duration_ms":15594,"concrete_test":"Construct a test set of 2000 queries containing diverse PII (standard, obfuscated, multilingual, and partial forms) disjoint from the original 600; measure PII recall. If recall drops below 99.99% on any subset, the structural-impossibility claim fails.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim states that PII queries route to local endpoints before any LLM computation, making residency violations structurally impossible. This holds only if the encoder classifier has zero false negatives for PII on every possible input. The reported 99.2% accuracy and near-perfect recall come from evaluation on 600 queries; the abstract provides no information on training distribution, OOD robustness, adversarial PII (e.g., encoded or partial identifiers), or formal error bounds. A single misclassification routes sensitive data to a non-local model, falsifying the structural guarantee. The architecture description does not add runtime verification or fallback mechanisms that would mitigate classifier error.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The paper proposes ComplianceGate, a classifier-gated multi-tier LLM routing architecture for regulated industries. A trained encoder classifier evaluates each incoming query for PII sensitivity and complexity before any decoder inference begins, routing PII queries exclusively to local endpoints and simple queries to smaller, cheaper models in appropriate geographic locations. The central claims are that this makes data-residency violations structurally impossible and yields 39% median latency reduction, 33-52% cost savings, and 122-200 tokens/s throughput on an evaluation of 600 queries, with the classifier achieving 99.2% accuracy and near-perfect PII recall at 7 ms overhead.","tokens_in":1912,"tokens_out":456,"duration_ms":18616,"significance":"If the empirical results hold under broader testing, the pre-inference classification approach would offer a practical compliance-by-design mechanism that current single-model or MoE deployments lack, while delivering measurable efficiency gains. The work highlights a concrete engineering pattern for separating compliance enforcement from model inference.","major_comments":[{"comment":"Abstract and Evaluation section: the manuscript reports aggregate metrics (99.2% accuracy, near-perfect PII recall, 39% latency reduction, etc.) on 600 queries but supplies no information on query sources, labeling process for PII, baseline system details, statistical significance testing, or error analysis. This leaves the central performance and compliance claims without verifiable support.","section":"Abstract / Evaluation"},{"comment":"Abstract: the claim that PII-containing queries make data residency violations 'structurally impossible' holds only if the classifier has zero false negatives on every possible input. The reported 99.2% accuracy and near-perfect recall on 600 queries does not establish this guarantee; OOD robustness, adversarial or encoded PII, and formal error bounds are not addressed, and no runtime verification or fallback is described.","section":"Abstract"}],"minor_comments":[{"comment":"The distinction between internal MoE routing and the proposed system-level routing could be stated more explicitly to avoid reader confusion.","section":"Introduction"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for their constructive comments on the manuscript. We address each of the major comments below and indicate the planned revisions.","responses":[{"response":"We agree that the current manuscript does not provide sufficient details on the query sources, PII labeling process, baseline system, statistical significance testing, or error analysis. These elements will be added to the Evaluation section and summarized in the abstract in the revised version to ensure the performance and compliance claims are fully supported and verifiable.","revision_made":"yes","referee_comment":"[Abstract / Evaluation] Abstract and Evaluation section: the manuscript reports aggregate metrics (99.2% accuracy, near-perfect PII recall, 39% latency reduction, etc.) on 600 queries but supplies no information on query sources, labeling process for PII, baseline system details, statistical significance testing, or error analysis. This leaves the central performance and compliance claims without verifiable support."},{"response":"We concur that the strong claim in the abstract requires qualification, as the classifier's performance is evaluated on a finite set of 600 queries and does not guarantee zero false negatives across all inputs. We will revise the abstract to clarify that data residency violations are rendered impossible for queries correctly classified as containing PII. Additionally, we will include a new Limitations subsection addressing OOD robustness, adversarial and encoded PII, the lack of formal error bounds, and the current absence of runtime verification or fallback mechanisms, framing these as important areas for future research.","revision_made":"yes","referee_comment":"[Abstract] Abstract: the claim that PII-containing queries make data residency violations 'structurally impossible' holds only if the classifier has zero false negatives on every possible input. The reported 99.2% accuracy and near-perfect recall on 600 queries does not establish this guarantee; OOD robustness, adversarial or encoded PII, and formal error bounds are not addressed, and no runtime verification or fallback is described."}],"tokens_in":1450,"tokens_out":428,"duration_ms":39113,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The paper's core proposal is a classifier that sits before any LLM inference, tags queries for PII and complexity, then routes them to the right model size and geographic location. That pre-inference gate is the main new piece; it combines PII detection with tiered routing in one system rather than handling compliance after data arrives or inside an MoE layer.\n\nThe reported numbers are the part that could matter in practice: 39% median latency drop, 33-52% cost savings, and the classifier adding only 7 ms while hitting 99.2% accuracy on 600 queries. Those figures give a concrete sense of the overhead and potential savings if the classifier holds up.\n\nThe soft spot is the central guarantee. The text states that PII queries route locally before any computation, making residency violations structurally impossible. That only follows if the classifier has zero false negatives on every input it will ever see. The 99.2% figure and near-perfect recall come from one test set of 600 queries with no reported details on query sources, labeling process, out-of-distribution cases, or adversarial PII. No runtime verification or fallback is described either. A single miss breaks the structural claim, and the evaluation does not rule that out.\n\nThe cost and latency results are harder to judge without baseline architecture details or statistical tests. The work is aimed at teams already running LLMs in finance or healthcare who need routing options. It deserves a referee to check whether the full methods section supplies the missing data and robustness checks; the idea is worth testing even if the strongest claim needs tempering.","headline":"The routing idea is practical but the 'structurally impossible' compliance claim rests on an unverified classifier with thin evaluation support.","tokens_in":2386,"tokens_out":396,"would_cite":false,"duration_ms":19534,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"A pre-inference encoder classifier routes PII queries to local endpoints and simple queries to small models, making data residency violations impossible by design.","keywords":["LLM routing","PII detection","compliance by design","inference optimization","data residency","classifier-gated architecture","regulated industries","multi-tier deployment"],"falsifier":"A held-out set of queries containing previously unseen PII patterns where the classifier fails to flag them, allowing data to reach a non-local endpoint.","tokens_in":2655,"feed_emoji":"🔒","tokens_out":663,"duration_ms":15066,"temperature":0.7,"pith_summary":"The paper shows that inserting a trained encoder classifier before any LLM decoder can evaluate each query for sensitivity and complexity, then direct it to the right model tier and location. This matters because regulated industries currently risk sending personal data to external endpoints before any compliance check occurs. If the classifier works as described, PII never leaves its jurisdiction and simple queries avoid full-size models. The reported results include 99.2 percent classification accuracy, near-perfect PII recall, 39 percent median latency drop, and 33-52 percent cost reduction on 600 test queries. The 7 ms overhead of the classifier is presented as small enough to keep the approach practical.","feed_headline":"Classifier routes PII queries locally before any LLM step","feed_subtitle":"Pre-inference check sends sensitive data only to local endpoints and simple queries to small models, cutting latency 39 percent.","key_machinery":"A trained encoder classifier that evaluates each query for complexity and data sensitivity before any decoder inference begins.","core_discovery":"A classifier-gated routing architecture places a trained encoder ahead of all decoder inference. The encoder labels each query for data sensitivity and complexity, then routes PII-containing queries exclusively to local endpoints and simple queries to smaller dense models. This ordering ensures no LLM computation begins until the routing decision is complete, so data residency rules are satisfied structurally rather than through post-hoc checks.","pith_inferences":["The same pre-inference gate could be extended to additional constraints such as model capability matching or regulatory jurisdiction without changing the core ordering.","If classifier errors occur mainly on edge cases, organizations could add a small human-review queue for low-confidence classifications.","The approach separates compliance logic from model architecture, so it could be applied to existing dense or mixture-of-experts deployments."],"forward_implications":["PII data never reaches non-local model endpoints because routing occurs before any LLM computation.","Simple queries incur only the cost and latency of small dense models rather than full-size ones.","Median latency falls 39 percent and generation throughput rises to 122-200 tokens per second.","Cost savings range from 33 to 52 percent depending on the mix of query types.","Compliance is enforced by the routing topology itself rather than by additional runtime verification."],"fun_headline_variants":["Classifier pre-routes PII queries to local models before inference","Encoder checks data sensitivity and complexity ahead of LLM processing","Routing places classifier before any decoder computation begins","Pre-inference classification keeps PII within jurisdictional bounds"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"The encoder classifier can correctly identify PII and query complexity on unseen inputs with 99.2 percent accuracy and near-perfect recall.","fun_headline_variants_meta":{"raw":{"variants":["Classifier pre-routes PII queries to local models before inference","Encoder checks data sensitivity and complexity ahead of LLM processing","Routing places classifier before any decoder computation begins","Pre-inference classification keeps PII within jurisdictional bounds"]},"model":"grok-4.3","cost_usd":0.003549,"raw_usage":{"total_tokens":1875,"prompt_tokens":696,"num_sources_used":0,"completion_tokens":61,"cost_in_usd_ticks":35487000,"prompt_tokens_details":{"text_tokens":696,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":1118,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":696,"tokens_out":61,"duration_ms":8384,"temperature":1.0,"reasoning_tokens":1118,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-02T20:14:10.598902+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"A held-out set of queries containing previously unseen PII patterns where the classifier fails to flag them, allowing data to reach a non-local endpoint.","supporting_citations":[],"review_version":2}