Paper: arXiv 2506.08026
Authors: Xibai Wang
Abstract
This paper proposes TIP-Search, a time-predictable inference scheduling framework for real-time market prediction under uncertain workloads. Motivated by the strict latency demands in high-frequency financial systems, TIP-Search dynamically selects a deep learning model from a heterogeneous pool, aiming to maximize predictive accuracy while satisfying per-task deadline constraints. Our approach profiles latency and generalization performance offline, then performs online task-aware selection without relying on explicit input domain labels. We evaluate TIP-Search on three real-world limit order book datasets (FI-2010, Binance BTC/USDT, LOBSTER AAPL) and demonstrate that it outperforms static baselines with up to 8.5% improvement in accuracy and 100% deadline satisfaction. Our results highlight the effectiveness of TIP-Search in robust low-latency financial inference under uncertainty.
Complexity vs Empirical Score
- Math Complexity: 6.0/10
- Empirical Rigor: 7.5/10
- Quadrant: Holy Grail — high math complexity, high empirical rigor
Why this score: The paper introduces a formal optimization formulation and a Pareto frontier concept for model selection under constraints, demonstrating moderate mathematical complexity. It provides strong empirical rigor by testing on multiple real-world LOB datasets, implementing dynamic scheduling, and reporting quantitative improvements (e.g., 8.5% accuracy gain, 100% deadline satisfaction) with ablation studies.
Research Flowchart
flowchart TD
A["Research Goal: Develop a time-predictable<br>inference scheduler for real-time<br>market prediction under uncertain load"] --> B["Data Preparation: Use Limit Order Book datasets<br>(FI-2010, Binance BTC/USDT, LOBSTER AAPL)"]
B --> C["Offline Profiling: Characterize model latency<br>and generalization performance"]
C --> D["Online Scheduling: Dynamic model selection<br>via TIP-Search to maximize accuracy<br>while meeting strict deadlines"]
D --> E{"Evaluation & Outcomes"}
E --> F["+8.5% Accuracy<br>vs. static baselines"]
E --> G["100% Deadline Satisfaction<br>under uncertain load"]
E --> H["Robust Low-Latency<br>Financial Inference"]
Evidence Card
What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card
Proposes
The paper proposes TIP-Search, a deadline-aware inference scheduling framework that dynamically selects a deep learning model per task to maximize predictive accuracy while meeting strict latency deadlines.
Source sentence
This paper proposes TIP-Search, a time-predictable inference scheduling framework for real-time market prediction under uncertain workloads.
Assumes 6
- Multiple pre‑trained models, each tailored to a specific domain, are available for selection. model
Source sentence
The system relies on the availability of multiple pre-trained models, each tailored to a specific domain or data distribution.
- Model generalization profiles remain stationary over time. model
Source sentence
TIP-Search currently assumes stationarity in model generalization profiles over time
- Each inference task must be completed within a hard 10 ms deadline. execution
Source sentence
runtime inference is triggered every 5ms, with a strict 10ms deadline per task.
- All models are deployed on a single GPU for inference. execution
Source sentence
These models are deployed on a single GPU
Show 2 more
- Offline profiling of latency and generalization performance accurately reflects online behavior. model
Source sentence
Our approach profiles latency and generalization performance offline
- Task arrivals follow realistic bursty patterns as simulated in the experiments. market
Source sentence
To simulate realistic online inference scenarios, we construct a heterogeneous workload using three distinct financial limit order book (LOB) datasets.
Data
- FI-2010 order-book public
Source sentence
comprises ten days of Nasdaq LOB data annotated with five‑class mid‑price movements.
- Binance BTC/USDT order-book public
Source sentence
The Binance BTC/USDT dataset contains a 2025/05/10 snapshot of raw cryptocurrency trade events labeled for three‑class price direction.
- LOBSTER AAPL order-book public
Source sentence
The LOBSTER AAPL dataset provides high‑frequency order book data for Apple Inc. over ten trading days, representing a more stable equity market.
Costs
Not applicable
Finds 5
- TIP-Search improves predictive accuracy by up to 8.5 % over static baselines while achieving 100 % deadline satisfaction. performance
Source sentence
Our results highlight the effectiveness of TIP-Search in robust low‑latency financial inference under uncertainty.
- TIP-Search consistently achieves higher accuracy than all three baselines (fixed‑model, random, and source‑aware). predictive-accuracy
Source sentence
TIP-Search consistently achieves higher accuracy than all three baselines.
- Model selection aligns with latent data characteristics even without explicit source labels during execution. other
Source sentence
The observed routing patterns confirm that model selection in TIP-Search aligns with latent data characteristics even without access to explicit source labels during execution.
- Classical real‑time scheduling policies (EDF, LLF) yield lower accuracy than TIP-Search under the same deadline constraints. performance
Source sentence
The resulting accuracy under deadline constraints was inferior to TIP-Search, suggesting that incorporating domain‑specific accuracy estimation provides meaningful gains over purely temporal heuristics.
- Static selection strategies either violate latency guarantees or suffer from model‑data mismatch, leading to lower overall performance. negative
Source sentence
strategies often lead to suboptimal performance: either violating latency guarantees or compromising predictive accuracy due to model–data mismatch.
Checked by others
No other paper in the archive has been found to support or contradict these findings (compared with 36 related papers). Coverage is partial, so this is neither replication nor refutation.
Code
No repository by the authors found in the paper.
Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.
Cited References (15)
- DeepLOB: Deep Convolutional Neural Networks for Limit Order Books (2019) — link
- Benchmark Dataset for Mid-Price Forecasting of Limit Order Book Data with Machine Learning Methods (2018) — link
- Scheduling Algorithms for Multiprogramming in a Hard-Real-Time Environment (1973) — link
- LOBSTER: Limit Order Book Reconstruction System (2011) — link
- LiT: Limit Order Book Transformer (2025) — link
- ViT-LOB: Efficient Vision Transformer for StockPrice Trend Prediction Using Limit Order Books (2024) — link
- TLOB: A Novel Transformer Model with Dual Attention for Stock Price Trend Prediction with Limit Order Book Data (2025) — link
- Deep Limit Order Book Forecasting — → 2403.09267
and 7 more
- Empowering Adaptive Early-Exit Inference with Latency Awareness (2021) — link
- BranchyNet: Fast Inference via Early Exiting from Deep Neural Networks (2016) — link
- Clipper: A Low-Latency Online Prediction Serving System (2016) — link
- Serving DNNs Like Clockwork: Performance Predictability from the Bottom Up (2020) — link
- Model Selection for Latency-Critical Inference Serving (2024) — link
- SneakPeek: Data-Aware Model Selection and Scheduling for Inference Serving on the Edge (2025) — link
- INFaaS: A Model-Less and Managed Inference Serving System (2019) — link
1 of 15 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2596 linked papers.