Paper: arXiv 2406.02969
Authors: Raeid Saqur, Anastasis Kratsios, Florian Krach, Yannick Limmer, Jacob-Junqi Tian, John Willes, Blanka Horvath, Frank Rudzicz
Abstract
We propose MoE-F - a formalized mechanism for combining \(N\) pre-trained Large Language Models (LLMs) for online time-series prediction by adaptively forecasting the best weighting of LLM predictions at every time step. Our mechanism leverages the conditional information in each expert’s running performance to forecast the best combination of LLMs for predicting the time series in its next step. Diverging from static (learned) Mixture of Experts (MoE) methods, our approach employs time-adaptive stochastic filtering techniques to combine experts. By framing the expert selection problem as a finite state-space, continuous-time Hidden Markov model (HMM), we can leverage the Wohman-Shiryaev filter. Our approach first constructs N parallel filters corresponding to each of the \(N\) individual LLMs. Each filter proposes its best combination of LLMs, given the information that they have access to. Subsequently, the N filter outputs are optimally aggregated to maximize their robust predictive power, and this update is computed efficiently via a closed-form expression, generating our ensemble predictor. Our contributions are: (I) the MoE-F plug-and-play filtering harness algorithm, (II) theoretical optimality guarantees of the proposed filtering-based gating algorithm (via optimality guarantees for its parallel Bayesian filtering and its robust aggregation steps), and (III) empirical evaluation and ablative results using state-of-the-art foundational and MoE LLMs on a real-world Financial Market Movement task where MoE-F attains a remarkable 17% absolute and 48.5% relative F1 measure improvement over the next best performing individual LLM expert predicting short-horizon market movement based on streaming news. Further, we provide empirical evidence of substantial performance gains in applying MoE-F over specialized models in the long-horizon time-series forecasting domain.
Complexity vs Empirical Score
- Math Complexity: 8.5/10
- Empirical Rigor: 7.0/10
- Quadrant: Holy Grail — high math complexity, high empirical rigor
Why this score: The paper introduces advanced stochastic filtering theory (e.g., continuous-time HMMs, Wonham-Shiryaev filter) with heavy theoretical derivations, indicating high math complexity. It is empirically rigorous with a real-world financial dataset, multiple LLM baselines, and detailed performance metrics (F1 improvements), though it lacks fully public code or raw backtest code.
Research Flowchart
flowchart TD
%% Research Goal
goal["Research Goal<br>Combine N LLMs for<br>Time-Series Prediction"]
%% Inputs
inputs["Input Data<br>Streaming News +<br>Time-Series"]
%% Methodology
methodology["Methodology: MoE-F<br>Stochastic Filtering-Based Gating"]
subgraph hmm_proc ["Hidden Markov Model Processing"]
hmm["Finite State-Space HMM<br>Expert Performance Forecasting"]
filters["Parallel Filters<br>Wohman-Shiryaev Filter<br>1 Filter per LLM"]
end
subgraph aggregation ["Aggregation Step"]
agg["Robust Aggregation<br>Closed-Form Update"]
end
%% Outcomes
outcomes["Key Findings<br>17% Abs / 48.5% Rel F1 Gain<br>State-of-the-Art Performance"]
%% Flow
goal --> inputs
inputs --> methodology
methodology --> hmm
hmm --> filters
filters --> agg
agg --> outcomes
Evidence Card
What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card
Proposes
The paper proposes MoE-F, a stochastic filtering-based gating mechanism to combine multiple pre-trained LLMs for online time-series prediction.
Source sentence
We propose MoE-F – a formalized mechanism for combining N pre-trained Large Language Models (LLMs) for online time-series prediction
Assumes 2
- The target process Y evolves according to a hidden Markov model with a hidden state w that follows a continuous-time Markov chain. model
Source sentence
... Assume that Y· evolves according to the dynamics in Eq. (4). We assume that w· is a hidden Markov process with values in the standard basis
- The input path x(t) is once continuously differentiable and the intensity matrix Q is uniformly bounded. statistical
Source sentence
Assumptions 2.1 (Regularity Conditions). The path x· : [0, ∞) →Rd is once continuously differentiable and there is a constant C > 0 such that, for each t ≥0 and i, j = 1, . . . , d, (Qt)i,j ≤C.
Data
- NIFTY text-news public
Source sentence
... is a processed and curated daily news headlines dataset for the stock (US Equities) market price movement prediction task.
- ETTh1 benchmark public
Source sentence
time-series forecasting (LTSF) using the ETTh1, ETTh2 [84] datasets with concurrent specialized LTSF models.
- ETTh2 benchmark public
Source sentence
time-series forecasting (LTSF) using the ETTh1, ETTh2 [84] datasets with concurrent specialized LTSF models.
Twitter / X Yahoo Finance
Costs
Not applicable
Finds 3
- MoE-F achieves a 17% absolute and 48.5% relative improvement in F1 score over the next best individual LLM on the Financial Market Movement task. performance
Source sentence
MoE-F attains a remarkable 17% absolute and 48.5% relative F1 measure improvement over the next best performing individual LLM expert predicting short-horizon market movement
- In Table 2, MoE-F shows a 43% F1 score compared to 52% for the best individual expert, confirming theoretical guarantees. performance
Source sentence
the overall performance on the FMM task is lower than from earlier: 43% compared to 52% (Table 2). This aligns with our proposed theoretical performance guarantee relative to experts.
- MoE-F consistently outperforms base Llama-2 and Llama-3 models across multiple datasets, achieving up to 0.43 F1 score. performance
Source sentence
Table 3: Performance of Llama-2-7b-chat and Llama-3-8B-Instruct
Checked by others
No other paper in the archive has been found to support or contradict these findings (compared with 18 related papers). Coverage is partial, so this is neither replication nor refutation.
Code
No repository by the authors found in the paper.
Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.
Cited References (340)
- Adjustment of an inverse matrix corresponding to a change in one element of a given matrix (1950)
- Introduction to Kalman filter and its applications (2018)
- Kalman filtering: Theory and Practice with MATLAB (2014)
- Patch-level routing in mixture-of-experts is provably sample-efficient for convolutional neural networks (2023)
- Active ranking of experts based on their performances in many tasks (2023)
- Merge, Then Compress: Demystify Efficient SM oE with Hints from Its Routing Policy (2024) — link
- From Sparse to Soft Mixtures of Experts (2024) — link
- Numerical approximations of stochastic differential equations with non-globally Lipschitz continuous coefficients (2015)
and 332 more
- Convolutional sequence to sequence learning (2017)
- Layer normalization (2016) — link
- Optimal projection filters with information geometry (2023)
- Approximate nonlinear filtering by projection on exponential manifolds of densities (1999)
- A differential geometric approach to nonlinear filtering: the projection filter (1998)
- Nonlinear filtering: Interacting particle resolution (1997)
- Optimal Estimation of Generic Dynamics by Path-Dependent Neural Jump ODEs (2022) — link
- GRU-ODE-Bayes: Continuous modeling of sporadically-observed time series (2019)
- Black-Box Variational Inference for Stochastic Differential Equations (2018) — link
- Neural Jump Ordinary Differential Equations: Consistent Continuous-Time Prediction and Filtering (2021) — link
- Existence of optimal strategies based on specified information, for a class of stochastic decision problems (1970)
- Optimum nonlinear systems which bring about a separation of a signal with constant parameters from noise (1959)
- Application of the Markov processes theory to optimal filtering (1960)
- Pricing credit derivatives under incomplete information: a nonlinear-filtering approach (2010)
- Statistical methods for financial engineering (2013)
- Compound Hawkes processes in limit order books (2019)
- Financial equilibrium with asymmetric information and random horizon (2018)
- Parameter estimation in commodity markets: A filtering approach (2007)
- Forecasting, structural time series models and the Kalman filter (1990)
- Stochastic filtering with applications in finance (2010)
- The Kalman filter in finance (1995)
- Linear and non-linear filtering in mathematical finance: a review (2011)
- Short-term variations and long-term dynamics in commodity prices (2000)
- The stochastic behavior of commodity prices: Implications for valuation and hedging (1997)
- Energy futures prices: term structure models with Kalman filter estimation (2002)
- Optimal trading strategy for an investor: the case of partial information (1998)
- Estimating and testing exponential-affine term structure models by Kalman filter (1999)
- Maximum likelihood estimation for a multifactor equilibrium model of the term structure of interest rates (1993)
- Fixed point theorems for metric spaces with a conical geodesic bicombing (2018)
- Restricted Riemannian geometry for positive semidefinite matrices (2023)
- Constructive approximation (1993)
- The phase diagram of approximation rates for deep neural networks (2020)
- Optimal approximation of continuous functions by very deep ReLU networks (2018)
- Deep vs. shallow networks: An approximation theory perspective (2016)
- Approximation bounds for random neural networks and reservoir systems (2023)
- Graph attention networks (2017) — link
- The curse of dimensionality in operator learning (2023) — link
- Near-optimal learning of Banach-valued, high-dimensional functions via deep neural networks (2022)
- Exponential relu neural network approximation rates for point and edge singularities (2023)
- Characterization of the variation spaces corresponding to shallow neural networks (2023)
- An Approximation Theory for Metric Space-Valued Functions With A View Towards Deep Learning (2023) — link
- Filling riemannian manifolds (1983)
- Bi-Lipschitz approximation by finite-dimensional imbeddings (2011)
- Entropy-regularized 2-Wasserstein distance between Gaussian measures (2022)
- Stochastic integration by parts and functional It ^o calculus (2016)
- Algebraic topology (1989)
- Efficient approximation of high-dimensional functions with neural networks (2021)
- Denise: Deep Robust Principal Component Analysis for Positive Semidefinite Matrices (2023) — link
- On the approximation of functions by tanh neural networks (2021)
- Searching for activation functions (2017) — link
- Implicit neural representations with periodic activation functions (2020)
- Sharp Analysis of a Simple Model for Random Forests (2021) — link
- Upper bounds for the density of solutions to stochastic differential equations driven by fractional Brownian motions (2014)
- Density for solutions to stochastic differential equations with unbounded drift (2019)
- Positivity of the density for rough differential equations (2022)
- On the support of Wiener functionals. Asymptotic problems in probability theory: Wiener functionals and asymptotics (Sanda/Kyoto, 1990), 3–34 (1993)
- Smoothness and asymptotic estimates of densities for SDEs with locally smooth coefficients and applications to square root-type diffusions (2011)
- Smooth densities for solutions to stochastic differential equations with jumps (2009)
- Existence of densities of solutions of stochastic differential equations by Malliavin calculus (2010)
- I-divergence geometry of probability distributions and minimization problems (1975)
- Optimizing Optimizers: Regret-optimal gradient descent algorithms (2021) — link
- Fisher-Rao geometry of Dirichlet distributions (2021)
- Riemannian geometry (1990)
- Deep learning for financial sentiment analysis on finance news providers (2016)
- Evaluation of sentiment analysis in finance: from lexicons to transformers (2020)
- Designing universal causal deep learning models: The geometric (Hyper)transformer (2023) — link
- Riemannian Lp center of mass: existence, uniqueness, and convexity (2011)
- Geodesic regression and the theory of least squares on Riemannian manifolds (2013)
- Deep arbitrage-free learning in a generalized HJM framework via arbitrage-regularization (2020)
- Modern portfolio theory, 1950 to date (1997)
- Universal approximation theorems for differentiable geometric deep learning (2022)
- Non-euclidean universal approximation (2020)
- A Riemannian framework for tensor computing (2006)
- Turing Completeness of Bounded-Precision Recurrent Neural Networks (2021) — link
- On the Turing Completeness of Modern Neural Network Architectures (2019) — link
- On the difficulty of training recurrent neural networks (2013)
- Portfolio selection: efficient diversification of investments (1968)
- Designing Universal Causal Deep Learning Models: The Case of Infinite-Dimensional Dynamical Systems from Stochastic Analysis (2022) — link
- Deep network approximation for smooth functions (2021)
- Universal Regular Conditional Distributions via Probabilistic Transformers (2023)
- Provable memorization via deep neural networks using sub-linear parameters (2021)
- Memory capacity of neural networks with threshold and rectified linear unit activations (2020)
- Discrete-time signal processing. Vol. 2 (2001)
- Neocognitron: A self-organizing neural network model for a mechanism of visual pattern recognition (1982)
- Theory of deep convolutional neural networks: Downsampling (2020) — link
- Nonlinear filtering in riemannian manifolds (1984)
- Principal geodesic analysis for the study of nonlinear statistics of shape (2004)
- Feedback particle filter on riemannian manifolds and matrix lie groups (2017)
- Particle filtering on Riemannian manifolds (2006)
- Random Toeplitz matrices: The condition number under high stochastic dependence (2021)
- Exponential-Wrapped Distributions on Symmetric Spaces (2022) — link
- Instance-Dependent Generalization Bounds via Optimal Transport (2022) — link
- Universal Approximation with Deep Narrow Networks (2020) — link
- Geodesic convolutional neural networks on riemannian manifolds (2015)
- Neural Machine Translation by Jointly Learning to Align and Translate (2016) — link
- The design of robust approximations to the stochastic differential equations of nonlinear filtering (1978)
- Matrix Analysis (2012)
- A Smooth Model of Decision Making under Ambiguity (2005) — link
- Structured inference networks for nonlinear state space models (2017)
- Deep Kalman Filters (2015) — link
- Statistics for Finance (2016) — link
- Sequential calibration of options (2008)
- Fast Simultaneous Calibration and Quadratic Hedging under Parameter Uncertainty (2014)
- Filtering in finance (2003)
- Generalised correlated batched bandits via the ARC algorithm with application to dynamic pricing (2022) — link
- Kalman filtering of generalized Vasicek term structure models (1999)
- The stochastic behavior of commodity prices: Implications for valuation and hedging (1997)
- Energy futures prices: term structure models with Kalman filter estimation (2002)
- Simple and extended Kalman filters: an application to term structures of commodity prices (2004)
- Optimal hedging and parameter uncertainty (2007) — link
- Scaling Learning Algorithms Towards AI (2007)
- A Fast Learning Algorithm for Deep Belief Nets (2006)
- Deep learning (2016)
- Regret-Optimal Federated Transfer Learning for Kernel Regression with Applications in American Option Pricing — → 2309.04557
- The entropic measure transform (2020)
- Deepseek-v3 technical report (2024) — link
- Deepseek-v2: A strong, economical, and efficient mixture-of-experts language model (2024) — link
- DeepSeek-Coder: When the Large Language Model Meets Programming–The Rise of Code Intelligence (2024) — link
- Augmented dickey fuller test (2011)
- Stanford Alpaca: An Instruction-following LLaMA model (2023)
- Alpaca: A strong, replicable instruction-following model (2023)
- Self-instruct: Aligning language model with self generated instructions (2022) — link
- PAC-Bayesian bounds for randomized empirical risk minimizers (2008)
- On the properties of variational approximations of Gibbs posteriors (2016)
- Mean-variance investing (2012)
- Arrows of Time for Large Language Models (2024) — link
- Exploring the Impact of Instruction Data Scaling on Large Language Models: An Empirical Study on Real-World Use Cases (2023) — link
- Training verifiers to solve math word problems (2021) — link
- Evaluating large language models trained on code (2021) — link
- Gaia: a benchmark for general ai assistants (2023) — link
- Measuring Massive Multitask Language Understanding (2021)
- Exact finite-dimensional filters for certain diffusions with nonlinear drift (1981)
- Multimodal datasets: misogyny, pornography, and malignant stereotypes (2021)
- On the opportunities and risks of foundation models (2021)
- Time series analysis: forecasting and control (2015)
- Rank analysis of incomplete block designs: I. The method of paired comparisons (1952)
- Language Models are Few-Shot Learners (2020)
- Sparks of artificial general intelligence: Early experiments with GPT-4 (2023)
- Filtering for stochastic processes with applications to guidance (2005)
- Finqa: A dataset of numerical reasoning over financial data (2021) — link
- PaLM : Scaling language modeling with pathways. 2022 (2022)
- Good debt or bad debt: Detecting semantic orientations in economic texts (2014)
- The pile: An 800gb dataset of diverse text for language modeling (2020) — link
- Introducing DBRX: A New State-of-the-Art Open LLM (2024)
- Advances in financial machine learning (2018)
- BERT : Pre-training of Deep Bidirectional Transformers for Language Understanding (2019) — link
- Particle filtering (2003)
- Implications of the Dodd-Frank act (2012)
- Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpus (2021) — link
- Direct preference optimization: Your language model is secretly a reward model (2023) — link
- GLaM: Efficient Scaling of Language Models with Mixture-of-Experts (2022)
- An empirical investigation of the challenges of real-world reinforcement learning (2020)
- UCI Machine Learning Repository (2017)
- Adaptive nonlinear system identification with echo state networks (2002)
- A framework for few-shot language model evaluation (2021) — link
- Bartscore: Evaluating generated text as text generation (2021)
- Bertscore: Evaluating text generation with bert (2019) — link
- Stochastic processes and filtering theory (2007)
- Billion-scale similarity search with GPUs (2019)
- Efficient capital markets: A review of theory and empirical work (1970)
- Efficient capital markets: reply (1976)
- Efficient capital markets: II (1991)
- Market efficiency, long-term returns, and behavioral finance (1998)
- Some applications of stochastic differential equations to optimal nonlinear filtering (1964)
- Stock movement prediction from tweets and historical prices (2018)
- Accurate Stock Movement Prediction with Self-supervised Learning from Sparse Noisy Tweets (2022)
- Hybrid deep sequential modeling for social text-driven stock prediction (2018)
- Www'18 open challenge: financial opinion mining and question answering (2018)
- SENTiVENT: enabling supervised information extraction of company-specific events in economic and financial news (2022)
- Good debt or bad debt: Detecting semantic orientations in economic texts (2014)
- NIFTY Financial News Headlines Dataset — → 2405.09747
- Is domain adaptation worth your investment? Comparing BERT and FinBERT on financial tasks (2021)
- Finbert: Financial sentiment analysis with pre-trained language models (2019) — link
- PIXIU: A Large Language Model, Instruction Data and Evaluation Benchmark for Finance (2023) — link
- Scaling instruction-finetuned language models (2022) — link
- Profitable momentum trading strategies for individual investors (2015)
- Pytorch fsdp: experiences on scaling fully sharded data parallel (2023) — link
- Short, colorful, and irreverent! a comparative analysis of new users on wallstreetbets during the gamestop short-squeeze (2022)
- Explaining Gamestop Short Squeeze using Intraday Data and Google Searches (2021)
- Gemini — link
- GPT-NeoX-20B : An Open-Source Autoregressive Language Model (2022) — link
- The Future of Financial Analysis (1963)
- Learning Word Vectors for 157 Languages (2018)
- BPE mb: Tokenization-free Pre-trained Subword Embeddings in 275 Languages (2018)
- Stochastic calculus (2005)
- Stop Uploading Test Data in Plain Text: Practical Strategies for Mitigating Data Contamination by Evaluation Benchmarks (2023)
- A deep reinforcement learning framework for the financial portfolio management problem (2017) — link
- Challenges and Applications of Large Language Models (2023) — link
- Deep kalman filters (2015) — link
- Understanding the Kalman filter (1983)
- A New Approach to Linear Filtering and Prediction Problems (1960) — link
- New Results in Linear Filtering and Prediction Theory (1961) — link
- Evaluating Sentiment Analysis in the Context of Securities Trading. (2016)
- BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding (2019)
- Neural sdes as infinite-dimensional gans (2021)
- Basel III: an overview (2011)
- Portfolio Transformer for Attention-Based Asset Allocation (2022) — link
- Decision support from financial disclosures with deep neural networks and transfer learning (2017)
- Quality at a Glance: An Audit of Web-Crawled Multilingual Datasets (2022) — link
- Sentiment analysis of financial news articles using performance indicators (2018)
- Lag-llama: Towards foundation models for time series forecasting (2023) — link
- Deduplicating Training Data Makes Language Models Better (2022) — link
- Rcv1: A new benchmark collection for text categorization research (2004)
- BART : Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension (2020) — link
- Holistic evaluation of language models (2022)
- Statistics of random processes: General theory (1977)
- Roberta: A robustly optimized bert pretraining approach (2019)
- Adaptive quantitative trading: An imitative deep reinforcement learning approach (2020)
- PAC-Bayes bounds for meta-learning with data-dependent prior (2021) — link
- LLaMA : Open and efficient foundation language models (2023)
- Textual analysis in accounting and finance: A survey (2016)
- Long short-term memory (1997)
- Individual choice behavior: A theoretical analysis (2005)
- Data Contamination: From Memorization to Exploitation (2022) — link
- Statistical and Machine Learning forecasting methods: Concerns and ways forward (2018)
- The M6 forecasting competition: Bridging the gap between forecasting and investment decisions (2023) — link
- A random walk down wall street [by] burton G. Malkiel (1973)
- Efficient market hypothesis (1989)
- Passive investment strategies and efficient markets (2003)
- Stochastic differential equations and applications (2007)
- Foundations of portfolio theory (1991)
- Handbook of Markov decision processes: methods and applications (2012)
- Learning to learn by gradient descent by gradient descent (2016)
- Learning to learn without gradient descent by gradient descent (2017)
- Mistral 7B (2023) — link
- Mistral 7B (2023) — link
- Deepseekmoe: Towards ultimate expert specialization in mixture-of-experts language models (2024) — link
- Megablocks: Efficient sparse training with mixture-of-experts (2023)
- Adaptive mixtures of local experts (1991)
- Mixtral of experts (2024) — link
- Hierarchical mixtures of experts and the EM algorithm (1994)
- Outrageously large neural networks: The sparsely-gated mixture-of-experts layer (2017) — link
- Mixture-of-experts meets instruction tuning: A winning combination for large language models (2023) — link
- Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity (2022)
- Mixture-of-experts with expert choice routing (2022)
- Unified Training of Universal Time Series Forecasting Transformers (2024) — link
- CLUES: few-shot learning evaluation in natural language understanding (2021)
- Neural ordinary differential equations (2018)
- NeuralForecast : User friendly state-of-the-art neural forecasting models. (2022) — link
- GPT-4 Technical Report (2023) — link
- GPT-4 Technical Report (2023)
- OpenAI Baselines (2017)
- Training language models to follow instructions with human feedback (2022)
- What In-Context Learning “Learns” In-Context: Disentangling Task Recognition and Task Learning (2023) — link
- Parameter-efficient transfer learning for NLP (2019)
- Gpt3. int8 (): 8-bit matrix multiplication for transformers at scale (2022)
- Lora: Low-rank adaptation of large language models (2021) — link
- Prefix-tuning: Optimizing continuous prompts for generation (2021) — link
- Qlora: Efficient finetuning of quantized llms (2023) — link
- Deep Contextualized Word Representations (2018) — link
- The analysis of permutations (1975)
- Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer (2020)
- Retrieval-augmented generation for large language models: A survey (2023) — link
- Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks (2019) — link
- Simple statistical gradient-following algorithms for connectionist reinforcement learning (1992)
- Reinforcement learning in financial markets - a survey (2018)
- Training a helpful and harmless assistant with reinforcement learning from human feedback (2022) — link
- A general language assistant as a laboratory for alignment (2021) — link
- Training language models to follow instructions with human feedback, 2022 (2022)
- Learning to summarize with human feedback (2020)
- Fine-tuning language models from human preferences (2019) — link
- PACOH: Bayes-optimal meta-learning with PAC-guarantees (2021)
- Scalable PAC-Bayesian Meta-Learning via the PAC-Optimal Hyper-Posterior: From Theory to Practice (2023)
- SAM former: Unlocking the Potential of Transformers in Time Series Forecasting with Sharpness-Aware Minimization and Channel-Wise Attention (2024) — link
- PrivySense: Price Volatility based Sentiment Estimation from Financial News using Machine Learning (2017) — link
- Bloom: A 176b-parameter open-access multilingual language model (2022)
- An analysis of verbs in financial news articles and their impact on stock price (2010)
- Stochastic differential equations in finance (1990)
- The Sharpe Ratio (1994)
- Megatron-lm: Training multi-billion parameter language models using model parallelism (2019)
- Impact of news on the commodity market: Dataset and results (2021)
- Some explicit formulae in a problem on ``disorder’’ (1965)
- MPNet: Masked and Permuted Pre-training for Language Understanding (2020)
- GlobalBench : A Benchmark for Global Progress in Natural Language Processing (2023)
- S parse TSF : Modeling Long-term Time Series Forecasting with 1k Parameters (2024) — link
- Beyond the imitation game: Quantifying and extrapolating the capabilities of language models (2022)
- Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer (2020) — link
- Ul2: Unifying language learning paradigms (2022) — link
- TF–IDF (2010) — link
- Llama: Open and efficient foundation language models (2023) — link
- Caltrans Performance Measurement System (PeMS) (2023)
- Chronos: Learning the Language of Time Series (2024) — link
- Ensemble Modeling for Time Series Forecasting: an Adaptive Robust Optimization Approach (2023) — link
- Periodicity and stochastic trends in economic time series (1996)
- Nhits: Neural hierarchical interpolation for time series forecasting (2023)
- Graph Deep Learning for Time Series Forecasting (2023) — link
- Are Language Models Actually Useful for Time Series Forecasting? (2024) — link
- Are transformers effective for time series forecasting? (2023)
- Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting (2022)
- Film: Frequency improved legendre memory model for long-term time series forecasting (2022)
- FITS: Modeling Time Series with $10 k $ Parameters (2024)
- The capacity and robustness trade-off: Revisiting the channel independent strategy for multivariate time series forecasting (2024)
- Position: What Can Large Language Models Tell Us about Time Series Analysis (2024) — link
- Generative time series forecasting with diffusion, denoise, and disentanglement (2022)
- Review of Time Series Forecasting Methods and Their Applications to Particle Accelerators (2022) — link
- Large language models are zero-shot time series forecasters (2024)
- Log neural controlled differential equations: The lie brackets make a difference (2024) — link
- Robust recurrent neural networks for time series forecasting (2023)
- The Rise of Diffusion Models in Time-Series Forecasting (2024) — link
- Language Models Still Struggle to Zero-shot Reason about Time Series (2024) — link
- Micn: Multi-scale local and global context modeling for long-term series forecasting (2023)
- N-BEATS: Neural basis expansion analysis for interpretable time series forecasting (2019) — link
- A time series is worth 64 words: Long-term forecasting with transformers (2022) — link
- Pyraformer: Low-complexity pyramidal attention for long-range time series modeling and forecasting (2021)
- Temporal fusion transformers for interpretable multi-horizon time series forecasting (2021)
- Long-term forecasting with tide: Time-series dense encoder (2023) — link
- Timesnet: Temporal 2d-variation modeling for general time series analysis (2023)
- Transformers in Time Series: A Survey (2023) — link
- Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting (2021)
- Time-series generative adversarial networks (2019)
- Informer: Beyond efficient transformer for long sequence time-series forecasting (2021)
- Stochastic calculus, filtering, and stochastic control (2007)
- Attention is All you Need (2017)
- Vicuna: An Open-Source Chatbot Impressing GPT-4 with 90%* ChatGPT Quality (2023) — link
- Efficient Memory Management for Large Language Model Serving with PagedAttention (2023)
- GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding (2019)
- SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems (2019)
- Self-Instruct: Aligning Language Model with Self Generated Instructions (2022) — link
- Text Embeddings by Weakly-Supervised Contrastive Pre-training (2022)
- Self-Instruct: Aligning Language Models with Self-Generated Instructions (2023) — link
- Finetuned Language Models are Zero-Shot Learners (2022)
- Chain-of-thought prompting elicits reasoning in large language models (2022)
- Emergent abilities of large language models (2022)
- Dumb Money (2024\mydate) — link
- No free lunch theorems for optimization (1997)
- Some applications of stochastic differential equations to optimal nonlinear filtering (1964)
- XLNet: Generalized Autoregressive Pretraining for Language Understanding (2019)
- Teacher Guided Training: An Efficient Framework for Knowledge Transfer (2022) — link
- On the optimal filtering of diffusion processes (1969)
- Zephyr: Direct distillation of lm alignment (2023) — link
- Opt: Open pre-trained transformer language models (2022) — link
- A survey of large language models (2023)
2 of 340 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2596 linked papers.