Paper: arXiv 2410.01831

Authors: Roman Belavkin, Panos Pardalos, Jose Principe

Abstract

The advances and development of various machine learning techniques has lead to practical solutions in various areas of science, engineering, medicine and finance. The great choice of algorithms, their implementations and libraries has resulted in another challenge of selecting the right algorithm and tuning their parameters in order to achieve optimal or satisfactory performance in specific applications. Here we show how the value of information (V(I)) can be used in this task to guide the algorithm choice and parameter tuning process. After estimating the amount of Shannon’s mutual information between the predictor and response variables, V(I) can define theoretical upper bound of performance of any algorithm. The inverse function I(V) defines the lower frontier of the minimum amount of information required to achieve the desired performance. In this paper, we illustrate the value of information for the mean-square error minimization and apply it to forecasts of cryptocurrency log-returns.

Complexity vs Empirical Score

  • Math Complexity: 8.5/10
  • Empirical Rigor: 3.0/10
  • Quadrant: Lab Rats — high math complexity, limited empirical testing

Why this score: The paper is heavily theoretical, deriving complex relationships using Shannon’s mutual information, Lagrange multipliers, and Legendre-Fenchel duality, indicating high mathematical complexity. Conversely, the empirical section is limited to a high-level application to cryptocurrency returns without specific code, backtesting procedures, or statistical metrics, resulting in low empirical rigor.

Research Flowchart

  flowchart TD
  A["Research Goal: <br>Use Value of Information V(I) <br>for ML Algorithm & Parameter Selection"] --> B["Input Data: <br>Financial Time-Series <br>e.g., Cryptocurrency Log-Returns"]
  B --> C["Key Methodology: <br>Estimate Mutual Information <br>between Predictors and Response"]
  C --> D["Computational Process: <br>Calculate V(I) <br>Upper Bound (Max Performance) <br>& I(V) Lower Frontier"]
  D --> E{"Outcomes & Application"}
  E --> F1["Theoretical Limit: <br>Define max achievable <br>prediction accuracy"]
  E --> F2["Selection Criteria: <br>Identify algorithms & params <br>approaching the bound"]
  E --> F3["Practical Result: <br>Optimized forecasts <br>for financial time-series"]

Evidence Card

What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card

Proposes

The paper proposes using the value of information framework for mean‑square error minimization and applies it to forecasting cryptocurrency log‑returns.

Source sentence
In this paper, we illustrate the value of information for the mean-square error minimization and apply it to forecasts of cryptocurrency log-returns.

Assumes 4

  • Log‑returns are often assumed to be independent (Markov process). model
    Source sentence
    they are often assumed to be independent (and hence prices s(t) are often modelled by a Markov process).
  • The Gaussian mutual‑information formula provides a lower bound for non‑Gaussian log‑returns. statistical
    Source sentence
    We use the approximate sign ≈, because the distributions of log-returns are generally not Gaussian (in fact, the above formula gives a lower bound for non-Gaussian random variables).
  • Training uses 100‑day windows and testing uses 25‑day windows with a rolling‑window procedure. data
    Source sentence
    Here we used 100 and 25 days data windows for training and testing respectively.
  • Trading returns are calculated without accounting for trading fees. execution
    Source sentence
    the value of MRR = .01 means 1% return per day without taking into account trading fees.

Data

Backtest sample: 2019-01 – 2021-01 · BTC/USD · crypto · daily

  • Daily cryptocurrency pair prices crypto-onchain public
    2019-01-01 to 2021-01-11 · BTC/USD, ETH/USD, DAI/BTC, XRP/BTC, IOT/BTC
    Source sentence
    The dataset used contains daily prices s(t) of several cryptocurrency pairs during the period between Jan 1, 2019 and Jan 11, 2021.

Costs

Not modeled A trading or execution result is reported without costs.

Source sentence
without taking into account trading fees.

Finds 3

  • Errors of fitting the training data closely follow the theoretical RMSE(I) curve; LM and NN achieve errors close to the theoretical bound. performance
    Source sentence
    Errors of fitting the training data closely follow theoretical curve RMSE(I). One can see that LM and NN achieve errors on the training data close to
  • Partial Least Squares (PLS) yields smaller and more stable forecast errors than linear regression or neural networks on the testing data. predictive-accuracy
    Source sentence
    PLS achieved smaller and more stable errors in forecasts than LM or NN in this experiment.
  • Some models generate mean daily returns of 0.3%–0.4%, slightly above the average 0.26% return of BTC/USD in the test period, while others can produce negative returns down to –0.5% per day. performance
    Source sentence
    Some models achieve mean rates of return .3% and .4% per day, which is slightly higher than the average rate of return of .26% from BTC / USD in the testing sets.

Checked by others

No other paper in the archive has been found to support or contradict these findings (compared with 28 related papers). Coverage is partial, so this is neither replication nor refutation.

Code

No repository by the authors found in the paper.

Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.

Cited References (21)

  1. Bounds of Optimal Learning (2009)
  2. Information trajectory of optimal learning (2010)
  3. Mutation and Optimal Search of Sequences in Nested H amming Spaces (2011)
  4. Dynamics of Information and Optimal Control of Mutation in Evolutionary Systems (2012)
  5. Optimal measures and M arkov transition kernels (2013)
  6. Asymmetry of Risk and Value of Information (2014)
  7. Relation Between the K antorovich- W asserstein Metric and the K ullback- L eibler Divergence (2018)
  8. Monotonicity of fitness landscapes and mutation rate control (2016)
and 13 more
  1. SIMPLS: An alternative approach to partial least squares regression (1993)
  2. Value of information and sufficient statistics during an observation of a stochastic process (1966)
  3. Conjugate Duality and Optimization (1974)
  4. A mathematical theory of communication (1948)
  5. Topics in the Theory of Random Noise (2014)
  6. Theory of Information and its Value (2020)
  7. On value of information (1965)
  8. Value of information during an observation of a stochastic process in systems with finite state automata (1966)
  9. Extreme problems of information theory and dynamic programming (1967)
  10. Theory of Information (1975)
  11. Value of information when an estimated random variable is hidden (1966)
  12. Game-theoretic problems with information constraints (1968)
  13. Analysis II (1990)

0 of 21 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2596 linked papers.