Using Machine Learning to Forecast Market Direction with Efficient Frontier Coefficients

ArXiv ID: 2404.00825 “View on arXiv”

Authors: Unknown

Abstract

We propose a novel method to improve estimation of asset returns for portfolio optimization. This approach first performs a monthly directional market forecast using an online decision tree. The decision tree is trained on a novel set of features engineered from portfolio theory: the efficient frontier functional coefficients. Efficient frontiers can be decomposed to their functional form, a square-root second-order polynomial, and the coefficients of this function captures the information of all the constituents that compose the market in the current time period. To make these forecasts actionable, these directional forecasts are integrated to a portfolio optimization framework using expected returns conditional on the market forecast as an estimate for the return vector. This conditional expectation is calculated using the inverse Mills ratio, and the Capital Asset Pricing Model is used to translate the market forecast to individual asset forecasts. This novel method outperforms baseline portfolios, as well as other feature sets including technical indicators and the Fama-French factors. To empirically validate the proposed model, we employ a set of market sector ETFs.

Keywords: portfolio optimization, efficient frontier, decision tree, inverse Mills ratio, Capital Asset Pricing Model, Equities (ETFs)

Complexity vs Empirical Score

• Math Complexity: 7.0/10
• Empirical Rigor: 6.5/10
• Quadrant: Holy Grail
• Why: The paper employs advanced mathematics, including mean-variance optimization, efficient frontier functional decomposition, and inverse Mills ratios for conditional expectations. It is highly empirical, using 24 years of ETF data, backtesting with portfolio metrics like Sharpe ratio, and comparing against multiple benchmarks (technical indicators, Fama-French factors).

  flowchart TD
    A["Research Goal:<br/>Improve asset return estimation<br/>for portfolio optimization"] --> B["Data/Inputs:<br/>Historical market sector ETF data"]
    B --> C{"Key Methodology Steps"}
    C --> D["Step 1: Efficient Frontier<br/>Decomposition<br/>Extract functional coefficients"]
    C --> E["Step 2: Monthly Market Direction<br/>Forecast using Online Decision Tree<br/>Features: Efficient Frontier Coefficients"]
    D --> E
    E --> F["Step 3: Conditional Return Estimation<br/>Integrate forecasts to portfolio optimization<br/>Methods: Inverse Mills Ratio & CAPM"]
    F --> G["Computational Process:<br/>Construct Actionable Portfolios"]
    G --> H["Key Findings/Outcomes:<br/>Outperforms baseline & other feature sets<br/>(Technical, Fama-French)"]
    H --> I["Conclusion:<br/>Validated model using Market Sector ETFs"]