Paper: arXiv 2411.06080

Authors: Sayyed Faraz Mohseni, Hamid R. Arian, Jean-François Bégin

Abstract

Portfolio diversification, traditionally measured through asset correlations and volatilitybased metrics, is fundamental to managing financial risk. However, existing diversification metrics often overlook non-numerical relationships between assets that can impact portfolio stability, particularly during market stresses. This paper introduces the lexical ratio (LR), a novel metric that leverages textual data to capture diversification dimensions absent in standard approaches. By treating each asset as a unique document composed of sectorspecific and financial keywords, the LR evaluates portfolio diversification by distributing these terms across assets, incorporating entropy-based insights from information theory. We thoroughly analyze LR’s properties, including scale invariance, concavity, and maximality, demonstrating its theoretical robustness and ability to enhance risk-adjusted portfolio returns. Using empirical tests on S&P 500 portfolios, we compare LR’s performance to established metrics such as Markowitz’s volatility-based measures and diversification ratios. Our tests reveal LR’s superiority in optimizing portfolio returns, especially under varied market conditions. Our findings show that LR aligns with conventional metrics and captures unique diversification aspects, suggesting it is a viable tool for portfolio managers.

Complexity vs Empirical Score

  • Math Complexity: 7.5/10
  • Empirical Rigor: 6.0/10
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper uses advanced information theory (Shannon entropy) and derived formulas for its new metric, indicating high mathematical density. It also conducts empirical tests on S&P 500 portfolios with backtest-ready comparisons to established metrics, though it lacks code or datasets in the excerpt.

Research Flowchart

  flowchart TD
  A["Research Goal: Introduce Lexical Ratio (LR) for portfolio diversification"] --> B["Methodology: Process textual & financial data from S&P 500"]
  B --> C["Data: Asset descriptions & sector-specific keywords"]
  C --> D["Computation: Compute LR using entropy-based analysis<br>(treating assets as documents)"]
  D --> E["Comparison: Benchmark LR against Markowitz volatility<br>& diversification ratios"]
  E --> F["Findings: LR enhances risk-adjusted returns & captures<br>unique non-numerical diversification aspects"]

Evidence Card

What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card

Proposes

The paper proposes a new diversification metric called the lexical ratio (LR) that uses textual data to capture diversification dimensions missed by standard approaches.

Source sentence
This paper introduces the lexical ratio (LR), a novel metric that leverages textual data to capture diversification dimensions absent in standard approaches.

Assumes 3

  • The lexical ratio is scale‑invariant, so scaling word counts does not affect its value. model
    Source sentence
    Given that LR is scale‑invariant, this scaling does not affect the numeric values, and this number is arbitrary.
  • News headline text adequately represents asset‑related information for diversification measurement. data
    Source sentence
    This study utilizes news headline data from the eodhd package.
  • Stock return data from yfinance accurately reflects asset price movements. data
    Source sentence
    Stock return data is sourced from the yfinance library.

Data

Backtest sample: 2018-01 – 2024-06 · S&P 500 stocks · equities

  • News headline data · eodhd package text-news public
    2018-2024 · S&P 500 constituents
    Source sentence
    Our dataset, spanning from 2018 to 2024, had some limitations regarding linguistic data availability
  • Stock return data · yfinance library market-prices public
    2018-2024 · S&P 500 constituents
    Source sentence
    Stock return data is sourced from the yfinance library.

Costs

Not modeled A trading or execution result is reported without costs.

Finds 3

Reported performance (Lexical Ratio (LR) optimized portfolio): Sharpe 2.80 · annual return 37.7%

  • LR consistently outperforms traditional diversification metrics across multiple market conditions, delivering higher risk‑adjusted returns. performance
    Source sentence
    ... consistently outperformed traditional metrics in identifying well‑diversified portfolios
  • On a random mixed portfolio 1, LR achieves a Sharpe ratio of 2.7954 versus 1.6615 for volatility‑based metrics. performance
    Source sentence
    Random mixed portfolio 1 Sharpe ratio 2.7954 ... 1.6615 ... 1.6618 ... 1.8384
  • LR aligns with conventional metrics while capturing unique diversification aspects not reflected by correlation‑based methods. theoretical
    Source sentence
    LR aligns with conventional metrics and captures unique diversification aspects, suggesting it is a viable tool for portfolio managers.

Checked by others

No other paper in the archive has been found to support or contradict these findings (compared with 18 related papers). Coverage is partial, so this is neither replication nor refutation.

Code

No repository by the authors found in the paper.

Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.

Cited References (51)

  1. Optimal versus naive diversification: How inefficient is the 1/$N$ portfolio strategy? (2007)
  2. Capital allocation to business units and sub-portfolios: The E uler principle (2007)
  3. Entropy-Based Financial Asset Pricing (2014)
  4. An improved portfolio optimization approach using cross-entropy (2013)
  5. Latent Dirichlet Allocation (2003)
  6. Why finance is deploying natural language processing (2020) — link
  7. Optimal portfolio diversification using the maximum entropy principle (2008)
  8. Machine learning for financial market prediction (2016)
and 43 more
  1. Elements of Information Theory (2012)
  2. Entropy and information theory (2011)
  3. On quadratic entropy measures (2004)
  4. Toward maximum diversification (2008)
  5. Short texts, best-fitting curves and new measures of lexical diversity (2002)
  6. Cutting the Gordian knot: The moving-average type–token ratio (MATTR) (2010)
  7. Lexical diversity and language development (2004)
  8. Risk parity, maximum diversification, and minimum variance: An analytic perspective (2013)
  9. Diversification quotients: Quantifying diversification via risk measures (2022)
  10. Optimising portfolio diversification and dimensionality (2022)
  11. Sequential quadratic programming for large-scale nonlinear optimization (2000)
  12. SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python (2020) — link
  13. Defining and Measuring Portfolio Diversification (2020) — link
  14. Value at Risk: The New Benchmark for Managing Financial Risk (2007)
  15. Modern Investment Management: An Equilibrium Approach (2003)
  16. Risk aggregation, dependence structure and diversification benefit (2008)
  17. On optimal portfolio diversification with respect to extreme risks (2010) — link
  18. Multivariate extremes and the aggregation of dependent risks: examples and counter-examples (2009)
  19. Risk concentration and diversification: Second-order properties (2010)
  20. Portfolio selection (1952)
  21. Coherent measures of risk (1999)
  22. Conditional value-at-risk for general loss distributions (2002)
  23. Parametric measures of variability induced by risk measures (2022)
  24. Diversification quotients based on VaR and ES — → 2301.03517
  25. Sector & Industry Performance (2024) — link
  26. Top companies by sectors (2024) — link
  27. The 8 Most Volatile Sectors (2024) — link
  28. How to Research Volatile Stocks (2024) — link
  29. Most Volatile Stocks Right Now (2024) — link
  30. Low Volatility – I ndices (2024) — link
  31. Low Volatility Stocks (2024) — link
  32. What is minimum volatility? (2024) — link
  33. A simple measure of conditional dependence (2021)
  34. EODHD API (2024)
  35. Daily Treasury Yield Curve Rates (2018) — link
  36. Measuring portfolio diversification (2011)
  37. Entropy-based financial asset pricing (2014)
  38. Portfolio diversification according to the principle of maximum entropy (2009)
  39. Portfolio Optimization with a Mean–Absolute Deviation–Entropy Model (2020)
  40. The analysis of diversification properties of stablecoins through the entropy-based portfolio optimization model (2024)
  41. Multi-Objective Mean-Absolute Deviation-Entropy on the JII Index (2023)
  42. yfinance - Yahoo! Finance market data downloader (2024) — link
  43. S&P Sector and Industry Indices (2024) — link

1 of 51 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2596 linked papers.