Paper: arXiv 2407.16437

Authors: Maksim Papenkov

Abstract

Accurate industry classification is critical for many areas of portfolio management, yet the traditional single-industry framework of the Global Industry Classification Standard (GICS) struggles to comprehensively represent risk for highly diversified multi-sector conglomerates like Amazon. Previously, we introduced the Multi-Industry Simplex (MIS), a probabilistic extension of GICS that utilizes topic modeling, a natural language processing approach. Although our initial version, MIS-1, was able to improve upon GICS by providing multi-industry representations, it relied on an overly simple architecture that required prior knowledge about the number of industries and relied on the unrealistic assumption that industries are uncorrelated and independent over time. We improve upon this model with MIS-2, which addresses three key limitations of MIS-1 : we utilize Bayesian Non-Parametrics to automatically infer the number of industries from data, we employ Markov Updating to account for industries that change over time, and we adjust for correlated and hierarchical industries allowing for both broad and niche industries (similar to GICS). Further, we provide an out-of-sample test directly comparing MIS-2 and GICS on the basis of future correlation prediction, where we find evidence that MIS-2 provides a measurable improvement over GICS. MIS-2 provides portfolio managers with a more robust tool for industry classification, empowering them to more effectively identify and manage risk, particularly around multi-sector conglomerates in a rapidly evolving market in which new industries periodically emerge.

Complexity vs Empirical Score

  • Math Complexity: 8.5/10
  • Empirical Rigor: 6.0/10
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper employs advanced mathematical techniques like Bayesian non-parametrics and Markov updating, requiring dense probabilistic modeling and LaTeX notation. It also includes out-of-sample testing and specific data pre-processing details for backtest-readiness, though it mentions future work for richer backtests.

Research Flowchart

  flowchart TD
  A["Research Goal:<br>Improve industry classification for multi-sector firms<br>like Amazon, moving beyond static GICS"] --> B["Data Inputs:<br>Company fundamental data & textual disclosures"]
  B --> C{"Key Methodology - MIS-2"}
  C --> D["Bayesian Non-Parametrics<br>Infers # of industries automatically"]
  C --> E["Markov Updating<br>Models temporal evolution of industries"]
  C --> F["Correlation Adjustment<br>Hierarchical structure (broad & niche)"]
  D & E & F --> G["Computational Process:<br>Probabilistic Multi-Industry Assignment"]
  G --> H["Key Findings/Outcomes"]
  H --> I["Out-of-sample test:<br>MIS-2 outperforms GICS in<br>future correlation prediction"]
  H --> J["Result:<br>Robust tool for risk management<br>in evolving markets"]

Evidence Card

What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card

Proposes

The paper proposes MIS-2, a probabilistic multi‑industry classification model that automatically infers the number of industries, updates them over time, and accounts for correlated and hierarchical industry structures.

Source sentence
we improve upon this model with MIS-2, which addresses three key limitations of MIS-1 : we utilize Bayesian Non-Parametrics to automatically infer the number of industries from data, we employ Markov Updating to account for industries that change over time, and we adjust for correlated and hierarchical industries allow

Assumes 3

  • Industries are correlated and hierarchical rather than independent. model
    Source sentence
    we adjust for correlated and hierarchical industries allowing for both broad and niche industries (similar to GICS).
  • Business description text reliably indicates industry membership. data
    Source sentence
    MIS-2 utilizes text from business descriptions to identify industry membership.
  • The set of firms includes over 3000 stocks with data available from 2021 onward. data
    Source sentence
    For our particular implementation, we only have data from 2021 to present

Data

  • Business description text of over 3000 stocks text-news unknown
    2021-present · over 3000 stocks
    Source sentence
    We perform this test for each firm in our universe of over 3000 stocks, and summarize the data below

Costs

Not applicable The paper reports no trading or execution result.

Finds 2

  • MIS-2 provides a measurable improvement over GICS in out‑of‑sample future correlation prediction. predictive-accuracy
    Source sentence
    we find evidence that MIS-2 provides a measurable improvement over GICS.
  • On average, MIS-2 outperforms GICS consistently across all sectors in the out‑of‑sample test. predictive-accuracy
    Source sentence
    You’ll notice that on average, MIS-2 outperforms GICS consistently across all sectors.

Checked by others

No other paper in the archive has been found to support or contradict these findings (compared with 7 related papers). Coverage is partial, so this is neither replication nor refutation.

Code

No repository by the authors found in the paper.

Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.

Cited References (41)

  1. Latent Dirichlet Allocation (2003)
  2. Finding Scientific Topics (2004)
  3. Multi-Industry Simplex : A Probabilistic Extension of GICS — → 2310.04280
  4. Dynamic Topic Models (2006)
  5. Mixed-Membership Models (2015)
  6. Hierarchical Dirichlet Process (2006)
  7. An Algorithm for Suffix Stripping (1980)
  8. Blocked Gibbs sampler for hierarchical Dirichlet processes (2023) — link
and 33 more
  1. Sharing clusters among related groups: hierarchical Dirichlet processes (2004)
  2. Hierarchical Topic Models and the Nested Chinese Restaurant Process (2003)
  3. Pachinko Allocation : DAG-Structure Mixture Models of Topic Correlations (2006)
  4. Measuring LDA Topic Stability from Clusters of Replicated Runs (2018)
  5. Automatic Evaluation of Topic Coherence (2010)
  6. Correlated Topic Models (2005)
  7. Gibbs Sampling for Logistic Normal Topic Models with Graph-Based Priors (2008)
  8. Distributed Algorithms for Topic Models (2009)
  9. Improving Topic Model Stability for Effective Document Exploration (2016)
  10. Improving Latent Dirichlet Allocation : On Reliability of the Novel Method LDAPrototype (2020)
  11. Exploring the Space of Topic Coherence Measures (2015)
  12. Term Weighting Schemes for Latent Dirichlet Allocation (2010)
  13. LDA-Based Industry Classification (2013)
  14. Improving Reliability of LDA by Assessing Its Stability Using Clustering Techniques on Replicated Runs (2020) — link
  15. Latent Dirichlet Allocation (2003)
  16. Dynamic Topic Models (2006)
  17. A New Approach to Linear Filtering and Prediction Problems (1960) — link
  18. Testing for Serial Correlation in Least Squares Regression:I (1950)
  19. Inference of Population Structure Using Multilocus Genotype Data (2000)
  20. Social-Network Analysis Using Topic Models (2013)
  21. Term Weighting Schemes for Latent Dirichlet Allocation (2010)
  22. Stochastic Relaxation, Gibbs Distributions and the Bayesian Restoration of Images (1993)
  23. Introduction to Mixed Membership Models and Methods (2015)
  24. Variational Inference : A Review for Statisticians (2017)
  25. Variational Inference: A Review for Statisticians (2017) — link
  26. The Global Industry Classification Standard (GICS) (2023) — link
  27. GPT-4 Technical Report (2023) — link
  28. LLaMA: Open and Efficient Foundation Language Models (2023) — link
  29. Visualizing Data using t-SNE (2008)
  30. Company Similarity using Large Language Models — → 2308.08031
  31. Company classification using zero-shot learning (2023) — link
  32. Automated Industry Classification with Deep Learning (2018) — link
  33. Industry classification based on supply chain network information using Graph Neural Networks (2023) — link

2 of 41 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2596 linked papers.