Principal Component Analysis

Machine Learning Based Stress Testing Framework for Indian Financial Market Portfolios ArXiv ID: 2507.02011 “View on arXiv” Authors: Vidya Sagar G, Shifat Ali, Siddhartha P. Chakrabarty Abstract This paper presents a machine learning driven framework for sectoral stress testing in the Indian financial market, focusing on financial services, information technology, energy, consumer goods, and pharmaceuticals. Initially, we address the limitations observed in conventional stress testing through dimensionality reduction and latent factor modeling via Principal Component Analysis and Autoencoders. Building on this, we extend the methodology using Variational Autoencoders, which introduces a probabilistic structure to the latent space. This enables Monte Carlo-based scenario generation, allowing for more nuanced, distribution-aware simulation of stressed market conditions. The proposed framework captures complex non-linear dependencies and supports risk estimation through Value-at-Risk and Expected Shortfall. Together, these pipelines demonstrate the potential of Machine Learning approaches to improve the flexibility, robustness, and realism of financial stress testing. ...

Axes that matter: PCA with a difference ArXiv ID: 2503.06707 “View on arXiv” Authors: Unknown Abstract We extend the scope of differential machine learning and introduce a new breed of supervised principal component analysis to reduce dimensionality of Derivatives problems. Applications include the specification and calibration of pricing models, the identification of regression features in least-square Monte-Carlo, and the pre-processing of simulated datasets for (differential) machine learning. Keywords: differential machine learning, principal component analysis, derivatives pricing, least-square Monte-Carlo, dimensionality reduction ...

Empirical Study on the Factors Influencing Stock Market Volatility in China ArXiv ID: 2501.08668 “View on arXiv” Authors: Unknown Abstract This paper mainly utilizes the ARDL model and principal component analysis to investigate the relationship between the volatility of China’s Shanghai Composite Index returns and the variables of exchange rate and domestic and foreign bond yields in an internationally integrated stock market. This paper uses a daily data set for the period from July 1, 2010 to April 30, 2024, in which the dependent variable is the Shanghai Composite Index return, and the main independent variables are the spot exchange rate of the RMB against the US dollar, the 10-year treasury bond yields in China and the United States and their lagged variables, with the effect of the time factor added. Firstly, the development of the stock, foreign exchange and bond markets and the basic theories are reviewed, and then each variable is analyzed by descriptive statistics, the correlation between the independent variables and the dependent variable is expanded theoretically, and the corresponding empirical analyses are briefly introduced, and then the empirical analyses and modeling of the relationship between the independent variables and the dependent variable are carried out on the basis of the theoretical foundations mentioned above with the support of the daily data, and the model conclusions are analyzed economically through a large number of tests, then the model conclusions are analyzed economically. economic analysis of the model conclusions, and finally, the author proposes three suggestions to enhance the stability and return of the Chinese stock market, respectively. Key Words: Chinese Stock Market, Volatility, GARCH, ARDL Model ...

Shocks-adaptive Robust Minimum Variance Portfolio for a Large Universe of Assets ArXiv ID: 2410.01826 “View on arXiv” Authors: Unknown Abstract This paper proposes a robust, shocks-adaptive portfolio in a large-dimensional assets universe where the number of assets could be comparable to or even larger than the sample size. It is well documented that portfolios based on optimizations are sensitive to outliers in return data. We deal with outliers by proposing a robust factor model, contributing methodologically through the development of a robust principal component analysis (PCA) for factor model estimation and a shrinkage estimation for the random error covariance matrix. This approach extends the well-regarded Principal Orthogonal Complement Thresholding (POET) method (Fan et al., 2013), enabling it to effectively handle heavy tails and sudden shocks in data. The novelty of the proposed robust method is its adaptiveness to both global and idiosyncratic shocks, without the need to distinguish them, which is useful in forming portfolio weights when facing outliers. We develop the theoretical results of the robust factor model and the robust minimum variance portfolio. Numerical and empirical results show the superior performance of the new portfolio. ...

Predicting Foreign Exchange EUR/USD direction using machine learning ArXiv ID: 2409.04471 “View on arXiv” Authors: Unknown Abstract The Foreign Exchange market is a significant market for speculators, characterized by substantial transaction volumes and high volatility. Accurately predicting the directional movement of currency pairs is essential for formulating a sound financial investment strategy. This paper conducts a comparative analysis of various machine learning models for predicting the daily directional movement of the EUR/USD currency pair in the Foreign Exchange market. The analysis includes both decorrelated and non-decorrelated feature sets using Principal Component Analysis. Additionally, this study explores meta-estimators, which involve stacking multiple estimators as input for another estimator, aiming to achieve improved predictive performance. Ultimately, our approach yielded a prediction accuracy of 58.52% for one-day ahead forecasts, coupled with an annual return of 32.48% for the year 2022. ...

“Microstructure Modes” – Disentangling the Joint Dynamics of Prices & Order Flow ArXiv ID: 2405.10654 “View on arXiv” Authors: Unknown Abstract Understanding the micro-dynamics of asset prices in modern electronic order books is crucial for investors and regulators. In this paper, we use an order by order Eurostoxx database spanning over 3 years to analyze the joint dynamics of prices and order flow. In order to alleviate various problems caused by high-frequency noise, we propose a double coarse-graining procedure that allows us to extract meaningful information at the minute time scale. We use Principal Component Analysis to construct “microstructure modes” that describe the most common flow/return patterns and allow one to separate them into bid-ask symmetric and bid-ask anti-symmetric. We define and calibrate a Vector Auto-Regressive (VAR) model that encodes the dynamical evolution of these modes. The parameters of the VAR model are found to be extremely stable in time, and lead to relatively high $R^2$ prediction scores, especially for symmetric liquidity modes. The VAR model becomes marginally unstable as more lags are included, reflecting the long-memory nature of flows and giving some further credence to the possibility of “endogenous liquidity crises”. Although very satisfactory on several counts, we show that our VAR framework does not account for the well known square-root law of price impact. ...

Distributional Reference Class Forecasting of Corporate Sales Growth With Multiple Reference Variables ArXiv ID: 2405.03402 “View on arXiv” Authors: Unknown Abstract This paper introduces an approach to reference class selection in distributional forecasting with an application to corporate sales growth rates using several co-variates as reference variables, that are implicit predictors. The method can be used to detect expert or model-based forecasts exposed to (behavioral) bias or to forecast distributions with reference classes. These are sets of similar entities, here firms, and rank based algorithms for their selection are proposed, including an optional preprocessing data dimension reduction via principal components analysis. Forecasts are optimal if they match the underlying distribution as closely as possible. Probability integral transform values rank the forecast capability of different reference variable sets and algorithms in a backtest on a data set of 21,808 US firms over the time period 1950 - 2019. In particular, algorithms on dimension reduced variables perform well using contemporaneous balance sheet and financial market parameters along with past sales growth rates and past operating margins changes. Comparisions of actual analysts’ estimates to distributional forecasts and of historic distributional forecasts to realized sales growth illustrate the practical use of the method. ...

Supervised Dynamic PCA: Linear Dynamic Forecasting with Many Predictors ArXiv ID: 2307.07689 “View on arXiv” Authors: Unknown Abstract This paper proposes a novel dynamic forecasting method using a new supervised Principal Component Analysis (PCA) when a large number of predictors are available. The new supervised PCA provides an effective way to bridge the gap between predictors and the target variable of interest by scaling and combining the predictors and their lagged values, resulting in an effective dynamic forecasting. Unlike the traditional diffusion-index approach, which does not learn the relationships between the predictors and the target variable before conducting PCA, we first re-scale each predictor according to their significance in forecasting the targeted variable in a dynamic fashion, and a PCA is then applied to a re-scaled and additive panel, which establishes a connection between the predictability of the PCA factors and the target variable. Furthermore, we also propose to use penalized methods such as the LASSO approach to select the significant factors that have superior predictive power over the others. Theoretically, we show that our estimators are consistent and outperform the traditional methods in prediction under some mild conditions. We conduct extensive simulations to verify that the proposed method produces satisfactory forecasting results and outperforms most of the existing methods using the traditional PCA. A real example of predicting U.S. macroeconomic variables using a large number of predictors showcases that our method fares better than most of the existing ones in applications. The proposed method thus provides a comprehensive and effective approach for dynamic forecasting in high-dimensional data analysis. ...

Non-parametric cumulants approach for outlier detection of multivariate financial data ArXiv ID: 2305.10911 “View on arXiv” Authors: Unknown Abstract In this paper, we propose an outlier detection algorithm for multivariate data based on their projections on the directions that maximize the Cumulant Generating Function (CGF). We prove that CGF is a convex function, and we characterize the CGF maximization problem on the unit n-circle as a concave minimization problem. Then, we show that the CGF maximization approach can be interpreted as an extension of the standard principal component technique. Therefore, for validation and testing, we provide a thorough comparison of our methodology with two other projection-based approaches both on artificial and real-world financial data. Finally, we apply our method as an early detector for financial crises. ...