Feature Selection

Partial multivariate transformer as a tool for cryptocurrencies time series prediction ArXiv ID: 2512.04099 “View on arXiv” Authors: Andrzej Tokajuk, Jarosław A. Chudziak Abstract Forecasting cryptocurrency prices is hindered by extreme volatility and a methodological dilemma between information-scarce univariate models and noise-prone full-multivariate models. This paper investigates a partial-multivariate approach to balance this trade-off, hypothesizing that a strategic subset of features offers superior predictive power. We apply the Partial-Multivariate Transformer (PMformer) to forecast daily returns for BTCUSDT and ETHUSDT, benchmarking it against eleven classical and deep learning models. Our empirical results yield two primary contributions. First, we demonstrate that the partial-multivariate strategy achieves significant statistical accuracy, effectively balancing informative signals with noise. Second, we experiment and discuss an observable disconnect between this statistical performance and practical trading utility; lower prediction error did not consistently translate to higher financial returns in simulations. This finding challenges the reliance on traditional error metrics and highlights the need to develop evaluation criteria more aligned with real-world financial objectives. ...

Orderbook Feature Learning and Asymmetric Generalization in Intraday Electricity Markets ArXiv ID: 2510.12685 “View on arXiv” Authors: Runyao Yu, Ruochen Wu, Yongsheng Han, Jochen L. Cremer Abstract Accurate probabilistic forecasting of intraday electricity prices is critical for market participants to inform trading decisions. Existing studies rely on specific domain features, such as Volume-Weighted Average Price (VWAP) and the last price. However, the rich information in the orderbook remains underexplored. Furthermore, these approaches are often developed within a single country and product type, making it unclear whether the approaches are generalizable. In this paper, we extract 384 features from the orderbook and identify a set of powerful features via feature selection. Based on selected features, we present a comprehensive benchmark using classical statistical models, tree-based ensembles, and deep learning models across two countries (Germany and Austria) and two product types (60-min and 15-min). We further perform a systematic generalization study across countries and product types, from which we reveal an asymmetric generalization phenomenon. ...

From On-chain to Macro: Assessing the Importance of Data Source Diversity in Cryptocurrency Market Forecasting ArXiv ID: 2506.21246 “View on arXiv” Authors: Giorgos Demosthenous, Chryssis Georgiou, Eliada Polydorou Abstract This study investigates the impact of data source diversity on the performance of cryptocurrency forecasting models by integrating various data categories, including technical indicators, on-chain metrics, sentiment and interest metrics, traditional market indices, and macroeconomic indicators. We introduce the Crypto100 index, representing the top 100 cryptocurrencies by market capitalization, and propose a novel feature reduction algorithm to identify the most impactful and resilient features from diverse data sources. Our comprehensive experiments demonstrate that data source diversity significantly enhances the predictive performance of forecasting models across different time horizons. Key findings include the paramount importance of on-chain metrics for both short-term and long-term predictions, the growing relevance of traditional market indices and macroeconomic indicators for longer-term forecasts, and substantial improvements in model accuracy when diverse data sources are utilized. These insights help demystify the short-term and long-term driving factors of the cryptocurrency market and lay the groundwork for developing more accurate and resilient forecasting models. ...

SAE-FiRE: Enhancing Earnings Surprise Predictions Through Sparse Autoencoder Feature Selection ArXiv ID: 2505.14420 “View on arXiv” Authors: Huopu Zhang, Yanguang Liu, Miao Zhang, Zirui He, Mengnan Du Abstract Predicting earnings surprises from financial documents, such as earnings conference calls, regulatory filings, and financial news, has become increasingly important in financial economics. However, these financial documents present significant analytical challenges, typically containing over 5,000 words with substantial redundancy and industry-specific terminology that creates obstacles for language models. In this work, we propose the SAE-FiRE (Sparse Autoencoder for Financial Representation Enhancement) framework to address these limitations by extracting key information while eliminating redundancy. SAE-FiRE employs Sparse Autoencoders (SAEs) to decompose dense neural representations from large language models into interpretable sparse components, then applies statistical feature selection methods, including ANOVA F-tests and tree-based importance scoring, to identify the top-k most discriminative dimensions for classification. By systematically filtering out noise that might otherwise lead to overfitting, we enable more robust and generalizable predictions. Experimental results across three financial datasets demonstrate that SAE-FiRE significantly outperforms baseline approaches. ...

Forecasting of Bitcoin Prices Using Hashrate Features: Wavelet and Deep Stacking Approach ArXiv ID: 2501.13136 “View on arXiv” Authors: Unknown Abstract Digital currencies have become popular in the last decade due to their non-dependency and decentralized nature. The price of these currencies has seen a lot of fluctuations at times, which has increased the need for prediction. As their most popular, Bitcoin(BTC) has become a research hotspot. The main challenge and trend of digital currencies, especially BTC, is price fluctuations, which require studying the basic price prediction model. This research presents a classification and regression model based on stack deep learning that uses a wavelet to remove noise to predict movements and prices of BTC at different time intervals. The proposed model based on the stacking technique uses models based on deep learning, especially neural networks and transformers, for one, seven, thirty and ninety-day forecasting. Three feature selection models, Chi2, RFE and Embedded, were also applied to the data in the pre-processing stage. The classification model achieved 63% accuracy for predicting the next day and 64%, 67% and 82% for predicting the seventh, thirty and ninety days, respectively. For daily price forecasting, the percentage error was reduced to 0.58, while the error ranged from 2.72% to 2.85% for seven- to ninety-day horizons. These results show that the proposed model performed better than other models in the literature. ...

Time Series Feature Redundancy Paradox: An Empirical Study Based on Mortgage Default Prediction ArXiv ID: 2501.00034 “View on arXiv” Authors: Unknown Abstract With the widespread application of machine learning in financial risk management, conventional wisdom suggests that longer training periods and more feature variables contribute to improved model performance. This paper, focusing on mortgage default prediction, empirically discovers a phenomenon that contradicts traditional knowledge: in time series prediction, increased training data timespan and additional non-critical features actually lead to significant deterioration in prediction effectiveness. Using Fannie Mae’s mortgage data, the study compares predictive performance across different time window lengths (2012-2022) and feature combinations, revealing that shorter time windows (such as single-year periods) paired with carefully selected key features yield superior prediction results. The experimental results indicate that extended time spans may introduce noise from historical data and outdated market patterns, while excessive non-critical features interfere with the model’s learning of core default factors. This research not only challenges the traditional “more is better” approach in data modeling but also provides new insights and practical guidance for feature selection and time window optimization in financial risk prediction. ...

PolyModel for Hedge Funds’ Portfolio Construction Using Machine Learning ArXiv ID: 2412.11019 “View on arXiv” Authors: Unknown Abstract The domain of hedge fund investments is undergoing significant transformation, influenced by the rapid expansion of data availability and the advancement of analytical technologies. This study explores the enhancement of hedge fund investment performance through the integration of machine learning techniques, the application of PolyModel feature selection, and the analysis of fund size. We address three critical questions: (1) the effect of machine learning on trading performance, (2) the role of PolyModel feature selection in fund selection and performance, and (3) the comparative reliability of larger versus smaller funds. Our findings offer compelling insights. We observe that while machine learning techniques enhance cumulative returns, they also increase annual volatility, indicating variability in performance. PolyModel feature selection proves to be a robust strategy, with approaches that utilize a comprehensive set of features for fund selection outperforming more selective methodologies. Notably, Long-Term Stability (LTS) effectively manages portfolio volatility while delivering favorable returns. Contrary to popular belief, our results suggest that larger funds do not consistently yield better investment outcomes, challenging the assumption of their inherent reliability. This research highlights the transformative impact of data-driven approaches in the hedge fund investment arena and provides valuable implications for investors and asset managers. By leveraging machine learning and PolyModel feature selection, investors can enhance portfolio optimization and reassess the dependability of larger funds, leading to more informed investment strategies. ...

CatNet: Controlling the False Discovery Rate in LSTM with SHAP Feature Importance and Gaussian Mirrors ArXiv ID: 2411.16666 “View on arXiv” Authors: Unknown Abstract We introduce CatNet, an algorithm that effectively controls False Discovery Rate (FDR) and selects significant features in LSTM. CatNet employs the derivative of SHAP values to quantify the feature importance, and constructs a vector-formed mirror statistic for FDR control with the Gaussian Mirror algorithm. To avoid instability due to nonlinear or temporal correlations among features, we also propose a new kernel-based independence measure. CatNet performs robustly on different model settings with both simulated and real-world data, which reduces overfitting and improves interpretability of the model. Our framework that introduces SHAP for feature importance in FDR control algorithms and improves Gaussian Mirror can be naturally extended to other time-series or sequential deep learning models. ...

Online High-Frequency Trading Stock Forecasting with Automated Feature Clustering and Radial Basis Function Neural Networks ArXiv ID: 2412.16160 “View on arXiv” Authors: Unknown Abstract This study presents an autonomous experimental machine learning protocol for high-frequency trading (HFT) stock price forecasting that involves a dual competitive feature importance mechanism and clustering via shallow neural network topology for fast training. By incorporating the k-means algorithm into the radial basis function neural network (RBFNN), the proposed method addresses the challenges of manual clustering and the reliance on potentially uninformative features. More specifically, our approach involves a dual competitive mechanism for feature importance, combining the mean-decrease impurity (MDI) method and a gradient descent (GD) based feature importance mechanism. This approach, tested on HFT Level 1 order book data for 20 S&P 500 stocks, enhances the forecasting ability of the RBFNN regressor. Our findings suggest that an autonomous approach to feature selection and clustering is crucial, as each stock requires a different input feature space. Overall, by automating the feature selection and clustering processes, we remove the need for manual topological grid search and provide a more efficient way to predict LOB’s mid-price. ...

Missing Data Imputation With Granular Semantics and AI-driven Pipeline for Bankruptcy Prediction ArXiv ID: 2404.00013 “View on arXiv” Authors: Unknown Abstract This work focuses on designing a pipeline for the prediction of bankruptcy. The presence of missing values, high dimensional data, and highly class-imbalance databases are the major challenges in the said task. A new method for missing data imputation with granular semantics has been introduced here. The merits of granular computing have been explored here to define this method. The missing values have been predicted using the feature semantics and reliable observations in a low-dimensional space, in the granular space. The granules are formed around every missing entry, considering a few of the highly correlated features and most reliable closest observations to preserve the relevance and reliability, the context, of the database against the missing entries. An intergranular prediction is then carried out for the imputation within those contextual granules. That is, the contextual granules enable a small relevant fraction of the huge database to be used for imputation and overcome the need to access the entire database repetitively for each missing value. This method is then implemented and tested for the prediction of bankruptcy with the Polish Bankruptcy dataset. It provides an efficient solution for big and high-dimensional datasets even with large imputation rates. Then an AI-driven pipeline for bankruptcy prediction has been designed using the proposed granular semantic-based data filling method followed by the solutions to the issues like high dimensional dataset and high class-imbalance in the dataset. The rest of the pipeline consists of feature selection with the random forest for reducing dimensionality, data balancing with SMOTE, and prediction with six different popular classifiers including deep NN. All methods defined here have been experimentally verified with suitable comparative studies and proven to be effective on all the data sets captured over the five years. ...