Data Preprocessing

Exploring Microstructural Dynamics in Cryptocurrency Limit Order Books: Better Inputs Matter More Than Stacking Another Hidden Layer ArXiv ID: 2506.05764 “View on arXiv” Authors: Haochuan Wang Abstract Cryptocurrency price dynamics are driven largely by microstructural supply demand imbalances in the limit order book (LOB), yet the highly noisy nature of LOB data complicates the signal extraction process. Prior research has demonstrated that deep-learning architectures can yield promising predictive performance on pre-processed equity and futures LOB data, but they often treat model complexity as an unqualified virtue. In this paper, we aim to examine whether adding extra hidden layers or parameters to “blackbox ish” neural networks genuinely enhances short term price forecasting, or if gains are primarily attributable to data preprocessing and feature engineering. We benchmark a spectrum of models from interpretable baselines, logistic regression, XGBoost to deep architectures (DeepLOB, Conv1D+LSTM) on BTC/USDT LOB snapshots sampled at 100 ms to multi second intervals using publicly available Bybit data. We introduce two data filtering pipelines (Kalman, Savitzky Golay) and evaluate both binary (up/down) and ternary (up/flat/down) labeling schemes. Our analysis compares models on out of sample accuracy, latency, and robustness to noise. Results reveal that, with data preprocessing and hyperparameter tuning, simpler models can match and even exceed the performance of more complex networks, offering faster inference and greater interpretability. ...

Optimizing Fintech Marketing: A Comparative Study of Logistic Regression and XGBoost ArXiv ID: 2412.16333 “View on arXiv” Authors: Unknown Abstract As several studies have shown, predicting credit risk is still a major concern for the financial services industry and is receiving a lot of scholarly interest. This area of study is crucial because it aids financial organizations in determining the probability that borrowers would default, which has a direct bearing on lending choices and risk management tactics. Despite the progress made in this domain, there is still a substantial knowledge gap concerning consumer actions that take place prior to the filing of credit card applications. The objective of this study is to predict customer responses to mail campaigns and assess the likelihood of default among those who engage. This research employs advanced machine learning techniques, specifically logistic regression and XGBoost, to analyze consumer behavior and predict responses to direct mail campaigns. By integrating different data preprocessing strategies, including imputation and binning, we enhance the robustness and accuracy of our predictive models. The results indicate that XGBoost consistently outperforms logistic regression across various metrics, particularly in scenarios using categorical binning and custom imputation. These findings suggest that XGBoost is particularly effective in handling complex data structures and provides a strong predictive capability in assessing credit risk. ...

The TruEnd-procedure: Treating trailing zero-valued balances in credit data ArXiv ID: 2404.17008 “View on arXiv” Authors: Unknown Abstract A novel procedure is presented for finding the true but latent endpoints within the repayment histories of individual loans. The monthly observations beyond these true endpoints are false, largely due to operational failures that delay account closure, thereby corrupting some loans. Detecting these false observations is difficult at scale since each affected loan history might have a different sequence of trailing zero (or very small) month-end balances. Identifying these trailing balances requires an exact definition of a “small balance”, which our method informs. We demonstrate this procedure and isolate the ideal small-balance definition using two different South African datasets. Evidently, corrupted loans are remarkably prevalent and have excess histories that are surprisingly long, which ruin the timing of risk events and compromise any subsequent time-to-event model, e.g., survival analysis. Having discarded these excess histories, we demonstrably improve the accuracy of both the predicted timing and severity of risk events, without materially impacting the portfolio. The resulting estimates of credit losses are lower and less biased, which augurs well for raising accurate credit impairments under IFRS 9. Our work therefore addresses a pernicious data error, which highlights the pivotal role of data preparation in producing credible forecasts of credit risk. ...