High-Frequency Data

Generalized Mean Absolute Directional Loss as a Solution to Overfitting and High Transaction Costs in Machine Learning Models Used in High-Frequency Algorithmic Investment Strategies ArXiv ID: 2412.18405 “View on arXiv” Authors: Unknown Abstract Regardless of the selected asset class and the level of model complexity (Transformer versus LSTM versus Perceptron/RNN), the GMADL loss function produces superior results than standard MSE-type loss functions and has better numerical properties in the context of optimization than MADL. Better results mean the possibility of achieving a higher risk-weighted return based on buy and sell signals built on forecasts generated by a given theoretical model estimated using the GMADL versus MSE or MADL function. In practice, GMADL solves the problem of selecting the most preferable feature in both classification and regression problems, improving the performance of each estimation. What is important is that, through additional parameterization, GMADL also solves the problem of optimizing investment systems on high-frequency data in such a way that they focus on strategy variants that contain fewer transactions so that transaction costs do not reduce the effectiveness of a given strategy to zero. Moreover, the implementation leverages state-of-the-art machine learning tools, including frameworks for hyperparameter tuning, architecture testing, and walk-forward optimization, ensuring robust and scalable solutions for real-world algorithmic trading. ...

FinML-Chain: A Blockchain-Integrated Dataset for Enhanced Financial Machine Learning ArXiv ID: 2411.16277 “View on arXiv” Authors: Unknown Abstract Machine learning is critical for innovation and efficiency in financial markets, offering predictive models and data-driven decision-making. However, challenges such as missing data, lack of transparency, untimely updates, insecurity, and incompatible data sources limit its effectiveness. Blockchain technology, with its transparency, immutability, and real-time updates, addresses these challenges. We present a framework for integrating high-frequency on-chain data with low-frequency off-chain data, providing a benchmark for addressing novel research questions in economic mechanism design. This framework generates modular, extensible datasets for analyzing economic mechanisms such as the Transaction Fee Mechanism, enabling multi-modal insights and fairness-driven evaluations. Using four machine learning techniques, including linear regression, deep neural networks, XGBoost, and LSTM models, we demonstrate the framework’s ability to produce datasets that advance financial research and improve understanding of blockchain-driven systems. Our contributions include: (1) proposing a research scenario for the Transaction Fee Mechanism and demonstrating how the framework addresses previously unexplored questions in economic mechanism design; (2) providing a benchmark for financial machine learning by open-sourcing a sample dataset generated by the framework and the code for the pipeline, enabling continuous dataset expansion; and (3) promoting reproducibility, transparency, and collaboration by fully open-sourcing the framework and its outputs. This initiative supports researchers in extending our work and developing innovative financial machine-learning models, fostering advancements at the intersection of machine learning, blockchain, and economics. ...

Wavelet Analysis of Cryptocurrencies – Non-Linear Dynamics in High Frequency Domains ArXiv ID: 2411.14058 “View on arXiv” Authors: Unknown Abstract In this study, we perform some analysis for the probability distributions in the space of frequency and time variables. However, in the domain of high frequencies, it behaves in such a way as the highly non-linear dynamics. The wavelet analysis is a powerful tool to perform such analysis in order to search for the characteristics of frequency variations over time for the prices of major cryptocurrencies. In fact, the wavelet analysis is found to be quite useful as it examine the validity of the efficient market hypothesis in the weak form, especially for the presence of the cyclical persistence at different frequencies. If we could find some cyclical persistence at different frequencies, that means that there exist some intrinsic causal relationship for some given investment horizons defined by some chosen sampling scales. This is one of the characteristic results of the wavelet analysis in the time-frequency domains. ...

Corporate Fundamentals and Stock Price Co-Movement ArXiv ID: 2411.03922 “View on arXiv” Authors: Unknown Abstract We introduce an innovative framework that leverages advanced big data techniques to analyze dynamic co-movement between stocks and their underlying fundamentals using high-frequency stock market data. Our method identifies leading co-movement stocks through four distinct regression models: Forecast Error Variance Decomposition, transaction volume-normalized FEVD, Granger causality test frequency, and Granger causality test days. Validated using Chinese banking sector stocks, our framework uncovers complex relationships between stock price co-movements and fundamental characteristics, demonstrating its robustness and wide applicability across various sectors and markets. This approach not only enhances our understanding of market dynamics but also provides actionable insights for investors and policymakers, helping to mitigate broader market volatilities and improve financial stability. Our model indicates that banks’ influence on their peers is significantly affected by their wealth management business, interbank activities, equity multiplier, non-performing loans, regulatory requirements, and reserve requirement ratios. This aids in mitigating the impact of broader market volatilities and provides deep insights into the unique influence of banks within the financial ecosystem. ...

The Efficient Tail Hypothesis: An Extreme Value Perspective on Market Efficiency ArXiv ID: 2408.06661 “View on arXiv” Authors: Unknown Abstract In econometrics, the Efficient Market Hypothesis posits that asset prices reflect all available information in the market. Several empirical investigations show that market efficiency drops when it undergoes extreme events. Many models for multivariate extremes focus on positive dependence, making them unsuitable for studying extremal dependence in financial markets where data often exhibit both positive and negative extremal dependence. To this end, we construct regular variation models on the entirety of $\mathbb{“R”}^d$ and develop a bivariate measure for asymmetry in the strength of extremal dependence between adjacent orthants. Our directional tail dependence (DTD) measure allows us to define the Efficient Tail Hypothesis (ETH) – an analogue of the Efficient Market Hypothesis – for the extremal behaviour of the market. Asymptotic results for estimators of DTD are described, and we discuss testing of the ETH via permutation-based methods and present novel tools for visualization. Empirical study of China’s futures market leads to a rejection of the ETH and we identify potential profitable investment opportunities. To promote the research of microstructure in China’s derivatives market, we open-source our high-frequency data, which are being collected continuously from multiple derivative exchanges. ...

Forecasting High Frequency Order Flow Imbalance ArXiv ID: 2408.03594 “View on arXiv” Authors: Unknown Abstract Market information events are generated intermittently and disseminated at high speeds in real-time. Market participants consume this high-frequency data to build limit order books, representing the current bids and offers for a given asset. The arrival processes, or the order flow of bid and offer events, are asymmetric and possibly dependent on each other. The quantum and direction of this asymmetry are often associated with the direction of the traded price movement. The Order Flow Imbalance (OFI) is an indicator commonly used to estimate this asymmetry. This paper uses Hawkes processes to estimate the OFI while accounting for the lagged dependence in the order flow between bids and offers. Secondly, we develop a method to forecast the near-term distribution of the OFI, which can then be used to compare models for forecasting OFI. Thirdly, we propose a method to compare the forecasts of OFI for an arbitrarily large number of models. We apply the approach developed to tick data from the National Stock Exchange and observe that the Hawkes process modeled with a Sum of Exponential’s kernel gives the best forecast among all competing models. ...

A nonparametric test for rough volatility ArXiv ID: 2407.10659 “View on arXiv” Authors: Unknown Abstract We develop a nonparametric test for deciding whether volatility of an asset follows a standard semimartingale process, with paths of finite quadratic variation, or a rough process with paths of infinite quadratic variation. The test utilizes the fact that volatility is rough if and only if volatility increments are negatively autocorrelated at high frequencies. It is based on the sample autocovariance of increments of spot volatility estimates computed from high-frequency asset return data. By showing a feasible CLT for this statistic under the null hypothesis of semimartingale volatility paths, we construct a test with fixed asymptotic size and an asymptotic power equal to one. The test is derived under very general conditions for the data-generating process. In particular, it is robust to jumps with arbitrary activity and to the presence of market microstructure noise. In an application of the test to SPY high-frequency data, we find evidence for rough volatility. ...

Probabilistic models and statistics for electronic financial markets in the digital age ArXiv ID: 2406.07388 “View on arXiv” Authors: Unknown Abstract The scope of this manuscript is to review some recent developments in statistics for discretely observed semimartingales which are motivated by applications for financial markets. Our journey through this area stops to take closer looks at a few selected topics discussing recent literature. We moreover highlight and explain the important role played by some classical concepts of probability and statistics. We focus on three main aspects: Testing for jumps; rough fractional stochastic volatility; and limit order microstructure noise. We review jump tests based on extreme value theory and complement the literature proposing new statistical methods. They are based on asymptotic theory of order statistics and the Rényi representation. The second stage of our journey visits a recent strand of research showing that volatility is rough. We further investigate this and establish a minimax lower bound exploring frontiers to what extent the regularity of latent volatility can be recovered in a more general framework. Finally, we discuss a stochastic boundary model with one-sided microstructure noise for high-frequency limit order prices and its probabilistic and statistical foundation. ...

Internet sentiment exacerbates intraday overtrading, evidence from A-Share market ArXiv ID: 2404.12001 “View on arXiv” Authors: Unknown Abstract Market fluctuations caused by overtrading are important components of systemic market risk. This study examines the effect of investor sentiment on intraday overtrading activities in the Chinese A-share market. Employing high-frequency sentiment indices inferred from social media posts on the Eastmoney forum Guba, the research focuses on constituents of the CSI 300 and CSI 500 indices over a period from 01/01/2018, to 12/30/2022. The empirical analysis indicates that investor sentiment exerts a significantly positive impact on intraday overtrading, with the influence being more pronounced among institutional investors relative to individual traders. Moreover, sentiment-driven overtrading is found to be more prevalent during bull markets as opposed to bear markets. Additionally, the effect of sentiment on overtrading is observed to be more pronounced among individual investors in large-cap stocks compared to small- and mid-cap stocks. ...

A Comparison of Cryptocurrency Volatility-benchmarking New and Mature Asset Classes ArXiv ID: 2404.04962 “View on arXiv” Authors: Unknown Abstract The paper analyzes the cryptocurrency ecosystem at both the aggregate and individual levels to understand the factors that impact future volatility. The study uses high-frequency panel data from 2020 to 2022 to examine the relationship between several market volatility drivers, such as daily leverage, signed volatility and jumps. Several known autoregressive model specifications are estimated over different market regimes, and results are compared to equity data as a reference benchmark of a more mature asset class. The panel estimations show that the positive market returns at the high-frequency level increase price volatility, contrary to what is expected from the classical financial literature. We attributed this effect to the price dynamics over the last year of the dataset (2022) by repeating the estimation on different time spans. Moreover, the positive signed volatility and negative daily leverage positively impact the cryptocurrencies’ future volatility, unlike what emerges from the same study on a cross-section of stocks. This result signals a structural difference in a nascent cryptocurrency market that has to mature yet. Further individual-level analysis confirms the findings of the panel analysis and highlights that these effects are statistically significant and commonly shared among many components in the selected universe. ...