Research on Optimizing Real-Time Data Processing in High-Frequency Trading Algorithms using Machine Learning

ArXiv ID: 2412.01062 “View on arXiv”

Authors: Unknown

Abstract

High-frequency trading (HFT) represents a pivotal and intensely competitive domain within the financial markets. The velocity and accuracy of data processing exert a direct influence on profitability, underscoring the significance of this field. The objective of this work is to optimise the real-time processing of data in high-frequency trading algorithms. The dynamic feature selection mechanism is responsible for monitoring and analysing market data in real time through clustering and feature weight analysis, with the objective of automatically selecting the most relevant features. This process employs an adaptive feature extraction method, which enables the system to respond and adjust its feature set in a timely manner when the data input changes, thus ensuring the efficient utilisation of data. The lightweight neural networks are designed in a modular fashion, comprising fast convolutional layers and pruning techniques that facilitate the expeditious completion of data processing and output prediction. In contrast to conventional deep learning models, the neural network architecture has been specifically designed to minimise the number of parameters and computational complexity, thereby markedly reducing the inference time. The experimental results demonstrate that the model is capable of maintaining consistent performance in the context of varying market conditions, thereby illustrating its advantages in terms of processing speed and revenue enhancement.

Keywords: High-Frequency Trading (HFT), Lightweight Neural Networks, Real-time Data Processing, Dynamic Feature Selection, Pruning Techniques, Equities (High-Frequency)

Complexity vs Empirical Score

• Math Complexity: 6.5/10
• Empirical Rigor: 7.0/10
• Quadrant: Holy Grail
• Why: The paper includes substantial mathematical derivations for feature selection and neural network pruning, indicating high math complexity, and it provides empirical results on real financial data (50 ETF) with implementation details like latency and performance metrics, showing high empirical rigor.

  flowchart TD
    A["Research Goal: Optimize Real-Time Data Processing<br>in High-Frequency Trading Algorithms"] --> B["Data Input:<br>Real-Time Market Data"]
    B --> C["Methodology Step 1:<br>Dynamic Feature Selection"]
    C --> D["Computational Process:<br>Clustering & Feature Weight Analysis"]
    D --> E["Methodology Step 2:<br>Lightweight Neural Networks"]
    E --> F["Computational Process:<br>Fast Convolution & Pruning"]
    F --> G["Key Findings:<br>Reduced Inference Time &<br>Enhanced Revenue"]