{"paper_id":"2412.01062","title":"Research on Optimizing Real-Time Data Processing in High-Frequency Trading Algorithms using Machine Learning","authors":["Yuxin Fan","Zhuohuan Hu","Lei Fu","Yu Cheng","Liyang Wang","Yuxiang Wang"],"paper_date":"2024-12-02","paper_type":"methodological","paper_url":"https://arxiv.org/pdf/2412.01062v1","page_url":"https://thequant.space/flowcharts/research-on-optimizing-real-time-data-processing-in-high-fre/","scores":{"empirical_rigor":7,"math_complexity":6.5,"quadrant":"Holy Grail"},"proposes":{"status":"available","statement":"The paper proposes a model that optimizes real-time data processing in high‑frequency trading algorithms using dynamic feature selection and lightweight neural networks.","quote":"The objective of this work is to optimise the real-time processing of data in high-frequency trading algorithms.","summary":"High-frequency trading (HFT) represents a pivotal and intensely competitive domain within the financial markets. The velocity and accuracy of data processing exert a direct influence on profitability, underscoring the significance of this field. The objective of this work is to optimise the…","strategy":null},"assumptions":{"status":"available","items":[{"assumption":"The dynamic feature selection module uses k‑means clustering with a fixed number of clusters k=10.","type":"data","stated":"explicit","quote":"The dynamic feature selection module uses the k‑means clustering algorithm, the number of clusters k is set to 10, and the feature weights are adjusted once in each training cycle."},{"assumption":"The pruning threshold is set to 0.01 and the regularization parameter to 0.1, implying these values are appropriate for the model.","type":"model","stated":"explicit","quote":"In a lightweight neural network, the kernel sizes of the convolutional layer are 3×3 and 5×5, the pruning threshold is set to 0.01, and the regularization parameter is 0.1."}]},"datasets":{"status":"available","items":[{"name":"50ETF dataset","provider":"Shanghai Stock Exchange","kind":"market-prices","access":"unknown","period":null,"universe":"50 ETF","quote":"We selected 50 ETF datasets for model validation and evaluation. The 50ETF dataset contains the historical trading data of the 50 ETF on the Shanghai Stock Exchange, including the opening price, closing price, highest price, lowest price, trading volume and other characteristics."}],"known":[],"sample":{"start":null,"end":null,"market":null,"asset_class":null,"frequency":null},"data_resolution":"unclear"},"transaction_costs":{"status":"not_extracted","modeled":"unclear","headline_net_of_costs":null,"detail":null,"quote":null,"source":"none","rule":"R0","candidates":{"front_matter":"unclear","evidence_card":"unclear","evidence_card_quoted":false},"meaning":"the classifier was not confident"},"results":{"status":"available","evaluation_type":"unclear","findings":[{"finding":"The proposed method achieves the lowest RMSE across all parameter settings, indicating superior prediction accuracy and stability.","kind":"performance","quote":"It can be seen that the Ours method exhibits the lowest RMSE across all parameters, indicating its superiority in prediction accuracy and stability."},{"finding":"The method consistently attains high R² scores across a range of mutual information thresholds, showing stable accuracy.","kind":"predictive-accuracy","quote":"As illustrated in Figure 3, the Ours method demonstrates consistent and high R² accuracy across a range of thresholds, exhibiting stability within the error range."},{"finding":"The model maintains consistent performance under varying market conditions, enhancing processing speed and revenue.","kind":"performance","quote":"The experimental results demonstrate that the model is capable of maintaining consistent performance in the context of varying market conditions, thereby illustrating its advantages in terms of processing speed and revenue enhancement."}],"reported_performance":null,"out_of_sample":{"method":"unclear","headline_out_of_sample":null}},"independent_evidence":{"status":"none_found","supports":[],"contradicts":[],"qualifies":[],"compared_with":17,"citations":{"cited_by_in_corpus":0,"by_independent_authors":0,"how_cited":{"used_as_baseline":0,"uses_method_or_data":0,"extends":0,"discusses_findings":0,"mentions":0,"unclassified":0},"independent_papers":[],"scope":"Citations between papers in this corpus only; citations from outside it are not counted."}},"implementation":{"status":"available","public":false,"code_url":null,"note":"No repository presented by the authors as their own was found in the paper. Absence here is not proof that no code exists."},"replication":{"status":"none_found","card":null,"note":"No replication of this paper has been run by thequant.space yet."},"extraction":{"text_source":"fulltext","model":"openai/gpt-oss-120b","version":1,"extracted_at":"2026-11-14T17:51:05Z"},"provenance":{"claims":"Everything here is what the authors state in the paper. Nothing was re-run or reproduced by the platform.","quotes":"Assumptions, datasets, cost detail and findings were extracted by a language model. Every quote is verbatim paper text. Where an item has `support`, \"full\" means every figure in it appears in the quoted sentences and \"partial\" means a figure is stated elsewhere in the paper; items with a figure found nowhere in the paper were discarded.","datasets_known":"datasets.known lists well-known vendors and public datasets by a stable id, found by a dictionary over the full text where the sentence says the data is used; it is independent of the model and covers only names in the dictionary.","flags":"transaction_costs.modeled, results.evaluation_type and results.out_of_sample.method are platform classifications (structured-decision models); see https://thequant.space/score-guide/#penalty-flags","comparisons":"supports / contradicts / qualifies are a language model's reading of two papers' quoted findings, restricted to closely related papers that share no author. They are pointers to read, not verified replications, and coverage is partial.","implementation":"code_url is a repository the paper itself presents as its own (rule-scored, ambiguous cases judged by a model), checked for liveness when it was recorded."},"docs":"https://thequant.space/agents/"}