{"paper_id":"2501.07580","title":"Assets Forecasting with Feature Engineering and Transformation Methods for LightGBM","authors":["Konstantinos-Leonidas Bisdoulis"],"paper_date":"2024-12-27","paper_type":"methodological","paper_url":"https://arxiv.org/pdf/2501.07580v1","page_url":"https://thequant.space/flowcharts/assets-forecasting-with-feature-engineering-and-transformati/","scores":{"empirical_rigor":7.5,"math_complexity":4,"quadrant":"Street Traders"},"proposes":{"status":"available","statement":"The paper proposes optimizing LightGBM for stock forecasting by introducing novel feature engineering and evaluating multiple transformation methods.","quote":"In this study we seek to optimize the performance of LightGBM, shifting away from the reliance on powerful hardware typically required for Deep Learning models.","summary":"Fluctuations in the stock market rapidly shape the economic world and consumer markets, impacting millions of individuals. Hence, accurately forecasting it is essential for mitigating risks, including those associated with inactivity. Although research shows that hybrid models of Deep Learning (DL)…","strategy":null},"assumptions":{"status":"available","items":[{"assumption":"LightGBM requires stationary input data for effective splitting.","type":"model","stated":"explicit","quote":"As LightGBM uses a tree-based algorithm, the input data needs to be stationary."},{"assumption":"An 80%-20% train-test split is used for evaluation.","type":"data","stated":"explicit","quote":"We use an 80%-20% train-test split of the dataset."},{"assumption":"Rolling train-validation split with TimeSeriesSplit avoids data leakage.","type":"execution","stated":"explicit","quote":"With the scope of robustly generalizing the model, we opt for a rolling train-validation split, using TimeSeriesSplit from scikit-learn to avoid any data leakage."},{"assumption":"Outlier handling may not significantly affect LightGBM performance.","type":"model","stated":"explicit","quote":"The proposed outlier handling method doesn’t offer much benefit for LightGBM, although testing was limited in that regard."},{"assumption":"Standardization of features is counterproductive for tree‑based models.","type":"model","stated":"explicit","quote":"The standardized variants of transformation methods slightly decreased accuracy but greatly increased training time, underscoring that standardization is counterproductive for tree‑based models like LightGBM."}]},"datasets":{"status":"available","items":[{"name":"AAPL daily prices","provider":null,"kind":"market-prices","access":"unknown","period":null,"universe":"AAPL","quote":"testing on the AAPL daily prices."}],"known":[{"id":"cboe","name":"Cboe","provider":"Cboe Global Markets","kind":"options","access":"commercial","quote":"TradingView uses data from Cboe One, which accounts for approximately 10% of the US equities market share [9].","mentions":1,"used_in":1}],"sample":{"start":null,"end":null,"market":null,"asset_class":null,"frequency":null},"data_resolution":"daily"},"transaction_costs":{"status":"not_applicable","modeled":"n/a","headline_net_of_costs":null,"detail":null,"quote":null,"source":"front_matter","rule":"R3","candidates":{"front_matter":"n/a","evidence_card":"n/a","evidence_card_quoted":false},"meaning":"the paper reports no trading or execution result"},"results":{"status":"available","evaluation_type":"forecast-accuracy","findings":[{"finding":"Log Returns, Returns and EMA Difference Ratio achieve the best overall performance, outperforming the random walk by more than 10% in every metric.","kind":"performance","quote":"Log Returns followed by Returns and then EMA Difference Ratio achieve the best performance overall, although not deviating notably from the other transformation methods."},{"finding":"Standardized transformation variants decrease accuracy and increase training time.","kind":"performance","quote":"The standardized variants of transformation methods slightly decreased accuracy but greatly increased training time, underscoring that standardization is counterproductive for tree‑based models like LightGBM."},{"finding":"Novel features, especially the overnight price gap, consistently have high feature importance across models.","kind":"predictive-accuracy","quote":"Features transformed with EMA Ratios, EMA Differenced Ratios and Returns scored particularly high in feature importance throughout the models. Of crucial importance proved the novel features, especially the overnight price gap, consistently outperforming conventional features in terms of importance."},{"finding":"All models reduce RMSE by more than 30% compared to the benchmark, mitigating large unexpected movements.","kind":"performance","quote":"In all models but the benchmark, RMSE decreased by more than 30% highlighting the models’ ability to mitigate the effects of large, unexpected movements such as crisis."}],"reported_performance":null,"out_of_sample":{"method":"unclear","headline_out_of_sample":null}},"independent_evidence":{"status":"available","supports":[],"contradicts":[],"qualifies":[],"compared_with":31,"citations":{"cited_by_in_corpus":1,"by_independent_authors":1,"how_cited":{"used_as_baseline":0,"uses_method_or_data":1,"extends":0,"discusses_findings":0,"mentions":0,"unclassified":0},"independent_papers":[{"paper":{"paper_id":"2601.22119","title":"Alpha Discovery via Grammar-Guided Learning and Search","paper_date":"2026-01-29","authors":["Han Yang","Dong Hao","Zhuohan Wang","Qi Shi","Xingtong Li"],"empirical_rigor":8,"page_url":"https://thequant.space/flowcharts/alpha-discovery-via-grammar-guided-learning-and-search/"},"how":"uses_method_or_data","quote":null}],"scope":"Citations between papers in this corpus only; citations from outside it are not counted."}},"implementation":{"status":"available","public":false,"code_url":null,"note":"No repository presented by the authors as their own was found in the paper. Absence here is not proof that no code exists."},"replication":{"status":"none_found","card":null,"note":"No replication of this paper has been run by thequant.space yet."},"extraction":{"text_source":"fulltext","model":"openai/gpt-oss-120b","version":1,"extracted_at":"2026-11-14T17:48:19Z"},"provenance":{"claims":"Everything here is what the authors state in the paper. Nothing was re-run or reproduced by the platform.","quotes":"Assumptions, datasets, cost detail and findings were extracted by a language model. Every quote is verbatim paper text. Where an item has `support`, \"full\" means every figure in it appears in the quoted sentences and \"partial\" means a figure is stated elsewhere in the paper; items with a figure found nowhere in the paper were discarded.","datasets_known":"datasets.known lists well-known vendors and public datasets by a stable id, found by a dictionary over the full text where the sentence says the data is used; it is independent of the model and covers only names in the dictionary.","flags":"transaction_costs.modeled, results.evaluation_type and results.out_of_sample.method are platform classifications (structured-decision models); see https://thequant.space/score-guide/#penalty-flags","comparisons":"supports / contradicts / qualifies are a language model's reading of two papers' quoted findings, restricted to closely related papers that share no author. They are pointers to read, not verified replications, and coverage is partial.","implementation":"code_url is a repository the paper itself presents as its own (rule-scored, ambiguous cases judged by a model), checked for liveness when it was recorded."},"docs":"https://thequant.space/agents/"}