{"paper_id":"2401.06179","title":"CNN-DRL for Scalable Actions in Finance","authors":["Sina Montazeri","Akram Mirzaeinia","Haseebullah Jumakhan","Amir Mirzaeinia"],"paper_date":"2024-01-10","paper_type":"empirical","paper_url":"https://arxiv.org/pdf/2401.06179v1","page_url":"https://thequant.space/flowcharts/cnn-drl-for-scalable-actions-in-finance/","scores":{"empirical_rigor":8,"math_complexity":3,"quadrant":"Street Traders"},"proposes":{"status":"available","statement":"The paper proposes a convolutional neural network (CNN) architecture as a deep reinforcement learning (DRL) agent for stock trading that can handle large, continuous action spaces up to 1000 shares.","quote":"In this work, we design a CNN architecture as a DRL agent to effectively discern salient features from financial data and achieve high rewards with wide-ranging buy/sell actions.","summary":"The published MLP-based DRL in finance has difficulties in learning the dynamics of the environment when the action scale increases. If the buying and selling increase to one thousand shares, the MLP agent will not be able to effectively adapt to the environment. To address this, we designed a CNN…","strategy":null},"assumptions":{"status":"available","items":[{"assumption":"The trading environment can be modeled as a Markov Decision Process with state represented as a matrix of the last 90 days of data.","type":"model","stated":"explicit","quote":"We have updated the environment developed by Finrl to enable the environment state representation in the form of a matrix. Our modified environment initializes the state matrix with data from the initial 90-day period of the dataset."},{"assumption":"The turbulence exclusion mechanism is not activated, i.e., the agent learns during turbulent periods.","type":"market","stated":"explicit","quote":"In contrast, we chose not to activate the turbulence exclusion and instead closely monitored and took action during this period as well as other period of time."},{"assumption":"Fixed transaction cost percentages from the FinRL framework are applied to all trades.","type":"execution","stated":"explicit","quote":"We employ identical fixed cost percentages as those developed in the FinRL framework."},{"assumption":"Reward values are scaled down to one for learning stability.","type":"model","stated":"explicit","quote":"This reward is scaled down to one as it returns from the environment to the agent to help the learning process"},{"assumption":"PPO and A2C are the chosen DRL algorithms, assumed to be appropriate for the task.","type":"model","stated":"explicit","quote":"We select PPO and A2C which are commonly used and show higher performance than other approaches."}]},"datasets":{"status":"available","items":[{"name":"Dow Jones 30 companies price data","provider":"Yahoo Finance and Wharton","kind":"market-prices","access":"public","period":"Jan 2015 - Sep 2023","universe":"30 Dow Jones constituents","quote":"We procure the stock prices of thirty companies listed in the Dow Jones from January 2015 through September 2023. This period encompasses the pandemic","known_id":"yahoo-finance"}],"known":[{"id":"yahoo-finance","name":"Yahoo Finance","provider":"Yahoo","kind":"market-prices","access":"public","quote":"This data is fetched from Yahoo Finance and Wharton [16] dataset.","mentions":1,"used_in":1}],"sample":{"start":null,"end":null,"market":null,"asset_class":null,"frequency":null},"data_resolution":"daily"},"transaction_costs":{"status":"available","modeled":"yes","headline_net_of_costs":null,"detail":"Identical fixed cost percentages as those developed in the FinRL framework are applied to all trades.","quote":"We employ identical fixed cost percentages as those developed in the FinRL framework.","source":"evidence_card","rule":"R2","candidates":{"front_matter":"unclear","evidence_card":"yes","evidence_card_quoted":true},"card_detail":"Identical fixed cost percentages as those developed in the FinRL framework are applied to all trades.","card_quote":"We employ identical fixed cost percentages as those developed in the FinRL framework.","meaning":"costs are part of the reported headline results"},"results":{"status":"available","evaluation_type":"trading-pnl","findings":[{"finding":"The CNN agent remains stable and learns effectively, achieving increased rewards compared to the MLP agent when action size is scaled to 1000 shares.","kind":"performance","quote":"... while our designed CNN remains stable, effectively learns the environment, and leads to an increase in rewards."},{"finding":"CNN achieves higher accumulated rewards and lower trading cost than MLP across both PPO and A2C algorithms.","kind":"performance","quote":"Figure 4 presents a comparison of key training statistics between the CNN and MLP agents. As shown, the CNN model achieved better accumulated rewards, while maintaining a lower cost compared to the MLP architecture."},{"finding":"CNN attains superior risk‑adjusted returns (higher Sharpe ratio) than MLP for both PPO and A2C.","kind":"performance","quote":"Figure 5 highlights the superior risk-adjusted returns (Sharpe ratio) attained by our proposed CNN across both PPO and A2C algorithms."},{"finding":"CNN learns the environment and increases its reward while the action size is increased to one thousand.","kind":"performance","quote":"Fig. 3 shows the learning reward collected by our CNN and prior MLP agents when two PPO and A2C algorithms are used. As it is demonstrated, our CNN agent is able to learn the environment and increase its reward while the action size is increased to one thousand."}],"reported_performance":null,"out_of_sample":{"method":"unclear","headline_out_of_sample":null}},"independent_evidence":{"status":"none_found","supports":[],"contradicts":[],"qualifies":[],"compared_with":49,"citations":{"cited_by_in_corpus":0,"by_independent_authors":0,"how_cited":{"used_as_baseline":0,"uses_method_or_data":0,"extends":0,"discusses_findings":0,"mentions":0,"unclassified":0},"independent_papers":[],"scope":"Citations between papers in this corpus only; citations from outside it are not counted."}},"implementation":{"status":"available","public":false,"code_url":null,"note":"No repository presented by the authors as their own was found in the paper. Absence here is not proof that no code exists."},"replication":{"status":"none_found","card":null,"note":"No replication of this paper has been run by thequant.space yet."},"extraction":{"text_source":"fulltext","model":"openai/gpt-oss-120b","version":1,"extracted_at":"2026-11-14T18:16:39Z"},"provenance":{"claims":"Everything here is what the authors state in the paper. Nothing was re-run or reproduced by the platform.","quotes":"Assumptions, datasets, cost detail and findings were extracted by a language model. Every quote is verbatim paper text. Where an item has `support`, \"full\" means every figure in it appears in the quoted sentences and \"partial\" means a figure is stated elsewhere in the paper; items with a figure found nowhere in the paper were discarded.","datasets_known":"datasets.known lists well-known vendors and public datasets by a stable id, found by a dictionary over the full text where the sentence says the data is used; it is independent of the model and covers only names in the dictionary.","flags":"transaction_costs.modeled, results.evaluation_type and results.out_of_sample.method are platform classifications (structured-decision models); see https://thequant.space/score-guide/#penalty-flags","comparisons":"supports / contradicts / qualifies are a language model's reading of two papers' quoted findings, restricted to closely related papers that share no author. They are pointers to read, not verified replications, and coverage is partial.","implementation":"code_url is a repository the paper itself presents as its own (rule-scored, ambiguous cases judged by a model), checked for liveness when it was recorded."},"docs":"https://thequant.space/agents/"}