{"paper_id":"2510.15929","title":"Comparing LLMs for Sentiment Analysis in Financial Market News","authors":["Lucas Eduardo Pereira Teles","Carlos M. S. Figueiredo"],"paper_date":"2025-10-03","paper_type":"empirical","paper_url":"https://arxiv.org/pdf/2510.15929v1","page_url":"https://thequant.space/flowcharts/comparing-llms-for-sentiment-analysis-in-financial-market-ne/","scores":{"empirical_rigor":8,"math_complexity":3.5,"quadrant":"Street Traders"},"proposes":{"status":"available","statement":"The paper conducts a comparative study of large language models for sentiment analysis of financial market news.","quote":"This article presents a comparative study of large language models (LLMs) in the task of sentiment analysis of financial market news.","summary":"This article presents a comparative study of large language models (LLMs) in the task of sentiment analysis of financial market news. This work aims to analyze the performance difference of these models in this important natural language processing task within the context of finance. LLM models are…","strategy":null},"assumptions":{"status":"none_found","items":[]},"datasets":{"status":"available","items":[{"name":"Financial Phrase Bank (FPB)","provider":null,"kind":"text-news","access":"public","period":null,"universe":null,"quote":"Financial Phrase Bank (FPB): It contains data in both Portuguese and English (which was used). Each record is a general financial news item. In total, there are 4,840 records (but the one used in the project contains 5 more records, in this case 4,845 records)","known_id":"financial-phrasebank"},{"name":"StockEmotions","provider":null,"kind":"text-news","access":"public","period":null,"universe":null,"quote":"StockEmotions: This dataset was built with the aim of detecting emotions in the financial market. It consists of 10,000 records, of which 8,000 are for training, 1,000 are for validation, and the remaining 1,000 are for testing."},{"name":"Tweet Financial News (TFN)","provider":null,"kind":"text-news","access":"public","period":null,"universe":null,"quote":"Tweet Financial News (TFN): The dataset contains financial news headlines collected in English. It is divided into training and validation, but only the validation data was used"}],"known":[{"id":"stocktwits","name":"StockTwits","provider":"StockTwits","kind":"text-news","access":"public","quote":"The data for its creation was collected through a platform similar to Twitter called StockTwits.","mentions":1,"used_in":1},{"id":"twitter","name":"Twitter / X","provider":"X Corp.","kind":"text-news","access":"public","quote":"The data for its creation was collected through a platform similar to Twitter called StockTwits.","mentions":1,"used_in":1}],"sample":{"start":null,"end":null,"market":null,"asset_class":null,"frequency":null},"data_resolution":"unclear"},"transaction_costs":{"status":"not_applicable","modeled":"n/a","headline_net_of_costs":null,"detail":null,"quote":null,"source":"front_matter","rule":"R3","candidates":{"front_matter":"n/a","evidence_card":"n/a","evidence_card_quoted":false},"meaning":"the paper reports no trading or execution result"},"results":{"status":"available","evaluation_type":"forecast-accuracy","findings":[{"finding":"Large language models outperform classical models in most cases.","kind":"performance","quote":"Results show that large language models outperform classical models in the vast majority of cases."},{"finding":"Gemini model achieved the highest accuracy among tested models, maintaining accuracy above 70% across datasets.","kind":"predictive-accuracy","quote":"Gemini was the most consistent model. Even though it did not have the highest accuracy among the three datasets, it still achieved good results compared to the other models, maintaining an accuracy above 70%."}],"reported_performance":null,"out_of_sample":{"method":"holdout","headline_out_of_sample":null}},"independent_evidence":{"status":"none_found","supports":[],"contradicts":[],"qualifies":[],"compared_with":25,"citations":{"cited_by_in_corpus":1,"by_independent_authors":1,"how_cited":{"used_as_baseline":0,"uses_method_or_data":0,"extends":0,"discusses_findings":0,"mentions":0,"unclassified":1},"independent_papers":[],"scope":"Citations between papers in this corpus only; citations from outside it are not counted."}},"implementation":{"status":"available","public":false,"code_url":null,"note":"No repository presented by the authors as their own was found in the paper. Absence here is not proof that no code exists."},"replication":{"status":"none_found","card":null,"note":"No replication of this paper has been run by thequant.space yet."},"extraction":{"text_source":"fulltext","model":"openai/gpt-oss-120b","version":1,"extracted_at":"2026-11-14T17:25:31Z"},"provenance":{"claims":"Everything here is what the authors state in the paper. Nothing was re-run or reproduced by the platform.","quotes":"Assumptions, datasets, cost detail and findings were extracted by a language model. Every quote is verbatim paper text. Where an item has `support`, \"full\" means every figure in it appears in the quoted sentences and \"partial\" means a figure is stated elsewhere in the paper; items with a figure found nowhere in the paper were discarded.","datasets_known":"datasets.known lists well-known vendors and public datasets by a stable id, found by a dictionary over the full text where the sentence says the data is used; it is independent of the model and covers only names in the dictionary.","flags":"transaction_costs.modeled, results.evaluation_type and results.out_of_sample.method are platform classifications (structured-decision models); see https://thequant.space/score-guide/#penalty-flags","comparisons":"supports / contradicts / qualifies are a language model's reading of two papers' quoted findings, restricted to closely related papers that share no author. They are pointers to read, not verified replications, and coverage is partial.","implementation":"code_url is a repository the paper itself presents as its own (rule-scored, ambiguous cases judged by a model), checked for liveness when it was recorded."},"docs":"https://thequant.space/agents/"}