Paper: arXiv 2412.10906
Authors: Qilong Wu, Xiaoneng Xiang, Hejia Huang, Xuan Wang, Yeo Wei Jie, Ranjan Satapathy, Ricardo Shirota Filho, Bharadwaj Veeravalli
Abstract
The rapid growth of the financial sector and the rising focus on Environmental, Social, and Governance (ESG) considerations highlight the need for advanced NLP tools. However, open-source LLMs proficient in both finance and ESG domains remain scarce. To address this gap, we introduce SusGen-30K, a category-balanced dataset comprising seven financial NLP tasks and ESG report generation, and propose TCFD-Bench, a benchmark for evaluating sustainability report generation. Leveraging this dataset, we developed SusGen-GPT, a suite of models achieving state-of-the-art performance across six adapted and two off-the-shelf tasks, trailing GPT-4 by only 2% despite using 7-8B parameters compared to GPT-4’s 1,700B. Based on this, we propose the SusGen system, integrated with Retrieval-Augmented Generation (RAG), to assist in sustainability report generation. This work demonstrates the efficiency of our approach, advancing research in finance and ESG.
Complexity vs Empirical Score
- Math Complexity: 3.0/10
- Empirical Rigor: 8.5/10
- Quadrant: Street Traders — empirical and applied, lower math complexity
Research Flowchart
flowchart TD A["Research Goal: Develop open-source LLM for finance & ESG"] --> B["Methodology: Dataset Creation & Model Training"] B --> C["Data/Inputs: SusGen-30K dataset"] C --> D["Computational Process: SusGen-GPT 7-8B training"] D --> E["Findings: State-of-the-art performance"] E --> F["Outcome: SusGen System w/ RAG for report generation"]
Evidence Card
What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card
Proposes
The paper introduces the SusGen-30K dataset, the TCFD-Bench benchmark, and a suite of fine‑tuned LLMs called SusGen‑GPT for financial and ESG NLP tasks.
Source sentence
In total, our contributions include the following: 1. SusGen-30K, a large-scale high quality dataset in both financial & ESG domain. 2. We propose and release a well-curated benchmark, tailored towards ESG report generation, TCFD-Bench. 3. A suite of fine-tuned LLMs, SusGen-GPT, shown to achieve comparable state-of-the
Assumes
No explicit assumptions found in the text.
Data
- SusGen-30K text-news public
Source sentence
The dataset is originally sourced from two primary sources: open-source datasets available on Hugging Face2 and annual reports sourced from TCFDHub3 Database.
- TCFD-Bench benchmark public
Source sentence
We propose a new benchmark, TCFD-Bench, specifically designed to assess models’ ability to generate concise and accurate ESG reports from annual reports.
Costs
Not applicable
Finds 2
- SusGen‑GPT achieves state‑of‑the‑art performance across six adapted and two off‑the‑shelf tasks, trailing GPT‑4 by only 2% while using only 7‑8B parameters versus GPT‑4’s 1,700B. performance
Source sentence
... trailing GPT‑4 by only 2% despite using 7-8B parameters compared to GPT‑4's 1,700B.
- Increasing the dataset size consistently improves performance across all eight financial NLP tasks. statistical
Source sentence
Overall, the results show a clear trend: increasing the dataset size consistently improves performance across all tasks.
Checked by others
No other paper in the archive has been found to support or contradict these findings (compared with 16 related papers). Coverage is partial, so this is neither replication nor refutation.
Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.
Cited References (274)
- The Theory of Parsing, Translation and Compiling (1972)
- Publications Manual (1983)
- Alternation (1981) — link
- Scalable training of L1 -regularized log-linear models (2007)
- Algorithms on Strings, Trees and Sequences (1997)
- Yara Parser: A Fast and Accurate Dependency Parser (2015) — link
- GPT-4 Technical Report (2023) — link
- Sparks of artificial general intelligence: Early experiments with gpt-4 (2023) — link
and 266 more
- Larger language models do in-context learning differently (2023) — link
- The Wall Street Neophyte: A Zero-Shot Analysis of ChatGPT Over MultiModal Stock Movement Prediction Challenges — → 2304.05351
- Can chatgpt forecast stock price movements? return predictability and large language models — → 2304.07619
- Are ChatGPT and GPT-4 General-Purpose Solvers for Financial Text Analytics? An Examination on Several Typical Tasks (2023) — link
- Holistic evaluation of language models (2022) — link
- Measuring massive multitask language understanding (2020) — link
- Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models (2023)
- BBT-Fin: Comprehensive Construction of Chinese Financial Domain Pre-trained Language Model, Corpus and Benchmark (2023) — link
- Transformers: State-of-the-Art Natural Language Processing (2020) — link
- Memory Sandbox: Transparent and Interactive Memory Management for Conversational Agents (2023) — link
- CodeHelp: Using Large Language Models with Guardrails for Scalable Support in Programming Classes (2023) — link
- Social Simulacra: Creating Populated Prototypes for Social Computing Systems (2022) — link
- Cognitive Architectures for Language Agents (2023) — link
- A Survey on Large Language Model based Autonomous Agents (2023) — link
- Improving Language Understanding by Generative Pre-Training (2018) — link
- RecAgent: A Novel Simulation Paradigm for Recommender Systems (2023) — link
- FinGPT: Instruction Tuning Benchmark for Open-Source Large Language Models in Financial Datasets — → 2310.04793
- An Information-theoretic Approach to Prompt Engineering Without Ground Truth Labels (2022) — link
- FinMem: A Performance-Enhanced LLM Trading Agent with Layered Memory and Character Design — → 2311.13743
- The Cattell-Horn-Carroll model of intelligence. (2012)
- CHC theory and the human cognitive abilities project: Standing on the shoulders of the giants of psychometric intelligence research (2009)
- Good Debt or Bad Debt: Detecting Semantic Orientations in Economic Texts (2013) — link
- Instruction Tuning for Large Language Models: A Survey (2023) — link
- InvestLM: A Large Language Model for Investment using Financial Domain Instruction Tuning — → 2309.13064
- XuanYuan 2.0: A Large Chinese Financial Chat Model with Hundreds of Billions Parameters (2023) — link
- F in BART : A Pre-trained Seq2seq Language Model for C hinese Financial Tasks (2023) — link
- CGCE: A Chinese Generative Chat Evaluation Benchmark for General and Financial Domains (2023) — link
- CFBenchmark: Chinese Financial Assistant Benchmark for Large Language Model (2023) — link
- DISC-FinLLM: A Chinese Financial Large Language Model based on Multiple Experts Fine-tuning (2023) — link
- Large Language Models in Finance: A Survey — → 2311.10723
- FinBERT: Financial Sentiment Analysis with Pre-trained Language Models (2019) — link
- FinBERT: A Pretrained Language Model for Financial Communications (2020) — link
- FinBERT: A Pre-trained Financial Language Representation Model for Financial Text Mining (2020) — link
- Future climate change significantly alters interannual wheat yield variability over half of harvested areas (2021) — link
- ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators (2020) — link
- A Survey of Large Language Models in Finance (FinLLMs) (2024) — link
- WWW'18 Open Challenge: Financial Opinion Mining and Question Answering (2018) — link
- S em E val-2017 Task 5: Fine-Grained Sentiment Analysis on Financial Microblogs and News (2017) — link
- Impact of News on the Commodity Market: Dataset and Results (2020) — link
- Domain Adaption of Named Entity Recognition to Support Credit Risk Assessment (2015) — link
- F i NER : Financial Numeric Entity Recognition for XBRL Tagging (2022) — link
- FinRED: A Dataset for Relation Extraction in Financial Domain (2023) — link
- The Financial Document Causality Detection Shared Task ( F in C ausal 2020) (2020) — link
- FinQA: A Dataset of Numerical Reasoning over Financial Data (2022) — link
- ConvFinQA: Exploring the Chain of Numerical Reasoning in Conversational Finance Question Answering (2022) — link
- MAEC: A Multimodal Aligned Earnings Conference Call Dataset for Financial Risk Prediction (2020) — link
- Multilingual and Cross-Lingual Intent Detection from Spoken Data (2021) — link
- M ulti F in: A Dataset for Multilingual Financial NLP (2023) — link
- Global Table Extractor (GTE): A Framework for Joint Table Identification and Cell Structure Recognition Using Visual Context (2021)
- Trade the Event: Corporate Events Detection for News-Based Event-Driven Trading (2021) — link
- StockEmotions: Discover Investor Emotions for Financial Sentiment Analysis and Multivariate Time Series — → 2301.09279
- CFGPT: Chinese Financial Assistant with Large Language Model (2023) — link
- MONOPOLY: Financial Prediction from MONetary POLicY Conference Videos Using Multimodal Cues (2022) — link
- Gemini: a family of highly capable multimodal models (2023) — link
- GLM: General Language Model Pretraining with Autoregressive Blank Infilling (2022)
- Qwen Technical Report (2023) — link
- Baichuan 2: Open Large-scale Language Models (2023) — link
- Internlm: A multilingual language model with progressively enhanced capabilities (2023)
- The falcon series of open language models (2023) — link
- Mixtral of experts (2024) — link
- Code llama: Open foundation models for code (2023) — link
- Select and Trade: Towards Unified Pair Trading with Hierarchical Reinforcement Learning — → 2301.10724
- Finetuned language models are zero-shot learners (2021) — link
- Bloom: A 176b-parameter open-access multilingual language model (2022) — link
- Multitask Prompted Training Enables Zero-Shot Task Generalization (2022)
- The flan collection: Designing data and methods for effective instruction tuning (2023) — link
- Mengzi: Towards lightweight yet ingenious pre-trained models for chinese (2021) — link
- BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding (2019)
- Good debt or bad debt: Detecting semantic orientations in economic texts (2014)
- Impact of news on the commodity market: Dataset and results (2021)
- Domain adaption of named entity recognition to support credit risk assessment (2015)
- Www'18 open challenge: financial opinion mining and question answering (2018)
- Self-Instruct: Aligning Language Model with Self Generated Instructions (2022) — link
- Vicuna: An Open-Source Chatbot Impressing GPT-4 with 90%* ChatGPT Quality (2023) — link
- FinQA: A Dataset of Numerical Reasoning over Financial Data (2021)
- Decoupled weight decay regularization (2017) — link
- GPT-NeoX-20B : An Open-Source Autoregressive Language Model (2022) — link
- Opt: Open pre-trained transformer language models (2022) — link
- Solving quantitative reasoning problems with language models (2022) — link
- Training verifiers to solve math word problems (2021) — link
- Accurate Stock Movement Prediction with Self-supervised Learning from Sparse Noisy Tweets (2022)
- Stock movement prediction from tweets and historical prices (2018)
- Hybrid deep sequential modeling for social text-driven stock prediction (2018)
- BELLE: Be Everyone’s Large Language model Engine (2023)
- Exploring the Impact of Instruction Data Scaling on Large Language Models: An Empirical Study on Real-World Use Cases (2023) — link
- Select and Trade: Towards Unified Pair Trading with Hierarchical Reinforcement Learning — → 2301.10724
- Enhancing stock movement prediction with adversarial training (2018) — link
- Accurate multivariate stock movement prediction via data-axis transformer with multi-level contexts (2021)
- Semeval-2017 task 5: Fine-grained sentiment analysis on financial microblogs and news (2017)
- FinanceBench: A New Benchmark for Financial Question Answering (2023) — link
- Bizbench: A quantitative reasoning benchmark for business and finance (2023) — link
- CALM-Bench: A Multi-task Benchmark for Evaluating Causality-Aware Language Models (2023)
- Disc-finllm: A chinese financial large language model based on multiple experts fine-tuning (2023) — link
- Fin-Eva Version 1.0 (2023)
- Trillion Dollar Words: A New Financial Dataset, Task & Market Analysis — → 2305.07972
- Fine-Grained Argument Understanding with BERT Ensemble Techniques: A Deep Dive into Financial Sentiment Analysis (2023)
- MultiFin: A Dataset for Multilingual Financial NLP (2023)
- Generating plausible counterfactual explanations for deep transformers in financial text classification (2020) — link
- Multi-Lingual ESG Issue Identification (2023)
- Finer: Financial named entity recognition dataset and weak-supervision model (2023) — link
- FinRED: A dataset for relation extraction in financial domain (2022)
- Financial document causality detection shared task (fincausal 2020) (2020) — link
- A probabilistic interpretation of precision, recall and F-score, with implication for evaluation (2005)
- Complementarity, F -score, and NLP Evaluation (2016) — link
- TAT-QA: A question answering benchmark on a hybrid of tabular and textual content in finance (2021) — link
- Financial Numeric Extreme Labelling: A dataset and benchmarking (2023)
- Textual analogy parsing: What’s shared and what’s compared among analogous facts (2018) — link
- How are We Detecting Inconsistent Method Names? An Empirical Study from Code Review Perspective (2023) — link
- Ectsum: A new benchmark dataset for bullet point summarization of long earnings call transcripts (2022) — link
- Rouge: A package for automatic evaluation of summaries (2004)
- Bertscore: Evaluating text generation with bert (2019) — link
- Bartscore: Evaluating generated text as text generation (2021)
- Statlog (German Credit Data) (1994)
- Statlog (Australian Credit Approval)
- Empowering many, biasing a few: Generalist credit scoring through large language models (2023) — link
- The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation (2020)
- Strategic management decision-making in a complex world: quantifying, understanding, and using trade-offs (2017)
- A monthly effect in stock returns (1987)
- The sharpe ratio (1998)
- Forecasting the equity premium: Do deep neural network models work? (2023)
- Tracking the fin trade: genetic stock identification in western Atlantic scalloped hammerhead sharks Sphyrna lewini (2009)
- Maximum drawdown (2004)
- Fingpt: Democratizing internet-scale data for financial large language models (2023) — link
- Dynamic Datasets and Market Environments for Financial Reinforcement Learning (2023) — link
- FinRL-Meta: Market Environments and Benchmarks for Data-Driven Financial Reinforcement Learning (2022) — link
- D'olares or Dollars? Unraveling the Bilingual Prowess of Financial LLMs Between Spanish and English (2024) — link
- LAiW: A Chinese Legal Large Language Models Benchmark (2024) — link
- Back to the Future: Towards Explainable Temporal Reasoning with Large Language Models (2023) — link
- Empowering Many, Biasing a Few: Generalist Credit Scoring through Large Language Models (2024) — link
- Mastering Pair Trading with Risk-Aware Recurrent Reinforcement Learning — → 2304.00364
- Qlora: Efficient finetuning of quantized llms (2024)
- Decoupled Weight Decay Regularization (2019) — link
- PaLM 2 Technical Report (2023) — link
- The Choice of Textual Knowledge Base in Automated Claim Checking (2023) — link
- chatClimate: Grounding Conversational AI in Climate Science (2023) — link
- Aggregate confusion: The divergence of ESG ratings (2022)
- Towards Continual Knowledge Learning of Language Models (2022)
- On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? (2021) — link
- It’s Not Just Size That Matters: Small Language Models Are Also Few-Shot Learners (2021) — link
- Large Language Models are Zero-Shot Reasoners (2023) — link
- Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigm (2021) — link
- A Survey on Automated Fact-Checking (2022) — link
- The Choice of Textual Knowledge Base in Automated Claim Checking (2023) — link
- Fact Checking: Task definition and dataset construction (2014) — link
- Attention is All you Need (2017) — link
- Ethical and social risks of harm from Language Models (2021) — link
- Improving language models by retrieving from trillions of tokens (2022) — link
- Towards a framework for evaluation and design of conversational agents (2016)
- The impact of chatbot characteristics on user satisfaction and conversational performance (2020)
- Climate-fever: A dataset for verification of real-world climate claims (2020) — link
- A Framework for Understanding and Evaluating Automated Systems (2020)
- In bot we trust: A new methodology of chatbot performance measures (2019) — link
- ChatGPT: A Large-scale Generative Language Model for Conversational AI (2022)
- SAINT $\mathplus$: Integrating Temporal Features for EdNet Correctness Prediction (2021) — link
- Towards Healthy AI: Large Language Models Need Therapists Too (2023) — link
- A Bibliometric Review of Large Language Models Research from 2017 to 2023 (2023) — link
- Towards a Human-like Open-Domain Chatbot (2020) — link
- ClimateBert: A Pretrained Language Model for Climate-Related Text (2022) — link
- Machine-learning-based evidence and attribution mapping of 100,000 climate impact studies (2021) — link
- Analyzing Sustainability Reports Using Natural Language Processing (2020) — link
- Low-Resource Adaptation of Open-Domain Generative Chatbots (2022) — link
- DERA: Enhancing Large Language Model Completions with Dialog-Enabled Resolving Agents (2023) — link
- HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in HuggingFace (2023) — link
- BloombergGPT: A Large Language Model for Finance (2023) — link
- CFBenchmark: Chinese Financial Assistant Benchmark for Large Language Model (2024) — link
- Enhancing Large Language Models with Climate Resources (2023) — link
- The importance of accurate and up-to-date information in the context of climate change (2020)
- Climate change and cities: challenges ahead (2021)
- Climate Change 2022: Mitigation of Climate Change (2022)
- Climate Change 2022: Impacts, Adaptation, and Vulnerability (2022)
- Climate Change 2021: The Physical Science Basis (2021)
- Survey of Hallucination in Natural Language Generation (2023) — link
- BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding (2019) — link
- Prompting Large Language Models with Answer Heuristics for Knowledge-based Visual Question Answering (2023) — link
- Leveraging Large Language Models for Multiple Choice Question Answering (2023) — link
- Self-Prompting Large Language Models for Open-Domain QA (2022) — link
- Plug-and-Play Adaptation for Continuously-updated QA (2022) — link
- Exploiting cloze questions for few shot text classification and natural language inference (2020) — link
- Visual chatGPT : Talking, drawing and editing with visual foundation models (2023) — link
- How good are GPT models at machine translation? A comprehensive evaluation (2023) — link
- Progressive-hint prompting improves reasoning in large language models (2023) — link
- Chain of thought prompting elicits reasoning in large language models (2022) — link
- Making pre-trained language models better few-shot learners (2020) — link
- Toolformer: Language models can teach themselves to use tools (2023) — link
- PIXIU: A Large Language Model, Instruction Data and Evaluation Benchmark for Finance (2023) — link
- Mistral 7B (2023) — link
- The Llama 3 Herd of Models (2024) — link
- Dictionary-based phrase-level prompting of large language models for machine translation (2023) — link
- Large language models are human-level prompt engineers (2022) — link
- Multitask prompted training enables zero-shot task generalization (2021) — link
- Cheap talk in corporate climate commitments: The role of active institutional ownership, signaling, materiality, and sentiment (2022)
- ClimateBERT: A Pretrained Language Model for Climate-Related Text (2022)
- Environmental Claim Detection (2023)
- Cheap talk and cherry-picking: What ClimateBert has to say on corporate climate risk disclosures (2022)
- Towards Climate Awareness in NLP Research (2022) — link
- Language models are few-shot learners (2020)
- Language Models are Unsupervised Multitask Learners (2019)
- Sustainable development through climate action (2019) — link
- Quantifying the human cost of global warming (2023)
- Toward a new understanding of environmental and financial performance through corporate social responsibility, green innovation, and sustainable development (2023)
- A sustainable development pathway for climate action within the UN 2030 Agenda (2021) — link
- Training language models to follow instructions with human feedback (2022)
- The Promise and Peril of Generative AI (2023)
- Attention is all you need (2017)
- Llama: Open and efficient foundation language models (2023) — link
- Quantifying the economic risks of climate change (2017)
- GPT-4 Technical Report (2023) — link
- Recommendations of the Task Force on Climate-Related Financial Disclosures (2017)
- Implementing the Recommendations of the TCFD (2021)
- Climate Change and Financial Stability (2021)
- The role of CO2 capture and utilization in mitigating climate change (2017)
- Task force on climate-related financial disclosures: Overview (2017)
- Machine-learning-based evidence and attribution mapping of 100,000 climate impact studies (2021)
- InstructGPT: AI for Generating Instructions (2023)
- RoBERTa: A Robustly Optimized BERT Pretraining Approach (2019) — link
- Dangers of AI: LLMs, Inaccuracy, and the Need for Responsibility (2021) — link
- Global Warming of 1.5°C. An IPCC Special Report on the impacts of global warming of 1.5°C above pre-industrial levels and related global greenhouse gas emission pathways, in the context of strengthening the global response to the threat of climate change (2018)
- Global Warming: The Complete Briefing (2015)
- The Economics of Climate Change: The Stern Review (2007)
- Uncertainty estimates in regional and global observed temperature changes: A new data set from 1850 (2006)
- Training a single AI model can emit as much carbon as five cars in their lifetimes (2019)
- Lora: Low-rank adaptation of large language models (2021) — link
- LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale (2022) — link
- 8-bit Optimizers via Block-wise Quantization (2022)
- Pause Giant AI Experiments: An Open Letter (2023)
- The real costs of financial efficiency when some information is soft (2016)
- Palm: Scaling language modeling with pathways (2022) — link
- TaskMatrix.AI: Completing Tasks by Connecting Foundation Models with Millions of APIs (2023) — link
- Ask BERT : How regulatory disclosure of transition and physical climate risks affects the CDS term structure (2020)
- Vicuna: An open-source chatbot impressing GPT -4 with 90%* chatgpt quality (2023)
- Instruction tuning with GPT -4 (2023) — link
- Large Language Models are not Fair Evaluators (2023) — link
- Lima: Less is more for alignment (2023) — link
- Automatic Evaluation of Attribution by Large Language Models (2023) — link
- Evaluating Verifiability in Generative Search Engines (2023) — link
- Check Your Facts and Try Again: Improving Large Language Models with External Knowledge and Automated Feedback (2023) — link
- Self- Checker : Plug -and- Play Modules for Fact - Checking with Large Language Models (2023) — link
- Judging LLM -as-a-judge with MT - Bench and Chatbot Arena (2023) — link
- Active Learning with Feedback on Both Features and Instances (2006)
- Active Learning for Graph Neural Networks via Node Feature Propagation (2021) — link
- Interscript: A dataset for interactive learning of scripts through error feedback (2021) — link
- Learning to repair: Repairing model output errors after deployment using a dynamic memory of feedback (2022) — link
- Large Language Models Are State -of-the- Art Evaluators of Translation Quality (2023) — link
- How is ChatGPT’s behavior changing over time? (2023) — link
- Llama 2: Open Foundation and Fine-Tuned Chat Models (2023) — link
- Stanford Alpaca: An Instruction-following LLaMA model (2023)
- WHEN FLUE MEETS FLANG: Benchmarks and Large Pre-trained Language Model for Financial Domain (2022) — link
- FinGPT: Open-Source Financial Large Language Models — → 2306.06031
- Instruct-FinGPT: Financial Sentiment Analysis by Instruction Tuning of General-Purpose Large Language Models — → 2306.12659
- Enhancing Financial Sentiment Analysis via Retrieval Augmented Large Language Models — → 2310.04027
- FinGPT: Instruction Tuning Benchmark for Open-Source Large Language Models in Financial Datasets — → 2310.04793
- Data-centric FinGPT: Democratizing Internet-scale Data for Financial Large Language Models (2023)
- The FinBen: An Holistic Financial Benchmark for Large Language Models (2024) — link
- Common 7B Language Models Already Possess Strong Math Capabilities (2024) — link
- RAFT: Adapting Language Model to Domain Specific RAG (2024) — link
- Scaling Laws for Neural Language Models (2020) — link
- Training Verifiers to Solve Math Word Problems (2021) — link
- Self-training Large Language Models through Knowledge Detection (2024) — link
- Lora learns less and forgets less (2024) — link
- Evaluating Topic Model on Asymmetric and Multi-Domain Financial Corpus (2024) — link
- Large Language Models as Financial Data Annotators: A Study on Effectiveness and Efficiency (2024) — link
- CHATREPORT : Democratizing Sustainability Disclosure Analysis through LLM -based Tools (2023) — link
- Glitter or gold? Deriving structured insights from sustainability reports via large language models (2024)
- ESGReveal: An LLM-based approach for extracting structured data from ESG reports (2023) — link
- METEOR: An automatic metric for MT evaluation with improved correlation with human judgments (2005)
- Bleu: a method for automatic evaluation of machine translation (2002)
15 of 274 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2596 linked papers.