Paper: arXiv 2411.03402
Authors: Aditya Dave, Mengchen Zhu, Dapeng Hu, Sachin Tiwari
Abstract
Corporate Greenhouse Gas (GHG) emission targets are important metrics in sustainable investing [“12, 16”]. To provide a comprehensive view of company emission objectives, we propose an approach to source these metrics from company public disclosures. Without automation, curating these metrics manually is a labor-intensive process that requires combing through lengthy corporate sustainability disclosures that often do not follow a standard format. Furthermore, the resulting dataset needs to be validated thoroughly by Subject Matter Experts (SMEs), further lengthening the time-to-market. We introduce the Climate Artificial Intelligence for Corporate Decarbonization Metrics Extraction (CAI) model and pipeline, a novel approach utilizing Large Language Models (LLMs) to extract and validate linked metrics from corporate disclosures. We demonstrate that the process improves data collection efficiency and accuracy by automating data curation, validation, and metric scoring from public corporate disclosures. We further show that our results are agnostic to the choice of LLMs. This framework can be applied broadly to information extraction from textual data.
Complexity vs Empirical Score
- Math Complexity: 3.0/10
- Empirical Rigor: 7.5/10
- Quadrant: Street Traders — empirical and applied, lower math complexity
Why this score: The paper focuses on practical NLP pipeline implementation and business process automation rather than theoretical finance, with some reported accuracy metrics and model comparison, placing it in the Street Traders quadrant.
Research Flowchart
flowchart TD
A["<b>Research Goal</b><br>Automate extraction of corporate<br>decarbonization metrics from disclosures"] --> B["<b>Data Input</b><br>Corporate Sustainability<br>Disclosures (Text)"]
B --> C["<b>LLM Pipeline</b><br>Climate AI CAI Model"]
C --> D["<b>Extraction Process</b><br>1. Locate emission targets<br>2. Extract linked metrics<br>3. Validate data"]
D --> E["<b>Validation</b><br>Subject Matter Expert<br>SME Review & Comparison"]
E --> F{"<b>Key Findings</b>"}
F --> G["<b>Outcome 1</b><br>Improved data collection<br>efficiency & accuracy"]
F --> H["<b>Outcome 2</b><br>Model agnostic to<br>choice of LLM"]
F --> I["<b>Outcome 3</b><br>Scalable framework for<br>textual information extraction"]
Evidence Card
What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card
Proposes
The paper proposes the Climate Artificial Intelligence for Corporate Decarbonization Metrics Extraction (CAI) model and pipeline, a novel LLM-based approach to extract and validate corporate emission metrics from public disclosures.
Source sentence
We introduce the Climate Artificial Intelligence for Corporate Decarbonization Metrics Extraction (CAI) model and pipeline, a novel approach utilizing Large Language Models (LLMs) to extract and validate linked metrics from corporate disclosures.
Assumes 3
- A context chunk size of 80 words provides the best recall for metric extraction. model
Source sentence
The optimal word count was found to be 80 with an overlap window of 20, which yielded the highest recall of 95% against 43 examples.
- The benchmark dataset (e.g., SBTi commitments) used for validation is accurate and reliable. data
Source sentence
The validation of the CAI model’s accuracy is conducted by aligning its outputs with a verified benchmark set, which is either assembled by SMEs or obtained from an external source such as SBTi.
- LLM choice does not affect the overall extraction performance. model
Source sentence
We further show that our results are agnostic to the choice of LLMs.
Data
- SBTi commitments benchmark public
Source sentence
For benchmarking, SBTi commitments for 35 companies are compiled, along with the reports containing the targets reported by SBTi as shown in Table 1.
Costs
Not applicable
Finds 5
- On the training set, the CAI model achieved 95% accuracy, 96% recall, and 90% precision. performance
Source sentence
The reconciliation of the training set yielded an accuracy of 95%, a recall of 96%, and a precision of 90%.
- On the validation set, the CAI model achieved perfect scores of 100% accuracy, recall, and precision. performance
Source sentence
The test set reconciliation, on the other hand, achieved a perfect score wit…
- Using a context chunk size of 80 words gave the highest recall of 95% across 43 examples. statistical
Source sentence
The optimal word count was found to be 80 with an overlap window of 20, which yielded the highest recall of 95% against 43 examples.
- Azure Open AI GPT-4 (gpt-4-0613) achieved the highest total recall of 97% among compared Gen AI models. performance
Source sentence
Azure Open AI GPT-4 (gpt-4-0613) 97% 100% 58%
- The CAI pipeline can identify over 95% of corporate carbon reduction commitments, meeting the primary metric of total recall. performance
Source sentence
The primary metric of focus is total recall, aiming for the CAI model and pipeline to identify over 95% of the commitments within a…
Checked by others
No other paper in the archive has been found to support or contradict these findings (compared with 24 related papers). Coverage is partial, so this is neither replication nor refutation.
Code
No repository by the authors found in the paper.
Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.
Cited References (35)
- Lost in the Middle: How Language Models Use Long Contexts (2023) — link
- Managing carbon aspirations: The influence of corporate climate change targets on environmental performance (2019)
- Aligning corporate greenhouse-gas emissions targets with climate goals (2015)
- How to Prompt? Opportunities and Challenges of Zero- and Few-Shot Learning for Human-AI Interaction in Creative Applications of Generative Models (2022) — link
- Glitter or Gold? Deriving Structured Insights from Sustainability Reports via Large Language Models (2024) — link
- ClimateBERT-NetZero: Detecting and Assessing Net Zero and Reduction Targets (2023) — link
- Pareto Optimal Learning for Estimating Large Language Model Errors (2024) — link
- Selective Annotation Makes Language Models Better Few-Shot Learners (2022) — link
and 27 more
- System 2 Attention (is something you might need too) (2023) — link
- Retrieval-Augmented Generation for Large Language Models: A Survey (2024) — link
- RoBERTa: A Robustly Optimized BERT Pretraining Approach (2019) — link
- ESGReveal: An LLM-based approach for extracting structured data from ESG reports (2023) — link
- Hallucination is Inevitable: An Innate Limitation of Large Language Models (2024) — link
- The Mann-Whitney U: A Test for Assessing Whether Two Independent Samples Come from the Same Distribution (2008) — link
- Parent Document Retriever (2023) — link
- The Transformer model family (2023) — link
- Autoregressive models (2023) — link
- What Are Autoregressive Models? How They Work and Example (2022) — link
- SBTi CORPORATE NEAR-TERM CRITERIA (2024) — link
- SBTi Criteria and Recommendations (2021) — link
- An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuning (2024) — link
- Attention Is All You Need (2023) — link
- Methodology: Developing an investible universe of climate adaptation and resilience companies (2023) — link
- ChatClimate (2023) — link
- PaLM 2 Technical Report (2023) — link
- ClimateQ&A: Bridging the gap between climate scientists and the general public (2024) — link
- Learning to Extract Structured Entities Using Language Models (2024) — link
- Information extraction meets the semantic web: a survey. Semantic Web (2020) — link
- ClimateBert: A Pretrained Language Model for Climate-Related Text (2022) — link
- Assessing the quality of information extraction (2024) — link
- Project Gaia: Enabling climate risk analysis using generative AI (2024) — link
- Limitations of information extraction methods and techniques forheterogeneous unstructured big data (2019) — link
- Challenges in information extraction from text for knowledge management (2001)
- Systematic Literature Review of Information Extraction From Textual Data: Recent Methods, Applications, Trends, and Challenges (2023) — link
- CHATREPORT: Democratizing Sustainability Disclosure Analysis through LLM-based Tools (2023) — link
0 of 35 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2596 linked papers.