Natural Language Processing

Shai: A large language model for asset management ArXiv ID: 2312.14203 “View on arXiv” Authors: Unknown Abstract This paper introduces “Shai” a 10B level large language model specifically designed for the asset management industry, built upon an open-source foundational model. With continuous pre-training and fine-tuning using a targeted corpus, Shai demonstrates enhanced performance in tasks relevant to its domain, outperforming baseline models. Our research includes the development of an innovative evaluation framework, which integrates professional qualification exams, tailored tasks, open-ended question answering, and safety assessments, to comprehensively assess Shai’s capabilities. Furthermore, we discuss the challenges and implications of utilizing large language models like GPT-4 for performance assessment in asset management, suggesting a combination of automated evaluation and human judgment. Shai’s development, showcasing the potential and versatility of 10B-level large language models in the financial sector with significant performance and modest computational requirements, hopes to provide practical insights and methodologies to assist industry peers in their similar endeavors. ...

Narratives from GPT-derived Networks of News, and a link to Financial Markets Dislocations ArXiv ID: 2311.14419 “View on arXiv” Authors: Unknown Abstract Starting from a corpus of economic articles from The Wall Street Journal, we present a novel systematic way to analyse news content that evolves over time. We leverage on state-of-the-art natural language processing techniques (i.e. GPT3.5) to extract the most important entities of each article available, and aggregate co-occurrence of entities in a related graph at the weekly level. Network analysis techniques and fuzzy community detection are tested on the proposed set of graphs, and a framework is introduced that allows systematic but interpretable detection of topics and narratives. In parallel, we propose to consider the sentiment around main entities of an article as a more accurate proxy for the overall sentiment of such piece of text, and describe a case-study to motivate this choice. Finally, we design features that characterise the type and structure of news within each week, and map them to moments of financial markets dislocations. The latter are identified as dates with unusually high volatility across asset classes, and we find quantitative evidence that they relate to instances of high entropy in the high-dimensional space of interconnected news. This result further motivates the pursued efforts to provide a novel framework for the systematic analysis of narratives within news. ...

Potential of ChatGPT in predicting stock market trends based on Twitter Sentiment Analysis ArXiv ID: 2311.06273 “View on arXiv” Authors: Unknown Abstract The rise of ChatGPT has brought a notable shift to the AI sector, with its exceptional conversational skills and deep grasp of language. Recognizing its value across different areas, our study investigates ChatGPT’s capacity to predict stock market movements using only social media tweets and sentiment analysis. We aim to see if ChatGPT can tap into the vast sentiment data on platforms like Twitter to offer insightful predictions about stock trends. We focus on determining if a tweet has a positive, negative, or neutral effect on two big tech giants Microsoft and Google’s stock value. Our findings highlight a positive link between ChatGPT’s evaluations and the following days stock results for both tech companies. This research enriches our view on ChatGPT’s adaptability and emphasizes the growing importance of AI in shaping financial market forecasts. ...

Enhancing Financial Sentiment Analysis via Retrieval Augmented Large Language Models ArXiv ID: 2310.04027 “View on arXiv” Authors: Unknown Abstract Financial sentiment analysis is critical for valuation and investment decision-making. Traditional NLP models, however, are limited by their parameter size and the scope of their training datasets, which hampers their generalization capabilities and effectiveness in this field. Recently, Large Language Models (LLMs) pre-trained on extensive corpora have demonstrated superior performance across various NLP tasks due to their commendable zero-shot abilities. Yet, directly applying LLMs to financial sentiment analysis presents challenges: The discrepancy between the pre-training objective of LLMs and predicting the sentiment label can compromise their predictive performance. Furthermore, the succinct nature of financial news, often devoid of sufficient context, can significantly diminish the reliability of LLMs’ sentiment analysis. To address these challenges, we introduce a retrieval-augmented LLMs framework for financial sentiment analysis. This framework includes an instruction-tuned LLMs module, which ensures LLMs behave as predictors of sentiment labels, and a retrieval-augmentation module which retrieves additional context from reliable external sources. Benchmarked against traditional models and LLMs like ChatGPT and LLaMA, our approach achieves 15% to 48% performance gain in accuracy and F1 score. ...

Effects of Daily News Sentiment on Stock Price Forecasting ArXiv ID: 2308.08549 “View on arXiv” Authors: Unknown Abstract Predicting future prices of a stock is an arduous task to perform. However, incorporating additional elements can significantly improve our predictions, rather than relying solely on a stock’s historical price data to forecast its future price. Studies have demonstrated that investor sentiment, which is impacted by daily news about the company, can have a significant impact on stock price swings. There are numerous sources from which we can get this information, but they are cluttered with a lot of noise, making it difficult to accurately extract the sentiments from them. Hence the focus of our research is to design an efficient system to capture the sentiments from the news about the NITY50 stocks and investigate how much the financial news sentiment of these stocks are affecting their prices over a period of time. This paper presents a robust data collection and preprocessing framework to create a news database for a timeline of around 3.7 years, consisting of almost half a million news articles. We also capture the stock price information for this timeline and create multiple time series data, that include the sentiment scores from various sections of the article, calculated using different sentiment libraries. Based on this, we fit several LSTM models to forecast the stock prices, with and without using the sentiment scores as features and compare their performances. ...

FinGPT: Open-Source Financial Large Language Models ArXiv ID: 2306.06031 “View on arXiv” Authors: Unknown Abstract Large language models (LLMs) have shown the potential of revolutionizing natural language processing tasks in diverse domains, sparking great interest in finance. Accessing high-quality financial data is the first challenge for financial LLMs (FinLLMs). While proprietary models like BloombergGPT have taken advantage of their unique data accumulation, such privileged access calls for an open-source alternative to democratize Internet-scale financial data. In this paper, we present an open-source large language model, FinGPT, for the finance sector. Unlike proprietary models, FinGPT takes a data-centric approach, providing researchers and practitioners with accessible and transparent resources to develop their FinLLMs. We highlight the importance of an automatic data curation pipeline and the lightweight low-rank adaptation technique in building FinGPT. Furthermore, we showcase several potential applications as stepping stones for users, such as robo-advising, algorithmic trading, and low-code development. Through collaborative efforts within the open-source AI4Finance community, FinGPT aims to stimulate innovation, democratize FinLLMs, and unlock new opportunities in open finance. Two associated code repos are https://github.com/AI4Finance-Foundation/FinGPT and https://github.com/AI4Finance-Foundation/FinNLP ...

FinBERT - A Large Language Model for Extracting Information from Financial Text ArXiv ID: ssrn-3910214 “View on arXiv” Authors: Unknown Abstract We develop FinBERT, a state-of-the-art large language model that adapts to the finance domain. We show that FinBERT incorporates finance knowledge and can bette Keywords: FinBERT, Natural Language Processing, Large Language Models, Financial Text Analysis, Technology/AI Complexity vs Empirical Score Math Complexity: 2.0/10 Empirical Rigor: 8.0/10 Quadrant: Street Traders Why: The paper focuses on fine-tuning a pre-existing transformer model (FinBERT) with specific financial datasets, which is primarily an empirical, implementation-heavy task with significant data preparation and evaluation metrics, while the underlying mathematics is standard deep learning rather than novel or dense derivations. flowchart TD A["Research Goal:<br>Create domain-adapted LLM for finance"] --> B["Data:<br>Financial Documents & Corpora"] B --> C["Preprocessing:<br>Tokenization & Formatting"] C --> D["Core Methodology:<br>BERT Architecture Adaptation"] D --> E["Training:<br>Domain-specific Fine-tuning"] E --> F["Evaluation:<br>Benchmark Testing"] F --> G["Outcome:<br>FinBERT Model"] F --> H["Outcome:<br>Improved Performance vs. General LLMs"] G --> I["Final Result:<br>State-of-the-art Financial NLP"] H --> I

Textual Analysis in Accounting and Finance: A Survey ArXiv ID: ssrn-2959518 “View on arXiv” Authors: Unknown Abstract Relative to quantitative methods traditionally used in accounting and finance, textual analysis is substantially less precise. Thus, understanding the art is of Keywords: Textual Analysis, Accounting Research, Finance Research, Natural Language Processing, General (Accounting & Finance) Complexity vs Empirical Score Math Complexity: 1.0/10 Empirical Rigor: 2.0/10 Quadrant: Philosophers Why: The paper is a survey of textual analysis methods, which are conceptually oriented and less mathematically dense, and while it discusses empirical applications, it lacks the specific implementation details, code, or backtests required for high empirical rigor. flowchart TD A["Research Goal:<br>Textual Analysis in Accounting & Finance"] --> B["Data Collection"] B --> C["Preprocessing & Normalization"] C --> D["Textual Analysis Methodology"] D --> E["Statistical & Computational Processing"] E --> F["Key Findings/Outcomes"] subgraph B ["Data/Inputs"] B1["Financial Statements"] B2["Regulatory Filings"] B3["Earnings Calls"] B4["News & Social Media"] end subgraph C ["Preprocessing"] C1["Tokenization"] C2["Stopword Removal"] C3["Stemming/Lemmatization"] end subgraph D ["Methodology"] D1["Linguistic Metrics"] D2["Sentiment Analysis"] D3["Topic Modeling"] D4["Machine Learning"] end subgraph E ["Computational Processes"] E1["Feature Extraction"] E2["Statistical Inference"] E3["Model Validation"] end subgraph F ["Outcomes"] F1["Financial Prediction"] F2["Risk Assessment"] F3["Market Efficiency Insights"] end

Textual Analysis in Accounting andFinance: A Survey ArXiv ID: ssrn-2504147 “View on arXiv” Authors: Unknown Abstract Relative to quantitative methods traditionally used in accounting and finance, textual analysis is substantially less precise. Thus, understanding the art is of Keywords: Textual Analysis, Accounting Research, Finance Research, Natural Language Processing, General (Accounting & Finance) Complexity vs Empirical Score Math Complexity: 3.0/10 Empirical Rigor: 2.0/10 Quadrant: Philosophers Why: The paper is a survey of textual analysis methods, focusing on conceptual frameworks and methodological ’tripwires’ rather than advanced mathematical derivations or empirical backtesting; it emphasizes understanding the art and science of text processing without presenting new quantitative models or implementation-heavy data. flowchart TD A["Research Goal: Quantify Text in Financial Contexts"] --> B["Data Sources<br>10-Ks, Earnings Calls, News"] B --> C["Methodology<br>Preprocessing &amp; Dictionaries"] C --> D["Computational Process<br>Sentiment/Readability Scoring"] D --> E{"Outcome"} E --> F["Findings: Sentiment predicts returns/volatility"] E --> G["Findings: Readability impacts cost of capital"]