Paper: arXiv 2212.14689

Authors: Aditya Pandey, Haseeba Fathiya, Nivedita Patel

Abstract

This paper presents a cross-domain trend analysis that aims to identify and analyze the relationships between stock prices, stock news on Twitter, and users’ behaviors on e-commerce websites. The analysis is based on three datasets: a US stock dataset, a stock tweets dataset, and an e-commerce behavior dataset. The analysis is performed using Hadoop, Hive, and Tableau, allowing for efficient and scalable processing and visualizing large datasets. The analysis includes trend analysis of Twitter sentiment (positive and negative tweets) and correlation analysis, including the correlation between tweet sentiment and stocks, the correlation between stock trends and shopping behavior, and the understanding of data based on different slices of time. By comparing different features from the datasets over time, we hope to gain insight into the factors that drive user behavior as well as the market in different categories. The results of this analysis can provide valuable insights for businesses and investors to inform decision-making. We believe that our analysis can serve as a valuable starting point for further research and investigation into these topics.

Complexity vs Empirical Score

  • Math Complexity: 2.0/10
  • Empirical Rigor: 3.5/10
  • Quadrant: Philosophers — conceptual discussion, limited math and data

Why this score: The paper outlines an interesting multi-domain data aggregation concept using big data infrastructure (Hadoop, Hive, Tableau), but lacks mathematical rigor and rigorous statistical backtesting. Its empirical validation relies mostly on high-level correlation and visual trend analysis rather than robust quantitative modeling.

Research Flowchart

  flowchart TD
    A[Research Goal: Analyze Cross-Domain Trends Between Stocks, Twitter, and E-Commerce] --> B[Input Data: US Stocks, Stock Tweets, & Shopping Behavior]
    B --> C[Scalable Data Processing using Hadoop & Hive]
    C --> D[Methodology: Sentiment & Multi-Domain Correlation Analysis]
    D --> E[Data Visualization via Tableau]
    E --> F[Key Outcome: Insights into Market Drivers & User Behavior for Business/Investors]