The Impact of Social Media on Stock Market
Empirical Analysis Based on Text Mining

Analysed and predicted the stock market using big data from social media by means of web crawler technology, text mining technology, machine learning algorithms, and econometric methods. Applied natural language processing and sentiment analysis to handle unstructured social media texts and build an emotion index. Using linear regression and neural network in deep learning, it empirically studied stock market performance, continuously monitored social media's impact, and provided investment advice and forecasts based on social media fluctuations.

Documents:

• 📄 Chinese Version (PDF) | Markdown
• 📄 English Version (PDF) | Markdown

Research Methodologies

• Big Data Analysis: Web crawling and large-scale data processing
• Text Mining: Natural language processing and sentiment analysis
• Machine Learning: Classification algorithms and neural networks
• Econometric Analysis: Statistical modeling and time series analysis

Technology Stack

• Python, R, SQL
• Pandas, NumPy, Scikit-learn, TensorFlow, Keras, LSTM
• NLTK, jieba, sentiment dictionaries
• Matplotlib, Seaborn
• SPSS, Stata, VAR models

Data Sources

• Twitter API
• Yahoo Finance
• S&P 500 Index data

Key Findings

• 88% accuracy in sentiment analysis with ML algorithms
• Negative correlation between sentiment and volatility
• LSTM outperforms linear regression for stock prediction
• Sentiment shows short-term predictive power