Open Source Automated Sentiment Analysis using Python Polygon.io and Open.ai

Greetings r/stocks I hope this post finds you well!

This tool is still in development and requires tuning! At this point I think Chat-gpt 4.0 code interpreter has taken me as far as we can get without expanding the effort with additional minds.

https://github.com/NerdyBurner/SAStocks

This tool is meant to replicate this research from the university of Florida:

https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4412788

Can ChatGPT Forecast Stock Price Movements? Return Predictability and Large Language Models

Article explaining the research:

https://www.artisana.ai/articles/chatgpt-trading-algorithm-delivers-500-returns-in-stock-market

ChatGPT Trading Algorithm Delivers 500% Returns in Stock Market

Problem I had to solve:

The university used backend financial data, we as individuals do not have access to that data. Enter polygon.io and their incredible toolkit of financial data!

Why did I build this tool?

I have had extensive conversations with chat-gpt4. One thing is clear, the best use of its abilities is to rapidly democratize its computational ability. Yes the tool likely requires refinement but now its in your hands. If you know python you can reach out to me to discuss refinements. If you are in finance and can help with the algorithms and analysis please also contribute if desired.

Just imagine a million people using AI intelligence to successfully inform their investing. The researchers achieved 500% returns.. if we are even half as successful it will be a huge win for all involved

Targeting with Tickers.csv

I deliberately made this flexible. I'm targeting the S&P 500 but you could target whatever you want in the stock market by editing this list...

How will I use the data?

The database generated by this output will be fed into GPT 4 Code Interpreter. Expect a second post that links back to this one with the data analytics. Right now the model is running - one thing I will say is due to rate limits it takes a while..

Putting my money where my mouth is:

We determined that the model will benefit from the performance of actual holdings so I am giving $3000 of my ROTH IRA to this process. Positions will be posted tomorrow by close of business.

Application Functional Summary

This application is a sentiment analysis tool for stock market news. It uses both the Vader sentiment analysis tool from the Natural Language Toolkit (NLTK) and the OpenAI API to analyze the sentiment of news articles related to specific stock tickers.

The application pulls API keys from a CSV file named api_keys.csv where each key is stored as a row. The keys that it uses are for OpenAI and Polygon.io. Stock tickers are obtained from a CSV file named Tickers.csv.

The application saves news articles and sentiment analysis results to SQLite databases, and it also prints a final report with the aggregated sentiment scores for all stock tickers.

Required or Suggested Programs

Python 3.6 or higher is required to run this application.

You'll also need several Python libraries, including pandas, nltk, requests, sqlite3, openai, and retrying.

API keys for the following services:

OpenAI: You can get an API key by creating an account on the OpenAI website (https://www.openai.com/). The cost is $20 per month plus additional charges based on usage.

Polygon.io: You can get an API key by creating an account on the Polygon.io website (https://polygon.io/). The cost is $30 per month.

Areas for Improvement

Aggregated Scores Algorithm:

The current algorithm for calculating the aggregated score is quite simple, and it might not accurately reflect the actual sentiment of the news articles. This could be improved by using a more sophisticated sentiment scoring algorithm, perhaps one that takes into account more nuanced aspects of the news articles.

GPT Prompt:

The prompt used for GPT-3 could potentially be improved. Currently, it asks the model to categorize the sentiment of an article as 'Good', 'Bad', or 'Unknown'. This could be expanded to include more nuanced sentiments, or to ask for a more detailed analysis of the article.

Expanding the Inputs to the Sentiment Analysis:

Currently, the application only considers the title and description of each news article for sentiment analysis. This could be expanded to include other elements of the articles, such as the main body text, or even comments on the article if available.

Expand to More Data Sources:

Currently, the application only uses news articles from Polygon.io. It might be beneficial to include more data sources to get a more comprehensive view of the sentiment around each stock ticker.

Error Handling and Logging:

While the application does some error handling, it could be improved by adding more detailed logging, so that if something goes wrong, it's easier to diagnose the problem.

Code Optimization:

Some parts of the code could potentially be optimized for better performance, especially the parts that involve making requests to external APIs or querying the database.

Remember to always keep your API keys secure and never share them publicly.

submitted by /u/NerdyBurner
[link] [comments] https://www.reddit.com/r/stocks/comments/13o7r8x/open_source_automated_sentiment_analysis_using/
Created 2y | May 22, 2023, 8:21:13 AM


Login to add comment

Other posts in this group

Rate My Portfolio - r/Stocks Quarterly Thread September 2023

Please use this thread to discuss your portfolio, learn of other stock tickers, and help out users by giving constructive criticism.

Why quarterly? Public comp

Oct 21, 2023, 1:20:16 AM | reddit stocks
Tax planning when selling one stock to buy another stock?

When you sell a stock to buy another stock, do you prefer to set the estimated amount of the capital gains taxes aside in a money market or do you think it better to

Oct 18, 2023, 6:20:11 AM | reddit stocks
Retirement Planning's-3 stocks could help power your investment portfolio and make you wealthier by retirement.

Saving for retirement is crucial, but relying solely on a 401(k) might not be enough due to high inflation. Consider investing in growth stocks, especially in the tec

Oct 18, 2023, 1:51:00 AM | reddit stocks
r/Stocks Daily Discussion Monday - Oct 16, 2023

These daily discussions run from Monday to Friday including during our themed posts.

Some helpful links:

Oct 18, 2023, 1:50:58 AM | reddit stocks
Discovery Financial Services (DFS). Any good ?

I’m think this is not a good investment as there is no chatter at all on the 52 week low. They are involved in a class action lawsuits by investors and credit card co

Oct 18, 2023, 1:50:56 AM | reddit stocks
Tax implications of selling one etf for a dividend etf?

Sorry if this is the wrong sub. Let’s say I had $1 million in VOO but I wanted to sell half of it to buy SCHD. It would suck to pay taxes on $500k. So how would you g

Oct 18, 2023, 1:50:53 AM | reddit stocks
Crocs Stock Analysis (CROX)

Hey guys, I did a deep dive into Crocs. In this analysis, I will do a brief breakdown of the company and go over some quantitative data, qualitative data and estimate

Oct 18, 2023, 1:50:51 AM | reddit stocks