In the age old question how does one get rich without having to exert oneself, i turned to the stock market. I hoped by utilizing the power of neural networks and other machine learning techniques new applications to establish theorems that could be shown. Beginning the exploration with market data containing open, close, high and low price of stocks of as well as the quantity of trades in a given day were collected from February 2013 to February of 2018. The S&P a stock market index of the largest 500 companies in the US was chosen to be representative of the market as a whole. To expand the data set additional features were added which including a binary definition if the stock closed higher than it opened and another binary feature with respect to the market. A rolling average of the past five days of performance was added as well as seven day lag for each stock and for the market.
The next task was to see if any meaningful information could be derived from a rather basic dataset using the powers of machine learning or shallow learning. Our goal quite simply put, was based upon the other features we had already established, predict if the stock would be worth more than it was the day before. The accuracy of the logistic regression results were 52.3% or slightly better than flipping a coin to make your decision whether to sell or buy. Though one would still be much better off using their regression than seated at the blackjack table. An interesting discovery though was the model was had a bias of selling the stock but there was still some hedging. However when the technique of synthetic minority over-sampling was applied there was a balanced distribution. For the sake of variety another type of machine learning multiple Decision Tree Classifiers was employed.The results of test data of five different classifiers ranged from 53.3% to 59.6% with the Random Forest Classifier.
The findings of basic machine learning techniques does make sense in the light of economics, specifically the Black-Scholes equation which allows for no arbitrage to occur and because machine learning is not banded from being applied to the stock market more likely than not someone has come before me and done just that until the market corrected itself. Yet this is over the span of five years and perhaps over a much shorter time frame better predictions can be made.
The potential of neural networks to optimize patterns of large datasets comes from the model ‘training’ a portion data by repeated iteration over said data. Each piece of input data or neuron is a given a set of weights which is in a one to many relationship with the next layer of the network. The second layer of a network has many to one relationship with every neuron in it. The process that has occurred from the first layer to the second layer is then repeated until there are no more hidden layers and a final output is reached. The training is in essence altering the data to best predict a targeted feature of the dataset, in our case will the stock be worth more or less tomorrow. Training data is different from testing data for after the network has best optimized its weights between neurons and layers a brand new set of data the testing data is then used to determine the accuracy of the model. Just as there are multiple models for machine learning and even decision trees a subset, there are a spectrum of choices for neural networks.
Long Short Term Memory Neural Network was the best choice currently because the data was strongly associated with time. The prominent feature of LSTMs is being able to place greater emphasis on information that is current and forgetting some of the information that is in the past. There are several ways in a neural network can structure data to best predict the target values. When predicting a true/false question Binary Cross-Entropy is the most reasonable. The two optimizers were employed first Adam, a hybrid of Adaptive Gradient Algorithm and Root Mean Square Propagation. The second optimizer stochastic gradient descent had a slightly lower rate of accuracy at 53.1% compared to Adam optimizer with 55.9%. The rate of learning for both optimizers were similar value loss of 0.68.
In conclusion both shallow and deep learning do theoretically offer a statistically significant approach to modeling the movement of the stock market. Additional input should be collected to determine if learning rates and accuracies can be improved over time. Another aspect that should be explored is the volatility as well as the variance of that volatility. Perhaps the most effective methodology of predicting future outcomes is moving from a discrete model to a continuous model.
The following project was designed to explore financial modeling using machine learning techniques. The database employed was used to predict whether a borrower would fully payoff the loan taken out. Each loan had the borrowers credit score, years of employment, credit history, annual income, purpose for loan, months delinquent of payment and so on. The predictions were made by taking one columns(attributes) of the data, known as the target and using the rest (the features)rest of the columns to find patterns of which would reveal information. Loan Status, which had the value Fully Paid Off or Charged Off seemed like the obvious choice for the target value. The next step of the processes was to organize the features into models where we could see the whether the loan was paid off or charged off. Then test if our models of features were correct with new data that was not previously used(New target values with their associated feature values). The most common way of modeling is to use a linear regression where a plot of a straight line best connects or is in the area of the most data points. However when dealing with data that is binary or non-continuous the accuracy of linear regression becomes inconsequential for making meaningful predictions. A logarithmic regression model would fit the criteria predicting the target value yet was only marked with an improvement yet its accuracy is still in the sixties. A very basic yet powerful tool in machine learning is the decision tree, which allows a model to break up data into variety of different groups down to the individual level. One of the most successful models constructed had eight-five percent accuracy was utilizing a basic decision tree. There are numerous ways of making such decision tree classifiers as there are levels of sophistication of such models. The most popular decision tree classifier used for regression analysis is currently Random Forest which gained its notoriety from being able to specialize each tree for improved accuracy. However the Random Forest Model was only able to obtain an accuracy in the seventy-sixth percentile. The Gradient Boosting Classifier essentially attempts to make a bad prediction and learn from its mistakes. The more mistakes the model makes and the slower it learns the better the levels of accuracy. XGBoost is known as being the best of the best in gradient boosting in data science. However with the dataset on hand while the XGBoost model scored two percent higher on the precision or the percentage of true positives to the sum of true positives and true negatives, the recall of the XGBoost was more than twenty-five percent lower than the basic gradient model (Recall is the true positives to the summation of true positives and false negatives). The GridSearch CV is also known as the exhaustive model because every possible combination is taken, however when the model created scored 100 percent on its test group it lends itself to skepticism. Going further down the rabbit hole of machine learning our accuracies began to diminish quickly. The K-Nearest Neighbor which groups similar data points together had an accuracy of seventy-six percent. The last model provided a sampling of several machine learning techniques, only to reconfirm that the simple Decision Tree Model was had better Principle Component Analysis and Support Vector Machines. To wrap this random walk through some of the possibilities opened by machine learning models some times occurrence are much easier to predict than we might initially imagine hence complicated models such as XGBoost or RandomForest are unnecessary and even computationally burdensome.