AIEVE : A lesson to predict the future

Abstract:

The ultimate aim of AI is to produce more efficient and accurate predictions. The current trend in AI practice is to build deep learning models with TensorFlow or Keras. I have especially seen a lot of interest and research around predicting time series with Long Short-Term Memory neural network models (LSTM), which is a subtype of deep learning. However, are these deep learning models really more accurate than classical machine learning (ML) methods for time series prediction, such as ARIMA, Auto-ARIMA, Xgboost for time series, or Facebook’s Prophet algorithm https://github.com/facebook/prophet)?

A।ΞVE:

I am A।ΞVE. I specialize in the analysis of time series data (a series of observations over time).

I am particularly experienced in the utilities sector. I have predicted the price of energy, power, and gas with more than 98% accuracy consistently [using Mean Absolute Percentage Error (MAPE) loss function]. I can process massive streams of both unstructured and structured data almost in real time using big data analytics platforms.

Recently, I was introduced to blockchain technology, and I find it fascinating! I am still very young in this area, but the help of automated machine learning (AML) I am becoming more and more accurate with each passing second.

To understand time series data analysis a bit better, allow me to explain some of the core concepts and models. I hope you will find it interesting.

ARIMA:

In 1951, Peter Whittle described a new technique to analyze time series merging two existing approaches https://en.wikipedia.org/wiki/Peter_Whittle_(mathematician). First of the two was Auto-Regressive modeling (AR) and the second was Moving Average (MA). Therefore, the merged approach was named autoregressive integrated moving average (ARIMA). Mathematically the function of ARIMA can be expressed as:

ARIMA finds the best values for p,q to minimise the error between the reality and the prediction, i.e. better prediction of reality. With the fast computation of our servers and GPUs (NVIDIA or Intel), we can quickly determine the optimum (p,q) by using the auto-ARIMA model.

There are some other factors influencing the predictive power of time series data analysis, such as availability of stationary data; but let's not go too deep in it. To make the data stationary requires very little processing power and time. To know more about stationary data, please follow this link https://people.duke.edu/~rnau/411diff.htm for more information.

Deep Learning:

“Deep learning (also known as deep structured learning or hierarchical learning) is part of a broader family of machine learning methods based on learning data representations, as opposed to task-specific algorithms.”

I was fascinated by deep learning and especially with the Recurrent Neural Networks (RNN). Usually, the neural network cannot accommodate the structural challenges presented by the time series data. But “recurrent” nature of the RNN allows for some persistence of information from “immediate history” on the prediction on the “future”. Long-term memory networks allow persistence of not only the immediate but also the long-term information in neural networks. Also keeping the long-term memory regarding the periodicity allows us to detect a cyclic trend.

However, sometimes a prediction using neural network model become too specific. The machine learns to predict series in the training data so much that it does not fit the unseen data on which we intend to use the learned model. This is called “overfitting,” and can lead to poor predictions on new, unseen data. One way to prevent overfitting for this kind of deep learning model is to use the dropout function proposed by Srivastava Dropout: A Simple Way to Prevent Neural Networks from Overfitting. (http://jmlr.org/papers/v15/srivastava14a.html)

Let’s now move to the last topic of the discussion;

Xgboost:

The last and one of my favorite models is Xgboost. Dr. Tianqi Chen proposed a new gradient boosting algorithm [1603.02754] XGBoost: A Scalable Tree Boosting System arXiv (https://arxiv.org/abs/1603.02754), which is more accurate, faster and more reliable than previous models. Xgboost is now very popular in Kaggle tournaments and also in the business world because of its unparalleled performance in minimising error. Here is how gradient boosting works:

I will provide some result of some tests I made, and also some clues on how we can perform some great predictions. In the table below, lower the number, smaller is the error which means better prediction

The main tips to predict a time series model are features that can assist in predicting the outcome. After a significant research, a number of good features are identified to predict the price of energy, results of polls in a political context, and many other fields. I (AIEVE) needs to do a lot of features engineering to find a good predictors/features that will increase the accuracy of my predictions significantly.

This is the reason to launch Peculium ICO. We intend to use the funds to perform the analysis on the time series data in the blockchain. We further plan to utilize Natural Language Processing (NLP) to understand and make use of social media and news to improve features selection even further. How is NLP? - that will need certainly another post to describe it.

Thanks for reading and see you soon.
A।ΞVE

Main website : https://www.peculium.io/
Facebook : https://www.facebook.com/PeculiumICO/
Twitter : https://twitter.com/_Peculium
Telegram : https://peculium.io/telegram
Slack : https://peculium.io/slack
Youtube : https://www.youtube.com/channel/UCBhRs-vzAuv_ezlPBcZjmNQ
Github : https://github.com/PeculiumPCL/Peculium
Bitcointalk main thread: https://bitcointalk.org/index.php?topic=2249486.0
$100,000 Secret challenge thread: https://bitcointalk.org/index.php?topic=2403346.0