The example uses tensorflow 1.0.0:
- our RNN graph is put together in
- with configurable parameters extracted to config yaml files.
- the config.yaml also defines where to read the data, label, etc.
- a few example configuration is stored in the
trainer.pytakes care of reading all config files, train on 80% of the data, and try to predict the last 20%
- the prediction are saved to
You are free to fork and test out different configs, different data series, etc. Sharing is caring!
Once again, the task is to predict the last 20% of a series, by looking only at the first 80%.
I’ve generated a simple series using modulus, and combination of
sin on this spreadsheet. It looks like below:
While we can only train on the first 80% of the data, we can run many many iterations (epochs) over that training data and continue to tune model weights, and improving the cost function. The number of epochs is also configurable in the yaml file.
The cost function here, for simplicity, is Mean Squared Error (MSE).
What params can be configured?
Configuration yaml’s in
configs/ directory can include a number of variables, as explained below:
In this post, though, we’ll take a look only at a few basic parameter tuning.
The following section illustrates how varying the RNN architecture, the
rnn_layer variable in the yaml (which is basically how many cells per layer, how many layer, etc.) affect the prediction power. The last 20% of the series is plotted against the predicted values.
This is the result of the configuration example found above; basic model architecture: 1 layer of 5 LSTM cells.
A single RNN layer with many many LSTM cells. This allows the network to inch closer to the targets.
A few layers of narrow RNN layers. Er, this time it seems the network didn’t even try.
If we increase the variable
time_steps so that more steps are bundled together as input for each (X, y) pair, we get a much better prediction. I found that matching the number of RNN cells with the number of steps gives good result:
There are actually a lot more parameters than just the RNN architechture that can be tweaked, but not illustrated here. When you run the configs, the MSE may vary, as for one thing, it depends on how the weights of the RNN get initialised.
Go ahead, clone/fork the repo and have fun!