Hello! I studied Computer Science, now I live in Bath! I write code, design games, and occasionally tweet. Why not subscribe?

Posts from 2016

Securing an terrible blogging platform

Posted December 1, 2016. Cybersecurity, Php, Web. 3162 words.

I was tasked with securing $BloggingPlatform. Here are my findings.

Continue Reading...

Machine Learning with MATLAB

Posted November 24, 2016. Machine-learning, Matlab, University. 4180 words.

I decided to investigate Machine Learning using MATLAB.

Posterior Probability

Posterior Probability 1Posterior Probability 2Posterior Probability 3

To compute the posterior probability, I started by defining the following two Gaussian distributions, they have different means and covariance matrices.

Using the definitions, I iterated over a N×N matrix, calculating the posterior probability of being in each class, with the function mvnpdf(x, m, C); To display it I chose to use a mesh because with a high enough resolution, a mesh allows you to see the pattern in the plane, and also look visually interesting.

Finally, I plotted the mesh and rotated it to help visualize the class boundary. You can clearly see that the boundary is quadratic, with a sigmodal gradient.


Classification using a Feedforward Neural Network

Classification using a Feedforward Neural Network 1Classification using a Feedforward Neural Network 2Classification using a Feedforward Neural Network 3

Next, I generated 200 samples with the definitions and the function mvnrnd(m, C, N);, finally partitioning it half, into training and testing sets. With the first of the sets, I trained a feedforward neural network with 10 hidden nodes; with the second, I tested the trained neural net, and got the following errors:

  • Normalized mean training error:
  • Normalized mean testing error:

These values are both small, and as the testing error is marginally larger than the training error, to be expected. This shows that the neural network has accurately classified the data.

Classification using a Feedforward Neural Network 4Classification using a Feedforward Neural Network 5Classification using a Feedforward Neural Network 6

I compared the neural net contour (At 0.5) to both a linear and quadratic Bayes’ optimal class boundary. It is remarkable how significantly better Bayes’ quadratic boundary is. I blame both the low sample size, and the low number of hidden nodes. For comparison, I have also included Bayes’ linear boundary, it isn’t that bade, but still pales in comparison to the quadratic boundary.

To visualize, I plotted the neural net probability mesh. It is interesting how noisy the mesh is, when compared to the Bayesian boundary.

Classification using a Feedforward Neural Network 7Classification using a Feedforward Neural Network 8Classification using a Feedforward Neural Network 9

Next, I increased the number of hidden nodes from 10, to 20, and to 50. As I increased the number of nodes I noticed that the boundary became more complex, and the error rate increased. This is because the mode nodes I added, the more I over-fitted the network. This shows that it’s incredibly important to choose the network size wisely; it’s easy to go to big!

After looking at the results, I would want to pick somewhere around 5-20 nodes for this problem. I might also train it for longer.

 Training ErrorTesting Error
10 Nodes
20 Nodes
50 Nodes

Macky-Glass Predictions

I was set the task of first generating a number of samples from the Mackey-Glass chaotic time series, then using these to train and try to predict their future values using a neural net.

Mackey-Glass is calculated with the equation:

Macky-Glass Predictions

For the samples, I visited Mathworks file exchange, and downloaded a copy of Marco Cococcioni’s Mackey-Glass time series generator: https://mathworks.com/matlabcentral/fileexchange/24390

I took the code, and adjusted it to generate samples, changing the delta from 0.1 to 1. If I left the delta at 0.1, the neural network predicted what was essentially random noise between -5 and +5. I suspect this was due to the network not getting enough information about the curve, the values given were too similar. You can see how crazy the output is in the bottom graph.

Next, I split the samples into a training set of 1500 samples, and a testing set of 500 samples. This was done with . I created a linear predictor and a feedforward neural network to look at how accurate the predictions were one step ahead.

Macky-Glass Predictions Error 1Macky-Glass Predictions Error 2Macky-Glass Predictions Error 3
  • Normalized mean linear error:
  • Normalized mean neural error:

This shows that the neural network is already more accurate, a single point ahead. If you continue, feeding back predicted outputs, sustained oscillations are not only possible, the neural net accurately predicts values at least 1500 in the future.

In the second and third graphs, you can notice the error growing very slowly, however even at 3000, the error is only 0.138


Financial Time Series Prediction

Using the FTSE index from finance.yahoo.com, I created a neural net predictor capable of predicting tomorrows FTSE index value from the last 20 days of data. To keep my model simpler and not overfitted, I decided to use just the closing value, as other columns wouldn’t really affect the predictions, and just serve to overcomplicate the model.

Financial Time Series Prediction 1

Feeding the last 20 days into the neural net produces relatively accurate predictions, however some days there is a significant difference. This is likely due to the limited amount of data, and simplicity of the model. It’s worth taking into account that the stock market is much more random and unpredictable than Mackey-Glass.

Financial Time Series Prediction 2

Next I added the closing volume to the neural net inputs, and plotted the predictions it made. Looking at the second graph, it’s making different predictions, which from a cursory glance, look a little more inline.

Financial Time Series Prediction 3

However, I wasn’t sure so I plotted them on the same axis, and, nothing really. It just looks a mess. Plotting the different errors again gives nothing but a noisy, similar mess. Finally, I calculated the total area, the area under the graph and got:

Financial Time Series Prediction 4

  • Normalized close error:
  • Normalized close+volume error:

This is nothing, a different of 0.011×10^5 is nothing when you are sampling 1000 points. It works out to an average difference of 1.131, or 0.059%.

From this I, can conclude that the volume of trades has little to no effect on the closing price, at least when my neural network is concerned. All that really matters is the previous closing values.

Overall, there is certainly an opportunity to make money in the stock market, however using the model above, I wouldn’t really want to make big bets. With better models and more data, you could produce more accurate predictions, but you still must contest with the randomness of the market.

I suggest further research before betting bit.

Continue Reading...

Aqua, an imperative language, for manipulating infinite streams.

Posted April 28, 2016. Languages, Ocaml, University. 1503 words.

This is the user manual for the Aqua programming language created as part of Programming Languages and Concepts. Visit the project on Github.

Aqua is a C­like imperative language, for manipulating infinite streams. Statements are somewhat optionally terminated with semicolons, and supports both block ( /* ... */) and line comments ( // ...).Curly brackets are used optionally to extend scope. Example code can be found in the Appendices.

Before continuing, it’s helpful to familiarise yourself with Extended BNF. Special sequences are used to escape.

Continue Reading...

Visiting New Zealand via Singapore

Posted April 7, 2016. Photos, Travel. 682 words.

This year I returned to New Zealand. Last time, I explored the country on a road trip, this time, I explore the North Island, and spend time with family.

Auckland

auckland1 1 auckland1 2

Continue Reading...

Experimenting with Microsoft Hyperlapse

Posted April 7, 2016. New-zealand, Singapore, Videos. 224 words.

Whilst abroad I put Microsoft Hyperlapse to the test, first in Singapore, then in New Zealand.

Riding the Singapore Mass Rapid Transit system (4x Hyperlapse)

 

Driving around Auckland (16x Hyperlapse)

Visiting Bloombergs London Headquarters for CodeCon Finals

Posted January 30, 2016. Algorithms, Competitions, Java, Photos. 203 words.

This week I had the pleasure of competing in the CodeCon Grand finals at Bloombergs London headquarters. While unfortunately I didn’t do that well, visiting the headquarters and exploring London was great fun. I didn’t take any pictures inside the event, I thought I would share a few photos taken on during my trip.

Museum of London

The day after the competition, I left the hotel and set of through the city. The Museum of London was nearby, so I checked it out. While there it started raining and a rainbow appeared.

St. Pauls Cathedral

Then, a quick walk lead me to St. Pauls.

Panorama 1

Panorama 2

While, crossing the Millennium Bridge I took two panoramas of the Thames. Unfortunately the day was overcast, and so the pictures are rather gray.

Tate Modern

Finally ending my trip inside the Tate Modern where I saw Abraham Cruzvillegas Empty Lot filling the Turbine room.


Finally, it was time to return home, heading back to Waterloo I boarded a train to Southampton. My day was over.

The Twelve Days of Rentmas

Posted January 24, 2016. Music. 461 words.

Let it snow, let it snow, let it snow!

On the first day of renting, My landlord gave to me: A junk pile in the garden.

Continue Reading...

How to play Grabble

Posted January 15, 2016. Games, Rules. 709 words.

Grabble is a fast paced, word game where players take it in turns to flip over tiles, forming words, and stealing words from other players.

I was first taught the game by Samson Danziger, and I am forming this list as from a brief glance, there is little writing on the topic.

Scrabble Tiles

Continue Reading...

StatCounter Pixel