Monday, 6 August 2012

Numerical Methods Self Inflicted Suffering

After making a solid enough matrix and vector class, a learning experience in itself, I decided to wrok on one of the more useful areas of numerical methods - linear algebra.  It probably is much simpler than some of the data and algorithms stuff I've done, at least once its broken into the fundamentals.  The problem with using a reference for experts ("Numerical Recipes") is the sheer amount of stuff is pretty intimidating.  The beginners numerical methods book is much more managable, but for me the explanations not as satisfying.  Ultimately though, I now have my own resources which focus on the bits I found hard, which will be the most useful.

I set about to implement a few different methods, the basic linear algebra methods like Gaussian elimination and LU decomposition.  The different algorithms aren't nearly as impressive as some of the gains seen in data structures and algorithms e.g. bubble sort vs quick sort, or the string matching algorithms, and the linear algebra algorithms verge on intractable (time complexity of O(n^3) for those that care).  Despite this, linear algebra is done all the time and clearly works well enough.  I think the secret is linear algebrists (not a word) are always looking to cut corners and spot patterns in the linear systems, where much more efficient algorithms will work.

For larger projects I like to start out on paper with a rough specification, maybe a hierarchy of classes and possibly even sample input and output.  This often triggers me to spot problems very early on, saving a lot of redundant coding.  It also helps me overcome writers block.  For this project I decided to keep more of a chronicalogical log, writing down major issues I get with the code, and the solution along with any doubts to the solution.  I did it because "The Practice of Programming" told me to, and I found it stopped me going around in circles.  In a few months, or years it will be interesting to look back over my thought process.

My second project was one I'd wanted to do for ages, and involved creating a simple neural net, following the guidance of "Games Programming Gems Vol 1", an awesome tutorial which had the perfect amount of information to get me started.  One thing that surprised me is the training that you have to give a neural net before it starts working.  That said, I really had no idea how they worked before the project.  It was quite satisfying how simple the simple neural nets are, and artificial intelligence is something that I've been fascinated with but at the same time haven't looked too much into it.

I did even more of an in-depth log, and it tracked every major development in the project.  Fortunately writing it only took 2 days and was quite straightforward.  The final product was a simple Windows application that lets the user create patterns and save them to the net.  In some intances the net works way better than I expected, in other instances the net is pretty dumb.  However, the tutorial tells you the algorithm is very simple and does badly in certain situations.

The brain in action
Next on the list is a long needed return to audio programming, and then maybe an attempt to try and finish my "Introduction to Numerical Methods".  I would like to get better at Python, learn some other languages and get familiar with Unix / Linux.

No comments:

Post a Comment