Faffle [faf-uhl]

I’m sitting here at home at 11pm, slowly recovering from the best weekend of my life (so far). I decided to write about it so that I can remember it forever. And then I decided to post it here, because I remembered that that is what a blog is meant to be for.

It started on the morning of Friday, my 29th birthday. Rather a few phone calls and voicemail messages wishing me happy birthday and singing the song. First was a voicemail message from Dad, at 8:30am (I’d put my phone on silent because he's made a habit in the past of calling at 7:30am on my birthday. I still wasn’t up at that time, mind you), then a message from Mel that came in whilst I was riding to uni. Phone calls from both Mum and Carm, voicemail message from Simon Liberta singing Happy Birthday very quietly so that no-one in his office would overhear him. Emails from Simon (Rose) and Kate Cousins, the latter wished me happy birthday and pointed out that I was getting closer to her age. I can only assume that she decided to stop ageing once she hit 30. And lots and lots of well-wishes on Facebook, including tonnes of people I hadn’t spoken to in ages and who I wouldn’t have bothered messaging when it was their birthday. Spent the day feeling totally loved.

So… several months too late, here’s the next section of my four-part series of blogs on my work. In today’s section, I’m going to give a quick overview on Radial Basis (Neural) Networks.

One of the common problems that Intelligent Systems tries to solve is that of classification. We have an item – it might be a flower, a book, a potential home-loan recipient, anything – and we want to determine into which of a set number of classes it belongs – for instance; species of flower, genre, likely/not-likely to default, etc. Usually these are tasks that a human can perform with training and experience, but this may be costly (in terms of wages) or take too much time (if there are a large number of items to be classified). Many approaches exist, most of which involve training a system based on a number of examples and their known class. The system must then be able to generalise from these known examples to determine with which group a new item belongs.

As you may have noticed, I allow anonymous commenting on this blog. It doesn't happen very often; I think only one legitimate comment has ever been posted by a non-registered member. I do get a fair amount of spam, though. It didn't bother me until the last few days, when I've been hit by about 50-odd comments a day.

To that end, I have installed the CAPTCHA and ReCAPTCHA modules to Drupal. The current settings are that it will only force you to confirm your humanity once, and after that you will be automatically accepted. Hopefully this will be sufficient - the sheer number of comments in a short amount of time (and the fact that they keep coming despite the fact that none of them ever get approved) implies to me that this is a bot attack rather than a human copy-pasting.

So, I apologise for the inconvenience. Hopefully you won't mind.

UPDATE: I found the setting to allow registered members to skip the CAPTCHA, so I have now set it to alway apply to non-members).

Look up at the robot in the corner if you don't see it.

If you want to comment on this blog, you really should sign up. That way, your comments will appear immediately. On the other hand, if you just go and post the comment without signing up, I have to go and approve it first.

In the last week, I've had four registrations from spammers. I can tell they're spammers and not just interested people I don't know, by checking them off a database of known spammer email addresses. After deleting and blocking them, I checked whether there were any comments waiting for approval and there were 30-odd that I'd never noticed. I think I had assumed that Drupal was going to email me when there was something for me to approve, so I'd never checked before.

Most of them were your standard SEO-style post; pretending to be an actual comment whilst talking about shoes, cars or fashion and, of course, posting a link to a 'relevant' website. Then I came across the following (links removed):

Thanks, nice read your post

ויאגרה
קניית ויאגרה
סיאליס
לויטרה

Makes me so glad that I worked out the character set issues a few months back, so this sort of thing could appear. At first I thought to myself "Oh, someone has found this site through my Limmud post and decided to comment". Then I looked closer and actually read the Hebrew words out loud:

ויאגרה - Viagra
קניית ויאגרה - Kniyit (buying) viagra
סיאליס - Sialis
לויטרה - Levitra

So that was a laugh. I wonder how many websites this guy had posted this on before stumbline across one where the owner can actually read Hebrew.

This was the side of a Lego box found in Target. (Originally spotted by Kale)

Take a look at what they're loading in to the truck. Lego recursion!

Oh, wow. I guess I forgot about this place for a while.

I'll make up for it with this.

I had quite a successful day of coding yesterday and yet I didn't actually write all that much code. If anything, the overall quality of my code was lower by the end of the day, but the ultimate goals were improved. It was one of those classic Engineering moments when you need to weigh up multiple goals and decide which is more important to a successful outcome. This post continues on from yesterday's, so read that first before continuing here. Once again, I'm trying to explain this at a layperson level, so please comment if you feel I need to explain something further.

'Dumb' Line Minimization

In yesterday's blog post about the Gradient Descent and Conjugate Gradient minimization techniques, I glossed over the details of how I was minimizing in a straight line. In fact, I coded two techniques. For the purposes of testing the larger GD/CG algorthims, I coded a simple, 'dumb' minimization function. It received the current point and a gradient and then stepped in the opposite direction (as it has to head downhill), taking 10,000 tiny steps and recording the best result found so far. Any computer scientist will tell you that this is grossly inefficient, given that this means that the formula needs to be calculated 10,000 times. A better technique will survey the whole range and try to make some sort of guess at where the best location to focus on will be. (Note: the best straight-line techniques often take in to account derivatives of the function so the gradient can give some sort of approximation of where the minimum might be. I couldn't do that for my application, as it only works where the function is fairly smooth and doesn't have any sudden changes in gradient, which I can't guarantee in a Neural Network context).

This blog post was originally going to be much, much longer. It started as a discussion on design decisions and the concept of success in a program's outcome, but I realised that I needed to first explain what I've been working on. That included both the overall project and the specific minimization techniques used to get it working. Then I realised that I would need to explain Neural Networks for my friends and well-wishers who haven't studied them. All of that would result in one absolutely massive post, so I decided to split it in to four. And because I'm wierd like that, they're all going to be out of order. So today I'm going to explain two minimization techniques without the explanation of how I'm going to be using them. You may ask yourself 'But why?'; just be patient and all will (hopefully) be explained as I churn out the other posts in the next few days. I'm trying to explain this at a layperson level, so please comment if you feel that I haven't been clear on any detail.

Minimization

Minimization is a massive field within computer science and has many applications; any context in which you can make a choice which will lead to either a better or a worse result can be considered a minimization task. Now, if you remember your Year 11 maths, you will know that it's easy to find the minimum of an equation with one parameter, for instance y=x²+7x-15. You just differentiate the equation (dy/dx=2x+7) and find the point where dy/dx = 0 (it's -3.5). But what if the equation has multiple parameters, like z=x²+3y²-12xy +7x-y? You can easily find the derivatives with respect to x and y seperately, but the values are dependant on each other; the minimum of x is only relevant for a given y and vice-versa. The only 100% reliable way to find the absolute minimum of a function is to evaluate z for every single possible value of x and y; an impossible task if they can take up any numerical value, and still an incredibly time-consuming task if they're restricted to a specific range, even with computerical assistance. If the formula is something more complicated, the priority becomes trying to evaulate it as few times as is possible.