RK4 in Scala

August 27, 2009

I’ve been looking at Scala recently and I’m impressed. It’s got a nice mix of imperative and functional styles that means that it can be really useful and easy to code in.

As an example I thought I’d knock up an implementation of RK4 in it.

The work-horse of RK4 is implemented as follows:

def rk4Iter(f: (Double, Double) => Double, 
            t : Double, 
            y : Double, 
            h : Double) = {
  val k1 = f(t, y)
  val k2 = f(t + h / 2, y + h * k1 / 2)
  val k3 = f(t + h / 2, y + h * k2 / 2)
  val k4 = f(t + h, y + h * k3)
  y + h * (k1 + k2 * 2 + k3 * 2 + k4) / 6
}

One thing I think we can all agree on is that having functions as first-class objects makes this incredibly succinct and easy to follow. I’m just calling the function f a whole bunch of times. Coming from a Java background this is pleasantly less noisy (compare with f.calculate(t, y) etc).

Another subtlety is the lack of specifying a return type. Scala is smart enough to infer the return type 🙂

This method is intended to be called from a loop that performs n steps of size h:

def rk4(f: (Double, Double) => Double, 
            t0 : Double, 
            y0 : Double, 
            n : Int, 
            h : Double) 
    : Double = {
  n match {
    case 0 => y0
    case _ => 
      val y1 = rk4Iter(f, t0, y0, h)
      rk4(f, t0 + h, y1, n - 1, h)
  }
}

For those coming from a Java background you’ll notice that match looks a lot like switch. You can think of it as switch on steroids (I’ll leave off explaining more as it’s a blog post in its own right!). It is sufficient to say that case _ is similar to default.

In functional programming the preferred way to do things is using recursion rather than iteration (when written properly the compiler can identify tail-recursive functions and make them just as fast as loops). We do this because it makes them easier to mathematically reason about them. Suppose the function works for:

  • the 0th case
  • the nth case if it works for the n -1th case

then the function works for all positive integers.

A recursive style also reduces the scope for off-by-one errors.

And now for an example that ties it together:

rk4(
  (t, y) => 4 * Math.exp(0.8 * t) - 0.5 * y, 
  0, 2, 10, 0.1)

I think anyone coming from a Java background will agree that passing around functions like this is lovely: goodbye interfaces and lots of curly braces, hello Functional Programming.

Advertisements

Launch of Two New Sites

August 20, 2009

Today I see the launch of two sites of mine that I’ve been working on in my free time over the past month or so. Neither of them are finished and well polished, but, the act of getting them out there will ensure I start taking them more seriously and start getting them used 🙂

So, I am proud to present: No Green Sheep and Self-review.com.

No Green Sheep

The idea behind this site is to promote being more green and friendly to our planet – but by understanding the effects of our actions instead of just doing whatever someone tells us.

The inspiration for this was a friend boiling a kettle on a gas hob and telling me it was more eco-friendly than boiling in an electric kettle. I asked “how?” and they couldn’t explain or give any numbers… unsurprisingly, I haven’t changed my kettle boiling habits. If my friend had known a few more details about this then she might well have had a convert out of me!

Self-review.com

When I worked at CHP Consulting I kept a monthly reminder in Outlook telling me to review how I’ve done over the past month. This process of self-reflection allowed me to move forward and improve myself in a positive manner. It also gave me complete ownership of it – it’s far better to set your own goals and achieve them than have someone else do it for you.

After leaving CHP I no longer had Outlook. I also decided I needed something a little more guided – something I could just tick a box to say I’d done what I wanted to do. I also wanted to gain feedback from my friends whenever possible (a self-criticising blog post of mine made me realise I wanted this). So, I made self-review.com. The commenting isn’t yet ready – but I’ll keep you posted!

My first review of myself is available at http://www.self-review.com/reviews/5

Twitter Accounts for Both

If you want to keep up-to-date with both of these projects then they have their own Twitter accounts: selfreview and nogreensheep


Why We Made Our Own Bug Tracker

August 11, 2009

Part of starting the development of Sonnet Models was choosing a bug tracker. Knowing that reinventing the wheel was a bad thing I decided that we should do it anyway.

Why would I do this? Knowing full-well of the existence and having used numerous bug trackers I decided that they all suffered from one common flaw: bloat.

Bloated with Fluff-Fields

If entering a bug requires more than 20 seconds of entering other fluff (such as priority, origin, area, etc etc) then small bugs will simply not get entered most of the time because it takes too long. It’s hard to admit that people are lazy but I’ve seen it happen too many times, “I’ll just add this bug onto this other issue to avoid trying to remember what to put in field X”.

I also believe that people tend to rush filling in such fluff-fields and the result is that much of the data is junk.

What do we Track?

In our bug tracker we have nine fields per bug.

Three of these are automatically generated and cannot be modified (created date, creator, modified date).

The other six are: type (bug, feature or admin task), summary, details, timebox, status (created, ready for testing, pending release, closed) and assignee. That’s it. I’m even considering binning the type field as we rarely use it (fortunately it defaults to bug so there isn’t an admin overhead here).

We allow commenting – but there is only one field – comment.

Drawback – No Sexy Reporting

Sure, I can’t query the bugs database to see the most common origin of bugs, but, as a start-up we have a very good feel for this and knowing the exact numbers wouldn’t justify the cost in time of collecting this data.

I also feel that the lack of sexy reporting is an advantage – I don’t spend half my afternoon trawling through meaningless reports looking for imaginary trends!

Question for Large Companies

Do you actually get any use from all the random data you collect? Or is it slowing you down? Are bugs slipping the net because people don’t want to have the burden of reporting bugs? (Or the e-mail deluge that normally follows from reporting a bug).

Avoiding the fluff is a hard task and sometimes we should learn to just say “no, we can’t add a field onto our bug reports saying what coffee the reporter was drinking at the time”.

Agile Zen

I’d like to finish this post by mentioning a new tracker: Agile Zen. This is an immensely light weight tracker and if it had commenting… we’d switch over as quickly as possible!


HtmlUnit – How to do a POST

June 24, 2009

Background

I’ve just started using HtmlUnit (I also use Selenium but find it too slow for 90% of my testing).

HtmlUnit is generally easy to work with. Generally, it doesn’t need documentation as the Javadoc is useful and the methods are obvious. However, this is not the case for POST… so, here is how you do a POST 🙂

How to do a POST in HtmlUnit

There really isn’t much to it – you just need to know what code to use:

final WebClient webClient = new WebClient();

// Instead of requesting the page directly we create a WebRequestSettings object
WebRequestSettings requestSettings = new WebRequestSettings(
  new URL("URL GOES HERE"), HttpMethod.POST);

// Then we set the request parameters
requestSettings.setRequestParameters(new ArrayList());
requestSettings.getRequestParameters().add(new NameValuePair("name of value to post", "value"));

// Finally, we can get the page
HtmlPage page = webClient.getPage(requestSettings);

Quite a lot of work for a simple… I imagine it won’t be hard to wrap this up into a neat POST method.


I feel good! (Follow-up on Lessons Learned!)

June 17, 2009

I feel good. I am more productive and happier in my life.

Why?

I’m working less.

Why?

The conclusion of my previous blog post was that I should try to work less. So, I’ve been forcing myself to ~7-8 hours every day.

Working less but getting more done?

Yes. I am motivated because I have had time to relax and reflect.

Win?

You bet.


Windows 7 (World) vs Ubuntu 9.04 Jaunty (World) for a developer

May 20, 2009

Background

This initially started of as a blog posting about how wonderful Ubuntu is for a developer and the various pitfalls I found. However, since installing Ubuntu and initially being wowed I have gone back to Windows. So, I thought I’d offer up my thoughts as to why I made this decision.

For the record: I still use Ubuntu on my laptop because some of my problems with Ubuntu aren’t evident on my laptop. I do also have Windows on dual-boot for the times I need it.

This is not Windows vs Ubuntu

The (world) in the title is a hint. I am comparing my world when running Windows  against my world when running Ubuntu. This means that I am looking at not just the operating system but how I can go about doing everything that I do (mostly development work with a bit of gaming thrown in)

Ubuntu – The Good

So, what makes the Ubuntu experience better than Windows?

  • Out the box support for multiple desktops
    • That said, the apps for Windows that do multiple desktops are much more stable than a few years ago
  • Customisation of… everything
    • For example, on my laptop I have compiz set up to remove title bars from maximised windows – screen estate is valuable on a laptop
  • Automatic updates for everything
    • Nearly everything I install I use apt for. This then means it gets auto-updated with new features / bug fixes
    • Nearly everything in Windows has its own update manager… and I kill most of them as I don’t want them running all the time

Ubuntu being free isn’t such a winner for me – it only adds to the experience when you first install it and you don’t continuously feel the benefits

Ubuntu – The Bad

And… what makes the Ubuntu experience worse?

  • Initial fonts (in my opinion) are horrible
    • This isn’t really Ubuntu’s fault. Many websites are designed using Windows fonts and due to licencing the fonts can’t be installed by default
    • Solution: install windows font pack and various other bits and pieces
  • GIMP. I’m not a designer. I want to take screenshots, resize them and draw cirlces / boxes around things. I don’t want to learn anything new to do this
    • GIMP by default is the equivalent of using Adobe Photoshop by default – it’s a powerful tool but not the right tool for my simple tasks
    • In Windows, Paint.net is exactly the right tool for 99% of the image editing I do, and, I haven’t had to spend time learning it
  • Dual-screen support
    • This was a huge chore to get working right in Ubuntu – it defaulted my left monitor as my primary monitor and setting it otherwise was non-trivial
  • User interface sluggish
    • After doing some reading it seems my ATI card is to blame
  • ATI drivers
    • Couldn’t get these working… but installing/uninstalling them was a lot of grief – if ATI want me to buy another ATI card then they need to support Ubuntu better (I hear NVidia has far better support)
  • SQLYog doesn’t work as nicely in Wine as it does in Windows and I couldn’t find a linux alternative
  • No equivalent to TortoiseSVN
  • Customisation of… everything
    • A blessing and a curse. I spent far more time than is healthy playing around with things 🙂
  • Gaming
    • I play Team Fortress 2 to unwind – it’s not as simple to get started as it is in Windows

Conclusion

So, what made me can the Ubuntu experiment? Dual screens and the sluggish reponse. These are the only things that affected me on a continual basis, everything else I could deal with.

In defence of Ubuntu: most of the problems are not the problem of the OS but a lack of support from other people and a lack of linux equivalents to my favourite Windows programs.

I hope this is of some use to anyone with a similar lifestyle when deciding what OS to use 🙂 As always, comments are welcome!


Redis vs MySQL vs Tokyo Tyrant (on EC2)

April 27, 2009

Update (28th April 2009) – Added Tokyo Tyrant

What is Redis?

Briefly, Redis is a key-value store that stores the values to disk periodically instead of after every set.

What is Tokyo Tyrant?

Tokyo Tyrant is similar to Redis – a key-value store.

Aim

This is a brief comparison of the performance of Redis and MySQL. It is not very thorough and is only to be used as an indication as to whether Redis may be worth your further investigation.

After a brief search I failed to find any performance comparisons of Redis and MySQL to determine if it was even worth my time looking at Redis. I appreciate they are very different beasts but in our database we have a lot of data that would fit very nicely into a key-value store.

So, the question I am trying to answer is “roughly how much performance can we gain from using Redis for storing simple data instead of MySQL?”. Then it is up to you to look at all the differences and decide whether it is worth pursuing a switch to Redis.

Note: I will be looking at other key-value stores in the near future 🙂

The Test

The two tests performed are:

  • Using 2 threads: set X items with the keys/values being consecutive numbers
  • Using 2 threads: randomly get these items Y times

Setup

  1. EC2 small instance.
  2. Default install of MySQL with a single InnoDB table for storing keys and values.
  3. Default make of Redis.
  4. Small Java benchmark programs sitting on the same box

Source Code

The numbers in these files are not the exact numbers I used (apart from using two client threads in both tests)

  1. Redis benchmark: http://pastebin.com/f5adb41b0
  2. MySQL benchmark: http://pastebin.com/m52fd35d1

Results

Mysql

3,030 sets/second.

4,670 gets/second.

Redis (In-memory only)

11,200 sets/second. (3.7x MySQL)

9,840 gets/second. (2.1x MySQL)

Tokyo Tyrant (Writing to disk)

9,030 sets/second. (3.0x MySQL)

9,250 gets/second. (2.0x MySQL)

Thoughts

Key-value stores are generally going to be faster than MySQL. They (generally) provide less functionality in the way of querying arbitrary columns and ACID compliance. This translates into faster performance on gets/sets.

Once again, it is worth pointing out that this is a very simple benchmark. My suspicion is that when you have lots of cross updates going on then Redis/Tyrant will have a greater performance boost over MySQL – but at the cost of transactionality.

My suspicion is that as the technology matures we are going to start seeing a big shift towards using key-value stores as the first port of call for data storage for websites. I expect it won’t be long before we start seeing hosts offering LAKP (Linux, Apache, a key-value store, PHP) as well as the more traditional LAMP offerings.

I would love to hear from anyone with real world experience and numbers of switching over to Redis… until I have some of my own to share anyway 🙂