Nemo Documents released!

Today I’m pleased to announce something we at IOLA have been working on for quite a while. In essence it deals with how one can create a more humane interface for managing files an documents. By humane I mean an interface that is build with people in mind instead of computers. I’ve written a bit more elaborately on the official blog about how and why we have designed the system in the way we did. If this short teaser was enough of an appetizer, you can also just go ahead and try our beta version right now for free.

Language design

When talking to people about the benefits of clojure often people point out that most modern languages have evolved to “support” the functional paradigm with lambda functions. The argument is that one can stay in the familiar safe environment of imperative programming and use the functional constructs when they fit the problem better. That is a very valid and good strategy but I’ll show in the following that it brings a whole lot of accidental complexity with it. Some of this is specific to the way .NET is implemented and some are a clash of the functional paradigm with the imperiative.

Exhibit 1:

My favorite example is what has been known in the office as the lambda bug. Coming from a background of having been coding in C++ for a couple of years, we switched to python in the last semesters of University. The lambda bug manifests itself when one combines two classical constructs of two different programming models: for-loop iteration and closures:

foreach (var i in list)
save_callback_function_for_later_use(x => System.Console.WriteLine(i));

Given that list contains the numbers 1,2,3,4,5 guess what the code will print when all the callback functions are called?

Exhibit 2:

In C# events seems to have been programmed to a pre-functional model and never updated when they added functional constructs. The interface for an event is += for adding events and -= for removing. This is all fine for point-and-click GUI programming. Meaning it works good on something like event += somefunc; and not event += delegate { use_the_power_of_closures_Luke ) :(

To add injury to insult, events doesn’t even support something like clear().

Exhibit 3:

In clojure all the seq library is lazy. Thus once one has figured that out (I must say it took a little while for me), everything behaves as would be expected. In C# some things are lazy (linq) while others are not. Imagine list contains 1,2,3. Then the following works:

var changed = list.ConvertAll(x => x * 2);
list.Clear();
list.Add(something)
list.AddRange(rest);

But the following doesn’t work because it behaves lazy:

var rest = observablelist.where(x => x != 1);
observablelist.Clear();
observablelist.Add(something)
observablelist.AddRange(rest);

Try guessing what the outcome of running the code above will be. I’ll confess that it was different from what I expected it to be.

Again it’s mixing two styles of programming, functional lazy code with imperative mutable objects.

Finally, my argument is that a language with clean design principles, even with relatively steep learning curve, far outweights the complexity of industry standard languages in the long run.

Concurrency the other way around

Clojure is built around concurrency and it clearly shows in the abstractions the language makes available. I would say that concurrency is pervasive in the language. The good thing about that it that it’s a bit harder to shoot yourself in the foot when doing programming with multiple threads. But the bad side is that it adds quite a bit of mental overhead in situations where concurrency is undesirable.

As an example in mucomp, there is a certain part of the code that deals with the audio player. This is inherently a resource that should only be handled by one thread at a time. Clojure comes with a very good abstraction for exactly this problem, agents. An agent is simply some state, that is manipulated by only thread. Using an agent is done by through sending a function to the agent that will take the old state and return a new state. So with that one gets everything that is needed to write an audio player: serialized access and safely mutable state.

The only bad thing about agents is that if one forgets to return something from a function that run on the agent, then the new state of the agent will be nil. After being bit by this two times I decided that enough was enough. One of the very nice things about languages in the lisp family, is that one can mold your own abstractions to make code better (easier to read and with less bugs in this case).

The following macro creates a new way to define functions. Functions defined in this way will check for nil on return, and return the old state instead. The only change that is needed in code is to use defa instead of defn :-)

Do we really need record labels?

I was very sad to hear that one of the better bands in metal, The Project Hate MCMXCIX, parted way with their record label, Vic Records, after only one record because they didn’t have the money to record their latest album. I guess it’s not easy to have a band on your roster who doesn’t tour.

The new album is written and just needs the funding to get recording. One great thing about Lord K, the main man behind the band, is that he cares a lot about the sound quality. The music sounds so much better in flac than in mp3 and proper speakers / headphones of course helps a lot :-) *hint* Slayer and Metallica.

The band has been searching for a new record label, but has decided to try a donation experiment to see if they can get the money needed to do the recording through generous metal heads instead of a record label. I suggested that he tried kickstarter, but I guess he just wanted a low-tech email solution.

It is going to be extremely exciting how this works out. The internet and p2p networks has to a large degree in the mainstream media been associated with destroying the music industry. This is our chance to show that it can also be used to create music.

Microfunding

I’ve recently joined flattr (that’s the icon on the right you can see :-) ), and just last week I read about some students from NYU who got almost 200.000$ in funding through kickstarter to write an open source facebook clone. Something is definitely buzzing in the micropayment world.

When you look at kickstarter and flattr, they are attacking the same problem, funding, at different angles. Kickstarter tries to get all the funding up front, while flattr is more of a tip jar model for something already produced. So in a way, they are complementary. I would argue that they both work best if the content is placed into the public one way or another. And that is where I think there is a huge potential.

Something like flattr creates an alternative to paywalls and an alternative to ads. And that is something I would very much like to see.

Google search is not a programmers best friend

I was playing around with google this weekend. The original problem I wanted to solve was that last.fm returns strange strange release dates for albums, so I was writing a small script that would extract the correct release date from various sources. I was aiming at www.metal-archives.com and wikipedia. Both of these sites have different search pages, and in general I’ve come to rely more on google’s site:xxx functionaly, than on individual pages own search engines. So I thought, why not just use google programmatically to search the sites. Seems easy enough.

Failure 1 (I’m feeling lucky):

Google has a very nice feature called “I’m feeling lucky” that will direct you to the first result. If I could specify my queries good enough, I could rely on that, and not have to parse google to get the url. It’s very simple, you just add &btnI at the end of your query and google will redirect. Sadly it works fine most of the time, but sometimes it just fails to redirect you. I couldn’t find any patterns to this randomness and a “works sometimes solution” is not a good one :|

Failure 2 (google ajax):

I then found out that google has a seemingly very nice api that lets you do queries and get JSON back. JSON is easy to work with and it also allows one to go through several results, in case google doesn’t return the right one as the first result. After a bit poking around I found out that google ajax randomly returns different results from the normal google. It’s like using Yahoo instead of google. A bit of poking around returns the following 2 year old bug report. Furthermore the TOS directly forbids using the API for this kind of activity. Oh well, it didn’t work anyway.

Failure 3 (parsing google results directly):

After two bitter defeats I thought screw it, I’ll just parse the damn google result pages, how hard can it be? At least I know that it gives me the right results. So I did that, coded everything up and checked that things was working. Then let it loose on my collection (2×275 requests) and around the middle it stopped working. I poked a bit around, and found out that google has identified my program as a bad boy and decided to spank it by returning a “Please identify yourself as a human” page back instead of the normal google result page.

As a side note, after 3 bitter defeats I was ready to jump ship and try bing or Yahoo. That was a quick detour though, as none of them where up for the challenge of returning good results.

Channel downmixing in MPlayer

Recently I have been playing with downmixing in MPlayer.  When I bought new speakers, I decided to go with stereo instead of surround since I mostly listen to music. As anyone using mplayer or any “derived” players such as vlc have discovered, there is a incredible annoying problem that the voices of the actors are very low, actually in general the sound is very low. It appears that when mixing to two speakers, the center channels is put very low in the mix. The same could be said about the subwoofer although it’s naturally not as easily recognized.

A quick google revealed that MPlayer has several tricks (audio filters) that might potentially work: volume, volnorm, pan, hrtf. I quickly discarded volume and volnorm since I don’t want to just boost the sound, I want it to distribute the channels properly. hrtf seemed like a good simple choice, since pan looked very complex. Sadly in the middle of Harry Potter I had to turn it off because it was making lots of clipping of the sound. So I was left with pan. It took a while to get a good default, but a bit of googling around revealed one with a decent default. I first just tried turning sub + center up to one but in one or the other movie introduced the dreaded clipping. So I had to keep it down a bit while still retaining decent boost of center and sub. After an afternoon of testing I came to the following “magic” formula:

-af pan=2:0.5:0:0:0.5:0.33:0:0:0.33:0.5:0.5:0.5:0.5

Please do note that one needs to add a -channels 6 in order for mplayer to decode all 6 channels so that it can mix it down to two. One can read more about the pan filter here.

mucomp released into the wild

Today I’m happy to announce that the world is one audio player richer! This is a personal project of mine that I have been working on for a little while. It’s written in clojure and javascript (jQuery) and uses alsaplayer as the audio player. I probably won’t have much time to hack on it, so consider this a code dump that hopefully someone else will find useful and play/run/do-whatever with. As for maturity I use it almost daily and it’s pretty stable. There are some known bugs and kinks (mostly due to that its using alsaplayer and that the java inotify library is buggy).

How far have we come?

Things like these makes me wonder, with all the advances in computer science how far have we really come?

  • 40 years after the invention of relational databases we are still manually defining indexes
  • 40 years after the invention of Unix, the scheduler in Android (= Linux) still does a terrible job at scheduling the tasks that really depend on it (games and audio)

AndNav2 is now Open Source

Wow this is quite a nice suprise. Nicolas gave in and released the source of AndNav2. So I can now spend my time hacking that instead of reverse-engineering it :) I rooted my HTC Hero so I actually don’t need to do that anymore, but that is really beside the point. With the source available there is no ends to what can be done :)

As I have said before, I really think this is a killer app for Android. Google released their navigation which is nice, but I’m not sure how well it works in offline mode.