Scuttlebutt development grant

Since January I have been working on a grant to add a benchmarking ci framework to scuttlebot to better understand the behaviour and make it easier to find regressions as the implementation matures and changes. Furthermore I have been spending time understanding and finding things that can be improved performance wise.

The grant is almost at an end so I’ll use this blog to summarize what has been achived.

  • Built a CI framework and visualization tool for the bench-ssb repo. I used the nci framework for this and have been quite happy with the result.
  • Found and fixed a memory leak that made level-post really slow over time.
  • Made several improvements to flume to improve performance. Some interesting links related to this: charwise  and json performance related to buffers in node.

I have kept a dev diary for the duration of the grant. It has been a very interesting to really get to understand a relatively large and real world distributed system.