One of the worst decision that was taken when designing language-puppet was to use the String type. It is well known that this type is horribly slow. I had a few tests to run and they were taking forever : it took about 11 seconds to compute 5 catalogs, which is annoying when you are doing this often. As a comparison, the puppetmaster on faster computer (it is a X5660, whereas my workstation uses a i7 860), running the latest Puppet, takes 23 seconds to compute those 5 catalogs sequentially.

A quick profiling revealed the following :

• The program couldn’t use more than a CPU.
• Almost all the time was spent computing templates.

I refactored the code a bit during lunch. The Daemon code was supposed to be multithreaded, and has been designed as such. The only problem was that I forgot to start several worker threads. This is now fixed.

The template computing time was reduced by spawning several threads for this task (see previous commit), and converting the String code to ByteString, using builders. The time is now mostly spent in the renderString function, defined as :

1
2

renderString :: String -> BB.Builder
renderString x = let !y = BB.stringUtf8 (show x) in y

I went with the definition in here, but it is much slower. If somebody has a better implementation, please let me know.

The ByteString move reduced the time it took to compute the catalogs to about 6 seconds, and parallelisation reduced it to about 2 seconds. It is a 5.5x speed upgrade for a few minutes of work, not too bad. The template generation still takes most of the running time (50% of the time is spent spawning and waiting for the Erb code, 20% preparing the inputs for the Ruby processes). Nice speedups could arise from parsing more complex Ruby expressions from the Haskell code, but it is not a priority now that the performance is acceptable.

You can grab the latest github repo to test it, but you will need a very recent bytestring, and you will need to fix the cabal file for luautils.