One of the worst decision that was taken when designing
to use the
String type. It is well known that this type is horribly slow. I
had a few tests to run and they were taking forever : it took about 11 seconds
to compute 5 catalogs, which is annoying when you are doing this often. As a
comparison, the puppetmaster on faster computer (it is a X5660, whereas my
workstation uses a i7 860), running the latest Puppet, takes 23 seconds to
compute those 5 catalogs sequentially.
A quick profiling revealed the following :
- The program couldn’t use more than a CPU.
- Almost all the time was spent computing templates.
I refactored the code a bit during lunch. The
Daemon code was supposed to be
multithreaded, and has been designed as such. The only problem was that I forgot
to start several worker threads. This is now
The template computing time was reduced by spawning several threads for this
task (see previous commit), and converting the
String code to
using builders. The time is now mostly spent in the
defined as :
I went with the definition in here, but it is much slower. If somebody has a better implementation, please let me know.
ByteString move reduced the time it took to compute the catalogs to about
6 seconds, and parallelisation reduced it to about 2 seconds. It is a 5.5x speed
upgrade for a few minutes of work, not too bad. The template generation still
takes most of the running time (50% of the time is spent spawning and waiting
for the Erb code, 20% preparing the inputs for the Ruby processes).
Nice speedups could arise from parsing more complex Ruby expressions from the
Haskell code, but it is not a priority now that the performance is acceptable.
You can grab the latest github repo to test it, but you will need a very recent bytestring, and you will need to fix the cabal file for luautils.