You have put your puppet master behind Passenger and that lasted for some time. But now that you're beginning to have quite a good number of hosts that checkin to your master regularly, performance has started to become an issue. What can be done about it?
There are multiple answers. The first and simplest would be to put more space between your puppet runs. Say you're currently running puppet every 30mins on each client, you could change that period to 45mins or 1h.
It's also possible to distribute runs via a central scheduler that would ensure to run every node before going back to square one. One such solutions is the "puppetcommander.rb" script that R.I. Pienaar wrote to use mcollective as the run scheduler.
Another method (that you can use in conjunction with the above) is to cache the responses that come from the puppet master.
Recent comments