Squeezing performance out of nginx

What, nginx is not fast enough for you? Well, if you have a very high load that's quite possible (although unprobable :P ). I've learned and tested quite a few things about nginx optimization by reading this article:

http://blog.martinfjordvald.com/2011/04/optimizing-nginx-for-high-traffic-loads/

Basics on Nginx configuration

Those posts by Martin Fjordvald are pretty interesting for those who want to start playing with nginx. They're a lot easier to read than anything I've seen out there yet, and still convey some pretty valuable information.

Tags:

programming python in pipelines

I always love to learn about new programming paradigms to make my expression potential better. One such paradigm that I've read about lately is about programming in pipelines. The idea in itself shouldn't be anything new (it's quite old and much talked about literature), but the way you can do this in python with generators and coroutines is really empowering.

Tags:

Slow-transitioning a web site to a new server

When a client of yours is moving a web site between two servers, of course the best option is to plan some time during which modification is prohibited and everything is moved as fast as possible. This ensures that you do only one operation and that everything follows.

However, in some cases, the client can desire to move more slowly in order to rearrange the new site or to filter out content prior to making it available on the new server, or there might be some transition issues that she doesn't want to revisit. In such cases, the web site will need to be available from two locations at once for some time.

In that case, you can help out by reverse proxying to the new server.

Caching puppet responses for greater scalability

You have put your puppet master behind Passenger and that lasted for some time. But now that you're beginning to have quite a good number of hosts that checkin to your master regularly, performance has started to become an issue. What can be done about it?

There are multiple answers. The first and simplest would be to put more space between your puppet runs. Say you're currently running puppet every 30mins on each client, you could change that period to 45mins or 1h.

It's also possible to distribute runs via a central scheduler that would ensure to run every node before going back to square one. One such solutions is the "puppetcommander.rb" script that R.I. Pienaar wrote to use mcollective as the run scheduler.

Another method (that you can use in conjunction with the above) is to cache the responses that come from the puppet master.

Wait.. what happened to my file? ... oh ... the puppet master blew out my changes.

A friend of mine found out an interesting plugin for those admins out there that use ViM. It makes it easier for you (and your colleages) to see when you are editing a file that is managed by puppet:

http://blog.mricon.com/2012/04/puppet-eyes.html

This module will only work when your editor has root privileges, since the file it parses, state.yaml, is accessible only to root (for relatively evident reasons). When you try to modify a file that is currently managed by puppet, you will see a white on red warning telling you "This file is managed by Puppet":

puppet-views warns you out when you open a file that is managed by puppet

Tags:

Managing your Tomboy/Gnote notes from the CLI

I personally have difficulty with GUIs and with how every one of them seems to approach the same problem differenlty. Mastered point-and-click is not as useful as the CLI because it's generally less direct. (e.g. you slide your mouse up there on the menu, click to open it, slide down, click to choose the option, click on a field in the newly opened window, type in the new value, slide down and click on the OK button. whereas with CLI you type in your command and pass in your arguments for the new values, and you're done)

I love Tomboy for how simple it is and how you can sync notes with ssh and other methods. However, I personally don't like how note management is achieved. It is very easy for newcommers, but lets no room for power users to get faster / better.

So, I've built a CLI interface to Tomboy. I called it Scout (after being advised to change name from Tomtom -- which was a slang synonym for Tomboy. Scout comes from the character from "To kill a mockingbird". She's that tomboy girl that likes to pick up a fight with other students when they make fun of her father).

Everything is done through the D-Bus API exposed by Tomboy. Luckily, Gnote (a port of Tomboy to C++ that became an entirely separate project) exposes the same API functions; this means Scout supports Gnote, too!

delaying disk decryption for manually giving the password

Long story short: both my 2.5" Caviar blue disks failed some days apart (GAH!). So I had to send them both back to WD for an RMA. But since all my stuff in there was accessible as-is, and also since I didn't have any other device to wipe out the disks (actually one had failed beyond the point where I could wipe out the disk even if I wanted to), I had to consider every ssh key, puppet SSL cert, passwords, etc, as compromised (double-GAH!!). Who knows what happens to the data after your disk is refurbished... I don't trust WD to wipe them out before shipping them to new clients.

Luckily, I had a recent backup of the entire /etc/puppet directory from the puppet master, and I was able with this to rebuild all VMs. Friends don't let friends run computers without a proper, regularly updated backup!

So! I decided I would add an encryption layer to the new disks, and since I don't have a screen plugged into the mini-ITX, I needed a way to delay the system from asking the password for decrypting the disk.

Iterating through all IP addresses in a puppet ERB template

Sometimes, it's relevant to iterate through all ip adrresses on a host to emit configuration for each one. Facter has a useful bunch of facts for this:

  • interfaces => lo,eth0,eth0_0,eth1
  • ipaddress_lo => 127.0.0.1
  • ipaddress_eth0 => 192.168.0.200
  • ipaddress_...
  • ... and so on

This is nice, but how can you use this in an ERB template?

Python is interpreted.... yes but it can be _fast_

I just listened to this presentation by Avery Pennarun about how Python can be very fast if you use the appropriate techniques, and how those techniques are actually graceful ones.

Pages

Subscribe to lelutin.ca RSS