The biggest problem that computers face comes from people.  Seriously.  It's not the fault of the people who manage the systems; they may be the most skilled and have the best intentions.  It's just that human nature lends itself to make mistakes, particularly when a person faces a monotonous, repeated task. 

Imagine now that you have 400 servers, or 50, or even 10, and a very important file has to be updated on all of them.  Perhaps it's the root password that needs to be changed on all of them, or the configuration file for the SMTP server has to be updated to reflect a new subdomain for your business.  An administrator can do this manually, which takes time and can create errors.  He or she could write a script using rsync or scp to push the file out across the systems and restart the services, but depending on the number of servers and the type of activity, this may take longer than it would take to do it manually.

What's the solution?

We turned to Puppet for configuration management, and then we made it better.  Puppet allows us to model the behavior of a system and break it down into roles.  These roles are then defined as a recipe of software packages, configuration files, services, and other actions that relate to the role.  A server can belong to one or more roles, and all of its behavior then adheres to the model.  Most organizations who use Puppet use it only for their systems, so their recipes are models of what they do now.  We use it for all of our clients, so our recipes are the evolution of our years of experience with administration and with Puppet itself.

What does this mean for you?