Hosties

Hosties is a ruby gem I’m working on to add some ease and safety my deployment tools. I really enjoy using ruby to build the tools that I use for deploying code to different product installations, and if I can’t have Chef, then I end up rolling my own. This is always packaged up nicely and exposed to the dev teams through a web frontend, giving something approaching click-to-deploy behavior. One problem I’ve ran into a few times is when a developer needs to update the list of machines, maybe to take one out of rotation, add more machines in, or change something about a machine. When the person making these changes isn’t familiar with the language it’s written in, that can prove to be an error-prone challenge. Previously, I had been using simple ruby hash instances with an expected set of keys to be present. Something like the following:

ProductA = {
  :dev => {
    :type_1_hosts => [
      { :hostname => "0.0.0.0", :web_port => 8080, :id => 1 },
      { :hostname => "0.0.0.0", :web_port => 8080, :id => 2 },
      { :hostname => "0.0.0.0", :web_port => 8080, :id => 3 }
    ],
    :type_2_hosts => [
      { :hostname => "0.0.0.0", :service_port => 1234, :id => 3 }
      { :hostname => "0.0.0.0", :service_port => 1234, :id => 3 }
      { :hostname => "0.0.0.0", :service_port => 1234, :id => 3 }
      { :hostname => "0.0.0.0", :service_port => 1234, :id => 3 }
    ]
  }
}

As you can see, this is a little less than friendly on the eyes. Worse, it’s prone to typos and other errors. What if somebody decides to add a :test environment to the above, but forgets to add any type_2_hosts, it won’t be noted until something deep in the guts of my deployment scripts break. This, of course, will happen at the worst time possible and shortly after my cell phone will ring.

To prevent this sort of mishap, and ease maintenance headaches, take a look at Hosties!

Installing

Hosties can either be built from source, so you can live on the edge of life and bravely speed forward into the future, or you can just grab the gem.

Build from source

$ git clone https://github.com/influenza/hosties
$ cd hosties
$ rake
$ gem build ./hosties.gemspec
$ gem install ./hosties-1.0.0.gem

Or just grab the gem!

Grab the gem

$ gem install hosties

Host and Environment Definitions

Before an environment can be declared, we provide a definition. This definition includes host definitions. By using this define-before-declare process, we can validate environments as they are declared and catch things like missing attributes or invalid values. Since environment definitions include host definitions, we define the host definitions first:

host_type :mutant_maker do
  have_services :chainsaw_arms, :laser_cannon
  have_attribute :brutality
end

So what’s happening here? The host_type function comes into scope when you require hosties. The provided symbol is used to reference the ‘type’ of host that we are defining, in this case a mutant maker service. Next, we provide a block that specifies what qualities a :mutant_maker type host should have. These are comprised of attributes and services. Attributes simply mean that when a host of this type is declared, it must have a value assigned to that attribute type to be valid. Services are a sort of specialization of attributes - they also must be set, but only integral values are considered valid. Additionally, the have_services and have_attributes are both aliased to their singular forms, have_service and have_attribute respectively.

Additionally, if there is an attribute that only has a small number of possible values, we can specify that as well!

host_type :mcmeal_creator do
  have_attributes :drink_size, :calorie_count
  where(:drink_size).can_be :small, :medium, :large, :ridulous
  where(:calorie_count).can_be 1000, 1500, 2000, 2500, 7500
end

To apply constraints to the possible values for an attribute, we use the where method. We specify the symbol used for the attribute name, then call can_be and provide it with a list of the possible values. These can be anything, as == will be used internally to compare values.

Now let’s take a look at how we define an environment:

environment_type :PostApocolypticFastFoodMMO do
  needs :mutant_maker, :mcmeal_creator
  has_attributes :environment, :region
  where(:environment).can_be :development, :testing, :production
  where(:region).can_be :asia, :europe, :north_america
  grouped_by :region
end

So above, we see the environment_type function. This works similarly to the host_type function - we provide it with a symbol to call the type and a block to configure it. Environments can have attributes, functioning the same way as hosts (they may be constrianed, etc), but also have a need method. This is where we specify what types of hosts must be present in an environment declaration for it to be valid. This is useful when you split out host definitions into a reusable component, and not all product environments will need all types of hosts. Finally, the grouped_by method is used. We specify an attribute and once an environment of this type is declared, it will be accessible in Hosties::GroupedEnvironments[type][groupingValue], for instance Hosties::GroupedEnvironment[:PostApocolypticFastFoodMMO][:asia] in the above example would give us a list of all PostApocolypticFastFoodMMO environments with a region of :asia.

Definition Error Handling

When an error is encountered in a definition, for instance an environment needing a host type that hasn’t been defined, or an attribute constraint for a non-existent attribute, the definition will be discarded and cannot be referenced. This will show up as environments are defined.

Environment Declarations

Once hosts and environment types have been defined, we can then proceed to declare them. Let’s define the production environment for our PostApocolypticFastFoodMMO product:

environment_for :PostApocolypticFastFoodMMO do
  environment :production
  region :asia
  mutant_maker "mutant1.hostname" do
    chainsaw_arms 80
    laser_cannon 22
    brutality "extreme"
  end
  mutant_maker "mutant2.hostname" do 
    chainsaw_arms 80 ; laser_cannon 22 ; brutality "medium" 
  end
  mcmeal_creator "meal.hostname" do
    drink_size :small
    calorie_count 1000
  end
end

To declare an environment, we use the environment_for function. It takes a symbol matching the type of a previously defined environment and a configuration block. Inside the config block, we assign values to declared attributes by using their names and providing a value. To create a host of one of the needed types, we use the name of the type and a hostname, then a config block. The config block for hosts works the same way, with some special case - services must have integer values (these represent the port the service is bound to).

Declaration Error Checking

When an error occurs with an environment declaration, an ArgumentError is raised providing (hopefully) descriptive test indicating what went wrong. This means that is someone has changed an environment declaration and they left out a required host or attribute, or set an attribute to an invalid value, it is caught immediately! Rather than having something deep in the guts of my deployment script break and scare off whoever was messing with it, things will break immediately and indicate that the environment declaration was invalid. Hopefully providing enough information for the user to perform some immediate action and fix the problem they introduced ;-)

Reading Environment Data

Now that we have a fully declared environment, let’s take a look at how you’d use it in your deployment scripts.

Updated 05May2013 21:21:43 for version 1.1.0.alpha

require 'hosties'
require 'our_mutant_mmo_environment'

asia_environments = Hosties::GroupedEnvironments[:PostApocolypticFastFoodMMO][:asia]
# Just grab the first one for example purposes
asia_environment = asia_environments.first

asia_environment.hosts_by_type(:mutant_maker).each do |host|
  puts "#{host.hostname} - Chainsaw arms available at #{host.chainsaw_arms}"
end

asia_environment.hosts_by_type(:mcmeal_creator).each do |host|
  puts "#{host.hostname} - drink size #{host.drink_size}, " +
      "total calories: #{host.calorie_count}"
end

# Or maybe we only want to interact with extemely brutal mutants
asia_environment.hosts_by_type(:mutant_maker).find_all { |host|
  host.brutality == "extreme"
}.each { |host|
  puts "Extremely brutal mutant @ #{host.hostname}. " +
        "Chainsaw arms available at #{host.chainsaw_arms}"
}
# Etc

More information

For more information, take a look at the RSpec files in /spec

Upcoming Changes

  • Comprehensive RDoc
  • Community feedback based changes

Conclusion

Hopefully someone other than myself can find a use for this. It was a great learning experience making this, as it involves some ruby metaprogramming black magic.