Automatic installation of commonly used packages

A number of years ago I was introduced to using Linux day to day. During this period of time I ran a large number of virtual machines to test its performance and various aspects which make it “linux”.

Typically when I first set up a system I would install a common set of packages. Then once this had been done, I would install packages relevant to the system I was installing. I wouldn’t install games on my server for example. During this period I wrote a piece of software called the Setup Spider.

What is the Setup Spider?

I wanted something really simple to both keep track of the packages that I cared about and install them for me automatically. This is what I have been using Setup Spider for.

At a basic level I list the packages that I want to install in the setup file, and the type of system. This type could be apt, yum or dnf and the packages are just a list of package names. Once added to my script it calls install() and the packages are installed according to the package manager.

Initially I was going to have a single list of packages for all operating systems. This worked for a very short period of time but I quickly hit a problem. For various reasons the naming of packages are not standard across major distributions. Even more annoying some packages are just not available for certain OS’s.

This led me to splitting up the packages into their own folders. As I upgraded Fedora I decided to branch my git repository for each version (fc17, fc18…). This was working until I started using other Linux distrubtions more.

Improving Setup Spider

Setup Spider was written a long long time ago so wasnt very well done. It worked well for what it did but wasnt entirely nice to use. Recently I decided to revisit the Setup Spider and add some improvements.

Now the scripts, instead of storing the package data in python files use json files to store the data. The python scripts are now used to manage the data in the json file.

I flattened all of the branches and moved them back into master. This added another level of directories to store files in {distribution}/{version}.json format.

I plan to be using some of this for the Raspberry Pi cluster. This is so I may quickly deploy packages to a new clone. However I am looking to in the future move to something more fully featured like Puppet. This should make managing the cluster a little easier.

Setup Spider is available on GitHub under the MIT licence. if you use it and its helpful please do contact me 🙂