Cloning multiple git repositories from a remote host with clone-all-the-repos

This post goes into details about what the clone-all-the-repos GitHub repository does and why it does it.

What is clone-all-the-repos GitHub repository

I have a number of git repositories on a couple of servers that I want to clone at once. Manually doing this each time I set up a development environment is tedious so I wanted to script it up.

Originally there was a quick script written for this but when using it recently I decided to clean it up and publish it.

The script follows the process:

1. Recursively loop through directories to find git repos (This is found using guessing if something is a git repository based on known files and folders)
2. Create “git clone” commands, that can be used by the remote machine
3. Create “mr register” commands, if that option has been provided

The newly cleaned script is now available on GitHub under the project clone-all-the-repos.

Using clone-all-the-repos

A basic example of the command is:

python get_clone_script.py ~/Documents/git/ hostname --mr-register

The first argument to the script is the root directory that the script will search from. From here it will search all subdirectories to see if It can find any git repositories.

The second argument of the script is the hostname. This will be used when constructing the clone command. The hostname is used to create a clone command such as

git clone foobar:~/Documents/git/repo1.git repo1.git

Here foobar is the value we passed to the script as the hostname parameter. Depending on your git setup this can be a fully qualified domain name, such as hostname.foobar.com, or a ssh alias such as foobarhost.

Finally I tell the script I want to output the mr register commands following the clone by passing --mr-register. If this isn’t provided the script will not print out the mr register commands.

A final optional argument is --git-clone-command which allows you to change what the script uses in place of git clone. This could be used to add additional flags to git clone, or adding a custom cloning command.

Future Improvements

One of the improvements I am going to add in the future is to add some better logic for finding and checking repositories.

The current logic for finding repositories is probably sufficient but I could add further logic to run a git command and check to see if it identifies it as a git repository.