How To Use Fabric for Automating Deployment and Database Tasksby Shahid Ayoub
Fabric is a Python library and command-line tool for streamlining the use of SSH for application deployment or system administration tasks. We usually use some basic commands again and again while deploying the projects. In order to reduce the overhead of repeating the same steps we use fabric for automating the steps. Here , I am going to focus mainly on automating deployment and database tasks. Application deployment also involves use of VCS (Version Control System) such as Git for source code management. The commands used by git can also be automated using fabric.
We usually come across various scenarios where we have to activate virtual environments, install requirements, push, pull and merge branches, drop old databases, create new ones based on the model changes in our project. Moreover, at times we have to take dump of databases and transfer it to our local machine or vice versa. All these tasks can be automated using fabric. We can achieve all this by following the below mentioned steps :
- Install fabric using below mentioned command.
pip install fabric
sudo apt-get install fabric
- Create a script file i.e., fabfile.py where all tasks to be executed are added .
- Define the functions for automating the deployment and database tasks .
Before defining the fab functions. Let us discuss some of the important features and helpers of fabric.
- run (fabric.operations.run) – Fabric’s run procedure is used for executing a shell command on one or more remote hosts.
- local (fabric.operations.local) – Fabric’s local operative is used for executing shell commands on a local machine .
- get (fabric.operations.get) – Get command is used to download the file from the remote machine .
- put (fabric.operations.put) – Put command is used to upload the file to the remote machine .
- lcd (fabric.context_managers.lcd) – In fabric we use lcd to set the directory state on the local machine where we can execute the desired commands .
- cd (fabric.context_managers.cd) – It is similar to lcd but is used to set the directory state of the remote machine.
- env (The environment dictionary) – It is a python dictionary where keys stand for the environment variables. Fabric have some in-built environment variables like env.host defining the remote hosts, but we can define our own variables as well.
Now let us see how can we use the above mentioned features in our frequently used tasks during deployment.
- At first, import the above mentioned features and helpers from fabric api.
from fabric.api import local from fabric.api import lcd,cd from fabric.api import run, env
- Define the env variables as per our requirements. For example define the remote host, password etc.
# defining IP address ,username and password of remote machine env.hosts = ['IP address of the remote machine'] env.user = "username" env.password = "password" # defining username and password for the database env.db_user = "username" env.db_passwd = "password" #setting up the project directory and virtual environment paths env.dir = "/path/to/remote or local directory" env.activate = "/bin/bash --rcfile '/path/to/virtual environment/bin/activate'"
- Prepare the deployment on local machine and push the changes to remote machine.
def prepare_deployment(): local('git add *') comment = raw_input("Enter your commit comment:") local('git commit -m "%s"' % comment) lbranch = raw_input("Enter the name of the Local branch you want to Push:") rbranch = raw_input("Enter the name of the Remote branch:") local('git push origin %s:%s' %(lbranch,rbranch))
- Deploy the changes on the remote machine.
def deploy(): with cd(env.dir): pull() cbranch = raw_input("Enter the name of the branch you want to checkout: ") mbranch = raw_input("Enter the name of the Branch you want merge:") run('git checkout %s && git merge %s' %(cbranch,mbranch)) restart() #restarting the remote server def restart(): run('sudo service apache2 reload') #pulling the changes from the remote server from a specific branch def pull(): pbranch = raw_input("Enter the name of the remote branch you want pull:") local('git pull origin %s' %(pbranch))
- Activate the virtual environment and install requirements.
def activate(): run(env.activate) #env.activate is already defined in step first. def install(): with cd(env.dir): run("pip install -r requirements.txt")
- Dropping the old database, creates a new database and synchronizes the new model changes.
def rebuild_db(): env.db_schema = raw_input("Enter the name of Database you want to drop: ") run('mysql -u %s -p%s -e "drop database if exists %s"' % (env.db_user, env.db_passwd, env.db_schema)) env.db_schema = raw_input("Enter the name of Database you want to create: ") run('mysql -u %s -p%s -e "create database %s"' % (env.db_user, env.db_passwd, env.db_schema)) with cd(env.dir): run('python manage.py makemigrations') run('python manage.py migrate') run('python manage.py syncdb')
(Caution : Be careful while using this fab function since it will delete all the data in the database.)
Take the dump of the remote database and transfer this on your local machine.
def remote_dump(): env.dbname = raw_input("Enter the name of Database you want to dump : ") env.dump = raw_input("Enter the name you want to assign to dump : ") run('mysqldump -u %s -p%s %s > %s/%s.sql' %(env.db_user,env.db_passwd, env.dbname,env.rdir,env.dump)) env.dump1 = raw_input("Enter the name you want to assign to local dump : ") get(env.rdir+"/"+env.dump+".sql",env.ldir+"/"+env.dump1)
All the above mentioned code snippets are to be included in the fabfile.py. Now, go to the directory where fabfile.py is located and run the fab function_name command on terminal to execute a particular fab function.
Stay tuned for more on fabric in upcoming blogs.