Building smaller secure docker images with Multi-Stage builds
April 10, 2019

Docker v17.05 introduced multi-stage builds giving us more ways to package and secure our applications. This post will show you how to use the power of the multi-stage builds for your projects. Check out the below Dockerfile and try to guess what we are trying to achieve here. We will go through each of the steps in detail right after that.

# Step 1
FROM node:10-alpine AS base
WORKDIR /app
COPY package.json .

# Step 2
FROM base AS dependencies
RUN npm install --only=production
RUN cp -R node_modules prod_node_modules
RUN npm install

# Step 3
FROM dependencies AS build
COPY . .
RUN npm run build

# Step 4
FROM base AS final
COPY --from=dependencies /app/prod_node_modules ./node_modules
COPY --from=build /app/dist ./dist
COPY --from=build /app/entrypoint entrypoint
RUN chmod +x entrypoint
EXPOSE 3000
ENTRYPOINT [ "sh", "entrypoint" ]

Now let us go through the multiple stages we have used to create our final image -

  1. Taking node:10-alpine image and naming it as base so that we can use it later in our build stage. This build stage will have only basic instructions. Here we have mentioned where our working directory will be an added our node projects package.json file to the working directory.

  2. We now take the base image and start building our dependencies image. Firstly we install only production dependencies then copy the node_modules directory to prod_node_modules directory. Then we install our dev dependencies on top of it. Step 4 will unravel why we did this.

  3. We then take the dependencies image and start building our build image. We copy all the files of our project to the working directory and start compiling/building our project. In this case, the source is written in Typescript and we are compiling it to Javascript.

  4. Now we pick only required directories for our final image. Which includes -

    • copying production node_modules from our dependencies image.
    • copying our compiled/built code form build image.
    • copying our entrypoint script from build image and giving it appropriate permissions.
    • exposing the port for the application.
    • adding the entrypoint. In this case, it is sh entrypoint.

Pros

The benefits of using multi-stage builds are -

  • only the necessary files are kept in the final image. In this case, source files are not copied when we prepare the final image. This reduces the size of image.
  • small image sizes reduce the deployment time.
  • keeping the files and dependencies to the minimum improves security.

Hope you learned something new today!

Related reads

Use multi-stage builds - https://docs.docker.com/develop/develop-images/multistage-build


Useful links - Software Orchestration Edition
June 22, 2017

git

Most commonly used git tips and tricks - Git-tips

ansible

Automating Django Deployments With Fabric and Ansible - Realpython.com blog

docker

A handy CLI in Docker container to make optimum use of git tips - Docker git-tips

Docker 101 workshop - introduction to Docker and basic concepts - Gravitational Workshops - docker

Build and deploying heroku apps with docker - Heroku + Docker

kubernetes

Kubernetes 101 workshop - introduction to Kubernetes and basic concepts - Gravitational Workshops - kubernetes


Python Gotchas
June 14, 2017

gotcha: n.

A misfeature of a system, especially a programming language or environment, that tends to breed bugs or mistakes because it is both enticingly easy to invoke and completely unexpected and/or unreasonable in its outcome.
For example, a classic gotcha in C is the fact that if (a=b) {code;} is syntactically valid and sometimes even correct. It puts the value of b into a and then executes code if a is non-zero. What the programmer probably meant was if (a==b) {code;}, which executes code if a and b are equal.

Here are some interesting python gotchas -

>>> value_or_values = 1, 
>>> # creates a tuple (1,)

>>> value_or_values[False]
>>> 1
>>> # returns value 1

>>> value_or_values[True]
Traceback (most recent call last):
    File "<stdin>", line 1, in <module>
IndexError: tuple index out of range
>>> # See what happened there?

>>> isinstance(True, int)
>>> True
>>> # I am sure you must have figured it out now.

>>> # Very bad example of getting a list of even numbers but still ...
>>> # It gets us the desired result here, but never remove elements from a running list.
>>> li = range(10)
>>> for value in li:
>>> ...:     if value % 2 != 0:
>>> ...:      li.remove(value)
>>> ...:
>>> print(li)
>>> [0, 2, 4, 6, 8]

>>> # more soon

References :

Definition: Gotcha


Setup Apache Spark and Initialize pyspark with ease
November 9, 2016

Recently I started working on Apache Spark and was having a hard time configuring spark on my machine. It became all try and catch while setting up the environment. Then I came across this gist which explains how to set up Apache Spark with IPython Notebook on MacOS. I love to work on IPython notebooks but most of the times I just need a simple ipython shell. So I skipped the IPython notebook portion of the setup.

After tweaking a bit I got the desired setup. I came up with the below bash alias to use pyspark with ipython -

alias ipyspark='PYSPARK_DRIVER_PYTHON=ipython /usr/local/bin/pyspark --master local[*] --driver-memory 2g'

You can read more about the above parameters in the documentation here and use the parameters as per your requirement.

Now comes the best part. Once you are inside ipython shell you would want to initialize pyspark as fast as possible. For that I found out findspark. Just install findspark using pip install findspark and start using pyspark with ease -

>>> import findspark
>>> findspark.init()
>>> import pyspark
>>> sc = pyspark.SparkContext(appName="myAppName")

Happy hacking!

Related Reads -

Getting started with spark? Check out this repo - http://jadianes.github.io/spark-py-notebooks

MOOC on edx.org - https://www.edx.org/course/big-data-analysis-apache-spark-uc-berkeleyx-cs110x


Hello, World
September 29, 2016

Welcome to my blog. I am a web developer based out of New Delhi, India. I will use this blog to write about python, django and a lot of development related stuff.

# views.py
from django.shortcuts import render

def about_this_blog(request):
    '''This page displays interesting features of this blog.
    '''
    return render(request, template_name='interesting_features.html')

Few things about this blog -

This platform is developed using Django which helps in building better web apps more quickly and with much less code. It is a framework built for perfectionists and I am very adamant to get there one step at a time.

I have used really great open source projects for building this website and I want to thank the developers by mentioning about their projects here. Lets check them out.

  • twitter-bootstrap with font-awesome - Creating web apps has never been easier for a backend developer who wants to spend less time worrying about how the frontend would look like. Bootstrap should be in your web development tools arsenal.

  • django-markdownx - It provides a very easy to use Markdown editor with a lot of features like raw editing, live preview, drag and drop image uploads and much more. You should definitely use it for your django projects.

  • pygments-css with pygments - Used for creating beautiful syntax highlighter like the one above. I have used the Monokai theme.

  • django-crispy-forms - creating forms in django has never been easier. crispy-forms helps us control the rendering behavior of Django forms in a very elegant and DRY way.


And of course this project is deployed on Heroku and will soon be on DigitalOcean.

Keep visiting for more.

Cheerio! o/