Useful links - Software Orchestration Edition
June 22, 2017

git

Most commonly used git tips and tricks - Git-tips

ansible

Automating Django Deployments With Fabric and Ansible - Realpython.com blog

docker

A handy CLI in Docker container to make optimum use of git tips - Docker git-tips

Docker 101 workshop - introduction to Docker and basic concepts - Gravitational Workshops - docker

Build and deploying heroku apps with docker - Heroku + Docker

kubernetes

Kubernetes 101 workshop - introduction to Kubernetes and basic concepts - Gravitational Workshops - kubernetes


Python Gotchas
June 14, 2017

gotcha: n.

A misfeature of a system, especially a programming language or environment, that tends to breed bugs or mistakes because it is both enticingly easy to invoke and completely unexpected and/or unreasonable in its outcome.
For example, a classic gotcha in C is the fact that if (a=b) {code;} is syntactically valid and sometimes even correct. It puts the value of b into a and then executes code if a is non-zero. What the programmer probably meant was if (a==b) {code;}, which executes code if a and b are equal.

Here are some interesting python gotchas -

>>> value_or_values = 1, 
>>> # creates a tuple (1,)

>>> value_or_values[False]
>>> 1
>>> # returns value 1

>>> value_or_values[True]
Traceback (most recent call last):
    File "<stdin>", line 1, in <module>
IndexError: tuple index out of range
>>> # See what happened there?

>>> isinstance(True, int)
>>> True
>>> # I am sure you must have figured it out now.

>>> # Very bad example of getting a list of even numbers but still ...
>>> # It gets us the desired result here, but never remove elements from a running list.
>>> li = range(10)
>>> for value in li:
>>> ...:     if value % 2 != 0:
>>> ...:      li.remove(value)
>>> ...:
>>> print(li)
>>> [0, 2, 4, 6, 8]

>>> # more soon

References :

Definition: Gotcha


Setup Apache Spark and Initialize pyspark with ease
November 9, 2016

Recently I started working on Apache Spark and was having a hard time configuring spark on my machine. It became all try and catch while setting up the environment. Then I came across this gist which explains how to set up Apache Spark with IPython Notebook on MacOS. I love to work on IPython notebooks but most of the times I just need a simple ipython shell. So I skipped the IPython notebook portion of the setup.

After tweaking a bit I got the desired setup. I came up with the below bash alias to use pyspark with ipython -

alias ipyspark='PYSPARK_DRIVER_PYTHON=ipython /usr/local/bin/pyspark --master local[*] --driver-memory 2g'

You can read more about the above parameters in the documentation here and use the parameters as per your requirement.

Now comes the best part. Once you are inside ipython shell you would want to initialize pyspark as fast as possible. For that I found out findspark. Just install findspark using pip install findspark and start using pyspark with ease -

>>> import findspark
>>> findspark.init()
>>> import pyspark
>>> sc = pyspark.SparkContext(appName="myAppName")

Happy hacking!

Related Reads -

Getting started with spark? Check out this repo - http://jadianes.github.io/spark-py-notebooks

MOOC on edx.org - https://www.edx.org/course/big-data-analysis-apache-spark-uc-berkeleyx-cs110x


Hello, World
September 29, 2016

Welcome to my blog. I am a web developer based out of New Delhi, India. I will use this blog to write about python, django and a lot of development related stuff.

# views.py
from django.shortcuts import render

def about_this_blog(request):
    '''This page displays interesting features of this blog.
    '''
    return render(request, template_name='interesting_features.html')

Few things about this blog -

This platform is developed using Django which helps in building better web apps more quickly and with much less code. It is a framework built for perfectionists and I am very adamant to get there one step at a time.

I have used really great open source projects for building this website and I want to thank the developers by mentioning about their projects here. Lets check them out.

  • twitter-bootstrap with font-awesome - Creating web apps has never been easier for a backend developer who wants to spend less time worrying about how the frontend would look like. Bootstrap should be in your web development tools arsenal.

  • django-markdownx - It provides a very easy to use Markdown editor with a lot of features like raw editing, live preview, drag and drop image uploads and much more. You should definitely use it for your django projects.

  • pygments-css with pygments - Used for creating beautiful syntax highlighter like the one above. I have used the Monokai theme.

  • django-crispy-forms - creating forms in django has never been easier. crispy-forms helps us control the rendering behavior of Django forms in a very elegant and DRY way.


And of course this project is deployed on Heroku and will soon be on DigitalOcean.

Keep visiting for more.

Cheerio! o/