Gprof2dot

Profiling

I want to share with you a nice trick which, even if not fundamental, is still very useful.
You can also use this to impress your girlfriend.

Sometimes profiling is necessary to understand which part of our code are taking so long.
Python offer different alternatives, I advice you to read the official python documentation to get an idea.

Gprof2Dot

Another very nice thing is gprof2dot .
This little program takes into input profiling data in many different formats and converts it to a dot file, keeping all the valuable informations.

As you can see for example here .
You also need to have graphviz to be able to actually generate the final graph.

Scripting and emacs integration

Given that I want to automate it and I want to use it also from emacs I wrote a simple bash script:

#!/bin/bash
FNAME=$1
if [ ! -f $FNAME ]
then
    echo "file $FNAME not found, executing from $PWD"
    exit 1
fi
# discard filename
shift

# change those variables as you prefer
STATS="output.pstats"
TYPE=png
OUT=output.$TYPE
OPEN="open"

# $@ collect all the arguments for the python script
# it doens't contain the file name after the shift
python -m cProfile -o $STATS $FNAME $@
gprof2dot.py -f pstats $STATS | dot -T$TYPE -o $OUT
rm $STATS
$OPEN $OUT

You can find this script also in my emacs configuration .

Now we’re almost done, just put this somewhere in the emacs exec-path and you’re ready to use it!
Just press Meta-! on a python buffer and launch “gprofile.sh python_file.py [extra args..]“

To automate even more this little elisp functions does that for you:

(defun gprof ()
  (interactive)
  (shell-command (concat "gprofile.sh " buffer-file-name)))

But at the moment it doesn’t take arguments for the script (will fix it soon).

elisp fun

Problem

As we all know programmers are lazy, repetition is the evil, we want to type as fast as possible.
Using C-like languages you always need to put the comma in the end.

Normally after the comma you also want to go to newline and indent, so why not put together those two things and automate it?

If you also like yasnippet then you’ll find this feature even nicer.
I took from this feature textmate , a wonderful text editor which comes with this setting by default.
As we’ll see we don’t need many lines of elisp code to end up with a much more powerful and flexible version of this nice feature.

Solution

newline-force

We first define a first function newline-force
Here it is:

(defun newline-force()
  "Goes to newline leaving untouched the rest of the line"
  (interactive)
  (progn
    (end-of-line)
    (newline-and-indent)))

This function goes to end of line and calls the predefined function newline-and-indent, which goes to newline and indent according to mode
In more detail:

• interactive: function is available for user
• progn: executes a sequence of disjointed operations

Github behind a proxy

Suppose you have an account to github and you want to use it even when you are behind a very restrictive proxy.
The problem arises “only” when you have to push new patches, given that you can clone/pull even using http out of the box.

Luckily the github guys give us the access to another server (ssh.github.com) which basically listen on the port 443 and forwards

The informations to set up your environment can be found here:

github over proxy

And here you find:
corkscrew

This for example is my configuration (on Macosx but on linux is the same):

SSH_CONFIG

Host gitproxy
  User git
  HostName ssh.github.com
  Port 443
  ProxyCommand corkscrew <proxy> <port> %h %p
  IdentityFile /Users/andrea/.ssh/github

Now in your git repository you can simply add the new remote

git remote add git@gitproxy:<repository>

Another nice way to bypass the proxy (thanks Christian) is to use the GIT_PROXY_COMMAND variable, in this way

export GIT_PROXY_COMMAND=/usr/local/bin/proxy-cmd.sh

#!/bin/bash
(echo "CONNECT $1:$2 HTTP/1.0"; echo; cat ) | socket proxy.unitn.it 3128 | (read a; read a; cat )

And now git for every push/pull will execute that script and bypass the proxy.

Multilanguage and multiparadigm navigator 3/n Python implementation

Now it starts the interesting part, we’ll actually go to see one language a time the real program.
The program that given the input we’ve seen in the previous chapter gives us a kind of useful output.

The program is divided in a few different parts:

Parsing command line options

Setting up data structures needed

Executing Floyd Warshall algorithm

Building the output and writing it to file

Optionally we can also graph the result using the library pydot and passing the “-d” flag from the command line.

Multilanguage and multiparadigm navigator 2/n Input generator

In this second post we’ll see how to implement correct and configurable input generator for our problem.

This script is used to generate two simple output files in the form:

Mapfile: [Number of cities] [List of cities] [Distance table]

PathFile: [List of cities]

both separated by dots.

For example the map file could be:

5.CASALBELTRAME.FONTANILE.GASPERINA.INVERSO.MEANA.0.4.-1.-1.-1.4.0.8.6.9.-1.8.0.-1.-1.-1.6.-1.0.-1.-1.9.-1.-1.0.

And the path file:

INVERSO.CASALBELTRAME.MEANA.GASPERINA.INVERSO.

The source code here below can also be downloaded together with a list of cities (necessary to make the randomization).

Get the k-th digit out of a number

One really common problem is to get the k-th digit out of a number.
For example given the number

10210

we want to know what it the second digit.

We’ll see how we can solve elegantly this problem in python.

Implementation of algorithms for inference of recursive types

I uploaded my thesis thesis and a shorter abstract , the last one in italian and english.

Multilanguage and multiparadigm navigator 1/n Introduction

In this series of post (where n > 1 and probably < 10) we’ll see how to implement a simple “navigator” in different languages.
This was a project I did for my university a long time ago that appeared to me interesting enough to improve it and publish it.
For navigator we mean a program that, given:

a table with distances between cities

a path

It returns the shortest path possible (if there’s one) with the total distance calculated.

The algorithm used is mainly Floyd Warshall which calculates iteratively the nth-distance between couple of cities and creates a table.

After that the only “problem” is building up the path of shortest paths.

The problem has been coded in very different languages (using the specified compiler):

Prolog: swi prolog

lisp: sbcl

Python: python

In plus there is an input generator (still in python), a performance analyzer (in bash + gnuplot) and a nice grapher (in python + pydot) to get a graphical view of the problem’s instances.
In this first chapter we’re just going to briefly analyze the how the input/output are formatted and the general structure of the program.

The full source code (and much more) that I’m going to show can be found (maybe more updated) on my github page navigator .

You can download everything by simply cloning

  git clone git://github.com/AndreaCrotti/navigator.git