MacVim setup for Python programming

There are already tons of blogs about setting up vim for Python programming. I've been going through these excellent articles this weekend. This blog is to record what steps I took to set up my own MacVim with Python.

(Disclaimer: In order to put together information from all different sources, I shamelessly copied/pasted content from other blogs. I'd try my best to give credit back to the authors by adding links to their original sources.)

MacVim as Python IDE

Install MacVim with Python

Either of the following two options can be used to install MacVim.

Option 1: Through Mac ports

Make sure Mac ports are the latest.

$ sudo port -v selfupdate

Install MacVim with Python.

$ sudo port install macvim +cscope +python26

(Source: Pietra's technical journals)

Option 2: "Make" your own

$ git clone git://github.com/b4winckler/macvim.git
$ cd macvim/src
$ ./configure --enable-pythoninterp --with-macsdk=10.6
$ make

To install MacVim, type:

$ open MacVim/build/Release

and drag the MacVim icon into your /Applications folder.

(source: MacVim Github)

Add the following to ~/.profile or ~/.bash_profile.

alias vim=/Applications/MacVim.app/Contents/MacOS/Vim

To test if MacVim has python integrated, type

:python print "Hello, world!"

in MacVim. It should respond "Hello, world!".

Look and feel

The default MacVim color scheme isn't that appealing. I found dark background is easier for my eyes. Light background could put a strain on my eyes after staring at screen more than one hour.

To install a color scheme, first, create a directory:

$ mkdir ~/.vim/colors

Then, go here for a list of color schemes. Download the *.vim file, and put it to the ~/.vim/colors directory.

The Lucius color scheme with dark background is amazingly beautiful. This color scheme also supports light and blue themes.

Move lucius to the colors directory. Edit .vimrc to turn on syntax highlighting and set color scheme:

$ mv lucius.vim ~/.vim/colors
$ vi ~/.vimrc

set nocompatible

syntax on
colorscheme lucius
"colorscheme mustang
set background=dark

Now, open MacVim to try it out. You can switch among dark, light, and blue themes by typing :LuciusLight, :LuciusDark, and :LuciusBlue individually in MacVim.

Lucius light

(Mustang2 is another great color scheme with dark background. You can find it here. For some people, if macvim cannot find mustang, you might need to rename Mustang.vim to mustang.vim.)

As for font, I prefer Monaco in size 12.

set gfn=Monaco:h12

Here are some other settings for tab, indent, search, and line numbers:

set tabstop=4
set shiftwidth=4
set expandtab
set softtabstop=4
set smartindent
set autoindent

set hlsearch
set incsearch
set showmatch

set number

Open new files in tabs

By default, "vi a_new_file" in terminal will open a new MacVim window. You might want to put the newly the opened file in a new tab.

Edit mvim:

$ vi /opt/local/bin/mvim

Add the following line at the top of the file below the comments.

tabs=true

And replace the if structure at the bottom of the file with the following:

# Last step:  fire up vim.
if [ "$gui" ]; then
  if $tabs && [[ `$binary --serverlist` = "VIM" ]]; then
    exec "$binary" -g $opts --remote-tab-silent ${1:+"$@"}
  else
    exec "$binary" -g $opts ${1:+"$@"}
  fi
else
  exec "$binary" $opts ${1:+"$@"}
fi

Use <gt> to switch tabs.

(Source: Web Expose)

CTags and Tag List

CTags and Tag list give you an outline of classes, members, and functions in a left-side panel. Quite handy for code navigation.

Install ctags:

$ port install ctags

Install taglist:

Download here. Copy taglist.vim to ~/.vim/plugin

Add these two lines in ~/.vimrc to turn on file type detection.

filetype on
filetype plugin on

Run ctags on your project folder to generate a tags file which contains locations of classes, members, and functions in your project. For example, here we generate a tags file for all python source code in the "my_django_project" directory and its sub-directories.

$ cd my_django_project
$ ctags -R *.py

Please notice: if you already have a directory with a exact same name as "tags" under "my_django_project", you will get a "ctags: Failure on attempt to read file : Is a directory" error message. You can either rename your "tags" directory to something else, or change the location where the tags file will be generated. This is not within the scope of this blog but you can find more details here.

ctags can be configured to skip indexing certain types of code. The following command has ctags skip indexing python import statements.

$ ctags -R --python-kinds=-i *.py

To see what else can be skipped, type:

$ ctags --list-kinds=python

In ~/.vimrc, bind F8 to ctags command so we can re-generate tags on the fly.

nnoremap <F8> :!/opt/local/bin/ctags -R --python-kinds=-i *.py<CR>

In MacVim, type :TlistToggle to open the tag list. Use <C-ww> to switch between windows, <C-]> to jump to tag when the cursor is over a word, and <C-t> to go back. Pressing <space> on a function name in the tag list shows the function signature. For a full list of tag list key bindings, check out this blog.

I bound F4 to :TlistToggle.

nnoremap <F4> :TlistToggle<CR>

If you'd like to open tag list on right, add this line to ~/.vimrc:

let Tlist_Use_Right_Window = 1

Omni Complete

If you ever used Visual Studio, Eclipse, or other modern IDEs, you probably already knew what Omni Complete does. Omni Complete is the equivalent IntelliSense or code autocomplete for vim.

Add this line to ~/.vimrc to enable omni complete for python.

autocmd FileType python set omnifunc=pythoncomplete#Complete

The key combo () to toggle omni complete is quite awkward. Here I changed it to .

inoremap <C-space> <C-x><C-o>

OmniComplete

Task list

It is a common practice for programmers to mark TODO and FIXME in code. The TaskList plugin shows a list of such marks.

Download it here. Copy the vim file to ~/.vim/plugin

Type :TaskList to open the list.

Task list showing TODO

Pyflakes

Pyflakes analyzes Python programs and detects various errors. It is a must-have plugin for me.

Download the plugin here. Unzip it to ~/.vim/

$ unzip pyflakes-vim.zip -d ~/.vim/

PyFlakes

SnipMate

One of TextMate's cool features is snippet. Type "for" then press the <tab> key, a block of for statement is automatically generated. Vim can have the same feature with the SnipMate plugin.

Download the plugin here. Unzip it to ~/.vim/

This video demonstrates SnipMate in action.

FuzzyFinder

Another extremely useful plugin. What does it do? See it your self: video.
You can download it here. The L9 library is also required because FuzzyFinder depends on it.

Type :FufFile to search in files. I added the following line in ~/.vimrc to bind <C-f><C-f> to :FufFile.

nnoremap <C-f><C-f> :FufFile<CR>

In FuzyFinder, type <CR> to jump to deeper directories or open a selected file in the current tab. Type <C-l> to open selected file in a new tab. For more details of FuzzyFinder usage, go here.

NERDTree

"The NERD tree allows you to explore your filesystem and to open files and
directories. It presents the filesystem to you in the form of a tree which you
manipulate with the keyboard and/or mouse. It also allows you to perform
simple filesystem operations." (Marty Grenfell, vim.org)

Type :NERDTreeToggle to open/close NERD Tree.

In NERDTree, type t to open the file in a new tab. Type :Bookmark to set a bookmark on a file or directory. <B> will open or hide a list of all bookmarks. In the bookmark list, type D to delete a selected bookmark. Type <C> on a directory will change the current working directory to that directory. More commands can be found in this article.

I bound the command to F3:

nnoremap <F3> :NERDTreeToggle<CR>

MatchIt

MatchIt is a handy plugin that lets you jump from a opening tag to its paired closing tag, and vice versa. To see it in action, check out MrTutcasts's awesome video.

Download it here. Unzip it to ~/.vim/

Move your cursor to a HTML tag, e.g. <div>, then type %. The cursor will jump to its closing </div> tag.

Save your fingers

All these wonderful plugins involve a lot of strokes on the Ctrl key. To make your typing more pleasant, it is recommended to swap Caps lock with the control key.

Change the key bindings at System Preferences -> Keyboard -> Modify keys.

Other interesting vim plugins

tComment: toggle comments on one or more lines.
MRU: list most recently used files.
SearchComplete: Tab to complete search words in / search.
SuperTab: Tab to activate auto completion.

JSONP -- a cross-domain alternative to AJAX

AJAX utilizes XMLHttpRequest (XHR) APIs to send HTTP(s) requests to a web server and load server response directly in client-side script. XHR is the backbone of AJAX. It is widely used in so called web 2.0 applications, e.g. Google Gmail, Google Maps, and Facebook. Many libraries such as JQuery and YUI build on top of XHR to abstract the details and provide easy-to-use APIs for web developers and designers.

Unfortunately, XHR has a limitation. Due to the same origin policy, the server that receives the XHR requests and the client that sends out the requests need to be in the same domain. For example, the JavaScript in the page at www.example.com/demo.html can send out XHR request to www.example.com/service.php, however, it cannot send XHR requests to www.anotherexample.com/service.php, because example.com and anotherexample.com are two different domains.

Although it is meant to enforce web security, this policy created a common problem for web applications that need to consume external data (the data from external domain).

One of the solutions is to inject JavaScript coming from the external domain to the client page of the targeted domain. Because the injected JavaScript is evaluated in the client page, the script is treated as being from the same domain.

The following script inserts a "<script>" element to the head. The source of the inserted script points to the feed service at www.externaldomain.com, and passes along the "tag" parameter.

<script type="text/javascript">

var elHead = document.getElementsByTagName("head")[0];         
var script = document.createElement('script');
script.type = 'text/javascript';
script.src = 'http://www.externaldomain.com/services/feed?tag=gaming';
elHead.appendChild(script);

</script>

The feed service at www.externaldomain.com takes the "tag=gaming" parameter as an input argument, retrieves a list of feeds related to gaming, and convert the gaming feeds into a JSON string. For example:

'{"feeds" : [ { "title" : "game1", "date" : "03-21-2011", "author" : "David Smith" }, { "title" : "game2", "date" : "03-22-2011", "author" : "Steve Yavorski" }, { "title" : "game3", "date" : "04-05-2011", "author" : "Kelly Lee" } ]}'

However, the service at externaldomain.com can not simply return this JSON string as response data. The "src" attribute of the script element that we're injecting should point to a JavaScript instead of a JSON string. The JSON string itself cannot be evaluated to lines of runnable JavaScript code. So what we need to do is to wrap the JSON string in JavaScript.

<script type="text/javascript">
var responseText = '{"feeds" : [ { "title" : "game1", "date" : "03-21-2011", "author" : "David Smith" }, { "title" : "game2", "date" : "03-22-2011", "author" : "Steve Yavorski" }, { "title" : "game3", "date" : "04-05-2011", "author" : "Kelly Lee" } ]}';
</script>

The above JavaScript code will be executed in the client page. JavaScript in the page is now able to parse variable responseText to get the gaming feeds.

<script type="text/javascript">
var feeds = parseJsonStr(responseText); // parseJsonStr is a pseudo function
</script>

What we did above can be summarized as below:

• Inject a "<script>" element to the HTML head
• Point the "src" attribute to an external service that takes parameters and gets response data
• Wrap the response data in JavaScript
• Reference and parse the response data in JavaScript

The 3rd step "Wrap the response data in JavaScript" is also called JSON Padding, and this is where JSONP comes from.

The above approach has two problems. First, we don't quite know when the injected script finishes loading and when the responseText variable is ready to be consumed. Second, we don't want to hardcode the variable name to "responseText". The server shouldn't dictate what name the variable should be. To fix these problems, we can implement a callback function that will be invoked when the response is ready.

<script type="text/javascript">
function callback(responseText /* or whatever name you want to give */) {
  var feeds = parseJsonStr(responseText);

  // Do something about feeds ...

}
</script>

On the server side, the generated JavaScript will call the callback and pass in the JSON string as an input argument to the function:

<script type="text/javascript">
callback('{"feeds" : [ { "title" : "game1", "date" : "03-21-2011", "author" : "David Smith" }, { "title" : "game2", "date" : "03-22-2011", "author" : "Steve Yavorski" }, { "title" : "game3", "date" : "04-05-2011", "author" : "Kelly Lee" } ]}');
</script>

This way, we captured the moment when the response is available, and removed the naming of the JSON string from the server side.

To make things better, the name of the callback function should not be hardcoded either. We can tell the server which callback function to call by passing the name of the callback function in the URL. Here we adjust the JavaScript injecting code a bit.

<script type="text/javascript">

function onDataReceived(responseText) {
  var feeds = parseJsonStr(responseText);

  // Do something about feeds ...

}

var elHead = document.getElementsByTagName("head")[0];         
var script = document.createElement('script');
script.type = 'text/javascript';

// callback=onDataReceived
script.src = 'http://www.externaldomain.com/services/feed?tag=gaming?callback=onDataReceived';

elHead.appendChild(script);

</script>

The server-side code takes the "callback=onDataReceived" parameter and passes the JSON string to the onDataReceived callback function:

<script type="text/javascript">
onDataReceived('{"feeds" : [ { "title" : "game1", "date" : "03-21-2011", "author" : "David Smith" }, { "title" : "game2", "date" : "03-22-2011", "author" : "Steve Yavorski" }, { "title" : "game3", "date" : "04-05-2011", "author" : "Kelly Lee" } ]}');
</script>

Implementation summary

What client side needs to do?

• Inject a "<script>" element to the HTML head
• Encode input arguments as query parameters into the URL of the script element's src attribute
• Include the name of the callback function to the URL
• Point the src attribute to an external service
• Define the callback function that takes JSON string as input argument
• Parse the JSON String in the callback function

What server side needs to do?

• Implement service code to answer the client requests, and expose the service through HTTP(s)
• Get all input arguments from the query parameters
• Get the callback function name
• Convert response data into JSON string
• Pass the JSON string to the callback function

Security concern

The technique of the JavaScript injection is also employed in some Cross-site Scripting (XSS) attacks. Since the consumer of the external service has no control of the returned script, the consumer can be vulnerable to XSS attacks that are introduced by the returned script from the external service. It is recommended only applying this technique for trusted external services.