see whatever…

jump to menu

May 17, 2008

Compiling Python modules extensions on Windows

Filed under: Python — see @ 8:56 pm

Obviously I have no real knowledge of C and am working mostly on Windows which does not have a C compiler installed normally. But I sometimes need a Python module which use C extensions (like lxml or simplejson). Problem is if no egg is available until now I had no way to install the module :(

I always wanted to get this to work and it is not that difficult…

A reminder for me (and maybe other folks who are interested) here the steps I used found (great overview) on various (additional hack) websites combined in a simple step by step overview. (Used on Windows Vista with Python 2.5.2 and the versions stated below.)

  1. Download MinGW, the Automated Installer is sufficient. I just selected “g++” and “make”, am not sure if that is needed afterall. Version I used is 5.1.4. Install it to “c:\mingw” (recommended for Vista and I only tried under Vista yet)
  2. Set MingW32 as the compiler for distutils by adding %PYTHONHOME%\lib\distutils\distutils.cfg with the following content:
    [build]
    compiler = mingw32
  3. The current version of MingW seems to trigger a small problem with distutils so you need to hack %PYTHONHOME%\lib\distutils\version.py and replace the definition of version_re (no actual functionality but just a new versioning system which seems to confuse distutils) with this:
    version_re = re.compile(r'^(\d+) \. (\d+) (\. (\d+))? (\. (\d+))? ([ab](\d+))?$',re.VERBOSE)
  4. save the following commands in a batch file which do set all needed environment vars etc:
    @echo off
    
    echo MinGW Enviroment Command Console
    echo ....
    echo ....
    @set MINGWROOT=C:\MinGW
    @set MINGWBIN=%MINGWROOT%\bin
    @set MINGWINCLUDE=%MINGWROOT%\include
    @set MINGWLIB=%MINGWROOT%\lib
    @set MINGWLIBEXEC=%MINGWROOT%\libexec\gcc\mingw32\3.4.5
    @set MINGWBIN2=%MINGWROOT%\mingw32\bin
    @set MINGWLIB2=%MINGWROOT%\mingw32\lib\ldscripts
    @set MINGW=%MINGWROOT%;%MINGWBIN%;%MINGWINCLUDE%;%MINGWLIB%;%MINGWLIBEXEC%;%MINGWLIB2%;%MINGWLIB2%
    
    rem to add more resource paths just use the set command like above and the and it to the set PATH like below
    
    @set Path=%MINGW%;%Path%

All is set so you can start to try it out. Start a command prompt and run the batch file created in step 4. Now use easy_install to install the lib you need (you may also use a simple setup.py install but I find easy_install so much easier and most projects use it nowadays):

easy_install "simplejson==1.9.1"

which should download simplejson version 1.9.1, compile the C extensions and save an egg to %PYTHONHOME%\Lib\site-packages\simplejson-1.9.1-py2.5-win32.egg.

Not that difficult after all (if you have the help of the pages linked to above ;) ). Guess I should have tried it earlier as I actually have used jsonlib in favor of simplejson for the simple fact that jsonlib was easier to install but simplejson seems to be in favor for going into the stdlib.

1 Comment »

  1. Very useful post indeed.

    I’m trying to get my head around python and compiling modules and I have to thank you for your post:)

    thanks

    Comment by Mihai — August 31, 2009 @ 8:07 pm

RSS feed for comments on this post. TrackBack URL

Leave a comment

Powered by WordPress