IronPython on MSDN TV

I recently taped a set of IronPython demos and they're now up on the MSDN web site.  It shows some interactive exploration and GUI building from the command-line along with a brief interlude on subclassing a C# class from Python.  It also shows how to simply embed a Python engine in a WPF application.  Take the MSDN folks seriously when they suggest disabling your pop-up blocker or you'll just get to see the fairly boring part of me talking and you'll miss out on the interesting part of the actual demos.  If you do watch the demos, please fill out the ratings as well as this helps MSDN decide what sort of content they want to present in the future.
posted by hugunin with 0 Comments

We shipped 2.0!

We shipped the .NET Framework 2.0!  Okay, I know that I’m not close to the first person to announce this, but I’m still excited.  .NET 2.0 is a great platform for dynamic languages like IronPython to build on top of with several great features:

  • System.Reflection.Emit.DynamicMethod fills the biggest hole in the CLR for supporting dynamic languages.  It allows cheap generation of code on the fly and most importantly the code it generates doesn't need to live for the full life of the application but can be reclaimed when there are no more references.  This ability to generate is reclaimable code is one that the JVM has had for a long time at the ClassLoader level and it's vital to effectively building a dynamic language.  What I like particularly about DynamicMethods is that it supports generating code on the granularity of a single method and then exposing that with delegates, the CLR's strongly typed managed function pointers.
  • Speaking of delegates, 2.0 saw some major improvements to their performance bringing them extremely close to regular method calls in performance.  I've found delegates to be very valuable for implementing Python and these performance improvements are the main reason that IronPython is considerably faster on .NET 2.0 than it was on 1.1.
  • Finally, generics were added to the CLR in a deep and consistent way.  There are lots of interesting arguments pro and con as to whether or not the benefits of generics outweigh the price of their complexity.  However, if you are going to add generics to a system it’s vital to do it in a consistent way.  The key element for dynamic languages is that generics are not implemented via compile-time tricks of type erasure, but are deeply embedded in the runtime so that System.Reflection and other runtime mechanisms can just do the right thing.  This meant that many generic methods just worked automatically in IronPython with proper type checking with no changes and that it only took about a dozen lines of code to completely support instantiating generic types.

Of course, what I’m most excited about with shipping .NET 2.0 is that I now get to focus full time on the future dynamic language story for .NET.  In the short-term that will mean a push to resolve the last few design issues before an IronPython-1.0 release.  In the longer-term it should mean even more interesting things…

posted by hugunin with 1 Comments

IronPython 0.7.1 is released to the world!

I'm excited to announce the public release of IronPython 0.7.1.  The community site is up on http://workspaces.gotdotnet.com/ironpython; however, you should know that most of the interesting discussions are happening on the old mailing list.  I find it really cool to go to a standard Microsoft download page and see a Python implementation, even if it is just an early alpha release.  Some of you might have already heard of and even downloaded this release.  Two weeks ago at PyCON I got to announce the first public release from Microsoft of IronPython 0.7.0.  I wasn't brave enough to start blogging again until we got 0.7.1 out the door.  The 0.7.1 release fixed over 80% of the bugs the community reported in 0.7.  Now that we've made two releases in two weeks I'm confident in our plan to make frequent releases of IronPython in response to community feedback and drive fast and hard to 1.0.

Nevertheless, this is still an early alpha release.  If you were at PyCON you'll know that I asked people to pretend that I released IronPython 0.6 and came to Microsoft only two months ago (instead of eight).  There hasn't been much technical work between 0.6 and 0.7 because I've only had two months to work on this and getting the details of the release together took most of that time.  Trust me, this is the easiest way to understand the 0.7 release.  This release doesn’t run any of the standard Python regression test suite out of the box.  There are several missing builtin functions from Python 2.3 and so far none of the new features in Python 2.4 have been implemented.  The path to fixing these issues is quite obvious and we’ve already started down it.  On our way to 1.0 we need to steadily run more and more of the standard Python regression tests so that by the time we reach 1.0 we’re running and passing almost all of this test suite.

Given the current early state of IronPython, who would want to use it right now?  Well, IronPython today is a pretty good platform for working with .NET libraries and for hosting inside of .NET applications.  I frequently use the IronPython interpreter when I want to experiment with a new .NET API and will often then move that interactive work to a .py file to save as a script for running again in the future.  This is also the area where early feedback is most important.  We know how all the standard Python features should behave and have good tests for this behavior in the standard Python test suite to make sure we get it right.  We still need to learn how best to interact with .NET libraries and code written in other .NET languages because this is new territory.  This is why this is the area we are looking for the most early feedback today.  If you use IronPython with your existing .NET code, let us know what works well and what is awkward.  If something feels awkward to you, it’s quite likely because you’ve discovered a case we didn’t consider and that we’d like to make work better.

Don’t take this current focus on .NET interaction to mean that Python compatibility isn’t important to us.  It is, and there will be a lot of work here as we proceed.  Before we can call IronPython 1.0, I think that it needs to run the standard Python test suite at least as well as Jython does.  Any tests that don’t pass will need to be carefully examined, understood and discussed in a similar way to how this was done for Jython.

I’m looking forward to a lot of fun as we work to build the best possible implementation of Python for .NET.

posted by hugunin with 6 Comments

Dynamic Language Position on the CLR team at Microsoft

You may have noticed that I've been really bad about writing in this blog since I joined Microsoft two months ago.  A big part of my problem has been that there are a TON of interesting things going on around this company that touch on the dynamic language space.  All of these distractions have left me without the time to spend talking to the external community.

The good news is that I now get to hire another member to join the CLR team working full-time on dynamic languages.  Having another person working on this project should give us enough time to really make a big impact on this exciting space.


Make the Common Language Runtime (CLR) an outstanding platform for dynamic languages and environments. You will be a key developer working on the CLR managed services team. Your first task will be to help finish the implementation of Python for the CLR (see http://ironpython.com/) so that we can show other developers how a high-performance production-quality dynamic language is implemented for this platform. Future work will likely involve addressing difficult implementation challenges for other dynamic languages and environments. A central part of this job will be working with the developer community. You will work both with developers who are trying to use dynamic languages on the CLR and those who are trying to develop dynamic languages for this platform. You will also work to identify and implement improvements to the CLR to enable us to provide the best possible experience for dynamic languages in the future.

We're looking for a talented developer with exceptional programming ability and great communication skills. It's a plus if you have expertise in compiler implementation, reflection, VM implementation and/or dynamic languages. Applicants should have at least 4 years experience in software development and a BS/MS in computer science or related field.


http://www.microsoft.com/careers/search/details.aspx?JobID=6391a54a-bfd7-4384-b18f-cecb0acf86e0


If you're interested in this position, I'd encourage you to both apply through the official channels and to send me your resume directly (jimhug at you know what company.com).

posted by hugunin with 2 Comments

Interactive Python Scripting Example

For my first blog entry I wanted to write an article explaining what a dynamic language is. Since I prefer to think concretely about things, I'm going to start with an example of using a dynamic language (Python) to play with an interesting API. For now, my working definition of a dynamic language is any language which could be easily used for the following example.

I’m going to use Windows.Forms as a standard API for this demo. (Note: This probably won't work on Mono-1.0, but it shouldn't be hard to port this example to GTK#. Here's an example of using IronPython with GTK# for those who are interested.) If you want to follow along, you can download IronPython-0.6. Remember that IronPython is at a pre-alpha 0.6 release, so don’t be surprised if it has bugs.

Formatting note: I'm sorry if these examples are a little hard to read.  I'm having trouble talking my blog editor into showing PRE blocks correctly.  The extra lines are an artifact of the blog editor and are not intentional.

The first step is to bring up the interactive interpreter. This should be simple. See README.html in the IronPython zip file for a few more hints on different platforms.

C:\IronPython\bin> IronPythonConsole 
>>>

Whenever I bring up a new interactive interpreter I feel compelled to get it to do some tricky math to let me know that it's really working.

>>> 2+2 
4

Next we’ll create a new Form and show the window.

>>> from System.Windows.Forms import * 
>>> from System.Drawing import * 
>>> f = Form(Text="Experiment #1") 
>>> f.ShowDialog()

You should see a nice little window popup now. However, there’s a big problem. f.ShowDialog has taken over the main thread and it’s impossible to type anything more at the interpreter prompt. Let’s fix that by first closing the window (click on the ‘X’). You should then see:

Cancel 
>>> 

Now let’s start the ShowDialog method in its own thread so that it doesn’t block the interpreter

>>> from System.Threading import * 
>>> Thread(f.ShowDialog).Start() 
>>> 

Now we have our window showing and the interpreter is still “live” for us to keep typing at. We can change different properties of the form like this:

>>> f.MinimizeBox = False 
>>> f.Size = Size(200,200) 
>>> f.Text = "New Title"

It’s fun playing with the form, but we’d like to add a button to give it some more functionality.

>>> b = Button(Text="Push Me") 
>>> f.Controls.Add(b) 
System.ArgumentException: Controls created on one thread cannot be parented to a contro <snip>

Oops! That didn’t work as well as we might have liked. This is our second experience of the interactions between the threading model in Windows.Forms and the interactive interpreter. There are a number of creative solutions to this problem (see …) To keep things simple for this blog, we’re going to just close the window again, add the control, and then restart the main event thread. Don’t forget to close the window before proceeding with the next two lines.

>>> f.Controls.Add(b) 
>>> Thread(f.ShowDialog).Start()

Now we should see the form appear again with a button on it. We can play with the button to make it bigger and brighter.

>>> b.Size = f.ClientSize 
>>> b.BackColor = Color.Yellow

Let’s make the button do something interesting. In preparation for this we’re going to create a list of the named colors in the Color class. This is the one piece of code that includes the most Python specific code. If you’re unfamiliar with Python you should just treat most of this code as magic. Dive Into Python is a good first resource for an experienced programmer who wants to become familiar with Python. It also has a very short section on indentation of blocks in Python. To make this example work, the one thing you should be aware of is that for each line which begins with "..." you will need to type a TAB to create the correct indentation and get your program to run correctly.

>>> colors = [] 
>>> for name in dir(Color): 
...    c = getattr(Color, name) # Use TAB to indent this line and below 
...    if type(c) == Color: colors.append(c) 
... 
>>> colors 
[Color [AliceBlue], Color [AntiqueWhite], Color [Aqua], Color [Aquamarine], Colo 
<snip> 
ke], Color [Yellow], Color [YellowGreen]]

Now that we have this list of colors, we can set the background color of the button to these colors in sequence. This requires creating a function to do the work and then setting that function to be run on the Click event. (Note: def is used in Python to define a new function, and just like above you should use TAB to indent each line in the body of the function.)

>>> index = 0 
>>> def push(b, e): 
...    global index # Use TAB to indent this line and below 
...    b.BackColor = colors[index] 
...    b.Text = colors[index].Name 
...    index += 1 
... 
>>> b.Click += push 
>>> 

Click on the button to cycle through all of the named colors. Finally, we need to exit the interpreter. This is done with ‘Ctrl-Z’.

>>> ^Z 
C:\IronPython\bin>

Now that we’re done playing with the interactive shell, we can put the results of our experiment into a file for running stand-alone. Now that we’re done experimenting, we can simplify the code a little bit by moving some of the property settings into the Form and Button constructors. We can also leave out all of the separate Thread nonsense and just call f.ShowDialog at the end of setting everything up.

------ colors.py ------ 
from System.Windows.Forms import * 
from System.Drawing import * 
f = Form(Text="Playing with colors", MinimizeBox=False, Size=Size(200,200)) 
b = Button(Text="Push Me", Size=f.ClientSize) 
f.Controls.Add(b) 
colors = [] 
for name in dir(Color): 
    c = getattr(Color, name) 
    if type(c) == Color: colors.append(c) 
 
index = 0 
def push(b, e): 
    global index 
    b.BackColor = colors[index] 
    b.Text = colors[index].Name 
    index += 1 
 
b.Click += push 
f.ShowDialog() 
------

Now you can run this as a stand-alone program:

C:\IronPython\bin> IronPythonConsole colors.py

I hope that this has helpful to give newcomers a feel for what IronPython feels like to code in and to give even experienced Python programmers a few more concrete examples of how to work with the CLR. As one last recommendation to anyone new to IronPython, I’d like to pass on this advice overheard on the ironpython users mailing list. IronPython will sometimes look more like VB than C# in the way that it uses CLR libraries and you should look at both C# and VB examples if you’re confused about how to call something in IronPython.

posted by hugunin with 36 Comments

Dynamic Languages and the Common Language Runtime (CLR)

Hi!  I’ve just finished my second week working at Microsoft. Now that I'm starting to settle in I can get to the fun work with the CLR and dynamic languages.  I’ve been fascinated by the power of programming languages and tools for many years, and I’ve built a number of tools in this space.  I’m the creator of Jython, a co-designer of AspectJ and the original lead for Numeric Python.  You can read more about these projects starting from my personal web site.

Over the past year, I’ve become a reluctant convert to the CLR.  My initial plan was to do a little work and then write a short pithy article called, "Why .NET is a terrible platform for dynamic languages". My plans changed when I found the CLR to be an excellent target for the highly dynamic Python language. Since then I've spent much of my spare time working on the development of IronPython.  The more time that I’ve spent with the CLR, the more excited I’ve become about its potential.  With my new position at Microsoft I have the pleasure of continuing to explore and extend this platform full-time surrounded by smart people who have a deep understanding of the CLR.

I announced the first public release of IronPython-0.6 during my talk at OSCon a couple of weeks ago.  The conference and the talk were great fun.  Before my talk at OSCon, IronPython had only ever run on my personal laptop under Windows XP.  Before my talk was even finished, Edd Dumbill had installed and run it successfully under Linux.  An hour later out in the halls I met several more people who had IronPython running successfully on Linux as well as two people who were running it under Mac OS X.  None of these people needed to make any modifications or even do a recompile to get it working on their systems.  Everyone could run exactly the same binary that I’d built using Visual Studio .Net on Win XP.  Of course, this is exactly how the CLR is supposed to work; nevertheless, it’s always a pleasant surprise when things work as advertised.

I'd like to thank all the people who've given me such positive feedback both about my IronPython announcement and my move to MS. 

This blog is going to give me an easy place to talk about technical issues with implementing languages for the CLR and about philosophical issues around how and where dynamic languages are most useful.  I’ve just started my new job and still have to work out my schedule and commitments, so don’t expect any comments from me about a next release for a little while.  Coming soon will be an entry similar to one of the live demos I gave during OSCon to show a little bit more about how IronPython can interact with interesting CLR libraries.

posted by hugunin with 13 Comments