Python-on-a-Chip

2010 State of the Subset

Author: Dean Hall

History

Development on the PyMite VM started in 2000 in my home in Austin, TX. I made the project open-source and presented it at PyCon 2003 in Washington D.C. and at a Austin PUG meeting. There were only two target platforms at the time: desktop and an Atmel ATmega103 based board of my own design, the Mega Mini Board 103 (mmb103). Features were few, development was slow and bugs were plenty. The first two releases were done on SourceForge in my old umbrella project area.

After PyCon 2003, the project didn't gain much traction with others, so the project crawled until 2006 when I caught the coding bug again. I moved the project to pymite.python-hosting.com so I could have a Subversion repository and a Trac project management system. Also at this time, two interested parties joined the discussion: one a Master's research student from Germany, another a PhD candidate from MIT. I got a lot of encouragement and excellent code from these guys.

The code really grew from 2006-2009 as seen in releases 03-08:

Release Release Date Package MD5
03 2006/09/17 pymite-03.tar.gz 145456f828172ac73dcb833f613abad9
04 2006/12/14 pymite-04.tar.gz 5af5da702c8fb70ff5193fc98705401d
05 2007/05/10 pymite-05.tar.gz 02f820d693595415c2e74ec5aaf582d8
06 2009/01/30 pymite-06.tar.gz 1efe8c327ad5f6920b5eba0341516e86
07 2009/03/08 pymite-07.tar.gz c75f8a03f9a26941756d3293b2369967
08 2009/04/20 pymite-08.tar.gz 13f07f28ef02b76b55face3be907e249

Then, I went to PyCon 2009 in Chicago and all hell broke loose.

2009 - The Turning Point

While at PyCon 2009, I gave a lightning talk on PyMite. It turns out a 5-minute lightning talk is a much better way to reach more Python developers than a 30 minute session. I also attended three Open-Space meetings: Python for embedded systems, Python in Robotics and one specific to PyMite. This is where a cadre of people grew the idea of PyMite into something bigger.

We decided on the name Python-on-a-Chip as a way to succinctly express the purpose of the project. The VM keeps the codename, PyMite. We developed some project goals: to build device drivers, libraries and ports to popular platforms. I was surprised to learn that an on-board compiler was not in high demand. Steve K. graciously obtained and continues to pay for the pythononachip.org domain.

To accommodate the new project name and handle the growing numbers on the mail list (189 as of 2010/09/01), I moved the software repository to http://code.google.com/p/python-on-a-chip/ and the maillist to http://groups.google.com/group/python-on-a-chip

Also, at PyCon 2009, I demonstrated the VM running on the mbed microcontroller platform. Several people were interested in that, so I coordinated a group purchase of 10 mbed devices and got them in the hands of developers 7 months before the general release of mbed.

I left PyCon 2009 with a ton of ideas, many feature requests and a lot of motivation.

Building a Solid Project

The larger audience of developers and users has led to the discovery and correction of defects, as well as the desire for more features, enhancements and platforms. 69 issues are mainlined and just a few more to go for release 09. Here is the list of some of the significant new features since PyCon 2009:

Since PyCon 2009, the list of supported platforms (those platforms included in the build) has grown from six to eleven (new platforms in bold):

The Flow

Since PyCon 2009, I've been slowly chipping away, adding features that are resulting in a robust and capable VM. I'm very pleased with the results. I never thought generators, coroutines and closures would take so little RAM. However, I have put off working on exceptions because I know its class hierarchy is going to take a lot of program memory and RAM. Because of these requirements, I want to first work on the next-generation VM which will define code objects in program memory. Doing so will free up RAM to allow the necessary hierarchy of exception classes. Once the next-generation VM and Exceptions are in place, I feel the PyMite VM will be in a good place and I will reduce my work on the VM, only performing small enhancements and bug fixes as needed. I will then ramp up on other areas of the project.

I have written a few ports: the mbed and econotag being the most significant. And I was very pleased to accept ports from a growing list of contributing developers. Dr. Bryan Jones is managing the port to the Microchip PIC24/dsPIC family. This port provides an interface to a significant number of peripherals. In so doing, Dr. Jones created a great set of macros that I hope to adopt and enhance to ease everybody's porting experience. I'd also like to give props to James Snyder for the port to STM32, Tyler Wilson for his port to the Windows platform and Angus Gratton for improvements to lib/avr.py and the port to the Teensy++. There are plenty of other people to thank who have contributed to p14p. I hope I've captured them all in the README (pleased let me know if I overlooked anybody).

I've also been surprised by the quantity of people that have provided quality feature requests, bug reports and code pathces. And then there are days when I'm blown away by an individual who hasn't made a peep on the maillist and drops a bombshell like a real-time incremental garbage collector. I think the word has gotten out that p14p is an active project that does what it advertises. The code base is well-commented, making it easy to hack.

Management

As the creator of most and copyright holder of all the code in the VM, I established a dual-licensing system for PyMite. I'm ambitous to keep building the software of p14p, but I'm not so keen on trying to build a business. So the idea of trying to support my development through software sales has languished. But I'm not giving up on the idea. The next-generation VM is going to have 2 features that will really make it worth paying for. So much so, that I've even been tempted to go closed-source.

For a project like p14p that must balance increasing features with available program memory and RAM, I think it's very important to have one very clear direction rather than a design-by-committe approach that has doomed other embedded software like the Bluetooth and Zigbee stacks. There is only one thing more important than clear direction and that is eliminating a single point of failure (a.k.a. What if Dean gets hit by a bus?). In the next 2 years, I'm really hoping to find a developer who really gets this project from both sides: the technical details of the codebase and the balance of wanting great features versus keeping things simple and small.

Two other areas where I welcome new developers are in ports to new platforms and a new area that's coming up: cross-platform features like an embedded filesystem or a networking library. I'm finding that a port to a new platform involves 3 distinct phases. (1) The initial port of the VM can be done in a short period. (2) Making or wrapping a platform's libraries is the longer, more tedious part of the process. The third phase is maintenence. (3) Major releases and some big VM changes require testing and sometimes changes to platform code. Since I don't own every platform, I definitely need help when such changes and releases occur. I want release 9 out before the end of 2010, so I'm going to be calling on volunteers to build and test the various platforms.

The Future

In the short term, here's what I'm working on:

  1. Feature freeze, fixing defects only for release 09
  2. Create release 09
  3. Enable Mercurial repository on Google Code
  4. Get rough-draft of next-gen VM done so I have Flash/RAM consumption numbers to share at PyCon
  5. Plan for PyCon 2011

I have a list of features for the next-generation VM, but they don't belong here. Long-term lists should have more generalities. So here's the plan for 2011:

  1. Implement the next-generation VM and migrate platforms
  2. Encourage new ports, such as one for CMSIS
  3. Create binaries that hardware vendors can use for direct sales
  4. Create quality, usable documentation for the user and the developer
  5. Continue promoting p14p at PyCon 2011 and online
  6. Maintain release 09 with bugfixes as warranted; release 0A if necessary.