FlowBasedProgramming | RecentChanges | Preferences

See StacklessPython

Conversation with WolfgangKeller? appended

Interesting post by CarlosRibeiro? in http://pythonnotes.blogspot.com/ (Dec. 15, 2004) reproduced in the following section. An interesting description of differences between the Java and Python mindsets can be found in http://dirtsimple.org/2004/12/python-is-not-java.html.

Conversation with WolfgangKeller? (questions are from PaulMorrison):

Stackless is excellent for every kind of application that needs to handle asynchronous threads. Especially for I/O-bound applications, where each thread usually wastes a lot of time waiting for something, you can enormously speed up things by using _lots_ (tens or hundreds of thousands) of threads in parallel and thus "interleaving" the waiting time, so that there's always at least one thread which has everything it needs to process.

Another issue (which you address in http://www.jpaulmorrison.com/fbp/method.htm if I understood it right):

Python is not only an excellent "middleware" language (as it interfaces with practically everything easily, including Java and .net), it's also an excellent prototyping language, with development times which are typically a fraction of those for the same application in Java. And once the Python prototype is finished, all you have to do to turn it into a productive application is to profile it and re-implement those modules which are considered too slow for example with Pyrex, which gives your "prototype" native C(++) speed.

 >> I would assume that [FlowBasedProgramming] ports and the network definition 
 >> would have to be added.  I didn't get much out of  ChristianTismer?'s 
 >> web site..

Unfortunately he's too busy earning his living and trying to keep the Stackless implementation in sync with the "mainstream" Python versions. :-(

 >> What on earth is Pickling?  :-) 

A very simple Python persistence/serialisation module.

 >> Also, as I posted on SourceForge and the wiki, what is so great about 
 >> getting rid of stacks - or am I reading too much into "stackless"?  :-) 

The name of "Stackless" is somewhat misleading (or at least bad PR ;-) . Getting rid of the C stack in Stackless is just a means for the aim of handling a lot of concurrent tasklets (=microthreads) with reasonable ressource requirements. IMHO Stackless should be renamed into something like "Microthreaded Python" "Parallel Python" or something like that... Or even better, the main Python implementation should be "Stackless"ed.

Flow Based Programming & the new Python coding style

Since being introduced in the language, [generators] (and their lesser cousins, [list comprehensions] are finally making it into the community mindset. In the long term, I think that this change will have a lasting effect on the way we write Python programs.

In the beginning, Python was a conventional scripting language that happened to have a exceptionally clean syntax and a solid object model. So most Python programs would resemble conventional sequential scripts. As the language evolved, the applications grew more complex, but their essence was still the same. The network-enabled modules pushed the envelope early on. Two architectures to handle multiple simultaneous requests were implemented, following (again) the conventional wisdom: [async] modules (based on the standard Posix select() call) and [threaded] execution.

[Stackless Python] was a interesting event in Python's history. It's still one of the most amazing pieces of Python-related software. I personally believe that the most important effect of Stackless was to break out the paradigm box. Once Stackless became available, people could see other ways of doing things. I personally believe that Stackless was a very strong driving force, even if only as a reference of what could be done once the conventional restrictions of the sequential processing model were lifted.

[List comprehensions] were added first to the language. The iterator protocol followed it, and soon after, [generators] were introduced. People started using then, at first slowly. [List comprehensions] in particular have led to a number of calls for help from people trying to grasp their syntax. But now, more and more systems rely on these new style constructs to implement complex programming patterns. While checking libraries, I can often see list-comps being used in places where loops would be used just a couple of years ago. More recently, the Web SIG defined a generator-based interface in the [WSGI spec], that will allow async-style calls between the web server and the application engine. And new modules and projects seem to be getting to grips with the async nature of generators, and are using them in highly interesting and innovative ways.

After following these changes over the past few years, I was surprised to be a late "rediscoverer" of [Flow Based Programming]. Just a quick reading on the topic shows how much can be done in terms of application modelling with Python. There are strong parallels between FBP and other paradigms, such as functional programming; a good discussion can be found on the C2 Wiki (FlowBasedProgramming). The basic premise is that business applications are data-driven by nature, and thus, poorly suited to the strictly sequential Von Neumann model. This may sound like old talk today, but it's interesting to note that this stuff was developed over 30 years ago, in a time when concepts like Object Oriented Programming were still an academic novelty. Critical applications written using FBP are still in use today, which proves its suitability for an extremely demanding task.

I personally believe that right now, more and more people are "rediscovering" how to think and write data centric code in Python. The newest features (specially [generator expressions]) will encourage this style of programming. This will lead to a change as the Python community incorporates this data-centric paradigm shift into new applications. The trend is already stablished; it's one more great application for Python, with a huge potential for success.

posted by Carlos Ribeiro @ 6:17 AM

FlowBasedProgramming | RecentChanges | Preferences
This page is read-only - contact owner for a password | View other revisions
Last edited January 14, 2005 3:15 pm by PaulMorrison (diff)