I would like to announce the creation of a Google group on Flow-Based Programming: http://groups.google.com/group/flow-based-programming .
Thanks, Justin!
Hot off the press!
A second edition of my book is now available from CreateSpace?'s eStore - https://www.createspace.com/3439170, and at Amazon.com - http://www.amazon.com/Flow-Based-Programming-2nd-Application-Development/dp/1451542321/. Take a look and see how FBP has changed in 15 years!
The new edition contains information about the Java and C# implementations, as well as the diagramming tool (DrawFBP), several sections contributed by Mike Beckerle, CTO of Oco Inc., plus a whole new chapter on projects that have appeared in the last 15 years and either build on FBP, or have strong similarities to it.
DrawFBP now allows users to choose two fonts: a variable-width font (for text) and a fixed-width font for component names, etc. The first time they use DrawFBP it will present them with two lists of the fonts on their machine, together with an indication of which ones support Chinese characters. DrawFBP will then store their choices in the its configuration file. This function is available to anyone using the web version of DrawFBP. 2010-05-11
DrawFBP has been enhanced to display the source code for components, if you double-click on a block associated with a component. As before, double-clicking on a subnet brings up the subnet in another frame. 2009-11-01
SvenSteinseifer has just sent me a link to a web site by SriramSrinivasan describing a system called Kilim - http://www.malhar.net/sriram/kilim/ - it is very close to FBP, but has a number of additional interesting features, among which are very fast context switching, and a type system that tracks ownership and containment. There is a video on YouTube? - http://www.youtube.com/watch?v=37NaHRE0Sqw . In the presentation Sriram shows how the performance compares favorably with that of Erlang. 2009-11-07
MikeBeckerle, who has given me much support over the years, sent some ideas about topics he would like to see in the next edition of the book - they can be found at MikeBeckerle. 2009-11-03
RichardHarter tells me that he will be writing a comparison of several languages. The ones that he has in mind are Hermes (now defunct but interesting), Erlang (an outlier but very much part of the scene), Axum (a new language being developed at Microsoft), and material drawn from the FBP book.
The point of all of this is to clarify his thinking... he says he is (slowly) working on developing a new language - but he is not sure what will be its final form. 2009-11-03
Leo Meyerovich at Berkeley has pointed me at the Capriccio work - http://capriccio.cs.berkeley.edu/index.html - looks interesting! 2009-10-21
Tzvi Tafet has just sent me a link to an article - http://reddevnews.com/articles/2009/10/01/net-rx-framework.aspx - about Microsoft's new .NET Reactive (Rx) Framework. Looks fascinating! Thanks, Tzvi! 2009-10-01
SvenSteinseifer has just sent me a link to his Master's Thesis: http://www.steinseifer-frdbg.de/thesis.pdf . Congratulations, Sven! 2009-09-29
Eric Gaumer has just announced Pypes - http://gaumer.blogspot.com/2009/08/flow-based-programming-with-pypes.html - an FBP implementation based on Stackless Python. Congratulations, Eric! 2009-08-16
SvenSteinseifer has discovered, and fixed, a couple of race conditions in JavaFBP. Thanks, Sven! I have made the same changes to C#FBP, and put the new zip file on my web site and on SourceForge? SVN. 2009-08-06
DrawFBP has now been enhanced to show a clickable outline on the various blocks - also it prompts you to enter a description on any new block. 2009-08-04
Interesting discussion showed up on Reddit: http://www.reddit.com/r/programming/comments/93vvi/flowbased_programming/.mobile . It includes some discussion about Sort, and the possibility of a game based on a factory concept. 2009-07-27
I have added an enhanced file write component to JavaFBP - it builds a 1/2 Mb buffer and only writes it out when it is full (or at end of job). It is considerably faster than the old one. 2009-07-15
I have just run across "Learning to Program with Lily" in "Message Passing", James Cordiner's blog - http://www.16kb.com/blog/?p=84 - looks fascinating! Lily is written in JavaScript? and runs as a Firefox plug-in. 2009-07-12
DrawFBP has now been enhanced to show alignment lines when a block is being dragged. The "teeth" have been removed as they were interfering with the alignment display! 2009-06-13
I have been doing some performance testing using sockets in JavaFBP and C#FBP - the results are described in SocketsPerformance. 2009-06-10
I have been dusting off the old 16-bit THREADS implementation written in C ("green" threads), and was advised to convert it to use Windows "fibers" - I have my first test case working, but I have to do a lot more testing! 2009-05-27
I have successfully sent data from a JavaFBP network over a socket to a C#FBP network, using general-purpose components! Excuse me for crowing a bit - I am pretty excited! 2009-04-28
DenisGarneau suggested I look at sockets for InterLanguageCommunication, as a result of which I have been able to run a JavaFBP network where some connections have been replaced by sockets (actually "socket write" and "socket read" components). 2009-04-28
DrawFBP has been enhanced to allow saving the current diagram as a JPEG file. The "drag to grid" display has been moved out of the display area, and will be preserved when saving and restoring diagrams to and from files. 2009-04-02
Tracing has now been enhanced in JavaFBP-2.4 as follows:
A fuller list of features can be found in FBPTracing. 2009-03-30
New versions of DrawFBP and JavaFBP, and now C#FBP, have been put up on my web site and on SourceForge. The source for both is now on SourceForge SVN (SubVersion) - thanks to DanDromereschi?. You can access the source code at http://flow-based-pgmg.svn.sourceforge.net/ . Please contact me if you wish to be added as a developer. 2008-03-10.
A new version of JavaFBP has been put up on my web site incorporating Sven Steinseifer's most recent error-handling logic, described in http://www.jpaulmorrison.com/cgi-bin/wiki.pl?SvenSteinseifer. The new version of JavaFBP can be found at http://www.jpaulmorrison.com/fbp/JavaFBP-2.4.jar . 2009-03-05
The following is way outside my area of expertise, but I suspect it might be of considerable interest to some of my readers - and you can just sense the enthusiasm! http://ecommconf.com/blog/2009/02/david-burgess-on-openbts.html 2009-03-02
This demo of MIT's Siftables isn't directly FBP-related, but it blew my mind! Watch it! http://www.ted.com/index.php/talks/david_merrill_demos_siftables_the_smart_blocks.html 2009-02-20
Flow-based programming will be discussed as part of COP4020, UCF, Spring 2009 - see http://www.eecs.ucf.edu/~leavens/COP4020Spring09/resources.shtml 2009-01-15
I have just received some very good feedback on JavaFBP from a Java expert at Giessen, Sven Steinseifer, in Germany. I have incorporated his suggestions and at his request recompiled JavaFBP for Java 5 (it was at 6 before), and promoted the result to my web site and to SourceForge SVN. This can accessed at http://flow-based-pgmg.svn.sourceforge.net/ . Sven has also added some powerful error-handling facilities to the SourceForge SVN version of JavaFBP, and found some minor bugs, e.g. in deadlock detection. These will be rolled out (e.g. to the jar file on my web site, and to the generate function of DrawFBP - these will move up to release 2.4) over the next few weeks. Thanks, Sven! 2009-02-12
A new version of the C# implementation has been put up on my web site - http://www.jpaulmorrison.com/fbp/CsharpFBP-2.3 - that incorporates the most recent scheduling changes from JavaFBP. No deadlocks were actually encountered in the previous C# implementation, but this logic seems more water-tight! 2009-01-06
DrawFBP has been enhanced to test "type compatibility" across arrows, at connect time, based on the type metadata specified for the component ports involved. A beta version of this capability is available on my web site (DrawFBP-2.3), and can be accessed via http://www.jpaulmorrison.com/graphicsstuff/DrawFBP.jnlp . 2009-01-04
KimKulak? presented his work with FBP at VanDev in Vancouver, BC, on Nov. 27, 2008. His presentation was called "Flow Based Programming, An Odyssey", and can be found at http://files.meetup.com/144695/VanDevPresentation.pdf , with the slides at http://files.meetup.com/144695/VanDevSlides.pdf. His presentation gives a good picture of the advantages of FBP, and good suggestions for future tools. 2008-12-07
We just got dugg! http://digg.com/programming/Tired_of_coding_try_FBP_Flow_Based_Programming . f3lheadhunter also pointed me at a diagramming tool at http://bubbl.us/edit.php - neat human factors!
Re DrawFBP: DuncanChild has convinced me that adding a Compile and Run facility was just causing confusion, not to mention adding a dependency on the JavaFBP core engine, so these functions are being removed. The assumption is that the generated code is to run on Eclipse, so this should be used for compiling and running generated code. You will have to do a Refresh after generating a JavaFBP network definition, to update the Eclipse libraries. 2008-11-21
I have been working on getting DrawFBP (see below) to work under Java Web Start (JWS) - it now seems to be working! For those interested in trying it out, click on http://www.jpaulmorrison.com/graphicsstuff/DrawFBP.jnlp . To find the "MergeandSort" test case, click on the "File/List Sample Networks" menu item. Please let me know if you have any problems. 2008-10-29
JustinBozonier has written a "knol" on FBP - it can be found at http://knol.google.com/k/justin-bozonier/flow-based-programming/3rcfjhor28t3o/2?locale=en# . I am sure he would welcome feedback! 2008-10-20
I have recently been working on getting JavaFBP running on a server, using open source web server software. I have a prototype working using JSPs, but I feel the interaction is too synchronous. Flex is a possibility for getting around this, and I have a prototype working using the same JSP approach, but I feel this has other disadvantages. Since then it has been recommended that I look at AJAX, so I have done some preliminary investigation - this looks more promising in this context. I can send the prototype to anyone who is interested. 2008-10-10
A brand new version of the diagramming tool DrawFBP has been made available as Open Source. It is written in Java, and can generate working JavaFBP programs, if the user provides the needed information. It supports multi-level diagramming, allowing the user to "zoom in" or "drill down" to lower levels, or pop back out. If a block on the diagram is assigned to an FBP "verb", it will display the "introspect" information of that verb. Since FBP is self-similar at all levels, this tool can be used for everything from high level data flows to actual running programs.
DrawFBP can generate an FBP network definition, prompting the user to enter component ("verb") names as needed, and will actually run the resulting network if the necessary jar files are available.
DrawFBP can be obtained from SourceForge (DrawFBP-2.3), or from the author's web site - http://www.jpaulmorrison.com/graphicsstuff/DrawFBPInstaller-2.3.zip. Here is a sample screen from this drawing tool:
.
There is more information about this tool in DrawFBP.
Note: (this does not apply if running JWS) jhall.jar (from the JavaHelp? facility) must be extracted to the same directory as the DrawFBP jar file - otherwise nothing happens! 2008-07-29
Another conversation with JustinBozonier, touching on some fundamental issues: JustinTalk - 2008-07-16
A minor change has been made to JavaFBP-2.2: the introspect function has now been made public. This is to support a function in the new diagramming tool being currently worked on (about which more soon). For anyone using JavaFBP, we suggest you either download the latest version (2.2), or change the introspect methods on all verbs.
I have just received a thought-provoking note from a software engineer named JustinBozonier?. His very interesting blog can be found at http://codelikebozo.com (web site changed 2011-12-16). I have added a portion of our discussion in AsynchronismAndUIs - Justin feel free to modify it! 2008-05-19
More news of a totally different kind: it seems that hot discussion has been bombing along on comp.lang.lisp, comparing FBP and Cells, started by one Frank Buss, in Germany I assume, on May 23, 2008. It can be found at http://groups.google.ca/group/comp.lang.lisp/browse_thread/thread/0719e128c11d7c3c?hl=en , and I found it very interesting! I got told about it yesterday, and already someone has been prompted to try out JavaFBP, and contact me about it. 2008-05-27.
Also, I realized that deadlock detection may have been switched off on a recent promotion - this has now been corrected. Also, methods have been provided to push and pop to and from component stacks - this was always in the architecture, but for some reason was missing from the Java implementation. 2008-05-24.
A mini-release of JavaFBP (http://www.jpaulmorrison.com/fbp/JavaFBP-2.2.jar) has been put on my web site and on SourceForge, containing a network which plays some bars of "Jingle Bells" :-) It also allows the priority of a component to be set using Java metadata. 2008-03-31
Another mini-release of C#FBP (http://www.jpaulmorrison.com/fbp/CsharpFBP-2.2.zip) has been put on my web site and on SourceForge, which includes subnet support; plus the component priority function has been changed to use attributes, e.g. [Priority(ThreadPriority?.Highest)]. 2008-03-31
A mini-release of C#FBP (http://www.jpaulmorrison.com/fbp/CsharpFBP-2.2.zip) has been put on my web site and on SourceForge, which has some rather annoying sporadic deadlocks fixed - also a simple concordance application, with key parts of the code contributed by DavidBennett, based on a suggestion by me, which displays the source text and the concordance output in TextBoxes?; plus some bars of "Jingle Bells" :-) As stated elsewhere, it uses "attributes" to specify certain characteristics of components. 2008-03-13
A new release of JavaFBP ((http://www.jpaulmorrison.com/fbp/JavaFBP-2.0.jar)) has been made available on my web site and on SourceForge. It has a number of minor improvements and internal changes, but its main feature is that components must have metadata attached, describing the ports and the MustRun attribute. Details of the new features are described in the Release Notes. 2008-03-13
While revisiting the JavaFBP code, I found and fixed 2 minor errors: one in the behaviour of components that have gone dormant, and one in the way the status of deadlocked components is displayed. The fixed version is 1.5.3, and can be obtained at http://www.jpaulmorrison.com/fbp/javaFBP-1.5.3.zip, as well as on SourceForge (http://sourceforge.net/projects/flow-based-pgmg). Please note: it is now a zip file, not a jar file.
I am currently working on an additional release of JavaFBP which supports the changes to IIP handling discussed in IIP, some simplified "connect" notations, an improved deadlock test - plus other types of general cleanup! This should be available some time in late Feb. or March 2008.
Between June and December 2007, I worked with DavidBennett on a C# implementation of FBP, which David called FlowBase. There are some differences between this and what might be called the "traditional" FBP approach, the most important of which is whether components with multiple input ports should be activated when data arrives at any input port (what might be called the "traditional" FBP approach) or when data is present at all input ports (David's approach). This has far-reaching implications for component scheduling - see AnyVsAll.
David and I have since agreed to work on separate implementations, although David has given me permission to use some packaging ideas from FlowBase. I am therefore starting to upgrade my own C# version, which I am calling C#FBP (not very original!). As a result of discussions with David, I am adding some new features to my version (see C#FBP features), and some older FBP features have been revisited (e.g. IIPs), which will be incorporated into a new version of JavaFBP. I hope to have a clean, working copy on SourceForge and this web site in late Feb. or March 2008. For an earlier (Aug. 2007) discussion between DavidBennett and myself, see FBPIdeas.
A friend from a few years ago, TomYoung, has just reappeared. He has an FBP-like execution environment called DFD, and an editor for it, called VDFD (is that right, Tom? [Yes, a graphical editor and development environment. twy]). They appear to allow for a wide variety of languages to interact. Interestingly, this also appears to be a goal of Ricardo Correa's effort described below.
Ricardo Correa has just written me (10 Nov. 2007) to say that he is planning on developing an FBP framework based on the Parrot VM (see ParrotCode), and making that work the basis for his thesis. We wish him all success!
Richard asked me to comment on the differences between OOP and FBP, and I have put my answer to him in OOPAndFBP2, as there was already a page called OOPAndFBP, which I had forgotten about! It seems relevant also.
The long-time supporter and promoter of FBP, Vorlath, is looking to create a list of components that should be available in FBP development environments, and their inputs and outputs. He has set up a wiki for this purpose, called http://cleo.no-ip.info/tw/tiki-index.php . Here are some comments by him.
There seems to be more components that deal with redirection than with other primitives. But that's perhaps because my system goes all the way down to the lowest levels. In your book, you used control streams to tell components how to handle certain decisions. I'm not doing that exactly. I have this built in as an ordinary input. With the use of generics, I'm hoping to gain more flexibility.
I'll be posting some descriptions of components soon. Feel free to pass this on to anyone interested and edit and create pages as you wish. I've enabled chat, forums, articles and blogs for any users. I'd like to get into a discussion of how exactly different architecture operate. What triggers a component? How many items it can deal with? How it deals with stream markers? Any relationships between scalar, vector and streams or their transformation between each. Stuff like that. More technical. For general discussion, we can use your site, no? ... my site cannot handle e-mail, so there can't be any notifications.
Many businesses incorporate data transformation, in one form or another, as a core business process. Current software development practices offer a large number of diverse solutions to this problem, but most require a large amount of work in order to begin focusing on implementing the transformations required, and are not easily maintained. We have developed an environment that offers a data-centric view of software development by combining the benefits of visual programming with the customization of source text. Each transformation network is a collection of loosely coupled objects (processes) developed in a one of several languages, and connected via bounded buffers (channels). This entirety of the environment is a client-server application communicating via HTTP and HTTPS. The client side of the application is a graphical shell providing ease of development, and acts as a communication medium to less technical contributors. The server is a queue of objects, and a minimal HTTP server that accepts RESTful requests from active objects. Data, encoded in JavaScript? Object Notation (JSON), is returned from the server to the requesting object. The use of this architecture allows for the possibility of distributing the transformation network as a remote service. Debugging is done through the inspection and modifying of the connecting channels. We allow the developer to stop a running transformation, increment and decrement state, and modify the values in each buffer to test the correctness the receiving process.
Just came across an Aug. 2007 article on Data Flow in the Java Developer's Journal by Jim Falgout and Matt Walker of Pervasive Software - http://java.sys-con.com/read/419717.htm - giving a historic survey of Data Flow programming approaches, and listing good areas in which to apply Data Flow. At the end it goes into some detail on Pervasive's DataRush? product, and describes results of a benchmark using it.
Also came across a somewhat older PowerPoint? presentation on Kepler by Bertram Ludaescher - http://daks.ucdavis.edu/~ludaesch/Paper/kepler-lbl.ppt - it describes some fascinating application areas, and how they are being approached using Data Flow.
Alan Harkreader has just sent me a link to a very thought-provoking paper by Gerry Sussman: http://swiss.csail.mit.edu/classes/symbolic/spring07/readings/robust-systems.pdf . Enjoy!
Leslie Sanford has just released a new C# Synth Toolkit for creating software based synthesizers using C#. He says, "a flow-based approach was perfect for creating the toolkit", although he says the approach taken is a bit different from what I describe in my book.
If you want to take a look at the Code Project articles, he says you can find the first two here:
I believe that the FBP scheduling rules could actually achieve the same results without the need for a special scheduler - perhaps modified slightly by making the actual sound generator module higher priority than the other components. This should have no effect on FBP scheduling, which is based primarily on the availability of data. This is perhaps borne out by a simple network that I have written that plays "Jingle Bells" :-)
A new version of JavaFBP-1.5 (JavaFBP-1.5.2) has replaced the older 1.5 version on SourceForge? - this now supports the subnet capability described in the chapter in my book on Composites - http://www.jpaulmorrison.com/fbp/compos.htm - see the network called TSS for an example. It also fixes some minor bugs. I would appreciate feedback on a) whether it works, and b) stylistic issues. (It is also available on my web site: http://www.jpaulmorrison.com/fbp/javaFBP-1.5.2.jar.) By the way, I have backed out the use of ArrayBlockingQueue as it turned out to be too hard to close down cleanly (IMO).
DavidBennett has also sent me some notes on a family of FBP-related stuff, grouped under the generic term PipeLines.
Microsoft is now on board in this general area with a product called VisualProgrammingLanguage? (VPL) - see http://msdn2.microsoft.com/en-us/library/bb483088.aspx - using the terms "activities" and "pins", probably inspired by LabView. Quote: "In addition, while its toolbox is tailored developing robot applications, the underlying architecture is not limited to programming robots and could be applied to other applications as well."
Comment from Dave Johnson:
I found out that MS is giving away evaluation copies of VPL for free, so I downloaded it. The editor is pretty slick, and the language allows things to be built from quite simple primitives, as well as permitting encapsulation of arbitrarily complex logic that can be invoked as a single block on a diagram.
The documentation is skimpy and not especially well written, and the runtime environment has both many layers of abstraction (very IBM-ish, I would say) and quite a bit of overhead. Of course, everything is locked into MS platforms also, although I think in principle that is not necessarily true. However, somehow I doubt that MS will be porting the runtime environment themselves. We'll see.
Still, like everything else from MS, many people will use this because it is obviously there, and it certainly puts a big, wet, sloppy Bill-Gates-kiss-of-approval on dataflow methods.
Comment from DaveBennett?:
I saw that. Looks like a nasty piece of work. I spent some time investigating visual programming languages a while back and became very dissatisfied with them. Trying to express the concepts of an iterative programming language using boxes and lines just doesn't work for me at all.
(I totally agree - all the way back to Nassi-Schneiderman! --pm)
The point about FBP is to leave the 3GL to code the stuff it's good at, and the couple higher order verbs and nodes into networks to describe the things 3GLs are not good at. I couldn't see that concept in VPL at all, although I shall have another look at some time.
(Totally agree with first sentence - second is disappointing news, though! --pm)
Mark Stahl, a student at a local state university in Savannah, GA (studying Computer Science and Philosophy), and one of his friends, have begun working on a potentially commercial project that implements many of the FBP ideas, using the .NET platform and writing the project in C#.
PhilippeMonnaie?, a student in computer sciences at Katholieke Universiteit Leuven in Belgium, has successfully defended his thesis which combines JavaFBP with Scala code of his own. His thesis contains sections on FBP and also on Scala - his thesis can be read on http://pmonnaie.dreamhosters.com/papers/A%20Survey%20of%20Support%20for%20Concurrency%20in%20Modern%20Programming%20Languages.pdf. See also http://en.wikipedia.org/wiki/Scala_%28programming_language%29 .
Donegal News: Fri, Jan 26, 2007
BT Young Scientist of the Year 2007 - prestigious competition which is now in its 43rd year. Quoting: "A number of other students and their projects were highly commended for the work they had entered into the competition.... They were: Stephen Dolan from Choláiste Cholmcille, Ballyshannon for his project 'Streams: flow-based programming in Python'; ...."
Congratulations to Stephen and his tutor/advisor! It seems the future is in good hands! --pm
![[Home]](/wiki.gif)