A second edition of my book is now available from CreateSpace's eStore - https://www.createspace.com/3439170, and at Amazon.com - http://www.amazon.com/Flow-Based-Programming-2nd-Application-Development/dp/1451542321/. Take a look and see how FBP has changed in 15 years!
The new edition contains information about the Java and C# implementations, as well as the diagramming tool (DrawFBP), several sections contributed by Mike Beckerle, CTO of Oco Inc., plus a whole new chapter on projects that have appeared in the last 15 years and either build on FBP, or have strong similarities to it. For obvious reasons, the new sections are not available on this web site - but just think, you could have a shiny new copy in your hands, printed with loving care by CreateSpace? - and IMHO it looks just as good as the original edition (if not better)!
The web site for FBP is http://www.jpaulmorrison.com/fbp .
This wiki was apparently last modified in 2010 - it is probably due for some updating!
There is now a Google Group on Flow-Based Programming: http://groups.google.com/group/flow-based-programming?lnk= - there are some lively discussions going on there!
For those of you unfamiliar with wikis, a wiki is a tree structure and you would normally follow the (underscored blue, green or mauve) links to get where you want to go, but for those who want to dive right in to a particular topic, you can either use the search facility, or click on http://www.jpaulmorrison.com/cgi-bin/wiki.pl?action=index. A more complex facility is the "links" facility: to show the links structure, click on http://www.jpaulmorrison.com/cgi-bin/wiki.pl?action=links.
The Flow-Based Programming Wiki uses the UseModWiki implementation. For information on how to edit text, see TextFormattingRules. For some background on wikis, and how they got started, see WelcomeVisitors. To see all changes, click on http://www.jpaulmorrison.com/cgi-bin/wiki.pl?action=rc&all=1&showedit=1.
The news portion of this page has now been moved into a separate page called FBPNews - it was getting too big! The most recent items will continue to be shown here (in reverse chronological sequence):
DrawFBP now allows users to choose two fonts: a variable-width font (for text) and a fixed-width font for component names, etc. The first time they use DrawFBP it will present them with two lists of the fonts on their machine, together with an indication of which ones support Chinese characters. DrawFBP will then store their choices in the its configuration file. This function is available to anyone using the web version of DrawFBP. 2010-05-11
DrawFBP has been enhanced to display the source code for components, if you double-click on a block associated with a component. As before, double-clicking on a subnet brings up the subnet in another frame. 2009-11-01
SvenSteinseifer has just sent me a link to a web site by SriramSrinivasan describing a system called Kilim - http://www.malhar.net/sriram/kilim/ - it is very close to FBP, but has a number of additional interesting features, among which are very fast context switching, and a type system that tracks ownership and containment. There is a video on YouTube? - http://www.youtube.com/watch?v=37NaHRE0Sqw . In the presentation Sriram shows how the performance compares favorably with that of Erlang. 2009-11-07
MikeBeckerle, who has given me much support over the years, sent some ideas about topics he would like to see in the next edition of the book - they can be found at MikeBeckerle. 2009-11-03
RichardHarter tells me that he will be writing a comparison of several languages. The ones that he has in mind are Hermes (now defunct but interesting), Erlang (an outlier but very much part of the scene), Axum (a new language being developed at Microsoft), and material drawn from the FBP book.
The point of all of this is to clarify his thinking... he says he is (slowly) working on developing a new language - but he is not sure what will be its final form. 2009-11-03
Leo Meyerovich at Berkeley has pointed me at the Capriccio work - http://capriccio.cs.berkeley.edu/index.html - looks interesting! 2009-10-21
Tzvi Tafet has just sent me a link to an article - http://reddevnews.com/articles/2009/10/01/net-rx-framework.aspx - about Microsoft's new .NET Reactive (Rx) Framework. Looks fascinating! Thanks, Tzvi! 2009-10-01
SvenSteinseifer has just sent me a link to his Master's Thesis: http://www.steinseifer-frdbg.de/thesis.pdf . Congratulations, Sven! 2009-09-29
EricGaumer has just announced Pypes - http://gaumer.blogspot.com/2009/08/flow-based-programming-with-pypes.html - an FBP implementation based on Stackless Python. Congratulations, Eric! 2009-08-16
SvenSteinseifer has discovered, and fixed, a couple of race conditions in JavaFBP. Thanks, Sven! I have made the same changes to C#FBP, and put the new zip file on my web site and on SourceForge? SVN. 2009-08-06
I have been doing some performance testing using sockets in JavaFBP and C#FBP - the results are described in SocketsPerformance. 2009-06-10
I have been dusting off the old 16-bit THREADS implementation written in C ("green" threads), and was advised to convert it to use Windows "fibers" - I have my first test case working, but I have to do a lot more testing! 2009-05-27
I have successfully sent data from a JavaFBP network over a socket to a C#FBP network, using general-purpose components! Excuse me for crowing a bit - I am pretty excited! 2009-04-28
I am reaching a point where the current Java and C# open source FBP implementations are nearing completion, and I am looking for experts in these two languages to help me with clean-up and packaging issues. If you want to give a hand, please contact me at the above email address.
The same thing is true for the diagramming tool DrawFBP, written in Java, which is able to generate (almost all of) a running FBP network.
The source for JavaFBP and DrawFBP has now been put up on SourceForge SVN (SubVersion) - http://flow-based-pgmg.svn.sourceforge.net/ . Feel free to peruse them!
What is now known as "Flow-Based Programming" (FBP) -- it was referred to previously as Dataflow Programming, but this term was felt to be too general -- was invented/discovered in 1969-71 by PaulMorrison at IBM Canada. An early implementation of these concepts has been in continuous use at a major Canadian bank since around 1975, running some very large batch applications, but it has remained largely unknown in the industry as a whole, although it has recently started to attract more attention as a result of PaulMorrison's 1994 book [ISBN 0-442-01771-5 (amazon.com, search)], and his web site (http://www.jpaulmorrison.com/fbp/).
Flow-Based Programming is an approach to developing applications, not in terms of the old von Neumann paradigm, but based on the concept of multiple asynchronous processes communicating by means of streams of data InformationPackets? passing over BoundedBuffer connections. In addition, a process identifies its related connections via Port names, rather than directly, and the network defining an application is defined externally to the internal logic of the processes. An application is viewed as a system of data streams being transformed by processes, rather than as a single "dumb" actor doing one step after another until all the data is processed. By "system", we mean that lots of processes are proceeding concurrently - it is the BoundedBuffers that make sure that all of this can still be done using finite resources. However, this "mechanism" quickly recedes into the background, and the developer just thinks in terms of flowing data streams.
This approach involves a paradigm shift which profoundly changes the way developers look at applications, and, although this concept is similar to ideas current in the area of distributed and parallel systems, up until now it has not been recognized that it is also an extremely productive approach to improving programmer productivity and application maintainability. The power and ease of application development and maintenance using FBP is an EmergentProperty of the attributes that make up FBP.
Most programming approaches, and discussions about programming, seem to assume a VonNeumannArchitecture, although people are beginning to network separate machines by means of data streams. However, it is very difficult to build complex applications on a (single) von Neumann machine, especially large data processing applications. Conversely, it turns out that, if you break your application up into small "machines" communicating via data streams travelling across BoundedBuffer connections, these kinds of applications suddenly become much simpler.
The addition of the concept of Port in turn allows each "machine" to potentially become an individual reusable component as it doesn't know the name of the other machine(s) it is communicating with - see ConfigurableModularity. This in turn means that the application becomes essentially a list of connections, which can be generated by a graphical tool if so desired. A "connection engine" (scheduler) is used to relate port names to the real network, and to drive the individual processes.
The connections constitute a DirectedGraph, usually acyclic, but not necessarily. Here is (a subset of) the network for a small interactive application - more information about it can be found in BrokerageApplication:
Wikipedia article: Flow-based_programming - feedback would be appreciated!
C2 (Ward's) Wiki page: FlowBasedProgramming - also has a link to a comparison with the Actors model - http://www.c2.com/cgi/wiki?ActorsModel (do find on FBP).
Patterns in FBP - some examples are described in Collate, ComplementaryComponentPairs (TelegramProblem), SubNets?, DeadLocks? (an AntiPattern), and a general page is PatternsInFBP, which includes a pointer to my recent paper. See also RichardVeryard's work on patterns in http://del.icio.us/dontpanic2/pattern.
For an example of an application which is very natural in FBP and CoRoutine based technologies, but extremely difficult without them, see SameFringeProblem (do a find on FlowBasedProgramming).
Related FBP topics: AsynchronismInFBP, StorageManagementInFBP, CoordinationLanguage, PortsAndConnections, ParametrizationOfComponents, FutureDirections, BrokerageApplication, FlowBasedProgramming, ExecutionEnvironments.
Comega:Microsoft Research is now working on a derivation of C# that has a lot of attractive features, including support for streams - see http://research.microsoft.com/Comega/
Patent applied for: http://www.freshpatents.com/Method-and-system-for-improving-performance-and-scalability-of-applications-that-utilize-a-flow-based-programming-methodology-dt20051229ptan20050289505.php?type=description, dated 28 Dec., 2005, #20050289505, by StanWilliams. Good luck, Stan! Should be interesting!
I also came across http://www.freepatentsonline.com/6282697.html, filed by Wylci Fables and Jore Park, in 1998 - not sure what its current status is. TomYoung has pointed out the following, by the same people:http://www.freshpatents.com/Computer-processing-and-programming-method-using-dynamic-block-streaming-and-synchronizing-data-handling-dt20070816ptan20070192768.php .
For more on FBP's legal status, see LegalStatus. Based on this, I would say that Stan's work is definitely non-obvious, and therefore very worthy of being patented. Since Stan has also extended the FBP work in novel directions that give very high performance, his work should definitely be looked at by anyone who has large volumes of data to process. Conversely, the Fables and Park filings are probably not be patentable, as they should have known, especially as FBP is listed in the references for the first filing.
Review: Nice review of FlowBasedProgramming (the book) by LispMeister? - http://lispmeister.com/blog/books/fbp.html
Recent article: http://www.eecs.berkeley.edu/Pubs/TechRpts/2006/EECS-2006-1.html ("The Problem with Threads" by Edward A. Lee)
From the same institution, one of Bertram Ludäscher's many excellent publications: http://chess.eecs.berkeley.edu/publications/talks/04/ludaescher9-28.pdf
Just came across http://denisbider.blogspot.com/2006/09/function-calls-considered-harmful.html - wonder what happened to his "Flow" project?
For projects that seem to be related to FBP, see FlowLikeProjects. This section has not been maintained, as FBP-like projects are multiplying faster than I can keep up with them! In general, FBP is now being used in lower case, e.g. http://www.infoq.com/news/2008/06/Einstein, http://softratty.com/article/51e1cd8bcf997a6c949f5fed61814dfc, so it might be fair to say it has become a new buzz-word!
Here is a new one, just added to the Wikipedia article: http://www.exept.de/en/products/expecco - described as a Flow-Based Programming IDE.
Google just came across "Eon: a language and runtime system for perpetual systems" - mostly by UMass Amherst people - http://portal.acm.org/citation.cfm?id=1322279. To quote from the abstract: "Paths through the program ('flows') may be annotated with different energy states. Eon's automatic energy management then dynamically adapts these states to current and predicted energy levels. " Intriguing!
There is some interesting stuff being written about Python, esp. StacklessPython, which seems to be developing commonalities with FlowBasedProgramming - see CarlosRibeiro?'s blog quoted in StacklessPython. However, for more reading on this topic, see GeneratorsAreNotCoroutines.
Interesting discussion of FBP and Unix pipelines in http://lambda-the-ultimate.org/node/1210.
For a pretty complete 2004 survey of the literature on dataflow programming languages: http://portal.acm.org/citation.cfm?id=1013208.1013209 .
XML stuff: http://xpipe.sourceforge.net/ - just stumbled across this one!
The following companies are among those selling products based on FBP concepts: http://www.trelliswerk.com, http://www.protosw.com, http://www.pervasivedatarush.com.
FBP and hardware: TheConvergence - thanks, SteveTraugott. Seems like these concepts are starting to pick up steam - see, e.g.
http://fractal.objectweb.org/ - very interesting! And it's been around since 2002!
FBP seems to be an implementation of a ServiceOrientedArchitecture.
Java has added some nice concurrent features - see java.util.concurrent in 5.0. Unfortunately they turned out to be too much trouble to use for FlowBasedProgramming. If you run across statements saying that I am using them - that's not true any longer (except for CountDownLatch?).
Vorlath: self-described software developer and futurist. Lots of material: check out http://my.opera.com/Vorlath/blog/ for his insights into the future of software development, especially with respect to his Project V.
See also Vorlath's comments for a free-wheeling discussion.
Some other people who IMO are tackling the fundamental questions of programming:
And now... Art Division of FlowBasedProgramming:
A couple of relevant (?) cartoons from "the old days":
See also WikiStructures for a discussion on using wikis for such things as FBP diagrams, org charts, etc. See also the description of <presub>...</presub> below.
WikiNode: FBPWiki is now officially a WikiNode - thanks TedErnst!
For answers to all your questions, visit WikiPedia's HomePage!
One difference in this wiki from the basic UseMod package is that InterWiki links are displayed in green, without the site indication (although you see them as links when you move the cursor to them). When editing them, use the convention 'site:XxxYyy', where 'site' is Wiki for the c2 wiki (Cunningham and Cunningham) (see FrontPage on c2.com), UseMod (see UseModWiki) or MeatBall with capital B (see MeatballWiki). If you wish to add other InterWiki links, please let PaulMorrison know.
I have also added a pair of tags <presub>...</presub> which behave like <pre>...</pre> except that WikiNames, italics and bold are supported. We plan to use this for various kinds of program documentation. See TelegramProblemWR for a simple example.''
Please add your name to the VisitorsList (consider using wiki-style names).
Wiki Spam is becoming a major headache. I had implemented an edit password as a means of defending against spambots, using a simple CaptchaTest. However, since then the activity dropped so low that I decided to make the wiki by invitation only. However, if you are interested in participating, I will gladly send you the edit password.
More information can be found on WikiSpam.
DrawFBP is a graphical FBP design tool - it can capture much of the information needed to build an FBP network definition. It has a number of icons which can be connected - in particular, a "composite" process that can be clicked on to reveal a subnet with "sticky" connections. A complete rewrite in Java has now been put up on SourceForge - see above under News.
Within IBM, involving PaulMorrison:
Outside IBM - all available on my web site, and (except for THREADS) on SourceForge:
APIs for THREADS, JavaFBP and C#FBP: FbpApi
Actually, a lot of the discussion is taking place on Ward's Wiki. DavidHopwood has done a great job of refactoring our discussion on the differences between FBP and Actors - see ActorsModel.
(Not sure where these came from!)
![[Home]](/wiki.gif)