Suppose you were told that an obscure programming technology that has been in continuous production use at one of Canada's major banks since the 1970s holds the key to a number of the major challenges facing today's programmers, including multicore machines and distributed computing, while providing improved maintainability, plus a much more seamless transition from structured design to running code, would you reject this as impossible... or just unlikely?
It's true though! This technology has enabled the applications using it to respond to changes in business needs and systems environment, while still exhibiting the high reliability required of banking applications, yet running in less elapsed time than programs written using conventional techniques. It was actually invented/discovered back in the late '60s, but it took us several decades to understand its potential! During that period we came to realize that this technology solves many of the problems that bedevil conventional programming, and that it also has the potential to simplify many that we are just now starting to run into. We have even figured out why! Hopefully we now have your attention!
Herakleitos, in about 500 BC, said
usually translated as "Everything changes". What he actually
said was: "Everything flows".
Somewhat more recently, the following phrase became popular:
"Go with the flow!"
For the Wikipedia article, see Flow-Based Programming.
For late-breaking news, see the FBP Wiki News.
Information about the author: J. Paul Morrison.
Flow-Based Programming is a new/old approach to application development, based on a completely different way of thinking about building applications. Some of its roots can be traced all the way back to the early days of computing, yet it offers solutions to many of the most pressing problems facing application development today.
Not surprisingly parts of the answer have been presaged many times over in the past, but my book on the subject, "Flow-Based Programming", van Nostrand Reinhold, 1994, ISBN 0-442-01771-5 is the first book to tie together many of these different efforts, and to point towards a very practical, powerful, yet perfectly realizable future of application development. New copies of this book are no longer available, but it is available on this web site in HTML and PDF form.
In "Flow-Based Programming" (FBP), applications are defined as networks of "black box" processes, which exchange data across predefined one-way connections. These black box processes can be reconnected endlessly to form different applications without having to be changed internally. It is thus naturally component-oriented. To describe this capability, Raoul de Campo of IBM Research coined the term "configurable modularity", and it was later adopted and developed by the distinguished IBM engineer, Nate Edwards, who described it as "the basis of all true engineered systems."
When using FBP, the application developer works with flows of data, being processed asynchronously, rather than the conventional single hierarchy of sequential, procedural code. This means that the precise timing of events can usually not be predicted exactly, which tends to make old-style programmers nervous. However, it turns out that this isn't really necessary (and never has been!), and in fact shifts the emphasis from sequences of actions to transformations performed on streams of data.
FBP is thus a good fit with multiprocessor computers, and also with modern embedded software. In many ways, an FBP application resembles more closely a real-life factory, where items travel from station to station, undergoing various transformations. Think of a soft drink bottling factory, where bottles are filled at one station, capped at the next and labelled at yet another one. FBP is therefore highly visual: it is quite hard to work with an FBP application without having the picture laid out on one's desk, or up on a screen! For an example, see a portion of a batch program (you have to zoom in to see some of the connections). This diagram, which was built using the earlier C++ DrawFBP diagramming tool, only shows part of an extremely complex program, which was running almost daily for over 30 years, processing millions of transactions a day (it is now in process of being replaced). Now imagine building the same function using a conventional programming language!
Strangely though, in spite of being at the leading edge of application development, it is also simple enough that trainee programmers can pick it up, and it is a much better match with the primitives of data processing than the conventional primitives of procedural languages.
In recent years, the concept seems to have started to take off - this seems to be coinciding with a feeling that Object-Oriented isn't the perfect solution to all our problems that it was originally believed to be. More and more projects and papers use the term Flow-Based Programming (often in lower case) - to the point where it has started to be a generic term (like Bandaid in N. America!). For a fairly lengthy list of projects, papers, and even companies, that seem to be using related concepts, see the Wiki page called "Flowlike Projects".
As you might guess from the logo at the top, a project has been registered on SourceForge. Source code for Java and C# implementations of the concepts has been uploaded to SourceForge, as well as a picture-drawing tool that supports many of its concepts (see DrawFBP).
An article on FBP has recently been added to Wikipedia - please feel free to extend it in any direction that you feel would enhance its usefulness.
The earliest implementations of FBP concepts are described in the Introduction to
the book, and were built for IBM mainframes.
For PCs, there was an early implementation called THREADS,
also described in the book, followed by Java (JavaFBP) and C# (C#FBP) implementations (developed
after the book was published). The network syntax and component
API are described in THREADS API and
Network Specification .
The latest version of JavaFBP is available on SourceForge and also on the FBP web site in non-executable jar file format - JavaFBP jarfile. Its network syntax and component API are described in "JavaFBP Network Syntax and Component API" .
The latest version of C#FBP is available on SourceForge and on the FBP web site - C#FBP zipfile. Its network syntax and component API are described in "C#FBP Network Syntax and Component API" .
A new feature of both of these releases is the addition of metadata (or "attributes") to components, giving port names and characteristics, and the "must run" attribute. This is intended to allow diagramming tools to display port names and use this information for various types of checking. If you are currently working with a previous version of JavaFBP, the metadata will have to be added to your existing components, but this only has to be done once, and hopefully will not be too onerous.
A new, much more powerful, Java version of the DrawFBP diagramming
tool is now available.
I have pulled together a lot of material in a FBP Wiki, either dredged from the Internet by myself, or contributed by interested readers. It's not very organized (no, it's very unorganized), so it would be fantastic if some energetic person would undertake to do what Wikizens call "refactoring". To add stuff to it, please contact me for an edit password.
The book is now out of print, but is still listed at Amazon.com , and second-hand copies become available from time to time.
A copy is now on-line in PDF format - to access it, click on on-line copy . To download it, you may have to shift and click. The diagrams are not as elegant as in the original publication, but hopefully they will suffice to get the ideas across. Maybe at some point in the future, they will be upgraded. Please let Paul know if there are any obvious omissions or formatting errors (apart from the diagrams!).
Two reviews of "Flow-Based Programming" appeared a few years ago in one of the Australian computer journals, one by Tony Stevenson, then at Monash University, and the other by Andrew Wenn, of Victoria University of Technology (Footscray). The latter has a pretty complete description of FBP and of the contents of the book, as well as some (well-deserved) criticisms! To see them, click on Stevenson, and Wenn, respectively.
Ed Yourdon has kindly added my book to his list of "cool books".
There are some papers on FBP on my web site. Not surprisingly, there is some duplication as it is generally necessary to give a short description of the basic concepts in each one, but each paper has some unique sections. Here they are:
"Asynchronous Component-Based Programming." (contains some stats on typical application component counts)
"Flow-Based Programming" (contains some statistics on reuse)
I have uploaded a slightly earlier version of my book to this web site. Unfortunately, as the book is out of print, this will be your only version, until we can arrange to have a paper version available again. As stated above, a copy is also now on-line in PDF format.
I should state that, as of 2009, the web site will start to diverge from the exact text of the book - in fact it always did! This is partly due to changes in the diagramming tools; also we will be starting to incorporate results from later Java and C# work as tools based on these languages evolve.
A few years ago I implemented a number of the concepts of FBP in C,
calling it THREADS. It used the Borland C(++) compiler. It is described
in my book, "Flow-Based
Programming" - in particular, in the Appendix
(see "THREADS API and Network Specification").
The code is available on this web site at THREADS.
It is in the process of being upgraded to use Win32 and "fibers"
- I have put it up on my web site, at THREADS_32,
but I freely admit that testing has only just started. I am also
uncertain how to implement component metadata, so this is currently
unsupported.
A Java implementation of Flow-Based Programming which we have called JavaFBP (formerly JFBP, formerly JavaFlow) is now available in open source form, and has been registered as a project on SourceForge, and the code has been uploaded onto the SourceForge File release System. It can be accessed by clicking here: SourceForge. To download the source code, click on "files", and then download the latest release. An earlier version was uploaded onto the CVS server on SourceForge, but it is not up-to-date.
The code is also available on this web site. To download it, click here: JavaFBP jar file (you may have to shift and click).
This implementation was written by John Cowan and tweaked by me. It allows a developer to specify an application as a network of nodes, which are long-running Java threads, connected by fixed-capacity connections, over which data objects travel. We feel this approach lends itself to server applications where high throughput is required. It is easy to multiplex components that are either I/O-bound or CPU-bound (in the latter case this allows us to take advantage of the multiple processors in current machines). The network is also specified as a Java program, which we feel is a more flexible approach than designing a special-purpose language for this purpose.
As JavaFBP did not exist when my book was written, a page has now been added to this site describing the syntax of a JavaFBP network definition and a sample JavaFBP component - information will be added to this page as time permits. To see it, click on JavaFBP Network Syntax and Component API.
The latest version, JavaFBP-2.4, has a number of new features, described on the FBP wiki.
JavaFBP also supports "static subnets" - an example can be found in the jar file, and there is a fairly complete description of how it works in Composite Components, particularly in the second half, including "substream sensitivity".
Please feel free to look at the code, and/or contact me for more information, or to give me feedback.
The following notice has been added to all the source files:
static final String copyright =
"Copyright 2007, 2008, 2009, J. Paul Morrison. At your option, you may copy, " +
"distribute, or make derivative works under the terms of the Clarified Artistic License, " +
"based on the Everything Development Company's Artistic License. A document describing " +
"this License may be found at http://www.jpaulmorrison.com/fbp/artistic2.htm " +
"THERE IS NO WARRANTY; USE THIS PRODUCT AT YOUR OWN RISK.";
As stated in the copyright notice, you can look at a document describing this license by clicking on Clarified Artistic License .
A couple of years ago, Amanda Ge kindly converted the pre-5.0 Java version to C#, but did not have time to test it. With input from David Bennett in Melbourne, Australia, I have upgraded my C# implementation and it is now available on SourceForge and on my web site. There is a readme file describing the new features.
The new version of C#FBP can be found at C#FBP
(you may have to shift and click), and
on SourceForge.
The source code is now available on SourceForge SVN.
Please feel free to play with it - and let me know how
you make out. I will of course post updates as required. If you have
already taken a copy, please download the latest version!
A page has been added to this site describing the syntax of a C#FBP network definition and a sample C#FBP component - information will be added to this page as time permits. To see it, click on C#FBP Network Syntax and Component API.
Several years ago, a picture-drawing tool which supports many of the concepts of FBP was written in C++ for Windows. It can still be obtained by clicking here: DrawFBP-C++ (you may have to shift and click), but it has now been superseded by a Java version, which can be obtained by clicking here: DrawFBP Installer (you may have to shift and click), and this is also available on SourceForge - look for the 2.3 version. The Java version is much more powerful, has fairly decent help facilities, and can generate runnable Java code, if it is given enough information. It can also be downloaded and executed immediately by clicking on JWS version of DrawFBP.
DrawFBP does not attempt to generate diagrams from text - we have seen too many failed attempts at this - instead it allows the diagrammer to lay out the flow as desired, capturing the information from the diagram, including x-y coordinates of blocks and line bends, in an XML file. From this, the diagram can be rebuilt, plus it captures the relationships between processes and their connections, so it has enough information to generate the lists of connections used by FBP (or FBP-like) schedulers. The new version of DrawFBP can generate working FBP networks, prompting the user to fill in any needed information.
DrawFBP also supports step-wise decomposition, and allows files and
reports to be added symbolically to a design. Here is a sample diagram,
showing an arrow in process of being drawn:
A Help facility is available, based on JavaHelp - this function does not require access to the Internet.
DrawFBP can be executed directly from the author’s web site, using Java Web Start (JWS), or can be downloaded as a jar file .
J. Paul Morrison,
8A Station Lane,
Unionville,
Ont., L3R 1R3
Canada
(905) 479-4056
Internet:
| @ | ||
| paul.morrison | rogers.com |
| 54,085 visits
|
