If you are wondering how a revolutionary new approach to application development, which has won acceptance from some of the leading thinkers in the IT industry, can have been in continuous use for over 30 years in key data processing applications at one of Canada's major banks, read on!
Over the last 30 years this technology has enabled the applications using it to respond to changes in business needs and systems environment, while still exhibiting the high reliability required of banking applications, yet running in less elapsed time than programs written using conventional techniques. It was actually invented/discovered back in the late '60s, but it took us 30 years to understand what we had! During that period we came to realize that this technology solves many of the problems that bedevil conventional programming, and, more importantly, we have even figured out why. Hopefully we now have your attention!
Herakleitos, in about 500 BC, said
usually translated as "Everything changes". What he actually
said was: "Everything flows".
Somewhat more recently, the following phrase became popular:
"Go with the flow!"
For late-breaking news, see the FBP Wiki.
Note the new version of JavaFBP available on SourceForge and also on the FBP web site - JavaFBP jarfile.
Also a new and improved version of C#FBP on SourceForge and on the FBP web site - C#FBP zipfile.
A new feature of both of these releases is the addition of metadata (or "attributes") to components, giving port names and characteristics, and the "must run" attribute. This is intended to allow future diagramming tools to display port names and use this information for various types of checking. If you are currently working with a previous version of JavaFBP, the metadata will have to be added to your existing components, but this only has to be done once, and hopefully will not be too onerous.
A new, much more complete, Java version of the DrawFBP diagramming tool has just been made available - for more information click on DrawFBP.
Flow-Based Programming is a new/old approach to application development, based on a completely different way of thinking about building applications. Some of its roots can be traced all the way back to the early days of computing, yet it offers solutions to many of the most pressing problems facing application development today.
Not surprisingly parts of the answer have been presaged many times over in the past, but my book on the subject, "Flow-Based Programming", van Nostrand Reinhold, 1994, ISBN 0-442-01771-5 is the first book to tie together many of these different efforts, and to point towards a very practical, powerful, yet perfectly realizable future of application development. New copies of this book are no longer available, but it is available on this web site in HTML and PDF form.
In "Flow-Based Programming" (FBP), applications are defined as networks of "black box" processes, which exchange data across predefined connections. These black box processes can be reconnected endlessly to form different applications without having to be changed internally. It is thus naturally component-oriented. To describe this capability, Raoul de Campo of IBM Research coined the term "configurable modularity", and it was later adopted and developed by the distinguished IBM engineer, Nate Edwards - he called it "the basis of all true engineered systems."
When using FBP, the application developer works with flows of data, being processed asynchronously, rather than the conventional single hierarchy of sequential, procedural code. This means that the precise timing of events can usually not be predicted exactly, which tends to make old-style programmers nervous. However, it turns out that this isn't necessary (and never has been!), and in fact shifts the emphasis from sequences of actions to transformations performed on streams of data.
FBP is thus a good fit with multiprocessor computers, and also with modern embedded software. In many ways, an FBP application resembles more closely a real-life factory, where items travel from station to station, undergoing various transformations. Think of a soft drink bottling factory, where bottles are filled at one station, capped at the next and labelled at yet another one. FBP is therefore highly visual: it is quite hard to work with an FBP application without having the picture laid out on one's desk, or up on a screen! For an example, see a portion of a batch program (you have to zoom in to see some of the connections). This diagram, which was built using the earlier C++ DrawFBP diagramming tool, only shows part of an extremely complex program, which has been running almost daily for over 30 years, processing millions of transactions a day. Now imagine building the same function using a conventional programming language!
Strangely though, in spite of being at the leading edge of application development, it is also simple enough that trainee programmers can pick it up, and it is a much better match with the primitives of data processing than the conventional primitives of procedural languages.
In recent years, the concept seems to have started to take off - this seems to be coinciding with a feeling that Object-Oriented isn't the perfect solution to all our problems that it was originally believed to be. More and more projects and papers use the term Flow-Based Programming (often in lower case) - to the point where it has started to be a generic term (like Bandaid in N. America!). For a fairly lengthy list of projects, papers, and even companies, that seem to be using related concepts, see the Wiki page called "Flowlike Projects".
As you might guess from the logo at the top, a project has been registered on SourceForge. Source code for a Java implementation of the concepts has been uploaded to SourceForge, as well as a picture-drawing tool that supports many of its concepts (see DrawFBP). There is more information about the Java implementation below at JavaFBP.
An implementation of FBP has recently been built using C# - it is on this web site (see below) and on SourceForge (under the same project name as the Java implementation).
An article on FBP has recently been added to Wikipedia - please feel free to extend it in any direction that you feel would enhance its usefulness.
Last but not least, I (Paul Morrison again!) am starting to have more time available, and I am now hoping to move more heavily into research, preferably involving Flow-Based Programming (of course). As the signs are that it is finally starting to take off, I am hoping that interested individuals will start to contact me about ideas and possibilities in this area. My email address can be found at the bottom of this page - I look forward to hearing from you!
You are hereby invited to participate in the FBP Wiki. A lot of material has been gathered onto this wiki, either dredged from the Internet by myself, or contributed by interested readers. It's not very organized, so it would be fantastic if some energetic person undertook to do what Wikizens call "refactoring". To add stuff to it, please contact me for an edit password.
The book is now out of print, but is still listed at Amazon.com , and second-hand copies become available from time to time.
A copy is now on-line in PDF format - to access it, click on on-line copy . To download it, you may have to shift and click. The diagrams are not as elegant as in the original publication, but hopefully they will suffice to get the ideas across. Maybe at some point in the future, they will be upgraded. Please let Paul know if there are any obvious omissions or formatting errors (apart from the diagrams!).
Two reviews of "Flow-Based Programming" appeared a few years ago in one of the Australian computer journals, one by Tony Stevenson, then at Monash University, and the other by Andrew Wenn, of Victoria University of Technology (Footscray). The latter has a pretty complete description of FBP and of the contents of the book, as well as some (well-deserved) criticisms! To see them, click on Stevenson, and Wenn, respectively.
Ed Yourdon has kindly added my book to his list of "cool books".
There are some papers on FBP on my web site. Not surprisingly, there is some duplication as it is generally necessary to give a short description of the basic concepts in each one, but each paper has some unique sections. Here they are:
"Asynchronous Component-Based Programming." (contains some stats on typical application component counts)
"Flow-Based Programming" (contains some statistics on reuse)
I have uploaded a slightly earlier version of my book to this web site. Unfortunately, as the book is out of print, this will be your only version, until we can arrange to have a paper version available again. As stated above, a copy is also now on-line in PDF format.
A Java implementation of Flow-Based Programming which we have called JavaFBP (formerly JFBP, formerly JavaFlow) is now available in open source form, and has been registered as a project on SourceForge, and the code has been uploaded onto the SourceForge File release System. It can be accessed by clicking here: SourceForge. To download the source code, click on "files", and then download the latest release. An earlier version was uploaded onto the CVS server on SourceForge, but it is not up-to-date.
The code is also available on this web site. To download it, click here: JavaFBP jar file (you may have to shift and click).
This implementation was written by John Cowan and tweaked by me. It allows a developer to specify an application as a network of nodes, which are long-running Java threads, connected by fixed-capacity connections, over which data objects travel. We feel this approach lends itself to server applications where high throughput is required. It is easy to multiplex components that are either I/O-bound or CPU-bound (in the latter case this allows us to take advantage of the multiple processors in current machines). The network is also specified as a Java program, which we feel is a more flexible approach than designing a special-purpose language for this purpose.
As JavaFBP did not exist when my book was written, a page has now been added to this site describing the syntax of a JavaFBP network definition and a sample JavaFBP component - information will be added to this page as time permits. To see it, click on JavaFBP Network Syntax and Sample Component.
The latest version, JavaFBP-2.1 has a number of new features, described on the FBP wiki.
JavaFBP also supports "static subnets" - an example can be found in the jar file, and there is a fairly complete description of how it works in Composite Components, particularly in the second half, including "substream sensitivity".
Please feel free to look at the code, and/or contact me for more information, or to give me feedback.
The following notice has been added to all the source files:
static final String copyright =
"Copyright 2007, 2008, J. Paul Morrison. At your option, you may copy, " +
"distribute, or make derivative works under the terms of the Clarified Artistic License, " +
"based on the Everything Development Company's Artistic License. A document describing " +
"this License may be found at http://www.jpaulmorrison.com/fbp/artistic2.htm " +
"THERE IS NO WARRANTY; USE THIS PRODUCT AT YOUR OWN RISK.";
As stated in the copyright notice, you can look at a document describing this license by clicking on Clarified Artistic License .
A few years ago I implemented a number of the concepts of FBP in C, calling it THREADS. It is described in my book, "Flow-Based Programming" - in particular, in the Appendix (see "THREADS API and Network Specification"). The code is available on this web site - just click on THREADS (you may have to shift and click). It has not been tested recently, and may need updating or modification to accommodate different compilers.
A couple of years ago, Amanda Ge kindly converted the pre-5.0 Java version to C#, but did not have time to test it. With input from David Bennett in Melbourne, Australia, I have upgraded my C# implementation and it is now available on SourceForge and on my web site. There is a readme file describing the new features.
The new version of C#FBP can be found at C#FBP (you may have to shift and click), and on SourceForge. Please feel free to play with it - and let me know how you make out. I will of course post updates as required. If you have already taken a copy, please download the latest version!
Several years ago, a picture-drawing tool which supports many of the concepts of FBP was written in C++ for Windows. It can still be obtained by clicking here: DrawFBP-C++ (you may have to shift and click), and is also available on SourceForge, but it has now been superseded by a Java version, which can be obtained by clicking here: DrawFBP Installer (you may have to shift and click), and this is also available on SourceForge - look for 2.0 version.
DrawFBP does not attempt to generate diagrams from text - we have seen too many failed attempts at this - instead it allows the diagrammer to lay out the flow as desired, capturing the information from the diagram, including x-y coordinates of blocks and line bends, in an XML file. From this, the diagram can be rebuilt, plus it captures the relationships between processes and their connections, so it has enough information to generate the lists of connections used by FBP (or FBP-like) schedulers. The new version of DrawFBP can generate working FBP networks, and also run them, once all diagram components have had FBP "verbs" assigned to them.
DrawFBP also supports step-wise decomposition, and allows files and
reports to be added symbolically to a design. Here is a sample diagram,
showing an arrow in process of being drawn:
A Help facility is available, using JavaHelp, where the help pages are held on the Internet.
Here is a sample XML network definition - the extension will be .drw. You will notice that the x-y coordinates are included - this allows both the diagram to be reconstructed from the XML, and also an executable form to be generated, which of course will ignore the x-y coordinates.
J. Paul Morrison,
8A Station Lane,
Unionville,
Ont., L3R 1R3
Canada
(905) 479-4056
Internet:
| @ | ||
| paul.morrison | rogers.com |
| 27,344 visits
|
