Flow-based Programming

Panta rhei (Panta rhei) - Everything flows.

Relationship with NoFlo

Some of my readers may have arrived at Flow-Based Programming (FBP) by way of NoFlo, based on JavaScript, which uses a number of the same terms and concepts as FBP. This project, started by Henri Bergius in 2012, attempts to combine FBP concepts with JavaScript, and has been creating a significant buzz world-wide since then, given impetus by the marketing skills of Henri, Dan Tocchini IV, and their talented team at The Grid.

In Aug. 1, 2013, they started a KickStarter project to raise $100,000 for future development, and had raised $115,677 by the end of the 45-day period. NoFlo has been recognized by Forbes, O'Reilly, FastCompany, and many other companies and publications.

NoFlo is integrated with a graphics tool called NoFlo-UI, developed by The Grid.

It is fantastic that NoFlo has been able to attract world-wide attention to FBP, but what they are calling FBP differs in a number of respects from what might be called "classical" FBP. Although NoFlo shares much of the terminology and indeed philosophy of FBP proper, it has a very different mindset. NoFlo is much more synchronous and procedural than FBP, and may be thought of as a halfway house between technologies now being referred to as "dataflow" and "classical" FBP. NoFlo and "classical" FBP address different problem spaces: NoFlo addresses user experience, human interfaces, and its configurable modularity certainly gives it great flexibility, while FBP addresses data processing applications (business or scientific), and has a very different way of thinking about computer applications. The latter really is a new paradigm - you can think of it as a literal "data factory" mental image, where the application is expressed as a series of transforms on data streams.

Because of FBP's asynchronous nature, it naturally results in components with lower granularity (coarser-grained), working with more complex data objects. For example, NoFlo has a Math package, in which math functions can be exposed at the network level, whereas in FBP users rapidly learn that they should not decompose records into fields, so a Math package would not be useful. A NoFlo network might easily exceed 300 nodes, whereas in FBP each one of these nodes would be an asynchronous process, so such a network would be extremely hard to visualize or manage.

However, both "classical" FBP and NoFlo are component-oriented, both support stepwise decomposition using "subnets", and both support "configurable modularity".

As "classical" FBP is highly asynchronous, it is well-adapted to supporting high data volume applications. All current FBP implementations use one thread per node, so they can take advantage of multiple core machines. FBP networks written in different languages can communicate by way of sockets. FBP provides a consistent application view from very high level (e.g. documents flowing between departments), down to the implementation level.

For those wishing to gain experience with "classical" FBP, there is no substitute for reading the book (Flow-Based Programming, 2nd Edition: A New Approach to Application Development), and then starting to use one of the FBP implementations such as JavaFBP or C#FBP, or even the C++/Boost implementation currently under development, as described on the FBP web site.

NoFlo-UI, the diagramming tool, was originally developed to support NoFlo, but is more general, and is in process of being extended to support other FBP, and FBP-like, development environments. NoFlo-UI is well-integrated with the NoFlo environment, and provides run-time interpretive facilities to help develop NoFlo applications.

Users wishing to work with "classical" FBP can use the older drawing tool, DrawFBP, written using Java Swing, which is also quite general. DrawFBP lacks run-time interpretive facilities, but, on the other hand, it can generate runnable networks for JavaFBP (immediately runnable) and C#FBP (with a bit of tweaking to add in component names). It can also generate the .fbp notation used by NoFlo.

DrawFBP diagrams are stored in XML format, and additional generators can be added easily, or users can build their own generators using the XML format as input. Although there is an extra "generate" step in the case of DrawFBP, the user can work both at the diagrammatic level, and at the level of the network definition in his or her chosen language - of course, once the generated network has been modified, the network diagram becomes less useful.