Paragraphs preceded by 2 angle brackets are from PaulMorrison (also some stuff in italics); other paras from Sean (including those with 4) - but you probably figured that out!
Sean speaks...
For the past two years i've been doing research on cognitive intuition and end-user programming, and have been very excited about the ruby programming language -- which i find to be alot like smalltalk polished with lessons learned from perl. (hence ruby is a japanese pun on perl.)
i've also been working with artisans who otherwise would never consider themselves programmers, who picked up and found graphical digital signal processing (dsp) programming such as the PureData and Max/MSP environments quickly, easily, and intuitively. i feel this shares much the same perspective as your design for flow-based programming.
also also, i've been very interested in the work of DougMcIlroy and the history of unix streams/sockets, particularly in the design of unix as an operating system built from compiled C "objects" (aka, the core utilities).
i hope to finish reading your book soon, between when i should be finishing NorbertWiener's "Cybernetics" (see http://pespmc1.vub.ac.be/), which with its message of feedback (http://pespmc1.vub.ac.be/FEEDBACK.html) and context you may find very interesting if you do not know of it already.
i would like to talk with you more about whether you think ruby could be a suitable language to implement fbp "properly", particularly as ruby may be of interest to you with your "SmartData types" because ruby is "pure OO". basically, objects are the primitives (although anonymous functions, aka lambda, are permitted for useful features such as closures).
differences from java... from my limited experience with java, major differences come from java language design trying to replace c++. ruby is more like smalltalk with friendlier ("more normal"?) syntax. by "pure OO", everything in ruby is an object, while java still allows for data type primitives.
i think you'll appreciate how ruby handles assignment.
according to a cache from the ruby website comparing it to ther languages:
Java is the language in the news one way or another. Now, let's compare Java with Ruby:
Ruby is an interpreted language. All data (including Integer, String, etc.) in Ruby are class instances. Ruby's module can be used to share implementation among classes. Ruby's variable and expressions are not statically typed.
>> 'Nother question: can I assemble an application from compiled pieces?
although i do not think anything technically stops ruby code from being compiled, no ruby compiler currently exists.
i'm not really interested in compilation because i think it is more imporant for human creative efficiency to change code dynamically and receive immediate feedback to the programmer. emacs and squeak/smalltalk work this way. "computer time is cheap, programmer time is expensive." -- esr
>> I >> got to thinking that I would rather not multithread inside Ruby (or >> Java), but have multiple light-weight engines (does Ruby have to be >> interpretive?) use FBP to do multithreading. This would allow >> coexistence with C++ modules, perhaps some kind of mini-Java, etc.
for myself, i'm not really interested in co-existence with c++. after programming for awhile in c++ i've become nothing but frustrated and fed-up with it's "multi-paradigm" use in modern software development (which amounts to no paradigm at all) as well as its history as a bastard language built on top of C as a political maneuver to convince the then existing C mindshare that object-orientation was a good idea. C as an abstraction for portable assembly is wonderful; ruby itself is written in C and can interoperate with C seemlessly. but i find c++ to be as ugly for complex (object-oriented) software as much as the csh is ugly to do shell scripting in. thus, i've been really put off by the commercial c->c++->java->.net? lineage. but that's just my opinion.
however, your question of multiple ruby engines is a good question. my personal design aesthetic is more inclined to provide the ruby interpreter as a daemon/service (ala HURD kernel design) and then work within "ruby space". but is the current ruby interpeter optimal for this task? that's a question for yukihiro "matz" matsumoto himself.
>> Have you explored this wiki - there are some fascinating >> discussions on it!
i *LOVE* the c2 / portland wiki repository, and indirectly ward cunningham. i felt unbelieveably refreshed and relieved of anxiety to finally find people discussing the history and design of programming languages, something too many snobby ivory tower computer scientists seem very determined to ignore, repeating "turing completeness" like a broken record -- as if the whole world should just speak english or chinese because of NoamChomsky.
c2 is how i discovered your FBP to begin with, though as i said i've had strong inklings about it from unix pipes and max/msp.
>> I have also tried to start an FBP wiki - early days yet!
wiki is a good technology idea, like the older cousin to the popular meme of weblogs. anything that gets people to publish content instead of just consuming it as with television will hopefully keep people exercising their brains.
>>>>i would like to talk with you more about whether you think ruby could be a >>>>suitable language to implement fbp "properly", particularly as ruby may be > >> of > >>>>interest to you with your "smart data types" because ruby is "pure OO". >>>> >> Yes, that sounds interesting! We need to get moving on SmartData.
yes, look into how "everything is an object" in ruby. this really hit me as a deep concept after a few days of learning ruby. sort of like a practical functional programming where "everything is an algorithm, even the data structures" as opposed to the c++/stroustrup OO model wish seems to push "everything is a data structure, even the algorithms". active versus passive.
>>>>basically, objects are the primitives (although anonymous functions, aka >>>>lambda, are permitted for useful features such as closures). >>>> > >> I tend to think of FBP as consisting of "passive" (data) objects and >> "active" (thread) objects. I find I am not really that fussed about >> whether the latter type is thought of as an object or not. I wish >> language like Smalltalk (and Ruby?) wouldn't talk about "message >> passing" - they are linearized indirect calls.
yes, either i'm really quite daft on the "message passing" of smalltalk, or it's simply the worst possible naming for something very simple.
so "data" objects as in information that gets processed, and "thread" objects which do the processing? do i understand you correctly?
Yes, exactly. I'd like to emphasize that several FBP "components" (threads) can exercise the same piece of (reentrant) code - in JavaFBP (the Java implementation of FBP - previously called JFBP) we called (had to call) these code chunks "classes", but I'd like to have a more specific name...
i think the "everything is an object" concept is very critical for the same reason "everything is a function" is the core tenant of functional programming. an example i'm vaguely thinking of off the top of my head would be distributed computation over a network -- if i wanted to share some objects/components with a friend to also run on his computer so we could collaborate on a larger project. i'll have to double-check the specifics of the "von neumann machine" (although i think the context typically referred to is sets of instructions that change the machine state) but something von neumann and turing both figured out is that the difference between the "stuff" and the information in the "stuff" is illusory. (as far as data and code go, code is also data.) in computers at least people can prove that they are the same thing. i think quantum physicists are proving the same with our shared reality, and hence my passion for information theory.
>- this conversation could conceivably be carried out on a wiki page - >> that way people could watch it, and when they get too upset, join in.
true enough.
now we're doing it!
>> - I quite agree about C++ - I'm afraid the only C++ program I have >> written was basically C, so I have the bad habit of not distinguishing >> them in my mind. But I think I said in my book that C++ struck me a >> rather weird hybrid.
many other people do this as well; too often they're the ones who write "C/ C++" on their resumes/cv. but according to BjarneStroustrup, C++ is really a completely different language built on top of C (but then by that definition, so are ruby, python, perl, etc.) but just happens to look like C. i think it's important people realize the difference between C as a portable assembly "low level language" and other programming languages as "high level languages", and clear up this muck about "very high level languages" and "4GL".
according to ruby inventor matz, he wrote ruby so it wouldn't surprise him (that it would be intuitive), whereas after years of programming in C++, it still surprised him. iirc, artima.com has some good interviews with matz.
>> - "everything is an object" has been devalued for me. After being >> beaten about the head for years by the OO folks, I have come to the >> conclusion that they use "OO" as a synonym for "good". Can we >> strengthen "object" back up again, or is it too late? Hey, can we ban >> primitive data types? That would be cool!
that's exactly what ruby does! that's a large reason why ruby is so cool, and why understanding that "everything is an object", aka "pure OO", finally clicked a reason in my brain for "objects are good", as opposed to just the hype everyone else seems to buy into. support for primitive data types are also, imho, one way how java snatches defeat from the jaws of victory.
seriously, check ruby out.
>> - yes to the comment about data objects and threads
word.
Quoting Paul Morrison <paul.morrison@rogers.com>:
>> I hadn't heard of Javaspaces, but I see there is an entry for it on the >> c2 wiki. Linda is an old friend - I quote quite liberally from G and >> C's writings in my book, including borrowing their phrase "coordination >> language". As far as I can see, the main difference between Linda >> (rinda) and FBP is that in the former tuples are retrieved >> associatively, while in the latter they arrive on tram lines!
i think a better url i could have sent you was the chapter from 'programming ruby': http://www.rubycentral.com/book/ospace.html
iirc, your train-car metaphor is functionally repeated here.
"Ruby calls this kind of serialization marshaling. [Think of railroad marshaling yards where individual cars are assembled in sequence into a complete train, which is then dispatched somewhere.] Saving an object and some or all of its components is done using the method Marshal::dump . Typically, you will dump an entire object tree starting with some given object. Later on, you can reconstitute the object using Marshal::load ."
>> I definitely need to download a copy of Ruby! But, you know, I'm >> thinking that, if we really want to prevent programmers from using >> integers, say, we will have to build our own language! That's even >> more fun than using it
i prefer not to reinvent the wheel wherever i can. i'm very satisfied (impressed even!) with the work done by matz and others with ruby, as an expressive language, already. but i feel that the larger goal of software development (and the complexity thereof) could be greatly improved by using the FBP pattern/style. i think this has already started to be realized on the whole by druby; again, particularly with introspection (vital for feedback -- relevant both to the systems theory work of cybernetics as well as education and human-computer interface to support end-user programming).
there's also something to be said for mindshare, particularly the history of c++ over smalltalk; outside of japan ruby is niche enough without creating another more esoteric programming language.
i think it's time to move on from "yet another programming language"; rather than standing on the shoulders of giants programmers tend to stand on each others toes, indeed. but i would LOVE to work with you on implementing FBP in ruby.
some other notes from reading your book:
] "data centric thinking" YES! particularly to both free "consumers" from "elite professional programmers" (what i think alvin toffler referred to in a historic context as the "priest class" -- i think you may have even said the same thing on the c2 wiki about a catholic context).
] "an FBP network is a directed graph"; know anything about graph theory and complexity? pick up a copy of barabasi's 'Linked' if you haven't already.
] "so FBP is a highly visual PatternLanguage"; yes, christopher alexander and the connection to PL via smalltalk/oopsla is quite interesting for developing style guides. or as you wrote, "FBP is an engineering-style discipline". also, again check out max/msp at http://www.cycling74.com/products/maxmsp.html
] according to your prologue you studied anthropology, this may be a granfaloon but so did bonnie a. nardi whose book 'a small matter of programming' (ISBN 0262140535 (amazon.com, search)) i enjoy, as well as her work on "activity theory" from the research of lev vygotsky is appealing from a cognitive science / cybernetics perspective.
] "any activity which bores people should be done by computers, and will probably be done better by them", you sound like a true Goodship personality.
I'm flattered!
] "to a logician the statement J = J + 1 is a contradiction", again check out how ruby handles assignment.
] "visualize an application built up of many such main-line programs running concurrently", yes i think this is the future where the distinction between applications (currently monolithic, single-purpose, usually compiled and thus generally static) and the operating system / development environment are blurred. the web over the internet started to do this through the publication of content (data-centric) but fell into "traditional software development" once commercial interests heated up. now weblogs and rss newsfeed aggregators (rdf / semantic web) is trying to get off the ground, but i think it needs to start with the tools for deployment, instead of yet again a priest class of programmers releasing applications for producing content (manilla perhaps being the worst example, and sadly moveabletype the best). as you said, "an application can alternatively be expressed as a network of simple programs"; this was the ideal unix once held. "this is the famous legoland programming which we have all been waiting for!"; precisely!!! as an avid childhood lego enthusiast, i've always tried to think of programming and reverse-engineering as a child would take apart and put together lego bricks.
] "taking advantage of developers' visual or "spatial" imagination", yes, i'm all about optimizing the human mind with computers to process information creatively. again, that's a major aspect of what goodship is all about as well. alan kay was also big on this. and it also relates to both the software engineering as well as the cognitive science aspects of cybernetics. ruby takes on a similar goal from a linguistics, or really a semiotics, perspective (you also expressed an interest in language and speaking several). whereas something more along the lines of max/msp, pure-data, or what's referred to as "fabrik" in the smalltalk circles (back in the late 80s). i feel it's important for users/developers to edit code with any editor that makes the most sense to them, but still produce code that makes sense in any other languge.
] do IIPs relate to smalltalk/ruby style closures at all? i'm wondering if the data stream should be yielded within an object method, or what... (i'm still trying to distill the terminology between ruby and your FBP).
] "it turns out that machines (and people) word more efficiently if you only retain the constraints that matter and relax the ones that don't, and you can do this without any loss of reliability." sounds just like Goodship!
] "another factor which makes me think it is timely for this technology to be made public is that we are facing a growing crisis in application development. at the same time as new requirements are appearing, the underlying technology is changing faster and faster." sounds exactly why i have an interest in christopher alexander's 'a pattern language' and end-user programming. frankly, only the end-users know what they really want, too much noise/entropy (see information theory & cybernetics) fills the channel from formalized specificaion and centralized management / software development by "expert" programmers. gnu/free software demonstrates a model by which users can then mutually share ("give back"/feedback) their "itch scratching" improvements to advance software. i believe in the future mechanisms will exist for people to collectively sponsor full-time software architects to generate valuable functionality they're less interested in thinking through themselves.
i think i understand what you're getting at with "black boxing" in terms of encapsulation... that to use a "concatinate" component i shouldn't have to figure out how it works in order to use it; i should be able to _just use it_.
however, when you discuss larger components -- "complex components" / components built up from smaller ones; i start to disagree only in the context of if a user/developer needs a complex component that doesn't exist yet and wants to learn how to assemble one. sometimes it's easier to glean a style from reverse-engineering a complex component and then applying the perceived pattern to designing the new component. in other words, to answer that powerful question of curiosity "how'd they do that?" ("i want to do something like that!", "what components did they use to do that?")
this is again why i think that interpreting source code and making upmost use of introspection to be so vital to both the education and practice of (data) flow-based programming (in ruby).
also, from your chapter 2 big vs. little customer vs. vendor observation, "a vendor can provide standard components, but the customer has to be able to write custom components as well" to be exactly true of end-users and full-time software development shops. i would say "end-users have to be able to write custom components as well", and in the same language and style as the skeleton -- no preferential treatment.
from chapter 2 of the FBP book on the subject of structured analysis: "there is a chasm, which nobody has been able to bridge in practice, although there are some theoretical approaches, such as the jackson inversion, which have been partially successful."
why are design specs and source code two different things? source code IS a design spec for telling the computer how to flip its transistors. authoring software IS the design (creative) process, and this is something that needs to be done organically, evolving with the entropy of changing human needs and unforseen circumstances. (which of course relates to end-user programming and ChristopherAlexander's PatternLanguage, as well as distributed ad-hoc software development efforts as exemplified by the gnu/free software "movement".) so the syntax sugar of a programming language (which is basically what defines a programming language over another vis-a-vis turing completeness) should encourage a best practices "meta" design pattern (just as for natural languages i believe the SapirWhorfHypothesis to be weakly true). this is of course the exact opposite of multiparadigm (no paradigm) languages encouraging "the freedom" to write bad code, such as BjarneStroustrup's C++; stroustrup ignores the economic principle of path-dependency. people forget that software (development) is a process not a product; you'll never write that one perfect (shipped) application and never look at it again -- perhaps think of all software authoring as "maintenance". again, this is why feedback is so critical to the development effort, it lets you know whether what you're doing is right or not -- as paul mentions in the FBP prologue "rapid prototyping is a process of reducing the uncertainties [see: KarlWeick? and organization theory] in the development process by trying things out [experimentation!]. i believe that anything you are uncertain about should be prototyped". i tend to refer to the problem of the mis-perception of software as a static commodity rather than a dynamic process as a major source for the problem of software brittleness -- again as paul refers to as the problem with building automotive parts with clay that then become fired in a kiln and brittle.
then in chapter 7 paul repeats from WayneStevens, "there was no change in viewpoint as you moved from design to implementation"; just that the problem remains to not try to "control flow" but to "data flow" in authoring software.
data flow, blocks, and closures
http://www.artima.com/intv/closures.html
as i asked, "do IIPs relate to smalltalk/ruby style closures at all? i'm wondering if the data stream should be yielded within an object method, or what... (i'm still trying to distill the terminology between ruby and your FBP)."
mostly more in terms of utilizing the context when yielding for objects to be truly flexible. both blocks and closures (as a special case) are really just lambda aka anonymous aka nameless functions (themselves aka methods aka subroutines aka messages).
p.s. this semantical nonsense in programming has got to stop. is it really a conspiracy to trick people into someone's ego sounding smart by creating new names for old ideas?
IIPs are data chunks that start off static and become dynamic - I believe Linda has something that does the reverse! IIPs address a problem we kept running into when using earlier FBP implementations: the distinction between parameters and data streams. For instance, you would like to put a parameter into an FBP network specification, but it would be nice to alternatively be able to get the same info off a file. In previous implementations, these used very different mechanisms - in our later implementations (THREADS and JavaFBP), this could be handled with a single mechanism from the point of view of the component being parametrized.
distributed ruby (druby), threads, and rinda
so browsing through the druby package, one of its requires is 'threads'. this forces me to confront another issue implementing FBP in ruby, am i creating a better replacement/alternative for existing ruby threads, or will i use/extend that package? how will this relate to druby's rinda vis-a-vis paul's research of linda? ah, word, rinda being an implementation of linda: LindaLanguage
threads and data, and ruby
>> I tend to think of FBP as consisting of "passive" (data) objects and >> "active" (thread) objects. I find I am not really that fussed about >> whether the latter type is thought of as an object or not. I wish >> language like Smalltalk (and Ruby?) wouldn't talk about "message >> passing" - they are linearized indirect calls.
so "data" objects as in information that gets processed, and "thread" objects which do the processing? do i understand you correctly?
>> - yes
lately i've been thinking about whether this maps directly to the difference between accessing a class description method (Class#method) and an actual object instance method (Object::method). so the "threads" are classes while the "data"/"information packets" are really object instances.
though josh huber reminds me "one last note though: classes are objects too! (their class is Class) In fact, everything in Ruby is an object, except for local variables. (which are merely references to an instance of some class)"
Von Neumann Architecture
many flaws of path-dependency have only shown themselves in recent history. aristotelian logic and platonian ideal forms particularly bother me; to a lesser extent, euclidian geometry... but for our mutual griping of von neumann architecture, i'd like to clarify the specifics point by point. correct me where i'm wrong:
1.) procedural programming. this needs to die a quick death in favor of a more functional/data-flow style. (if only academics had pushed functional programming for business/real-world solutions in the 70s... functional programming to this day still is flailing as "academic toy languages", not helped by lack of well-defined libraries.)
I agree totally. But functional is orthogonal to asynchronous data-linked: the combination would be pretty powerful!
Somewhere I said that the only things wrong with procedural languages are 1) procedural logic, 2) variables, and 3) constants. I still think that!
2.) hardware data storage. many of your (paul's) arguments and my subsequent use of ruby after years of C++ have brought this into sharp focus. the problem stems from too much human attention to the difference between storing actual numeric data and then pointers to memory (storage) addresses which are numeric data themselves (i.e. 0x828472). for example, in C think of both how strings are really arrays of ascii-referenced numbers, and how when passing arrays into functions in C++ (i'm assuming this is also from C) the compiler automatically assumes you're really passing a pointer (*variable) to the memory address of the beginning of the array, usually (unless you force it) not a copy of the actual numeric data itself ("that would be inefficient"). another failure by most compsci programs to fully detail the importance of understanding pointers and references methinks; but then again, also massive failures to educate students on the history and etymology of languages. again, ruby neatly shields the user from these reference details, while in the interpreter handling them neatly/cleanly in pure C. so of course my biased opinion remains that ruby offers a cleaner separation between "high-level programming" and "close to the metal" "low-level programming" of C, assembly, and compiler design.
I think you're saying that references are another thing that should recede into the background when you're coding. I found it very important in C to know whether I'm passing a structure or an array or string (pointer) to subroutines. Then there are pointers to pointers, structures of pointers, and so on ad nauseam. is this not a task better suited to the authors of the code interpreter itself rather than each and every program author? but of course you are referring to C specifically, and within the context of a portable assembly (particularly perhaps for embedded devices) this makes perfect sense to me. when talking about human expression in communicating to a computer what you would like it to do, such as i would like to realize with flow-based ruby as an end-user (or "cottage industry") programming platform, i feel we can -- as you say in chapter one (introduction) of the FBP book -- "it turns out that machines (and people) work more efficiently if you only retain the constraints that matter and relax the ones that don't, and you can do this without any loss of reliability". i suspect this is also true of organization systems, that with ("grown") tools to take advantage of the internet as a rapid semi-autonomous communication medium, we can retain and probably even improve upon efficiency of centralized/hierarchial management with the freedom of diverse and distributed ad hoc "cottage industry" shops. there's a line in the TomDeMarco book PeopleWare that goes something like "people feel good carefully crafting something, yet don't really get any satisfaction out of turning out more widgets today than yesterday", and considering the cost of reproduction of code is near-zero this makes more sense to me than factory-themed economic models. i'll even be so bold as to suggest that failure of organizational structures to adapt to new economic conditions of *process over product* results in the current legal/political debate of copyright, DMCA, and both developing and selling media (including software) as a commodity rather than a service. ok, that was a little off-topic. :-p
My point about data storage (I think) was that it is the combination of having to preplan the timing of events in procedural code, plus a uniform array of pigeon-holes with non-destructive readout, that makes conventional programming so difficult and error-prone. It's like never knowing whether you've retrieved the same pigeon twice, or stuffed a different pigeon in on top of a previous occupant... Then you have to coordinate all this in a complex program with zillions of alternative paths.
Here's a simple example of what I mean: take the two statements (using '=' to mean 'set one variable to the value of another variable):
A = B B = CNow change it to the following, and the meaning becomes totally different:
B = C A = B
i may be wrong about this. i'm going to play with irb in a moment to confirm or deny. but what i've been trying to communicate is that in ruby, the meaning is not totally different. this is why ruby somewhat encourages (or at least allows) chained assignment (as well as other operators i imagine...). it's only different in the C (what i've termed as "portable assembly") model because assignment represents actual memory storage.
in ruby, all variable names are only pointers to objects. (this is much more intuitive to me as a student of linguistics and semiotics; variables are simply names for things, and the same thing can be a "clock", a "timepiece", and a "chronometer". as with lambda functions in lisp, objects may also be unnamed, although this makes referencing them slightly more difficult (but not very, thanks to introspection (aka reflection in java)).
of course i must point out the caveat that we are not assigning to different instances of "something" (in ruby, an object). also, i again must test in irb if ruby allows for variables that reference to a nil, or whether this throws an error (as it is a bit peculiar, and perhaps unintuitive -- at least to matz). ok, yes, ruby doesn't like undefined variable names offhand; "NameError: undefined local variable or method". so in ruby you've gotta be talking about real things, not non-existent stuff.
Now imagine this expanded exponentially - with hundreds of variables and thousands of possible paths in the logic. Of course, this whole area of complexity disappears altogether in a true applicative/functional language, as it doesn't think in terms of plugging values into pigeon-holes. Someone once said that, in an ideal language, you should be able to drop the cards (what are cards, grandfather?!) and the program would continue to run. This will actually be the case if your language is just a list of connections, as in FBP.
iirc, "dropping the cards" is called "referential transparency" in functional programming circles. at some point i would like to further clarify how FBP and functional programming are orthogonal to each other. i think in my brain i'm still somewhat thinking of it as an easier to digest (perhaps more distilled) version of "object-functional programming". http://www.mm.informatik.tu-darmstadt.de/staff/kuehne/fps/ i should double-check Kuehne's patterns, but i do not think they are asynchronous; i think async is vital, and certainly orthogonal to functional programming.
so getting over the data storage issue i think is where lazy (aka late) evaluation, aka streams (aka pipes & filters), aka coroutines (aka bounded buffers), aka data flow, enters the picture. take a look at luis casillas' lazy evalution version of the Enumeration class (from which many other classes inherit) at http://newson-pc.stanford.edu/~casillas/lazyenum.rb . basically with this he's just passing pointers to be evaluated on call rather than trying to do it all at once up front with the data itself; hence lazy evaluation.
See also the description of Friedman and Wise's work on 'lazy cons' in http://www.jpaulmorrison.com/fbp/recurs.htm.
3.) ...? profit! ;-)
p.s. someday i'll learn how to properly edit a wiki, really...
random thought number 42
a big key to data flow programming i would think is the feedback from the environment -- the code's own awareness of currently active (constructed) objects including the 'Class'es themselves -- to then author code (classes) which are DYNAMIC in nature. again, i think this is precisely what paul is on point about with black-boxing. you shouldn't have to know, as a programmer, what exists -- you should be able to query the environment itself for what you have to work with. (emacs has a similar help feature for discovering elisp functions. as 'C-h' is generic for "help"; 'C-h k' for 'describe-key', 'C-h c' for 'describe-key-briefly', and 'C-h a' for 'command-apropos'. and functions can be tab-completed.) rather than necessarily knowing the exact name (alias/assignment) of objects you wish to use, it's safer/easier/better to write code that asks what objects of a certain class are available to use. oooh, does this avoid the need for array/list processing completely; or rather than array/lists are generated dynamically from filters of the total available objects?
no more brittle hand-crafted data in code! no more rolls-royce applications, bring on henry ford interchangeable parts (components)! We're still waiting for BradCox's SoftwareIndustrialRevolution! Or another TedNelson who realizes that software and the software development process itself must Intertwingle (often this is what RichardStallman talks about with his memories of the pre-commercial AI Lab, and hence the GPL as one way to intertwingle code) as well as utilizing, especially with popular broadband, that everyone who currently uses a web-browser could be authoring and serving content as well from that same terminal (even remotely loading and storing program state rather than loading and unloading large applications). i dig on TheStructureOfScientificRevolutions and the idea of Fabrik, though i think i agree more with AlvinToffler that software development represents a *return* to cottage industry; NorbertWiener makes the same recommendation (in 'The Human Use of Human Beings', p.127) referring to research conducted by scientists of the world."
or in management and organizational theory, the reason why the japanese automotive industry switched to just-in-time (kanban) delivery production to eliminate warehousing.
maybe this should go up in the black boxing comments? Sure!
Contrasting the FBP style of programming with the hand-crafted style most other people practise, I have a strong intuition that almost all business applications can be done using a mix of precoded, pretested, components and logic specified using domain-specific mini-languages - not necessarily procedural - encoding business rules. In my book I gave some stats on component reuse that we collected over a series of projects. Look for the phrase "productivity gains" in http://www.jpaulmorrison.com/fbp/reuse.htm - especially Project "C".
the truth is out there
from http://www.jpaulmorrison.com/cgi-bin/wiki.pl?BasicConceptsofFBP :
"InformationPackets (IP) can be thought of flowing between functions, but more exactly we want to think of handles to the IP's that are flowing, the IP's remain unchanged 'out there'"
so yes, ruby does this already with how it handles assignment for object, and everything is an object. i'm thinking ruby's definition of an object will be suitable for an earnest information packet (IP). see #2 in 'von neumann architecture'.
Yes, I think so too - for data chunks, the object model seems fine (Java would be the same, no?). For the asynchronous "components", I see less advantage to the object model (but I'm willing to be convinced otherwise!).
what disadvantages do you see? offhand, primarily, i'm thinking it's better to not create a dichotomy where one need not exist. although java is more popular, i have concerns about code using primitive data structures and people distributing byte-code only software. i think this applies more to my own interest in end-user programming and code intertwingularity, than to flow-based programming.
BTW, I am wondering if we could call them "agents" - has that term been appropriated by any other branch of IT/computer science?
yes, "agent-based programming" has already been coined by others.
darn!
check out http://cliki.tunes.org/Actor
also, ActorsAndFlowBasedProgrammingDiscussion
Also BTW, what distinction are you making with the word "earnest"?!
only that i don't want to bastardize your definition of what an IP is until i really intuitively understand it. particularly in the history of programming languages, this happened too many times already.
Came across Vista out of the Univ. of Linz, Germany. They use the term "processors", e.g. "Processors are objects with distinguishing structure and communication facilities." The URL is http://www.se.jku.at/people/schiffer/se-94-05/se-94-05.htm. What do you think of this term, as opposed to "agents" or "actors"?
kishotenketsu
i've independently come back to the idea of "kishotenketsu programming" (as an antithesis to the outline style of imperative programming), and remembered slashdot fielding comments on this subject:
http://ask.slashdot.org/article.pl?sid=03/02/03/155259&mode=nested
kinda seems fitting for ruby as well.
Could you expand on this thought? Is it like going from the general to the particular rather than vice versa? If so, perhaps we could draw parallels with RapidPrototyping?...
part of it is my entry on IPC below. part of it is the functional-programming mindset. part of it is how ruby treats objects as just "out there" as paul also recommends in his FBP book.
as for RapidPrototyping?, my opinion on that remains that code -- software as a process -- like all technology, is a tool to think with, to extend the human capacity for creativity. i tend to think of RapidPrototyping? as a buzzword, or at least a misnomer, because software is never "done". software development, like all creative acts, is a PROCESS not a product. (or as other examples: what people want is not food the product, but alleviation of hunger the process. people want not water, but quenching of thirst. people want not shelter, but to rest safely and comfortably.) so in essence, all software at any given moment is only a prototype -- like any lifeform at any given moment, hence evolution (see Richard Dawkins or Stephen Jay Gould). the question i believe people developing Rapid Prototyping are actually trying to ask is "how do we increase the FEEDBACK between computers and humans?", in the process they both play a hand in for software development. (of course, read some Ray Kurzweil in addition to Cybernetics and understand that technology is merely the extension of humanity. without pushing too far into the realm of hinduism/buddhism, the computer is also a part of you as much as your eyes, your hands, and your brains; someday scifi-themed cybernetics, like star trek's "the borg" or Masamune Shiro's 'Ghost in the Shell', will make that plainly clear. for increasing the feedback, i believe popularizing introspection (aka reflection), particularly in the education of students in programming, will act as a major step forward. and i believe this model will be most effective for flow-based programming (fbp), as paul has already discussed in visualizing connections in asynchronous data-flow code.
unix tradition, inter-process communication, and the case against threads
i've been thinking long and hard about 'processes vs. threads', and i think i've come to a conclusion that threads in the traditional operating systems sense (skimping on resources by joining them under one parent process) is plainly a bad idea (except for, of course, the rare scenario of scientific computing). the C2 portland wiki repository has a a few entries for running debates on "threads considered harmful" (ThreadsConsideredHarmful ; i seriously started reading wikipedia on how to edit wiki too... sorry. this very topic distracted me from it.) of course, i argue against procedural programming, unlike the original article.
for a system to be robust, parts of it are going to fail, but the core system maintaining the integrity of the running data should not fail. if you're at all like me and have umpteen tabs open in your web browser for various thoughts that entered your brain for a short time only (such as suddenly surfaced subconscious / "back burner" thoughts for which you suddenly thought of a possible solution) before your conscious thinking takes on a new task from sensory motivation (what you're looking at), and then suddenly have the browser, as a monolithic application, crash, you too know how damaging this can be to your productivity for the day, week, or possibly month. hacks exist which cache or otherwise backup your tab urls, but sometimes you're writing an email in webmail or otherwise using a hack upon a hack upon a hack. for this reason is why a more robust software development method like flow-based programming is necessary to begin with. the application is too brittle a metaphor for computer-enhanced human workflow: computers as tools to think with. "old school" unix, squeak/smalltalk (and self), emacs share this design pattern.
for ruby and data-flow programming, each object i believe must be a process just as each small single-purpose compiled C utility in "old school" unix spawns as its own process. yet, for now, i think this can be done within "ruby space"... back to this after i test how the ruby kernel handles this.
i should also double-check HURD design to understand if any sub-hurd can recover for a crashed initial hurd kernel.
(oops, i thought i had thought of an answer to the "how ruby threads" issue, but actually more side-stepped it. but that's what i intend to address here.)
mach microkernel and kernel servers
so... in trying to solve the "ruby space" problem, i've been thought experimenting with the what-if of the ruby interpreter / runtime environment as a kernel server in parallel to a unix "personality" server under the mach microkernel.
unlike the design of GNU/hurd, most microkernel unix operating systems implement Mach for the basics (multi-tasking/processes, threads, multi-processor on single machine or over a network, inter-process communication / IPC, memory protection) and then a single "unix kernel server" to fill out the rest of the unix functionality found in monolithic kernels such as linux.
so if the ruby interpreter were another "kernel server" parallel to the "unix kernel server", ruby would still interact with the traditional unix environment, but each object in "ruby space" could be its own process (as compiled C utilities are in traditional unix) for truely reuseable and fault-resistent components. (this essentially creates a virtual "ruby machine" not unlike "smalltalk machines" and "lisp machines" of yore.)
the reason i mention such a wild idea here, other than continuing the discourse on threads and processes, is because of how similar the Mach design feels compared to flow-based programming FBP:
it's almost as if the Mach kernel just needs a friendly interface to running code through it, and ruby could very well be that friendly syntactic sugar.
of course this grossly underestimates the difficulty of writing kernel software, particularly for a microkernel such as Mach. paul, your expertise seems very C heavy, perhaps you have a better gauge of approachability.
This is interesting. I take it that Mach is the same Mach that evolved out of Accent out of RIG (see http://www.jpaulmorrison.com/fbp/cognates.htm). The similarities with FBP are quite striking. BTW I have been playing with getting a FBP-like kernel working on IBM mainframes. It would use preemptive multitasking, so presumably should be a lot more robust - I will post some info about it as it develops.
word up! http://www.kernelthread.com/mac/oshistory/8.html
i suppose independent verification never hurts. :-)
an update of sorts
i've been reading up on carl hewitt and the actor model versus FBP. i've also been intrigued by the extension of the 'capability model' for managing/securing distributed objects. http://www.erights.org/history/actors.html & http://www.erights.org/elang/index.html in the nitty gritty, FBP seems less "restricted", but i would prefer more elaboration on the "pure actor" model as ruby is pure-OO. reading http://www.agorics.com/Library/joule.html was quite clarifying from a "pure actor" perspective
i've been trying to integrate the diversely described feature sets of FBP, actor model, capability model, et. al. into a coherent feature list of what exactly needs extending in ruby and what that syntax will look like.
I assume you have been reading DavidHopwood's refactoring of our discussion in ActorsModel - what do you think of it?
everyone building upon java makes me grumpy. but, i finally decided to throw open the java implementation of FBP.
Continuations
i'm on the road at the moment, and once again i'm reminded that i need to majorly refactor this page...
but until then, i would like to point out something quite interesting that turned up in my reading of carl hewitt, henry baker, and the actor model (but perhaps should have turned up sooner in my reading on coroutines?). that something is continuations.
http://www.lcs.mit.edu/publications/specpub.php?id=762
http://www.lcs.mit.edu/publications/specpub.php?id=765
http://www.erights.org/elib/concurrency/index.html
http://rubygarden.org/ruby?ContinuationExplanation
that last link especially pleases me, that the ruby community is already keen on continuations, such that if the only tweak really necessary to turn pure-OO of ruby now into pure-actors / FBP is to change the process of methods/subroutines calling and returning.
Maybe you can make continuations more palatable for me! I saw several descriptions that talked about counters generating series of numbers, and my reaction is :-(. IMHO it is much cleaner to have an FBP process that has the following logic:
set local count variable "c" to 0 do until desired limit reached create an IP set field/element/attribute in IP to "c" send IP to some output port increment "c" enddo close output port
This just uses the usual FBP primitives, doesn't require any special language features, and has really clean use of variables, and you can parametrize it to make it a nice reusable component. Am I missing something?
![[Home]](/wiki.gif)
(define (make-counter)
(let ((c 0))
(lambda ()
(let ((current c))
(set! c (+ c 1))
current))))
the problem i have with this is that you've basically described a for-loop. and my (rather strong) impression now is that for-loops are a cheat for procedural/imperative programming to reduce redundancy. it's the first major crack in the casing of "maybe we're all wrong about this do-step-by-step-by-step linear model of programming". such loop syntax obscures the real nature of what you're trying to do -- which is call the same block of code (blocks are anonymous functions/procedures in ruby, fortunately) repeatedly but while passing in different context into it. in other words, recursive calling of the same function/subroutine with different arguments.
the advantage of keeping this methodology clear i think becomes evident from a data-centric/dataflow perspective. we want to follow where the data is going and what's happening to it, we DON'T want to worry about the imperative instructions used to achieve that dataflow.
i don't know whether i'm making this clear or not... but examples crop up such as spreadsheet programming, where that linearity doesn't exist yet we want to couple procedures (subroutines/functions) together to manipulate information. i tend to think of it as linking together segments of firehose with various filters in it. or perhaps a more proper analogy would be some sci-fi fibre-optics where the segments filter or bend the light to produce different colors at the end than at the beginning. plug in yellow and get green out. from a functional programming perspective this is typically the 'collect & select' idiom -- aka 'pipes & filters'. UseEnumerationsNotForLoops
I think you have put your finger on an interesting philosophical issue, and, just maybe, the reason why the academic community has been cool to FBP. You see, I don't care how a component is written - what matters is how it behaves. FBP components talk data streams to each other, not procedure calls. The counter above could have been written functionally, procedurally, or some other way, and it would still produce a stream of 'n' IPs containing numeric values. The pipes and filters (or plumbing) you mention are exactly how you hook together FBP components, which incidentally may be atomic or composite - again, it doesn't matter how they were built. A composite component is just a network with some sticky connections. Think black boxes, where you can add to the collection, but with a sufficiently rich collection, you should almost never have to (once you reach a certain critical mass).
(p.s. people throw around the term "messages" mostly as yet another way to say functions/methods/subroutines/etc as has been discussed here earler. however, from carl hewitt (who taught alan kay, who then invented OO and smalltalk), his use of the word "messages" seems to be more about the method "call" and emphasizing that at the low-level subroutines are a process -- syntactic sugar for gotos/jumps. this also says something about the debate over iterators, loops, and closures.)
I think of subroutines as another kind of building block, but one that is best used for synchronous purposes. FBP components are little "main lines", which can call subroutines to do synchronous jobs, because they are compatible with that use.
<older topic> I have felt for a long time that referring to linearized subroutine calls as "messages" is a bit of a cheat. Can you shed light on the history behind this? Was it intentional or just confusion?
so far my murky understanding is that somewhere between carl hewitt --> alan kay --> smalltalk community, that messages are simply the "hey, i'm asking for you by name" got abused in colloqy to mean the whole shebang of "calling a function". in most conversation i could see how this could happen, especially when you start to make assummptions about always using functions which return (implicit call back) as alan kay did with objects/smalltalk. carl hewitt's papers reveal this disctinction, as do the use of continuations.
i hope to make all this clear in my upcomming paper on the actors model and flow-based programming.
I look forward to seeing that - have you looked at the discussion in ActorsModel? I'd be interested in your reaction.