An Architect's View

In all of the post-acquisition chatter, I have had several folks comment to me that they're glad I'm staying around because of my community contributions. I've said it before (several times!) but it clearly needs saying again: My community involvement is not part of my job. The talks at CFUGs? Not part of my job. The conferences? Not part of my job (I have to apply to attend CFUN / CFUNITED for example just like any other conference - I choose CFUNITED as one of the conferences I want to go to each year!). The Concurrency library and the IRC bot? Not part of my job. Similarly, my contributions to Mach II, Model-Glue and all the other frameworks communities. Strange as it may seem, I do all this - almost entirely in my spare time - because I enjoy it! I like the ColdFusion community! You're a weird, mouthy bunch at times but I really do enjoy your company. I maintain a couple of websites outside work and they're written in ColdFusion. By choice. I know lots of languages but I choose to write websites in ColdFusion - because I like ColdFusion! If I find myself no longer an Adobe employee at some point, you can expect me to continue to participate in the ColdFusion community - because it's not my job!

I have uploaded my Concurrency library to riaforge.org and made the 1.2 release available as a download there. Visit the Concurrency for ColdFusion MX project on riaforge.org. The code in the CVS repository includes the beginnings of the TaskVector component I promised a while back and the beginnings of some reworked documentation (in HTML format).

Joe Rinehart has officially released the much anticipated 1.0 version of Model-Glue. Lots of great enhancements in this release - read his blog for details! Things I like: built-in redirects with event state management, event beans, ability to plugin SES directly into the framework, simplified configuration - lots of defaults now! And of course the ability to declare asynchronous messages (relies on my Concurrency library).

ColdFusion MX 7 makes it so easy to run code in parallel that you're probably going to go mad with it once you start. If you're using my Concurrency library, it can be very tempting to run many of your methods concurrently. You'll do it the name of performance, to make a request run faster. When I started this series of blog entries, I mentioned that you don't want too many simultaneous requests to be processed by ColdFusion because it can cause two much contention for CPU resources and you can end up thrashing your server as it constantly switches from request to request trying to give each some CPU time. That's why the "3-5 requests per CPU" rule of thumb is useful. The same consideration is true of your event gateways. If you try to run everything in parallel, you may well end up thrashing your server as it spends more time switching between threads than actually getting any work done. The default setting for the number of event gateway threads is 10. You might think that setting this higher will mean better performance but you probably don't want it set higher than 5 times the number of CPUs you have if your event gateway tasks are CPU intensive. If your event gateway tasks are I/O-bound, or are mostly waiting for external systems (e.g., running a slow web service call in the background), then you can afford to have more of them executing at once and you can set the number of event gateway threads higher. We had a 4 CPU server and someone had pushed the number of threads up to 40. The result was that under load, all the threads were active and the server started thrashing. We cut the number of threads back to 20 and the performance improved dramatically. We are actually still able to thrash the server under load so we are going to throttle the parallelism a little more at some point. Oh, the load test scenario we'd forgotten? If you've read all three of these Asynchronous Development entries, you've probably guessed by now: we did not test the scenario where we have a lot of data queued up, ready to be consumed, and you start all of the event gateways at the same time. We had too much parallelism and too many automatically started gateways - when we brought up ColdFusion, it immediately started consuming data and overwhelmed the server. We now know how much of a backlog of data we can consume without causing a problem and we have also tested a process for consuming larger backlogs: starting and stopping the event gateway to let it process part of the backlog each time. One very useful thing to note about the event gateways: if you ask them to stop, they do so gracefully, accepting no more entries on the queue but waiting for active threads to complete before actually stopping.

Mostly for Roland Collins, this release of the Concurrency library adds a TaskPool CFC that lets you execute a group of tasks asynchronously which saves you a lot of typing. The README has full details. I've also changed the FutureTask constructor so that you can specify tasks as CFC names and methods, e.g., "MyTask.execute()", which means you no longer have to instantiate task objects yourself if they have no specific initialization code. Task arguments can also be specified at initialization now, instead of just when you run() tasks. Coming soon, release 1.3, which will add a TaskVector, allowing you to execute a method on an array of values in parallel and gather up the results. Suggestions for additions to the library are always welcome!

You'll probably noticed my blogging has been a little lighter than usual over the last few weeks. That's because the order management project that I've been working on for many months is now approaching the integration testing stage so we're assembling all of the data paths and running realistic data end to end in every direction. The project relies heavily on the event gateways in CFMX 7. I'm focused on the ERP servers where we're running nine event gateways but both the web application servers and the CRM application servers are also running CFMX 7 with about half a dozen event gateways on each. Naturally all that is keeping me extremely busy and leaving me with less time to read and respond to mailing lists and less time to blog stuff. It's going pretty well tho' and we've seen throughput of 1,000 operations a minute (reading a message from Oracle AQ using a custom event gateway, transforming the XML rowset to a query, generating a more structured XML message from the query and publishing it to a JMS event gateway - that's all one "operation"). I did find a little time to update my Concurrency library and I've had some great feedback from Roland Collins that will make it into release 1.2 some time in the next few weeks. Despite all that, I've started hanging out in my Breeze meeting much more. That's mostly because Breeze 5 now supports Mac screen sharing as well as a number of very useful presenter features. Some folks watched me upgrade our QA ERP servers to CFMX 7 on Thursday morning. Other folks watched me work on Concurrency 1.1 earlier in the week. I tend to have my webcam turned on all day at work (not at home, although I do have a webcam at home too!) and I often share my screen while I'm working - if it's something that is not confidential, of course! - just so folks can get a feel for what I do and how I do it. Drop in any time and say "Hi!"... Even outside work hours I'm often logged into Breeze now!

With the new event gateway feature in ColdFusion MX 7, you can execute CFML asynchronously by using sendGatewayMessage() and a CFC with an onIncomingMessage() method, defined as part of an event gateway instance. However, if you want to return results, you have to do a little bit of work, sleeping and checking for completion (per Damon Cooper's threads example). At SD West, I saw the Java 5 concurrency library for the first time and also saw a generic C++ library that worked in much the same way. The code would be very clean for CFML: pass.cfc simply extends Concurrency.Callable and defines a single method, call(), that returns a result, e.g., All of the asynchronous machinery is hidden inside the Concurrency.Future CFC, along with a generic Concurrency.FutureEvent CFC which is the only gateway instance you need to define. You create a 'future' for each CFC you want to 'call'. You 'run' the 'future' and then 'get' the result. You can also test whether the result is available (calling f.isDone() in the above example) and you can wait for a specified time for the result (calling f.getWithTimeout(numMillis), which throws an exception if the result does not become available in the specified time). You can read the Java 5 API documentation for more information about Future, FutureTask and Callable (I rolled FutureTask's functionality into Future since ColdFusion has no interfaces). Would folks be interested in this concurrency library?

Subtitled "Programming without a call stack", this was a bit of an odd talk. I expected more generality, I think, or... well, I don't know. But it was really about a specific set of Java classes Gregor had developed to solve some specific problems and, whilst it was interesting, it didn't really have much breadth. He started out by explaining what a call stack provides: Continuation (return), Context (local vars) and Coordination (synchronized execution). This means one at a time sequential execution, in a determined order, on a single machine. If you want concurrency (and scalability), you need to move away from the simple call stack model. He looked at threads but said they are hard to work with and fussy - lots of low-level housekeeping code. The Java 5 standard library adds a Future generic / template for simplified concurrency programming model. It has an isDone() and a get() method - isDone() tests whether the thread is finished, get() blocks until the result is available so it is a join operation. He showed more of the Java 5 library framework and it uses the Command and Executor patterns, just like Kevlin's generic C++ concurrency library. In fact, much of this section of Gregor's talk mirrored Kevlin's, despite the C++ / Java difference. He noted that even with this easier programming model, it is still related to the call stack architecture: it has the same command and control scheme with a primary execution thread and explicit synchronizatio etc. At this point he introduced his Event-Driven Architecture which, to be honest, looked a lot like JMS with publishers and subscribers and queues - except that it all operates locally under control of Java's threading model. Because of his specific approach, that are several downsides: type safety is reduced (he relies on reflection to some extent, looping over the inheritance tree and using try/catch to find event matches) so testing is more important; dependencies are critical and can be hard to visualize. One truism of systems like this is that loosely coupled systems are harder to diagnose and debug, no matter what techniques you use to implement them. Overall it was an interesting little technique but it seemed rather fragile. However, between this and Kevlin's talks, I'm getting some good ideas for how to build an easy to use concurrency framework around CFMX 7's event gateway mechanism!