An Architect's View

CFML, Clojure, Software Design, Frameworks and more...

An Architect's View

Transfer ORM and memory usage

March 14, 2009 ·

Some people - myself included - have observed runaway memory usage and apparent memory leaks with applications built with certain combinations of CFML frameworks that include Transfer ORM. We spent a lot of time tuning the JVM and looking at code and database usage in our Broadchoice Community Platform (CMS), we worked with Mike Brunt on load testing and tuning (highly recommended - if you have any performance problems, get Mike on your case!) as well as working with Mark Mandel directly on Transfer itself. All that work led to a much more stable system and we decided to just continue investigating as a background task.One of the key problems for us was that we could never reproduce the memory issues outside of our production system. We set up all sorts of test environments and ran load tests, to no avail. We tried a number of tests that eliminated pieces of the architecture to try to identify exactly what was the root cause. We use JMS to synchronize the cache on multiple instances so we turned that off, piece by piece, and eventually eliminated that as a source of the leak. That was good news - it meant we could continue using JMS to keep multiple instances synchronized! Recently we've been running a number of Java profiling and memory analysis tools to look at JVM heap usage (we = one of my team, not me, so if you ask for details I'll have to get my colleague to provide them!) and established once and for all that we had some large cyclic graphics of objects that did not seem to be GC'd and that new cyclic graphs were being added fairly quickly. And those chains of objects were deep inside Transfer ORM. We'd finally "proved" that the problem was somehow in our usage of Transfer ORM. I went back to Mark and asked about specific bug fixes in the caching machinery. It turned out that between the 1.1 RC1 release (which we had been using since it appeared) and the final 1.1 "Stable" release, Mark had made a couple of small tweaks to the two parts of the caching engine. We upgraded from 1.1 RC1 to 1.1 Stable and we've been watching memory usage in production ever since. So far - and I don't really want to jinx it by commenting - the results look very, very promising. Memory looks pretty stable after several days of runtime. The irony is that I've always been an advocate of using the "BER" (Bleeding Edge Release) of frameworks and in pretty much the one case that I stayed with a particular release, I fell victim to bugs that were subsequently fixed! So, if you're using Transfer and you're still on the 1.1 Release Candidate, you really should upgrade to the 1.1 Stable release!

Tags: broadchoice · coldfusion · orm

4 responses

  • 1 Elliott Sprehn // Jun 11, 2009 at 2:37 AM

    I've been noticing a lot of memory usage from our Transfer based apps compared to the ones that used fairly basic home rolled ORM before. It seems like it uses almost double, and it also increases the load times noticeably as well. Especially first load which might have taken 4 seconds before and now takes 48 seconds to load the 70 objects (+ relationships) for something like Speakers.

    The architecture benefits of Transfer are great, but there is absolutely a performance hit from it, and a memory one too.

    Transfer also does some pretty silly things performance wise like repeatedly querying for lazy objects (making them not so lazy). :P

    I've considered rolling my own ORM that's a lot simpler and smarter about pulling in objects in groups and not doing so much work per object, but there's also CF9's Hibernate support on the Horizon that looks promising.
  • 2 Sean Corfield // Jun 11, 2009 at 10:55 AM

    @Elliott, sounds like you haven't tuned Transfer much - I've had great performance from it with judicious use of the Transfer cache and a combination of lazy loading and proxy objects. I've never seen load times close to a minute for such a small number of objects. Or maybe the problem is the server you're running on?

    I've used Transfer in high-performance production apps for years and the only problem I've seen - now fixed as of 1.1 - is the memory leak.
  • 3 Elliott Sprehn // Jun 11, 2009 at 12:00 PM

    Proxy objects don't seem to work properly for us. Any updates that happen on the proxies seem to get lost when we save(). I've talked to Mark about it, and there's some kind of bug inside Transfer that causes it, but that's still not fixed.

    I've also tried lazy loading, and that mostly works, but sometimes is even worse because Transfer will requery for lazy relationships repeatedly even if the relationship is already loaded. This is especially true when you clone objects and Transfer will requery for every relationship that's Lazy.

    There's also no way to control what gets cloned so cloning say, an Attendee to update some information, clones the parent Transaction too, which then clones every other attendee in the transaction (which is slow). Updating a speaker record is particularly expensive as a result since you all were registered together and the transaction has 70 attendees on it. If I make that lazy performance is worse since it queries more often than it used to, and if I make it proxies then updates to Attendees fail to save. (doh!)

    Part of it is the server which isn't very fast (1GB of ram and 2Ghz Celeron) but we have another server with a 3Ghz Xeon and 2GB of ram. Loading a page of say, 50 attendees takes 10 seconds on this machine. A single smart query against the database brings the request time down to 400ms.

    I'd be really curious to see what other people's setups are, because I've seen nothing but performance hits from Transfer compared to building a simple architecture yourself. I don't doubt that you can do it, since apparently people do, but I've not seen it.
  • 4 Sean Corfield // Jun 11, 2009 at 12:27 PM

    @Elliott, good to know about the proxy bug. I'm surprised that's not more widely discussed.

    Cloning shouldn't be used for every update - just ones where there might be a consistency problem. I hardly ever clone objects unless I'm changing relationships - but I know what you mean about the problems with it also cloning related objects.

    Once you have the data cached - and the data behind CFUnited is small so it should easily all fit in memory - Transfer is extremely fast (within the limitations of Adobe's CFC implementation).