To ORM or not to ORM

[tdktank59]

Well im planning out a new project for work and I have the option to use ORM or not to use ORM....

Ive seen some pros and cons here and there and they all point at each other and laugh per say.

Anyways I was hoping to get some input from some people who have used both for large systems...

(Mind you ive played with a bit of ORM on past projects but this one will be a large system that should in the end be accessed by thousands of people worldwide or at least thats the goal...)

 

[devman]

Abstraction of the database api makes your business code cleaner and generally more scalable. YMMV, of course depending on the ORM vendor. It can also make your code portable between database vendors with no rewriting needed if you purely use the ORM vendor's API. Imo I think it makes a larger system more manageable.

 

[Ankle]

Portability vs Performance. Which is more important to you? I'd lean to performance particularly if its something with a heavy traffic load.

 

[ameoba]

What sort of 'project' are we talking about?

I'd lean to performance particularly if its something with a heavy traffic load.

Assuming we're talking about the web, 'thousands' of users is not a heavy traffic load.

 

[Xeth]

I've worked on large projects written in Java and ASP.NET using Hibernate/NHibernate/Habanero. The most important and painful lesson I've learned is that using an ORM is only worth it if all of your developers understand its internals well and is prepared to use it "the right way". I've seen many occasions where:

1. Coder writes some stuff against ORM not knowing what it actually does under covers.
2. Coder sees horrible performance.
3. Coder blames horrible performance on ORM, proceeds to hack around ORM, defeating its purpose in the first place.

If this "large" project only involves a couple of developers then I would definitely use ORM. Otherwise it may be easier to just roll your own dataaccess layer (or use a very lightweight one), or a code generator that doesn't require additional runtime libraries.

 

[deuce868]

In the end I say wrap the ORM. That way you can keep a stable API but replace the parts of the ORM that you outgrow. Once a certain method gets too slow due to the way the ORM builds the queries/etc you can replace it with a custom SQL function and move on.

90% of the code is nice ORM code that is simpler and the other 10% looks like ORM code, but it's custom done for speed.

 

[devman]

I've worked on large projects written in Java and ASP.NET using Hibernate/NHibernate/Habanero. The most important and painful lesson I've learned is that using an ORM is only worth it if all of your developers understand its internals well and is prepared to use it "the right way". I've seen many occasions where:

1. Coder writes some stuff against ORM not knowing what it actually does under covers.
2. Coder sees horrible performance.
3. Coder blames horrible performance on ORM, proceeds to hack around ORM, defeating its purpose in the first place.

If this "large" project only involves a couple of developers then I would definitely use ORM. Otherwise it may be easier to just roll your own dataaccess layer (or use a very lightweight one), or a code generator that doesn't require additional runtime libraries.

This. This +1000. ORM isn't databases for newbies, its a useful tool for scalability and portability. You still need to know what your doing, and knowing how the internals work under the hood helps your design process.

 

[:LJ:]

If you're thinking of casting ORM aside on the basis of performance, stop and think. Good ORM frameworks make a lot of use of caching, and so can speed things up considerably in some cases; however, worst case is that you'll add maybe 5ms to an operation. In a web context, contrast this with the cumulative effect of network latency for all the objects on a page as well as the page itself, and most of the time it'll turn out that the worst-case penalty is negligible.

 

[Xeth]

If you're thinking of casting ORM aside on the basis of performance, stop and think. Good ORM frameworks make a lot of use of caching, and so can speed things up considerably in some cases; however, worst case is that you'll add maybe 5ms to an operation. In a web context, contrast this with the cumulative effect of network latency for all the objects on a page as well as the page itself, and most of the time it'll turn out that the worst-case penalty is negligible.

Agreed, ORMs don't perform badly by default, but if you want good performance out of an ORM with a non-trivial schema or large data sets, you had better know what you're doing.

 

[:LJ:]

Agreed, ORMs don't perform badly by default, but if you want good performance out of an ORM with a non-trivial schema or large data sets, you had better know what you're doing.

Yes - I'm making the assumption that the people who are using the ORM framework are "good" developers who don't jump into an implementation without having researched and tested it properly first. Without wishing to sound inflammatory, complaints about the quality of ORM implementations generally come from people with an insufficient understanding of the implications and details of the framework they're working with.

 

[tdktank59]

Well thanks for the input guys!

Cant say it changed my mind since i've found the uses of ORM to be very helpful.

But wanted to see what all the issues where and why some people hate it... (kinda like vista.. Even thou vista does kinda suck. So I guess thats a bad analogy lol).

Anyways I have not been using ORM for a long time so im still a newbie when it comes to it. But I am looking into the "proper" way to use it to get the most out of it.

 

[Ankle]

I would argue that performance could be fixed easily with caching and proper coding.

Maybe if the ORM is written in C/C++ since you could optimize the code to avoid any extra memory allocation by using the db c API directly. The overhead with an interpreted language could potentially involve a lot of individual memory allocations.

 

[:LJ:]

I was thinking PHP above...since that is what I program in. I don't exactly understand your comment. You are saying the only way to optimize an ORM is to write some DB extension to handle it? Yes C/C++ would be faster by all means, but it would not be worth the time (most likely) to write, test, and implement.

Precisely. People seem to think that the only way to optimise data access is to write it all in C/C++/some other compiled language. That really isn't the case - the heavy lifting in ORM-style data persistence is in two places: the database itself, and the reflection usually required to shoehorn the relational data back into the business model. Assuming that your database manager is as optimised as possible and you're not going to rewrite it, and that the job of actually creating a business object from said data has to be done in the language you're using for the rest of the system...writing your ORM layer in another language ain't gonna give you much of a benefit, and certainly not enough to make a difference that the user would notice.

You're much better off making sure that your application scales well (in a web context, at least) rather than trying to shave a few milliseconds off the data retrieval process. The big open source ORM frameworks are all very mature now, and generally run within an order of magnitude of straight database access in terms of performance; the law of diminishing returns has already kicked in there. Now, improving how your application architecture handles concurrency...that is where the big gains will be made.

 

[devman]

Precisely. People seem to think that the only way to optimise data access is to write it all in C/C++/some other compiled language. That really isn't the case - the heavy lifting in ORM-style data persistence is in two places: the database itself, and the reflection usually required to shoehorn the relational data back into the business model. Assuming that your database manager is as optimised as possible and you're not going to rewrite it, and that the job of actually creating a business object from said data has to be done in the language you're using for the rest of the system...writing your ORM layer in another language ain't gonna give you much of a benefit, and certainly not enough to make a difference that the user would notice.

You're much better off making sure that your application scales well (in a web context, at least) rather than trying to shave a few milliseconds off the data retrieval process. The big open source ORM frameworks are all very mature now, and generally run within an order of magnitude of straight database access in terms of performance; the law of diminishing returns has already kicked in there. Now, improving how your application architecture handles concurrency...that is where the big gains will be made.

Well said. Though I would say that the ORM is probably even in the same order of magnitude as the straight database access as the differences are likely only to be scalar.

Most ORM systems still allow you to pass through a hand built native query if you really wanted to for optimization purposes, but more often your time is better spent improving the business code.

 

[:LJ:]

Maybe if the ORM is written in C/C++ since you could optimize the code to avoid any extra memory allocation by using the db c API directly. The overhead with an interpreted language could potentially involve a lot of individual memory allocations.

I just thought I'd address this, since I forgot to earlier. How are you going to invoke that C/C++ uber-optimised code from something else? You could use something similar to JNI in whatever platform you're developing for, but those are usually buggy and result in a lot more code and chatter, and impose a penalty which usually negates the gains from writing it in C/C++/assembly in the first place (unless you're doing a lot of heavy computation, which doesn't happen in an ORM framework). Alternatively, you could invoke it as a shell command and read the output CGI-style, but that brings a massive penalty in terms of the loading the shell environment first.

There's no way around it - unless your app is written in C/C++/assembly, there is precisely zero (or so close as to be negligible) benefit in 99% of situations to writing your ORM framework in such a language. It simply isn't appropriate to the problem.

 

[Ankle]

I was thinking PHP above...since that is what I program in. I don't exactly understand your comment. You are saying the only way to optimize an ORM is to write some DB extension to handle it? Yes C/C++ would be faster by all means, but it would not be worth the time (most likely) to write, test, and implement.

I wouldn't say its the only way but writing the ORM in C/C++ would alleviate any overhead from creating objects and copying data with an interpreted language. I can definitely see it being too much extra work for most projects for the gains but if you are writing a application framework focusing on performance and scalability it could be.

Something as complicated as ORM could potentially be quite difficult to bridge since the ORM necessarily relies very heavily on the OO aspect of languages and would be very deeply ingrained in the language it is using. For example you can use C++ objects from Perl but they don't work the same other than the basics and the potential for confusion is high. In the end its hard to say if a highly optimized ORM in your interpreted language of choice or trying to bridge a C/C++ ORM would be better without trying it first and seeing some results for comparison.

I just thought I'd address this, since I forgot to earlier. How are you going to invoke that C/C++ uber-optimised code from something else? You could use something similar to JNI in whatever platform you're developing for, but those are usually buggy and result in a lot more code and chatter, and impose a penalty which usually negates the gains from writing it in C/C++/assembly in the first place (unless you're doing a lot of heavy computation, which doesn't happen in an ORM framework). Alternatively, you could invoke it as a shell command and read the output CGI-style, but that brings a massive penalty in terms of the loading the shell environment first.

Perl XS. I suppose one could use SWIG but I can't see why anyone would choose it over XS if using Perl. Inline C/C++ is a nice approach which would compile and cache the code at first startup while creating bindings for it. Shell CGI would definitely be too slow - Fastcgi waiting for requests through a socket on the local machine would be far better.

 

[tdktank59]

Seems I created a monster here lol. But the general census seems to be go with ORM

Other wise thanks for the great information!