Ajug hibernate-dos-donts

When Hibernate Attacks!
Hibernate Best Practice and Pitfalls

© 2011 Altisource Portfolio Solutions. All rights reserved. Altisource™, Altisource Portfolio Solutions™, the Altisource Logo, the "REAL" family of trademarks and service marks, and all other marks identified herein are trademarks or service marks of Altisource
© 2011 S.A. or its subsidiaries and may be S.A. All with the United States Patent and Trademark Office and in other countries. Proprietary and Confidential.
Portfolio Solutions Altisource Portfolio Solutionsregistered rights reserved. Proprietary and Confidential.

Page | 1

Agenda
– Me
– Hibernate in two slides
– Hibernate
 The Good, The Bad
 Sanity Check
 My Hibernate Divorce

© 2011 Altisource Portfolio Solutions S.A. All rights reserved. Proprietary and Confidential.

Page | 2

Me
–
–
–
–

Roy Russo
Former JBoss Portal Co-Founder
LoopFuse Co-Founder
Senior Software Architect @ AltiSource
 We’re Hiring… and we don’t suck.


Page | 3

Hibernate in Two Slides
– Addresses the ‘Object-Relational Impedance
Mismatch’
– Persistence classes using idiomatic Java
– Transparent
 No interfaces or build dependencies
– Querying Facilities:
 HQL, Criteria API, JPAQL, SQL
– Database-Agnostic
– Performance:
 Caching , Lazy initialization, fetching
strategies, versioning, etc…
– Automatic DDL generation


public class User implements Serializable
{
private long userid = -1;
private String userName;
private String firstName;
private String lastName;
private String password;
private Map props;
private Set roles;
…

Page | 4

Hibernate in 2 Slides
<hibernate-mapping>
<class
name="com.loopfuse.domain.auth.User"
table="LF_USERS">
<id
name="userid"
column="USER_ID"
type="java.lang.Long">
<generator class="native"/>
</id>
<property
name="userName"
column="USERNAME"
type="java.lang.String"
unique="true"/>
…
<set
name="roles"
table="LF_ROLE_MEMBERS"
lazy="false"
inverse="false"
cascade="none"
sort="unsorted">
<key column="USER_ID"/>
<many-to-many
class="com.loopfuse.domain.auth.Role"
column="ROLE_ID"
outer-join="true"/>
</set>
</class>

public class User implements Serializable
{
private long userid;
private String userName;
private String firstName;
private String lastName;
private String password;
private String email;
private Map props;
private Set roles;
…

public Task getUserByID(long ID, Session session) throws DataSourceException
{
Transaction tx = session.beginTransaction();
User user = null;
try
{
Query query = session.createQuery("from User where userID=? ");
query.setParameter(0, ID);
user = (User) query.uniqueResult();
tx.commit();
}
catch (RuntimeException e)
{…}
return user;
}


Page | 5

First… The Basics
– It’s JDBC
– It’s a Database
– (Please, work with a DBA)
“Some developers come to using a tool like
Hibernate because they are uncomfortable
with SQL and with relational databases. We
would say that this is exactly the wrong
reason to use Hibernate. You should be very
comfortable with SQL and JDBC before you
start using Hibernate - Hibernate builds on
JDBC, it does not replace it. “
 Gavin King, Hibernate Founder


Page | 6

Why we love Hibernate
– Lazy developers love Hibernate:
 Near-flat learning curve
 Little knowledge of JDBC or DB schema
design
 Little knowledge of Hibernate mapping
– Lazy loading, cache, session mgmt

 Little knowledge on how to tune Hibernate or
its JDBC parameters
 Ignore Hibernate SQL Query log


Page | 7

The Good and the Bad
– Hibernate can rock:
 Rapid development
 Cache
 Pooling
 Support / Docs
– Hibernate can suck:
 Performance problems
 Session Management
 Development Time


Page | 8

Problem: Schema-Ownership
Development 101:
1. Developers begin project with a clean slate.
2. Developers define schema in the DB AND *.hbm.xml
3. Developers ship product
4. DBAs held accountable for performance
5. DBAs call for refactoring / denormalization
6. Now what?

– Who owns the DB schema?
– Marketing called and needs 500 reports written.
Who delivers?
– Sales needs integration with Crystal Reports
– Dual metadata: Changes to either schema requires
changes in the other.
 (IDEs won’t refactor/migrate/adapt DB schema)

Page | 9

Solution: Schema-Ownership, cont.
– Work with a DBA
 Be prepared: Likely minimize reliance on Hibernate
 DBA provides Query plan.
– Consider Hibernate for simple CRUD
– Set up a mirror DB specifically for reports
 Possible silo


Page | 10

Problem: n+1 SELECT
So you set lazy=“true”…
Retrieve all Items for User:
Iterator items = session.createCriteria(Item.class)
.add( Expression.eq("item.seller", user) )
.list()
.iterator();

The n+1 problem is difficult to
detect, as it is usually hidden
inside application logic.

Find maximum Bid for Items:
List maxAmounts = new ArrayList();
while (items.hasNext()) {
Item item = (Item) items.next();
BigDecimal maxAmount = new BigDecimal("0");
for ( Iterator b = item.getBids().iterator(); b.hasNext(); ) {
Bid bid = (Bid) b.next();
if ( bid.getAmount().compareTo(maxAmount) == 1 )
maxAmount = bid.getAmount();
}
maxAmounts.add( new MaxAmount( item.getId(), maxAmount ) );
}

Hibernate issues 1 SELECT per Bid…

Page | 11

Solution: n+1 SELECT
Enable Batch Fetching:
<set name="bids" lazy="true" inverse="true" batch‐size="10">

– Hibernate pre-fetches the next 10 collections
– Problem is now reduced to n/10+1 SELECTs
– A little better, but now other transactions will fetch collections
unnecessarily.
HQL Aggregation:
String query = "select MaxAmount( item.id, max(bid.amount) )" +
" from Item item join fetch item.bids bid" + " where item.seller = :user group by item.id";
List maxAmounts = session.createQuery(query).setEntity("user", user).list();

– Possible solution, unless we want to do complex processing of Bids.
Enable Eager Fetching:
<set name="bids" inverse="true" outer‐join="true">

– Note: HQL ignores outer-join, but you can use the Criteria API
– You should be fired.

Page | 12

Solution: n+1 SELECT
Runtime Declaration of Fetch Strategy:
List results = session.createCriteria(Item.class)
.add( Expression.eq("item.seller", user) )
.setFetchMode("bids", FetchMode.EAGER)
.list();

Iterator items = new HashSet(results).iterator();
List maxAmounts = new ArrayList();
for ( ; items.hasNext(); ) {
Item item = (Item) items.next();
BigDecimal maxAmount = new BigDecimal("0");
for ( Iterator b = item.getBids().iterator(); b.hasNext(); ) {
Bid bid = (Bid) b.next();
if ( bid.getAmount().compareTo(maxAmount) == 1 )
maxAmount = bid.getAmount();
}
maxAmounts.add( new MaxAmount( item.getId(), maxAmount ) );
}

– No guarantee of distinct Items returned.


Page | 13

Problem: LazyInitializationException
– Cause:
 Thrown when an unitialized collection (or proxy)
is accessed outside of the scope of the Session.
– Detached collections can’t be initialized

 Lazy initialization turned on by default v3.
– Recommended best practice to leave it turned on.


Page | 14

Solution: LazyInitializationException
– If Session is open, use Hibernate.initialize(item), forcing initialization, as
long as the Session is open.
– Keep the Session open:
 Open Session in View pattern: Use a servlet filter to close the Session
at the end of a request.
– You must alter application architecture
– You must address exception handling

 Call Hibernate.initialize(item) in the business layer before returning to
the web tier, or resort to eager fetching.
 May not be a feasible strategy in n-tier environments


Page | 15

Problem: “Hibernate is Slow”
– Hibernate addresses performance:
 Cache
 Amount of data loaded
– Finding the Cause:
 Monitor the DB
– In MySQL:
•
•
•
•

show processlist;
log_slow_queries = /var/log/mysql/mysql-slow.log &
long_query_time = 1
log-queries-not-using-indexes

 Monitor Hibernate logging/queries:
– hibernate.show_sql=true / log4j.logger.org.hibernate.type=debug
– The output will scare you, but you can use it to execute a test query…
• EXPLAIN [SQL]

 Use a profiler

Page | 16

Addressing: “Hibernate is Slow”
– Are tables properly indexed? Call the DBA.
– Slow INSERT?
 Consider “batch” INSERT
– Explicit flushing:
• Dirty checking of objects can take a lot of time

– Slow SELECT?
 SELECT on PK. Avoids DB call – uses cache
 Are you loading everything in memory and looping through it? Don’t.
– 2nd Level cache on read-only entities
– Use SQL ;-)
– If appropriate, consider batch processing…


Page | 17

Problem: OOM on Batch Processing

Session session = sessionFactory.openSession();
for ( int i=0; i<100000; i++ ) {
Customer customer = new Customer(.....);
session.save(customer);
}
tx.commit();
session.close();

– Above code will lead to OOM
 Hibernate caches all newly created Customer instances


Page | 18

Solution: OOM on Batch Processing
Session session = sessionFactory.openSession();

StatelessSession session = sessionFactory.openStatelessSession();

for ( int i=0; i<100000; i++ ) {
Customer customer = new Customer(.....);
session.save(customer);
if ( i % 20 == 0 ) { //20, same as the JDBC batch size
//flush a batch of inserts and release memory:
session.flush();
session.clear();
}
}
tx.commit();
session.close();

ScrollableResults customers = session.getNamedQuery("GetCustomers")
.scroll(ScrollMode.FORWARD_ONLY);
while ( customers.next() ) {
Customer customer = (Customer) customers.get(0);
customer.updateStuff(...);
session.update(customer);
}
tx.commit();
session.close();

– Flush and Clear the Session regularly
– Use the StatelessSession Interface
 Bypass cache
 No dirty-checking
 Ignores collections on entities
 Bypass Hibernate interceptors and event model

Page | 19

Sanity Check
–

–

–

Addresses the ‘Object-Relational Impedance
Mismatch’
 Good marketing. ORM is the impedance
mismatch. It creates the inefficient connection.
 DBs provide permanent storage. Programming
languages process data in step-wide fashion.
Morphing both is a mismatch. Promoting their
strengths is the ideal scenario.
Transparent
 You’re in control of the level of depth
Hibernate permeates your codebase.
 Expect some magic.
Querying Facilities (HQL, Criteria API, JPAQL, SQL)
 In the end, they output SQL. Ugly SQL.


Page | 20

Sanity Check
– Database-Agnostic
 Who cares?
– “Performance”: Lazy initialization, fetching
strategies, versioning, etc…
 Faster than what?
 If performance is a concern, raw JDBC/SQL
is your answer.
– Caching:
 Databases already offer this.
 Roll your own cache
 Beware of cached data modified by other
applications.
– Automatic DDL generation
 Never use this!

Page | 21

My Hibernate Divorce
– Scenario:
 Analytics company processing several million
hits, email clicks, email opens, and form posts
per day.
 Over 800 customer in multi-tenant-style
schema
 75+ tables per schema
 2 MySQL servers in Master-Slave
configuration
 Application servers from 8GB to 16GB physical
memory.
– Hibernate Use:
 100%
 Originally JSF+Hibernate = Rapid Development


Page | 22

– Problem #1:
 Random OOMs
– Diagnosis:
 Boot up Jprofiler. Nothing there.
 Analyze heap dump file:
– Hibernate holding 5MB per SessionFactory in memory, ~4GB per server

 Reporting and other intensive operations cause OOM
 By design, lazy-loading was disabled.


Page | 23

– Solution:
 Add more RAM. Could not scale horizontally.
 Load balance customers across servers (ala SFDC)
 Rethink lazy-loading strategy:
– Massive re-architecture
– Suboptimal in certain parts of the system.
• Recording hits/clicks/opens
• Reporting

 Rip out Hibernate for memory intensive operations.
–
–
–
–

Table per Class – this was going to take a while.
Perform processing in DB, not code! That’s what they’re designed to do!
Some stored proc.
Get a DBA.


Page | 24

– Problem #2:
 Slow boot time: ~20 minutes.
 With hibernate update 2+ hours. (and system crashes)
 Marketing people aren’t patient.
– Diagnosis:
 Hibernate SessionFactory creation across 800 schema can be slow. ;-)
 hbm2ddl.auto=update is evil
– Solution:
 hbm2ddl.auto=none
– Handle schema updates manually (yay, PHP!)
– Migration and adapting data a manual process.

 Load SessionFactories on-demand.
– When user logs in, load SessionFactory

 Reap stale SessionFactories

Page | 25

Final Thoughts
Anti-Pattern:
1. Some repeated pattern of action, process or structure
that initially appears to be beneficial, but ultimately
produces more bad consequences than beneficial
results, and
2. A refactored solution exists that is clearly
documented, proven in actual practice and repeatable.

If you really must use Hibernate:
– Never let Hibernate design your schema
– Your schema will outlive your application
– Learn SQL, Stored Proc, and database design principles.
– Proper indexing is paramount! Get a DBA.
– Consider a mix: 50% Hibernate (75% or 90%)

Page | 26

And Today…
“... ‘a lot of open source projects just outgrow
Hibernate. Hibernate was great to get
started and that although it has a richer set
of capabilities than iBatis, sometimes it just
gets in the way’.”
- John Newton, CTO, Alfresco


Page | 27

Questions?


Page | 28

Ajug hibernate-dos-donts

More Related Content

Viewers also liked

Similar to Ajug hibernate-dos-donts

More from Roy Russo

Recently uploaded

Ajug hibernate-dos-donts

Editor's Notes