Evolved Binary (EB-843) reference counted objects - Java interface proposal

[alanpaxton]

Overview

Experimental re-imagining of how the Java API maintains consistency with internal RocksDB objects.

The current RocksObject / AbstractImmutableNativeReference / AbstractNativeReference implementation expects a single Java object to be the single owning handle (isOwningHandle()) for an underlying C++ object, for instance a database handle or a column family handle.

Multiple Java objects sharing the same underlying object can exist, and then the owning protocol must be observed in order that neither double-deallocation nor a resource leak occurs.

We want to do away with the problem by introducing a form of reference counting so that the last owner of an object always close()s it for real, and no other owner does.

Our idea is that we can do this by changing the handle from being a C++ T* to being a reference to a C++ shared_ptr *, where each Java object has its own companion shared_ptr. On closing a Java object, the shared pointer is reset() and deallocated.

There is a pull request with the changes WIP - #10736

Implementation Detail (Java)

We have introduced a new class, RocksNative which wraps the native reference to a RocksDB C++ object
in a modified way to the old RocksObject.

• Naming of methods is systematically different; we can adopt an incremental transition of some classes but not all, for instance we have not yet converted options classes to derive from RocksNative.
• All accesses are performed behind an AtomicBoolean isOpen. There is no concept of having disowned the native reference at the Java side; ownership and lifetime is dealt with entirely by smart pointers on the C++ side.
• The class hierarchy is simplified, we don't need to distinguish AbstractImmutableNativeReference because all references are necessarily to an immutable shadow object.

Implementation Detail (C++)

In rocksdb repo branch we have added a number of API files to the java/rocksjni directory, named api_<item>.h thus

• api_base.h - common support functions
• api_rocksdb.h - the shadow database handle
• api_columnfamilyhandle.h - the shadow column family handle
• api_iterator.h - the shadow iterator handle
• api_wrapper.h - a standard wrapper for less commonly used objects which have shadows

These shadow objects all implement variations of the same pattern:

api_rocksdb.h - the shadow database handle

The shadow database handle contains shared pointers to the RocksDB itself (TDatabase is usually ROCKSDB_NAMESPACE::DB)
and to the column family handles open within the database.

std::shared_ptr<TDatabase> db;
std::vector<std::shared_ptr<ROCKSDB_NAMESPACE::ColumnFamilyHandle>>
      columnFamilyHandles;
  ...

The shadow database handle is created on a database open() JNI call, and its reference is cast to long and returned as the handle
in the parallel Java object (class RocksDB in RocksDB.java). Previously, a pointer to the ROCKSDB_NAMESPACE::DB was returned.

When the RocksDB class, is accessed in native code, the handle is cast to the shadow object, so we define the
shorthand in C++ using API_DB = APIRocksDB<ROCKSDB_NAMESPACE::DB>; and then

auto* dbAPI = reinterpret_cast<API_DB*>(jdb_handle);

this replaces the previous common idiom

auto* db = reinterpret_cast<ROCKSDB_NAMESPACE::DB*>(jdb_handle);

The shadow database handle also defines the standard pointer methods on itself,
so that it can be used as if it were a pointer

TDatabase* operator->() const { return db.get(); }
std::shared_ptr<TDatabase>& operator*() { return db; }
TDatabase* get() const { return db.get(); }

and this leads to the idiom (*dbAPI)->Method(...) being used as a replacement for db->Method(...)
throughout the changed codebase.

The shadow object is deleted in the Java_org_rocksdb_RocksDB_nativeClose method.
When this happens, reference counts on the std::shared_ptr members are reduced,
and by the magic of shared pointers the ROCKSDB_NAMESPACE::DB and ROCKSDB_NAMESPACE::ColumnFamilyHandle
objects are deleted (with the C++ delete() method) on their final reference.

Crucially, however, these references will persist when the shared pointers have been copied into other
shadow objects. This gives us the guarantees that we will be able to access certain structures (e.g. iterators)
where their lifetime persists beyond that of the database, while still cleaning them up when all references
to them are closed.

This means that we can use idioms like closing objects out of nested order, or passing objects out of a loop, while still being secure that our calls will work, or at the very worst fail cleanly with an obvious java exception.
What they won't ever do is crash in C++ when invoked from Java (practically speaking, not never but much less frequently and in much more corner cases).

api_iterator.h - the shadow iterator handle

Iterators are used with shadow objects in a similar way to databases. Because iterators are created with a
database in hand, we can copy the database (and if there is one, column family handle) from the RocksDB shadow
object into the iterator shadow object. The shadow object for an iterator then consists of the following:

std::shared_ptr<TDatabase> db;
std::shared_ptr<ROCKSDB_NAMESPACE::ColumnFamilyHandle> cfh;
std::unique_ptr<TIterator> iterator;

the std::unique_ptr<TIterator> iterator is all that is necessary because we never have multiple reference to a single iterator. The std::shared_ptrs maintain the lifetime of the database and column family handle for the
lifetime of the shadow object, so that the iterator is always valid until it is closed. The only requirement on the
user is to close() the iterator when they are done with it, most often just by using Java's try-with-resources
mechanism, but reserving the option to close() it manually if its lifetime is more sophisticated.

api_columnfamilyhandle.h - the shadow column family handle

Column family handles also have shadow objects. We choose to make their lifetime different from that of iterators.
When the corresponding database (and all other objects which implicitly use it) are closed, the column family handle becomes invalid. We take this view because usually a column family is accessed with a database in hand anyway,
so it is wrong to do so if that database is closed.

When the column family handle is used closed, we return a clean Java exception.

This behaviour is achieved using the C++ std::weak_ptr mechanism. Each weak pointer is associated with a
std::shared_ptr and can be lock()-ed to yield a std::shared_ptr as long as there are other outstanding copies of the shared pointer (its reference count has not fallen to 0). A successful lock() yields a std::shared_ptr<ROCKSDB_NAMESPACE::ColumnFamilyHandle> which is valid at least until it goes out of scope, and then in C++ we dereference that to make the same calls to the database as before. An unsuccessful lock() of the database weak pointer tells us that all database and iterator handles are already closed.

std::weak_ptr<TDatabase> db;
std::weak_ptr<ROCKSDB_NAMESPACE::ColumnFamilyHandle> cfh;

So when accessing column families in client code, we must access the column family while we have an open database.
If the database has been closed, we will fail with a RocksDBException with the message "column family already closed".

Performance

The new code does what might be considered as some extra work in 2 ways

1. checking the AtomicBoolean isOpen before every access of the native handle
2. Dereference of RocksDB objects via a std::shared_ptr or a std::weak_ptr rather than a raw pointer.

For smart pointers, we devised a worst case performance test based on existing JMH tests for JNI performance of alternative implementations of get() methods.
The RocksDB JNI Performance branch has been
adapted to provide an extra test which simulates the access pattern using a std::shared_ptr to access a
simulated database object, and a std::weak_ptr to access a simulated column family object.

The performance of 4 separate implementations of an implementation of get() which uses the JNI SetRegion methods
to return data in a java byte[] are compared. A distinct 5th implementation using SetRegion to get into an indirect
ByteBuffer is compared for reference.

The 4 implementations are:

1. Use a compiled static reference to the model database in get() calls
2. Pass a reference to the model database to get() calls
3. Pass a shadow object containing a std::shared_ptr holding a reference to the model database.
4. Pass a shadow object containing std::shared_ptr and std::weak_ptr references; fetch both,
   check that they are equal (of course this is true), and then use the reference.

• As the buffer size grows, get() performance becomes indistinguishable between all the SetRegion into byte[] implementations.

• SetRegion into an indirect ByteBuffer is significantly slower.

• SetRegion into an indirect ByteBuffer is significantly slower.

• For small buffer sizes there is a statistically significant but still extremely small difference between (1) and
  (2,3,4). This may be attributable to optimizations possible when the address of the mock database is a known, static value or it may be due to passing 1 less parameter to the get() method.

Otherwise, dereferencing a shadow object via a smart pointer, even where locking is involved, seems to have negligible performance impact. Bear in mind that the performance being tested is entirely stripped down to the
nuts and bolts of buffer transfer, so it would exaggerate any performance difference compared to what we would
see in RocksDB, which has to go through a longer code path before it can access buffers.

Status

A large number of RocksObject classes have been converted to RocksNative, and the corresponding C++ code updated. All Java unit tests are passing in this branch. As well as the fundamental classes (RockDB, ColumnFamilyHandle, Iterator) this has necessitated the conversion of a large number of derivative or related classes.

The classes that remain to be converted include the various Options classes.

TODO

• Identify objects that we have not yet converted
• Code Review
• Broader testing - major public Java API clients