Enum JNIMemoryManager.MemoryModel
- java.lang.Object
-
- java.lang.Enum<JNIMemoryManager.MemoryModel>
-
- com.avpkit.ferry.JNIMemoryManager.MemoryModel
-
- All Implemented Interfaces:
java.io.Serializable,java.lang.Comparable<JNIMemoryManager.MemoryModel>
- Enclosing class:
- JNIMemoryManager
public static enum JNIMemoryManager.MemoryModel extends java.lang.Enum<JNIMemoryManager.MemoryModel>
The different types of native memory allocation models Ferry supports.Memory Model Performance Implications
Choosing theJNIMemoryManager.MemoryModelyou use in Ferry libraries can have a big effect. Some models emphasize code that will work "as you expect" (Robustness), but sacrifice some execution speed to make that happen. Other models value speed first, and assume you know what you're doing and can manage your own memory.In our experience the set of people who need robust software is larger than the set of people who need the (small) speed price paid, and so we default to the most robust model.
Also in our experience, the set of people who really should just use the robust model, but instead think they need speed is much larger than the set of people who actually know what they're doing with java memory management, so please, we strongly recommend you start with a robust model and only change the
JNIMemoryManager.MemoryModelif your performance testing shows you need speed. Don't say we didn't warn you.Model Robustness Speed JAVA_STANDARD_HEAP(default)+++++ + JAVA_DIRECT_BUFFERS_WITH_STANDARD_HEAP_NOTIFICATION+++ ++ NATIVE_BUFFERS_WITH_STANDARD_HEAP_NOTIFICATION+++ +++ JAVA_DIRECT_BUFFERS(not recommended)+ ++++ NATIVE_BUFFERS+ +++++ What is "Robustness"?
Ferry objects have to allocate native memory to do their job -- it's the reason for Ferry's existence. And native memory management is very different than Java memory management (for example, native C++ code doesn't have a garbage collector). To make things easier for our Java friends, Ferry tries to make Ferry objects look like Java objects.
Which leads us to robustness. The more of these criteria we can hit with a
JNIMemoryManager.MemoryModelthe more robust it is.- Allocation: Calls to
make()must correctly allocate memory that can be accessed from native or Java code and calls todelete()must release that memory immediately. - Collection: Objects no longer referenced in Java should have their underlying native memory released in a timely fashion.
- Low Memory: New allocation in low memory conditions should first have the Java garbage collector release any old objects.
What is "Speed"?
Speed is how fast code executes under normal operating conditions. This is more subjective than it sounds, as how do you define normal operation conditions? But in general, we define it as "generally plenty of heap space available"
How Does JNIMemoryManager Work?
Every object that is exposed from native code inherits from
RefCounted.Ferry works by implementing a reference-counted memory management scheme in native code that is then manipulated from Java so you don't have to (usually) think about when to release native memory. Every time an object is created in native memory it has its reference count incremented by one; and everywhere inside the code we take care to release a reference when we're done.
This maps nicely to the Java model of memory management, but with the benefit that Java does all the releasing behind the scenes. When you pass an object from Native code to Java, Ferry makes sure it has a reference count incremented, and then when the Java Virtual Machine collects the instance, Ferry automatically decrements the reference it in native code.
In fact, in theory all you need to do is make a finalize() method on the Java object that decrements the reference count in the native code and everyone goes home happy.
So far so good, but it brings up a big problem:
- Turns out that video, audio and packets can be fairly large objects. For example, a 640x480 YUV420P decoded video frame object will take up around 500K of memory. If those are allocated from native code, Java has no idea it got allocated; in fact the corresponding Java object will seem to only take up a few bytes of memory. Keep enough video frames around, and your Java process (that you expect to run in 64 Megs of heap memory) starts to consume large amounts of native memory. Not good.
- The Java virtual machine only collects garbage when it thinks it needs the space. However, because native code allocated the large chunks of memory, Java doesn't know that memory is being used. So it doesn't collect unused references, which if Ferry just used "finalize()" would mean that lots of unused memory might exist that clog up your system.
- Lastly, even if Java does do a garbage collection, it must make sure that all methods that have a finalize() method are first collected and put in a "safe-area" that awaits a second collection. On the second collection call, it starts calling finalize() on all those objects, but (don't ask why just trust us) if needs to dedicate a separate finalizer thread to this process. The result of this is if you allocate a lot of objects quickly, the finalizer thread can start to fall very far behind.
RefCountedimplementation solves all these problems for you.How you ask:
- We use Java Weak References to determine if a native object is no longer used in Java. Ferry objects allocated from native code do not finalizers.
- Then every-time you create a new Ferry object, we first make sure we do a mini-collection of all unused Ferry objects and release that native memory.
- Then, each Ferry object also maintains a reference to another object that DOES have a finalize() method and the only thing that method does is make sure another mini-collection is done. That way we can make sure memory is freed even if you never do another Ferry allocation.
- Lastly, we make sure that whenever we need large chunks of memory (for IPacket, IFrame and IAudioSamples interfaces) we can allocate those objects from Java, so Java ALWAYS knows just how much memory it's using.
In the event you need to manage memory more expicitly, every Ferry object has a "copyReference()" method that will create a new Java object that points to the same underlying native object.
And In the unlikely event you want to control EXACTLY when a native object is released, each Ferry object has a
RefCounted.delete()method that you can use. Once you call "delete()", you must ENSURE your object is never referenced again from that Java object -- Ferry tries to help you avoid crashes if you accidentally use an object after deletion but on this but we cannot offer 100% protection (specifically if another thread is accessing that object EXACTLY when youRefCounted.delete()it). If you don't callRefCounted.delete(), we will call it at some point in the future, but you can't depend on when (and depending on theJNIMemoryManager.MemoryModelyou are using, we may not be able to do it promptly).What does all of this mean?
Well, it means if you're first writing code, don't worry about this. If you're instead trying to optimize for performance, first measure where your problems are, and if fingers are pointing at allocation in Ferry then start trying different models.
But before you switch models, be sure to read the caveats and restrictions on each of the non
JAVA_STANDARD_HEAPmodels, and make sure you have a good understanding of how Java Garbage Collection works.- Author:
- aclarke
-
-
Enum Constant Summary
Enum Constants Enum Constant Description JAVA_DIRECT_BUFFERSLarge memory blocks are allocated as DirectByteBufferobjects (as returned fromByteBuffer.allocateDirect(int)).JAVA_DIRECT_BUFFERS_WITH_STANDARD_HEAP_NOTIFICATIONLarge memory blocks are allocated as DirectByteBufferobjects (as returned fromByteBuffer.allocateDirect(int)), but the Java standard-heap is informed of the allocation by also attempting to quickly allocate (and release) a buffer of the same size on the standard heap..JAVA_STANDARD_HEAPLarge memory blocks are allocated in Java byte[] arrays, and passed back into native code.NATIVE_BUFFERSLarge memory blocks are allocated in native memory, completely bypassing the Java heap.NATIVE_BUFFERS_WITH_STANDARD_HEAP_NOTIFICATIONLarge memory blocks are allocated in native memory, completely bypassing the Java heap, but Java is informed of the allocation by briefly creating (and immediately releasing) a Java standard heap byte[] array of the same size.
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Modifier and Type Method Description intgetNativeValue()Get the native value to pass to native codestatic JNIMemoryManager.MemoryModelvalueOf(java.lang.String name)Returns the enum constant of this type with the specified name.static JNIMemoryManager.MemoryModel[]values()Returns an array containing the constants of this enum type, in the order they are declared.
-
-
-
Enum Constant Detail
-
JAVA_STANDARD_HEAP
public static final JNIMemoryManager.MemoryModel JAVA_STANDARD_HEAP
Large memory blocks are allocated in Java byte[] arrays, and passed back into native code. Releasing of underlying native resources happens behind the scenes with no management required on the programmer's part.
Speed
This is the slowest model available.
The main decrease in speed occurs for medium-life-span objects. Short life-span objects (objects that die during the life-span of an incremental collection) are relatively efficient. Once an object makes it into the Tenured generation in Java, then unnecessary copying stops until the next full collection.
However while in the Eden generation but surviving between incremental collections, large native buffers may get copied many times unnecessarily. This copying can have a significant performance impact.
Robustness
- Allocation: Works as expected.
- Collection: Released either when
delete()is called, the item is marked for collection, or we're in Low Memory conditions and the item is unused. - Low Memory: Very strong. In this model Java always knows exactly how much native heap space is being used, and can trigger collections at the right time.
Tuning Tips
When using this model, these tips may increase performance, although in some situations, may instead decrease your performance. Always measure.
- Try different garbage collectors in Java. To try the parallel
incremental collector, start your Java process with: these options:
-XX:+UseParallelGC
The concurrent garbage collector works well too. To use that pass these options to java on startup:-XX:+UseConcMarkSweepGC -XX:+UseParNewGC
- If you are not re-using objects across Ferry calls, ensure your objects are short-lived; null out references when done.
- Potentially try caching objects and reusing large objects across multiple calls -- this may give those objects time to move into the Tenured generation and reduce the copying overhead.
- Explicitly manage Ferry memory yourself by calling
delete()on everyRefCountedobject when done with your objects to let Java know it doesn't need to copy the item across a collection. You can also usecopyReference()to get a new Java version of the same Ferry object that you can pass to another thread if you don't know whendelete()can be safely called. - Try a different
JNIMemoryManager.MemoryModel.
-
JAVA_DIRECT_BUFFERS
public static final JNIMemoryManager.MemoryModel JAVA_DIRECT_BUFFERS
Large memory blocks are allocated as DirectByteBufferobjects (as returned fromByteBuffer.allocateDirect(int)).This model is not recommended. It is faster than
JAVA_STANDARD_HEAP, but because of how Sun implements direct buffers, it works poorly in low memory conditions. This model has all the caveats of theNATIVE_BUFFERSmodel, but allocation is slightly slower.Speed
This is the 2nd fastest model available. In tests it is generally 20-30% faster than the
JAVA_STANDARD_HEAPmodel.It is using Java to allocate direct memory, which is slightly slower than using
NATIVE_BUFFERS, but much faster than using theJAVA_STANDARD_HEAPmodel.The downside is that for high-performance applications, you may need to explicitly manage
RefCountedobject life-cycles withRefCounted.delete()to ensure direct memory is released in a timely manner.Robustness
- Allocation: Weak. Java controls allocations of direct memory from a separate heap (yet another one), and has an additional tuning option to set that. By default on most JVMs, this heap size is set to 64mb which is very low for video processing (queue up 100 images and see what we mean).
- Collection: Released either when
delete()is called, or when the item is marked for collection - Low Memory: Weak. In this model Java knows how much direct memory it has allocated, but it does not use the size of the Direct Heap to influence when it collects the normal non-direct Java Heap -- and our allocation scheme depends on normal Java Heap collection. Therefore it can fail to run collections in a timely manner because it thinks the standard heap has plenty of space to grow. This may cause failures.
Tuning Tips
When using this model, these tips may increase performance, although in some situations, may instead decrease performance. Always measure.
- Increase the size of Sun's Java's direct buffer heap. Sun's Java
implementation has an artificially low default separate heap for direct
buffers (64mb). To make it higher pass this option to Java at startup:
-XX:MaxDirectMemorySize=<size>
- Paradoxically, try decreasing the size of your Java Heap if you get
OutOfMemoryErrorexceptions. Objects that are allocated in native memory have a small proxy object representing them in the Java Heap. By decreasing your heap size, those proxy objects will exert more collection pressure, and hopefully cause Java to do incremental collections more often (and notice your unused objects). To set the maximum size of your java heap, pass this option to java on startup:-Xmx<size>
To change the minimum size of your java heap, pass this option to java on startup:-Xms<size>
- Try different garbage collectors in Java. To try the parallel
incremental collector, start your Java process with: these options:
-XX:+UseParallelGC
The concurrent garbage collector works well too. To use that pass these options to java on startup:-XX:+UseConcMarkSweepGC -XX:+UseParNewGC
- If you are not re-using objects across Ferry calls, ensure your objects are short-lived; null out references when done.
- Potentially try caching objects and reusing large objects across multiple calls -- this may give those objects time to move into the Tenured generation and reduce the copying overhead.
- Explicitly manage Ferry memory yourself by calling
delete()on everyRefCountedobject when done with your objects to let Java know it doesn't need to copy the item across a collection. You can also usecopyReference()to get a new Java version of the same Ferry object that you can pass to another thread if you don't know whendelete()can be safely called. - Try the
JAVA_DIRECT_BUFFERS_WITH_STANDARD_HEAP_NOTIFICATIONmodel.
-
JAVA_DIRECT_BUFFERS_WITH_STANDARD_HEAP_NOTIFICATION
public static final JNIMemoryManager.MemoryModel JAVA_DIRECT_BUFFERS_WITH_STANDARD_HEAP_NOTIFICATION
Large memory blocks are allocated as DirectByteBufferobjects (as returned fromByteBuffer.allocateDirect(int)), but the Java standard-heap is informed of the allocation by also attempting to quickly allocate (and release) a buffer of the same size on the standard heap..This model can work well if your application is mostly single-threaded, and your Ferry application is doing most of the memory allocation in your program. The trick of informing Java will put pressure on the JVM to collect appropriately, but by not keeping the references we avoid unnecessary copying for objects that survive collections.
This heuristic is not failsafe though, and can still lead to collections not occurring at the right time for some applications.
It is similar to the
NATIVE_BUFFERS_WITH_STANDARD_HEAP_NOTIFICATIONmodel and in general we recommend that model over this one.Speed
This model trades off some robustness for some speed. In tests it is generally 10-20% faster than the
JAVA_STANDARD_HEAPmodel.It is worth testing as a way of avoiding the explicit memory management needed to effectively use the
JAVA_DIRECT_BUFFERSmodel. However, the heuristic used is not fool-proof, and therefore may sometimes lead to unnecessary collection orOutOfMemoryErrorbecause Java didn't collect unused references in the standard heap in time (and hence did not release underlying native references).Robustness
- Allocation: Good. Java controls allocations of direct memory from a separate heap (yet another one), and has an additional tuning option to set that. By default on most JVMs, this heap size is set to 64mb which is very low for video processing (queue up 100 images and see what we mean). With this option though we inform Java of the allocation in the Direct heap, and this will often encourage Java to collect memory on a more timely basis.
- Collection: Good. Released either when
delete()is called, or when the item is marked for collection. Collections happen more frequently than under theJAVA_DIRECT_BUFFERSmodel due to informing the standard heap at allocation time. - Low Memory: Good. Especially for mostly
single-threaded applications, the collection pressure introduced on
allocation will lead to more timely collections to avoid
OutOfMemoryErrorerrors on the Direct heap.
Tuning Tips
When using this model, these tips may increase performance, although in some situations, may instead decrease performance. Always measure.
- Increase the size of Sun's Java's direct buffer heap. Sun's Java
implementation has an artificially low default separate heap for direct
buffers (64mb). To make it higher pass this option to Java at startup:
-XX:MaxDirectMemorySize=<size>
- Paradoxically, try decreasing the size of your Java Heap if you get
OutOfMemoryErrorexceptions. Objects that are allocated in native memory have a small proxy object representing them in the Java Heap. By decreasing your heap size, those proxy objects will exert more collection pressure, and hopefully cause Java to do incremental collections more often (and notice your unused objects). To set the maximum size of your java heap, pass this option to java on startup:-Xmx<size>
To change the minimum size of your java heap, pass this option to java on startup:-Xms<size>
- Try different garbage collectors in Java. To try the parallel
incremental collector, start your Java process with: these options:
-XX:+UseParallelGC
The concurrent garbage collector works well too. To use that pass these options to java on startup:-XX:+UseConcMarkSweepGC -XX:+UseParNewGC
- If you are not re-using objects across Ferry calls, ensure your objects are short-lived; null out references when done.
- Potentially try caching objects and reusing large objects across multiple calls -- this may give those objects time to move into the Tenured generation and reduce the copying overhead.
- Explicitly manage Ferry memory yourself by calling
delete()on everyRefCountedobject when done with your objects to let Java know it doesn't need to copy the item across a collection. You can also usecopyReference()to get a new Java version of the same Ferry object that you can pass to another thread if you don't know whendelete()can be safely called. - Try the
JAVA_STANDARD_HEAPmodel.
-
NATIVE_BUFFERS
public static final JNIMemoryManager.MemoryModel NATIVE_BUFFERS
Large memory blocks are allocated in native memory, completely bypassing the Java heap.It is much faster than the
JAVA_STANDARD_HEAP, but much less robust.Speed
This is the fastest model available. In tests it is generally 30-40% faster than the
JAVA_STANDARD_HEAPmodel.It is using the native operating system to allocate direct memory, which is slightly faster than using
JAVA_DIRECT_BUFFERS, and much faster than using theJAVA_STANDARD_HEAPmodel.The downside is that for high-performance applications, you may need to explicitly manage
RefCountedobject life-cycles withRefCounted.delete()to ensure native memory is released in a timely manner.Robustness
- Allocation: Weak. Allocations using
makeand releasing objects withRefCounted.delete()works like normal, but because Java has no idea of how much space is actually allocated in native memory, it may not collectRefCountedobjects as quickly as you need it to (it will eventually collect and free all references though). - Collection: Released either when
delete()is called, or when the item is marked for collection - Low Memory: Weak. In this model Java has no idea how
much native memory is allocated, and therefore does not use that
knowledge in its determination of when to collect. This can lead to
RefCountedobjects you created surviving longer than you want to, and therefore not releasing native memory in a timely fashion.
Tuning Tips
When using this model, these tips may increase performance, although in some situations, may instead decrease performance. Always measure.
- Paradoxically, try decreasing the size of your Java Heap if you get
OutOfMemoryErrorexceptions. Objects that are allocated in native memory have a small proxy object representing them in the Java Heap. By decreasing your heap size, those proxy objects will exert more collection pressure, and hopefully cause Java to do incremental collections more often (and notice your unused objects). To set the maximum size of your java heap, pass this option to java on startup:-Xmx<size>
To change the minimum size of your java heap, pass this option to java on startup:-Xms<size>
- Try different garbage collectors in Java. To try the parallel
incremental collector, start your Java process with: these options:
-XX:+UseParallelGC
The concurrent garbage collector works well too. To use that pass these options to java on startup:-XX:+UseConcMarkSweepGC -XX:+UseParNewGC
- Use the
JNIMemoryManager.startCollectionThread()method to start up a thread dedicated to releasing objects as soon as they are enqued in aReferenceQueue, rather than (the default) waiting for the next Ferry allocation orJNIMemoryManager.collect()explicit call. Or periodically callJNIMemoryManager.collect()yourself. - Cache long lived objects and reuse them across calls to avoid allocations.
- Explicitly manage Ferry memory yourself by calling
delete()on everyRefCountedobject when done with your objects to let Java know it doesn't need to copy the item across a collection. You can also usecopyReference()to get a new Java version of the same Ferry object that you can pass to another thread if you don't know whendelete()can be safely called. - Try the
NATIVE_BUFFERS_WITH_STANDARD_HEAP_NOTIFICATIONmodel.
- Allocation: Weak. Allocations using
-
NATIVE_BUFFERS_WITH_STANDARD_HEAP_NOTIFICATION
public static final JNIMemoryManager.MemoryModel NATIVE_BUFFERS_WITH_STANDARD_HEAP_NOTIFICATION
Large memory blocks are allocated in native memory, completely bypassing the Java heap, but Java is informed of the allocation by briefly creating (and immediately releasing) a Java standard heap byte[] array of the same size.It is faster than the
JAVA_STANDARD_HEAP, but less robust.This model can work well if your application is mostly single-threaded, and your Ferry application is doing most of the memory allocation in your program. The trick of informing Java will put pressure on the JVM to collect appropriately, but by not keeping the references to the byte[] array we temporarily allocate, we avoid unnecessary copying for objects that survive collections.
This heuristic is not failsafe though, and can still lead to collections not occurring at the right time for some applications.
It is similar to the
JAVA_DIRECT_BUFFERS_WITH_STANDARD_HEAP_NOTIFICATIONmodel.Speed
In tests this model is generally 25-30% faster than the
JAVA_STANDARD_HEAPmodel.It is using the native operating system to allocate direct memory, which is slightly faster than using
JAVA_DIRECT_BUFFERS_WITH_STANDARD_HEAP_NOTIFICATION, and much faster than using theJAVA_STANDARD_HEAPmodel.It is worth testing as a way of avoiding the explicit memory management needed to effectively use the
NATIVE_BUFFERSmodel. However, the heuristic used is not fool-proof, and therefore may sometimes lead to unnecessary collection orOutOfMemoryErrorbecause Java didn't collect unused references in the standard heap in time (and hence did not release underlying native references).Robustness
- Allocation: Good. With this option we allocate large, long-lived memory from the native heap, but we inform Java of the allocation in the Direct heap, and this will often encourage Java to collect memory on a more timely basis.
- Collection: Good. Released either when
delete()is called, or when the item is marked for collection. Collections happen more frequently than under theNATIVE_BUFFERSmodel due to informing the standard heap at allocation time. - Low Memory: Good. Especially for mostly
single-threaded applications, the collection pressure introduced on
allocation will lead to more timely collections to avoid
OutOfMemoryErrorerrors on the native heap.
Tuning Tips
When using this model, these tips may increase performance, although in some situations, may instead decrease performance. Always measure.
- Paradoxically, try decreasing the size of your Java Heap if you get
OutOfMemoryErrorexceptions. Objects that are allocated in native memory have a small proxy object representing them in the Java Heap. By decreasing your heap size, those proxy objects will exert more collection pressure, and hopefully cause Java to do incremental collections more often (and notice your unused objects). To set the maximum size of your java heap, pass this option to java on startup:-Xmx<size>
To change the minimum size of your java heap, pass this option to java on startup:-Xms<size>
- Try different garbage collectors in Java. To try the parallel
incremental collector, start your Java process with: these options:
-XX:+UseParallelGC
The concurrent garbage collector works well too. To use that pass these options to java on startup:-XX:+UseConcMarkSweepGC -XX:+UseParNewGC
- Use the
JNIMemoryManager.startCollectionThread()method to start up a thread dedicated to releasing objects as soon as they are enqued in aReferenceQueue, rather than (the default) waiting for the next Ferry allocation orJNIMemoryManager.collect()explicit call. Or periodically callJNIMemoryManager.collect()yourself. - Cache long lived objects and reuse them across calls to avoid allocations.
- Explicitly manage Ferry memory yourself by calling
delete()on everyRefCountedobject when done with your objects to let Java know it doesn't need to copy the item across a collection. You can also usecopyReference()to get a new Java version of the same Ferry object that you can pass to another thread if you don't know whendelete()can be safely called. - Try the
NATIVE_BUFFERS_WITH_STANDARD_HEAP_NOTIFICATIONmodel.
-
-
Method Detail
-
values
public static JNIMemoryManager.MemoryModel[] values()
Returns an array containing the constants of this enum type, in the order they are declared. This method may be used to iterate over the constants as follows:for (JNIMemoryManager.MemoryModel c : JNIMemoryManager.MemoryModel.values()) System.out.println(c);
- Returns:
- an array containing the constants of this enum type, in the order they are declared
-
valueOf
public static JNIMemoryManager.MemoryModel valueOf(java.lang.String name)
Returns the enum constant of this type with the specified name. The string must match exactly an identifier used to declare an enum constant in this type. (Extraneous whitespace characters are not permitted.)- Parameters:
name- the name of the enum constant to be returned.- Returns:
- the enum constant with the specified name
- Throws:
java.lang.IllegalArgumentException- if this enum type has no constant with the specified namejava.lang.NullPointerException- if the argument is null
-
getNativeValue
public int getNativeValue()
Get the native value to pass to native code- Returns:
- a value.
-
-