[JNI Beginner] GC of Java arrays returned by the native code

jduprez

Hello all,
I am beginning with JNI, to integrate a C library that pilots an industrial equipment, into a java UI. This library enables to exchange various proprietary PDUs (protocol data units), with the equipment, up and down (request/replies). Both requests and replies are arrays of bytes (+char*+).
"Request" byte arrays are constructed by Java code, which passes them to the JNI code that glues with the lib. "Reply" byte arrays are returned to the Java code, which analyzes them.

The "return reply" part is very similar to this [tutorial example|http://java.sun.com/developer/onlineTraining/Programming/JDCBook/jniexamp.html] , which returns bytes read from a file. However there's something I don't understand with regard to garbage collection of the returned byte array:
- in this stock example, the C code creates a Java byte array fills it, and simply returns it (example code stripped to highlight only the parts relevant to my question):

    jByteArray=(*env)->NewByteArray(env, size);
    (*env)->SetByteArrayRegion(env, jByteArray, 0, size, (jbyte *)sourceBytes);
    ....
    return (jByteArray);

What will happen to this Java array (jByteArray) with regard to garbage collection?
- if it's no more referenced (the example Java code just systemouts it and forgets it), will it be eligible to GC?
- if it is referenced by a Java variable (in my case, I plan to keep a reference to several replies as the business logic requires to analyze several of them together), do regular Java language GC rules apply, and prevent eligibility of the array to GC as long as it's referenced?

That may sound obvious, but what mixes me up is that the same tutorial describes memory issues in subsequent chapters: spécifically, the section on "passing arrays states that:

[in the example] the array is returned to the calling Java language method, which in turn, garbage collects the reference to the array when it is no longer used

This seems to answer "yes" to both my questions above ) But it goes on:

The array can be explicitly freed with the following call:
{code} (*env)-> ReleaseByteArrayElements(env, jByteArray, (jbyte *)sourceBytes, 0);{code}

Under what circumstances would one need to explicitly free jByteArray when it's about to be returned to the Java calling method? Or does this sentence apply to completely different situations (such as, when the array is +not+ returned as is to a Java method)?

The tutorial's next section has a much-expected +memory issues+ paragraph, from which I quote:

By default, JNI uses local references when creating objects inside a native method. This means when the method returns, the references are eligible to be garbage collected.

I assume this means, +unless the references are assigned, in the Java code, to a Java variable+, right?

If you want an object to persist across native method calls, use a global reference instead. A global reference is created from a local reference by calling NewGlobalReference on the the local reference.

I assume this enables the C code to maintain a global reference to a java object even if it's not referenced anymore from the Java variables, right?

I also checked the [JNI specification|http://download-llnw.oracle.com/javase/6/docs/technotes/guides/jni/spec/design.html#wp1242] , but this didn't clear the doubt completely:

*Global and Local References*

The JNI divides object references used by the native code into two categories: local and global references. Local references are valid for the duration of a native method call, and are automatically freed after the native method returns. Global references remain valid until they are explicitly freed.
Objects are passed to native methods as local references. All Java objects returned by JNI functions are local references. The JNI allows the programmer to create global references from local references. JNI functions that expect Java objects accept both global and local references. A native method may return a local or global reference to the VM as its result

Again I assume the intent is that Global references are meant for objects that have to survive across native calls, regardless of whether they are referenced by Java code. But what worries me is that combining both sentences end up in +All Java objects returned by JNI functions are local references (...) and are automatically freed after the native method returns.+.

Could you clarify how to make sure that my Java byte arrays, be they allocated in C code, behave consistently with a Java array allocated in Java code (I'm familiar already with GC of "regular" Java objects)?

Thanks in advance, and best regards,

J.

jschellSomeoneStoleMyAlias

jduprez wrote:
Hello all,
I am beginning with JNI, to integrate a C library that pilots an industrial equipment, into a java UI. This library enables to exchange various proprietary PDUs (protocol data units), with the equipment, up and down (request/replies). Both requests and replies are arrays of bytes (+char*+).
"Request" byte arrays are constructed by Java code, which passes them to the JNI code that glues with the lib. "Reply" byte arrays are returned to the Java code, which analyzes them.

The "return reply" part is very similar to this [tutorial example|http://java.sun.com/developer/onlineTraining/Programming/JDCBook/jniexamp.html] , which returns bytes read from a file. However there's something I don't understand with regard to garbage collection of the returned byte array:
- in this stock example, the C code creates a Java byte array fills it, and simply returns it (example code stripped to highlight only the parts relevant to my question):

    jByteArray=(*env)->NewByteArray(env, size);
(*env)->SetByteArrayRegion(env, jByteArray, 0, size, (jbyte *)sourceBytes);
....
return (jByteArray);

What will happen to this Java array (jByteArray) with regard to garbage collection?

It will be collected when it is no longer referenced.
The fact that you created it in jni doesn't change that.

The array can be explicitly freed with the following call:
(*env)-> ReleaseByteArrayElements(env, jByteArray, (jbyte *)sourceBytes, 0);

Under what circumstances would one need to explicitly free jByteArray when it's about to be returned to the Java calling method? Or does this sentence apply to completely different situations (such as, when the array is not returned as is to a Java method)?

Per what the tutorial says it is either poorly worded or just wrong.

An array which has been properly initialized it a just a java object. Thus it can be freed like any other object.

Per your original question that does not concern you because you return it.

In terms of why you need to explicitly free local references.

[http://download-llnw.oracle.com/javase/6/docs/technotes/guides/jni/spec/design.html#wp16785]

The tutorial's next section has a much-expected memory issues paragraph, from which I quote:

By default, JNI uses local references when creating objects inside a native method. This means when the method returns, the references are eligible to be garbage collected.

I assume this means, unless the references are assigned, in the Java code, to a Java variable, right?

As stated it is not precise.

The created objects are tracked by the VM. When they are eligible to be collected they are.

If you create a local reference and do NOTHING that creates an active reference elsewhere then when the executing thread returns to the VM then the local references are eligible to be collected.

>

If you want an object to persist across native method calls, use a global reference instead. A global reference is created from a local reference by calling NewGlobalReference on the the local reference.

That is not precise. The scope is the executing thread. You can pass a local reference to another method without problem.

I assume this enables the C code to maintain a global reference to a java object even if it's not referenced anymore from the Java variables, right?

It enables access to it to be insured across multiple threads in terms of execution scope. Normally you should not use them.

EJP

    jByteArray=(*env)->NewByteArray(env, size);
(*env)->SetByteArrayRegion(env, jByteArray, 0, size, (jbyte *)sourceBytes);
....
return (jByteArray);

What will happen to this Java array (jByteArray) with regard to garbage collection?

Same as any other object.

- if it's no more referenced (the example Java code just systemouts it and forgets it), will it be eligible to GC?

Yes.

- if it is referenced by a Java variable (in my case, I plan to keep a reference to several replies as the business logic requires to analyze several of them together), do regular Java language GC rules apply, and prevent eligibility of the array to GC as long as it's referenced?

Yes, of course.

The array can be explicitly freed with the following call:

  (*env)-> ReleaseByteArrayElements(env, jByteArray, (jbyte *)sourceBytes, 0);

That is referring to sourceBytes, i.e. the native byte array obtained from GetByteArrayElements(), as per the JNI Specification. It is not referring to releasing jByteArray.+

Under what circumstances would one need to explicitly free jByteArray when it's about to be returned to the Java calling method?

None, see above.

Or does this sentence apply to completely different situations (such as, when the array is not returned as is to a Java method)?

See above.

I assume this means, unless the references are assigned, in the Java code, to a Java variable, right?

Of course.

If you want an object to persist across native method calls, use a global reference instead. A global reference is created from a local reference by calling NewGlobalReference on the the local reference.

I assume this enables the C code to maintain a global reference to a java object even if it's not referenced anymore from the Java variables, right?

It also enables the C code to refer to the same object across multiple JNI methods. You can't save a local reference, e.g. a jobject, and use it again in a subsequent JNI method, because the object might have moved or been garbage-collected. You have to go via a GlobalRef or a WeakRef. This is a common trap for young players. Any time you see static jobject ... or static jclass ... or either of those as members of a C++ class which is preserved across JNI method calls, you are looking at a major bug waiting to happen. However this is all irrelevant to your ReleaseByteArrayElements() question.

jduprez

Thank you both for your clarifications. I'm not sure I understood jschell correctly (perdon my French, litterally), but in light of your clarifications I'd venture that the tutorial extracts I quoted are, for the beginner, poorly worded, not precise, and misleading.

The Java object created by JNI code] will be collected when it is no longer referenced.
The fact that you created it in jni doesn't change that. (...)
An array which has been properly initialized it a just a java object. (...)
The created objects are tracked by the VM. When they are eligible to be collected they are. (...)
If you create a local reference and do NOTHING that creates an active reference elsewhere then when the executing thread returns to the VM then the local references are eligible to be collected.

From your replies, I take it that the usual understanding of regular object referencing and garbage collection apply equally to Java objects created by JNI code (they are plain regular Java objects).
The provisions for the special cases are at the C-level (not Java):

[Global Reference] enables access to it to be insured across multiple threads in terms of execution scope. Normally you should not use them.

It also enables the C code to refer to the same object across multiple JNI methods. You can't save a local reference, e.g. a jobject, and use it again in a subsequent JNI method, because the object might have moved or been garbage-collected.
You can't save a local reference, e.g. a jobject, and use it again in a subsequent JNI method, because the object might have moved or been garbage-collected. You have to go via a GlobalRef or a WeakRef.

Those provisions merely impact the way my JNI code should use what I would call "JNI references to Java objects" (such as, a C variable of type jobject), which for the reasons highlighted cannot be just saved in C code and reused across threads or native method calls.

Any time you see static jobject ... or static jclass ... or either of those as members of a C++ class which is preserved across JNI method calls, you are looking at a major bug waiting to happen. However this is all irrelevant to your ReleaseByteArrayElements() question.

Thanks for those advices. Much appreciated.

This is a common trap for young players.

I could show you my beard and my scars, but Mom's calling me for lunch... )


Thank you again, and best regards,

J.

jschellSomeoneStoleMyAlias

jduprez wrote:
Thank you both for your clarifications. I'm not sure I understood jschell correctly (perdon my French, litterally), but in light of your clarifications I'd venture that the tutorial extracts I quoted are, for the beginner, poorly worded, not precise, and misleading.

The Java object created by JNI code] will be collected when it is no longer referenced.
The fact that you created it in jni doesn't change that. (...)
An array which has been properly initialized it a just a java object. (...)
The created objects are tracked by the VM. When they are eligible to be collected they are. (...)
If you create a local reference and do NOTHING that creates an active reference elsewhere then when the executing thread returns to the VM then the local references are eligible to be collected.

From your replies, I take it that the usual understanding of regular object referencing and garbage collection apply equally to Java objects created by JNI code (they are plain regular Java objects).

Yes.

Might note that I do not suggest using JNI except as a last resort. Instead create and executable around the C code, add a comm api, using Java Runtime.exec() to manage the exe and talk to it via the comm API.

Easier to maintain and debug and it can't crash the VM.