[Rxtx] rxtx moving from JNI to JNA (was Re: About JRE crashes)

[Kustaa Nyholm]

eam that the SerialPort returns:
>> 
>> public int read(byte[] b, int off, int len) throws IOException {
>>     mDataAvailableNotified = false;
>>     int n = 0;
>>     if (mUseSelectForTimeouts && mReceiveTimeOutEnabled) {
>>         mPOSIX.FD_SET(mFD, read);
> 
> yeah ok, if someone implemented FD_SET in native code...
> but such tricks don't count I think.
> BTW, initial FD_ZERO is missing ;-)

No, if you look carefully fdset 'read' is not created here
but elsewhere (so we don't have to create it everytime we do
read) where it zeroed, and since as long as the port is
open mFD does not change there is no need to zero it because
after selectt the fdset will either be all zero (or more
properly nothing is FD_ISSET) or just mFD flag is set.

Anyway, this is just a sketch, never run.

> 
>>         timeout.tv_sec = 0;
>>         timeout.tv_usec = mReceiveTimeOutValue * 1000;
> 
> couldn't this overflow?

Yeah, at least with insane timeout values, in production
code this would need to be handled but this is just a sketch,
I was more interested in getting the overall picture of
what is involved.

> 
>>         int srv = mPOSIX.select(mFD + 1, read, null, null, timeout);
>>         if (srv == 0)
>>             return 0;
>>         if (srv < 0)
>>             throw new IllegalStateException();
>>         }
> 
> IllegalStateException on EINTR? Does this make sense?
> Where is the error handling (strerror etc)?

Error handling would need to be added, sure, but when (if)
I try to bring this baby up my first concern is just to be
informed if select fails when it should not.
> 
>>     n = mPOSIX.read(mFD, b, off, len);
> 
> POSIXs read has four parameters?

No, but my POSIX wrapper takes four parameters. But this is
a design decision ie the wrapper could just as well look
as much POSIX as possible and we could do the pointer
arithmetic (b+off) here. I have not actually ever done any
pointer arithmetic on Java side but I com.sun.jna.Pointer.share
exists just for this purpose.


> loop around is mandatory, because read may return just part of
> the requested data.

I thought that InputStream does not guarantee to read more
than one byte and a javacomm InputStream with time out
does not even guarantee that? So there will be looping
in the client side anyway so to me it would be better just
to return what is available.
 
>>     if (n < 0)
>>         throw new IllegalStateException();
>>     return n;
>>     }
> 
> well, I don't know what mPOSIX is, is this coming with default
> JNA (i.e. without needing to install any native code)?
> Then OK, it seems to have special wrappers for special functions
> and thus is not generic.

No, JNA does not provide any POSIX (or any  other OS) stuff by itself.
JNA just allow me to map C calls to Java call directly. So mPosix
is a light weight Java wrapper that provides the POSIX constant and
macros that cannot be accessed from Java and allows calling
of different native APIs.



> 
> I think `pure JNA' cannot have e.g. FD_SET (which is a define) or
> EINTR (also usually a define) which are no symbols in libraries.
> Problem is to know which integer value EINTR has on the current
> system. How does the mPOSIX above knows that?

Like I said later in my previous response, this info would have
to be copied from the appropriate OS's header. Nuisance, can't be
avoided but not really an issue as the value for a particular OS
is by now pretty much fixed.

> 
> I though it would simply offer to call any dynamic library
> function in a generic way, like the ld/ABI stuff works
> internally. In other words, I though JNA is something like libdl,
> not with a special socket wrapper library.

That is right, JNA just enables calling a binary C library directly from
Java without any glue code that needs to be compiled.
> 
> Are the special termios wrapper libraries avialable for all
> supported platforms?

No, it would need to be created.

>> My point is that just getting C code to compile cleanly in all
>> platforms is a major headache in many projects. And setting up
>> the toolchain is often pain too, even for maintainers.
> 
> I see and agree.
> But this won't be solved by JNA I think, because now the code
> compiles but still does not work.

Well, a code that does not compile cannot be deployed but there
is a chance that if you can run the code you will not use the
part that is failing. More, the compile issue maybe related to
a platform in which the library is not being deployed so
it could just work it you could just get it to run, and that
is where JNA helps.

>  This can be worse because more
> difficult to find out.
> Let's assume you would have an OS with:
> 
> int get_errno();
> #define errno get_errno()
> 
> the `extern int errno;' code won't compile (link), but in JNA it
> would not work but as long as not testing if raised errors are
> detected you would not even notice that.

You say that like it was a negative thing ;-)

It is true that 'link' errors are detected later but if this is
and issue, the visioned POSIX library could exercise each function
once in a static initializer to detect any missing functions
and thus provide fail fast operation.

> 
>> And having to mentally switch between two languages that look
>> the same but have subtly different semantics is also difficult.
> 
> OK, but others (like me) might consider it pleasant.
> 

:-)


> 
> I think, easiest is to use the most appropriate language, be it
> Java, C or even something else. But I see your points, also with
> building and deployment.
> 

I don't subscribe to the point of view that you should always use
the most appropriate language for a task, of course it depends
on the definition of 'appropriate', but still. To me the language
is dictated by popularity of the language. Thus I mostly code in C/C++ or
Java, even if a more 'appropriate' language existed. Of course
this is not stead and fast and I may have to take up ObjC for
obvious reasons. But I won't be doing Haskell before it is
very popular.

> Yes, I think such discussions are `being considered', discussing
> pros and cons, hear about others opinions, getting new ideas,
> evolving them etc. I enjoyed the discussions so far and learned
> from them, which always is good :)

Same here, nice to interact with people who are more intelligent
and knowledgeable than one self...

> 
>  [...]
>> I implemented, just for fun, JNA interface for POSIX ioctl,
>> tcgetattr,select,read,write etc like this (omitted the 'structs' for brevity
>> here):
>> 
>> public class MacOSXAPI {
>>     static public IMacOSXAPI mMacOSXAPI = (IMacOSXAPI)
>> Native.loadLibrary("c", IMacOSXAPI.class);
>> 
>>     public interface IMacOSXAPI extends Library {
> 
> this is supposed to work on all platforms or just on MacOS?

Basically only on Mac OS but should work as long as the SOS ABI
for the functions in questions is compatible with Mac OS, and
for the few functions and given the predominance of Intel arch,
there probably are other operating systems that this would work
with. However the proper strategy would be to have a separate
class for each OS.

> 
>  [...]
>> With this POSIX calls can be made as almost as easily from Java as from C,
>> and the code that uses the POSIX API looks almost identical in C and Java,
>> only that with JNA I did not  have to write a JNI POSIX wrapper or have
>> access to C tool chain and 32/64 bit architecture issues are supporsed to be
>> handled by JNA.
>> 
>> And I don't have to compile and ship native libraries for all platforms.
>> 
>>> let's say
>>> {
>>>    int  flags = 0, syserr;
>>>    syserr = ioctl(file, TIOCMGET, &flags);
>>>    if (syserr == -1) {
>>>        snprintf(&log[strlen(log)], logSize,
>>>                "%s at %s\n",  strerror(errno), "TIOCMGET, TIOCM_CTS");
>>>    }
>>>    return(flags & TIOCM_CTS);
>>> }
>>> 
>>> can be done easier in Java?
>> 
>> This is what this would look with my POSIX JNA library binding:
>> 
>> // not compiled nor tested
>> 
>>    int[]  flags = {0}; int syserr;
>>     syserr = mPosix.ioctl(file, TIOCMGET, flags);
>>     if (syserr == -1) {
>>         log += String.format( "%s at %s\n",  mPosix.strerror(errno),
>> "TIOCMGET, TIOCM_CTS");
>>     }
>>     return(flags[0] & TIOCM_CTS);
> 
> what are TIOCMGET and TIOCM_CTS? Where do the values come from?
> errno really exists?

These are 'static int' fields in the posix library class.

> 
> If this is (more or less) really that simple, than I'm surprised.
> Would be great. It would mean that JNA knows and actively
> supports termios etc (by defining TIOCM_CTS etc).

Well, it is that simple but yes you do not get TIOCM_CTS automatically
you have to copy those from OS headers.

> 
> you said it was not compiled... is the example code simplied or
> is it really close to a working example?
> 

I think I did not say that. I does compile, so it can be run, but
I've yet to run it and it is bound to need some more work before
it has a chance to function.


> behind the logSize placeholder hoping no one would notice :-)

:-)

> 
> based on platform, architecture, even platform version?

Whatever is required.

> Hard to know, hard to test.

Testing is the issue, you would need the platform to actually test
and then you would know the values.

> EINTR probably won't change on linux, but FD_SET maybe...

Might, but not very likely.

> 
>> There are about 80 of these required for implementing rxtx and while it is
>> trivial to look them up from the headers, it needs to be done and done
>> correctly.
> 
> ok, what is EAGAIN on cygwin 2.21 and how to handle it?
> :-)

Yeah, would have to fire up my WinXP laptop with Cygwin installed and
I can't be bothered for this discussion but point taken that it is
a chore and there maybe difficulties.


> 
> ahh ok. I wondered how to implement them.
> 
> Does this work? I would assume that `int[] set' isn't binary
> compatible to `struct { unsigned long int fd_set[] }'.

IIRC (looked it up when I wrote that piece) I think it is compatible,
but of course if it is not then this would just have to take some
slightly different form.

> On
> winsock, it even is:
> 
>   typedef struct fd_set {
>           u_int   fd_count;               /* how many are SET? */
>           SOCKET  fd_array[FD_SETSIZE];   /* an array of SOCKETs */
>   } fd_set;
> 
> I think JNA cannot know whether there is a fd_count as first
> element in fd_set or not.

Correct, JNA does not know. And that is why my posix wrapper actually
uses an abstract Java class FDSet to represent the set on Java side
and an actual implementation class for each platform. (Just a reminder
that I've just sketched an implementation on Mac OS X, not implying that
there exist a full POXIS wrapper for multiple platforms).

> 
> Performance would be interesting. Linux even uses inline
> assembler to speed them up... if they do for a reason, this java
> code could be too slow :)
> 
I doubt that very much, the Java code gets compiled at runtime by
the JIT which does pretty good job in many cases. And given the
context of serial communication and looking at the amount of
code involved for example in the InputStream.read I wont believe
this is an issue until it is demonstrated.


Ok, thanks for the discussion, need to go and earn some money.

bye for now, Kusti