[Rxtx] close != flush and may != close!
Joachim Buechse
joachim at buechse.de
Thu Sep 28 00:56:57 MDT 2006
Well,
I had a hard night because of this;-) At first sight Gregg and I seem
to completely disagree on the matter of close. In reality I don't
think that this is the case. I'm sure that Gregg has some practical
experience that makes him advocate his standpoint, so do I.
What are the facts:
- FilterOutputStream (which is the Basis for BufferedOS, ObjectOS,
etc) calls flush in close.
- FileOutputStream makes no guarentees about data beeing on disk.
- HttpUrlConnection.OutputStream makes no guarentees about data
beeing transmitted at all (it's just a buffer).
It is quite likely that application programmers will wrap the
Port.OutputStream in a BufferedOutputStream, Writer, etc. The library
has no access to those objects which contain there own buffer.
Calling Port.OutputStream.flush() in Port.close() has no usefull
effect in those cases.
I would hence suggest the following behaviour:
1) Port.OutputStream.close() MAY block until all data has been
"sent". (in RXTX it will call Port.OS.flush())
2) Port.OutputStream.flush() SHOULD block until all data has been
"sent" (in RXTX it will call tcdrain).
3) Port.close() MUST NOT block.
[Gregg, you'd probably want 1) to be SHOULD instead of MAY. I still
think it's rather MAY then SHOULD, but SHOULD is OK for me as well.]
[2) can not be stronger than SHOULD with some hardware]
[The MUST NOT in 3) must be guarenteed even in the face of driver/
device errors.]
To me, this behaviour seems the best match of what is in the API and
what is expected by the programmer:
- Guarenteed flush on close can be implemented by calling
Port.OS.flush(), Port .OS.close(), Port.close()
- Best effort flush on close can be implemented by calling
Port.OS.close(), Port.close()
- Best effort, non-blocking abort can be implemented with Port.close
() (similar to HttpUrlConnection.disconnect()).
With this implementation Port.finalise() MAY call Port.close()
without risking to deadlock the virtual machines finalizer thread.
[It is a whole new discussion if the garbage collector should be
responsible to close leftover ports. For FileOutputStream this is the
case, for HttpUrlConnections I have seen it, however I never checked
where the Socket.close() is triggered. For the infamous animated GIF
it is not the case (the GIF animator thread keeps them alive forever).]
Best regards,
Joachim
---
Joachim Büchse
Softwarelösungen und Beratung
Hadlaubsteig 2
CH-8006 Zürich
On 27.09.2006, at 15:48, Dr. Douglas Lyon wrote:
> Hi All,
> I think Joachim might be correct.
> Flush and close may not appropriate to combine into the
> implementation.
>
> The semantics of close are already complex. If you
> close and get a PortInUseException, then you simply will not
> be able to close. Other exceptions that might occur include
> an IOException.
>
> Would it make sense to have these exceptions appear in the
> specification? Are there other exceptions that we might need?
> Thanks!
> - DL
>
>> I strongly opt to NOT call OutputStream.flush() in the implementation
>> of Port.close(). We COULD call it in OutputStream.close() even though
>> I would argue not to do it to motivate clean coding.
>>
>> Flush has no timeout sematics. Calling OutputStream.flush() in
>> Port.close() effectively kills the only abort mechanism that exist.
>>
>> I am aware, that many Java stream implementations call flush in the
>> close. Especially the higher level ones. This is all nice and handy
>> until you see the results in the real world. Missing flush calls and
>> flushes at the wrong moment are the source of 50% of performance
>> problems in network applications. I think it's a very bad idea
>> because relying on flush to get data delivered doesn't work anyway.
>> In most out of lab scenarios only one assumption holds: If the other
>> side (receiving application, not network stack) hasn't checked and
>> confirmed it, it didn't arrive.
>>
>> In my opinion a SerialPort should react like a Socket. This is the
>> closest widely known equivalent existing in the Java standard
>> distribution. Calling close on a TCP-Socket (not the Outputstream of
>> the Socket!) closes this socket immediately without calling flush.
>>
>> Best regards,
>> Joachim
>>
>> ---
>> Joachim Büchse
>> Softwarelösungen und Beratung
>> Hadlaubsteig 2
>> CH-8006 Zürich
>>
>>
>> On 27.09.2006, at 14:22, Dr. Douglas Lyon wrote:
>>
>>> Hi All,
>>> Joachim suggests that:
>>> "This is the one point where I really disagree. Port.close() should
>>> never be used in the meaning of OutputStream.flush(). Port.close
>>> () is
>>> the only abort mechanism available. The native implementation
>>> should
>>> be free to throw away unsent data on close, abort reads and release
>>> the system resource as soon as possible."
>>>
>>> This may be a good point.
>>> http://java.sun.com/j2se/1.4.2/docs/api/java/io/
>>> OutputStream.html#close()
>>> Says that:
>>> Closes this output stream and releases any system
>>> resources associated with this stream. The
>>> general contract of close is that it closes the
>>> output stream. A closed stream cannot perform
>>> output operations and cannot be reopened.
>>>
>>> This last bit, a closed stream cannot be reopened is really
>>> telling.
>>> The roles of "flush" and "close" are often combined in an
>>> implementation,
>>> implicitly. They are also often (but not always) left out of the
>>> specification!
>>>
>>> I would say that is not an optimal treatment and that we can do
>>> better.
>>>
>>> For example:
>>> http://java.sun.com/j2se/1.4.2/docs/api/java/io/
>>> ObjectOutputStream.html#close()
>>> Shows no flush close. But the code at the top of the document
>>> shows:
>>> FileOutputStream fos = new FileOutputStream("t.tmp");
>>> ObjectOutputStream oos = new ObjectOutputStream(fos);
>>>
>>> oos.writeInt(12345);
>>> oos.writeObject("Today");
>>> oos.writeObject(new Date());
>>>
>>> oos.close();
>>>
>>> There is no flush in the example! Technically, according to the
>>> spec,
>>> this should write out NOTHING! But it DOES WORK!! Why?
>>>
>>> But from the source code of the ObjectOutputStream.java, I see:
>>> public void close() throws IOException {
>>> flush();
>>> clear();
>>> bout.close();
>>> }
>>> So, the flush is not in the specification, but it is in the
>>> implementation.
>>>
>>> As a Java programmer, I have gotten used to making the assumption
>>> that flush and close are combined and that the flush is implicit in
>>> the close.
>>>
>>> I am now starting to question my assumption. Whatever we decide,
>>> it should probably be explicit in the specification.
>>>
>>> There is no consistency on close implementations in the IO package.
>>> Check out the close on the ByteArrayOutputStream:
>>> /**
>>> * Closing a <tt>ByteArrayOutputStream</tt> has no effect. The
>>> methods in
>>> * this class can be called after the stream has been closed
>>> without
>>> * generating an <tt>IOException</tt>.
>>> * <p>
>>> *
>>> */
>>> public void close() throws IOException {
>>> }
>>>
>>> Now check the FilterOutputStream:
>>> * The <code>close</code> method of <code>FilterOutputStream</
>>> code>
>>> * calls its <code>flush</code> method, and then calls the
>>> * <code>close</code> method of its underlying output stream.
>>> *
>>> * @exception IOException if an I/O error occurs.
>>> * @see java.io.FilterOutputStream#flush()
>>> * @see java.io.FilterOutputStream#out
>>> */
>>> public void close() throws IOException {
>>> try {
>>> flush();
>>> } catch (IOException ignored) {
>>> }
>>> out.close();
>>> }
>>>
>>> Perhaps something like the FilterOutputStream makes sense for
>>> us. But
>>> should we be ignoring exceptions on close? Perhaps we can close by
>>> trying to flush, then release the resource in a finally
>>> statement and
>>> STILL throw an exception to the caller.
>>>
>>> We may need a different exception for flush than for the internal
>>> close, to facilitate debugging. Thus
>>> public void close() throws IOException, PortInUseException
>>>
>>> At present, we have:
>>> public interface CommPortInterface {
>>> /**
>>> * @return the name of this port.
>>> */
>>> String getName();
>>>
>>> /**
>>> * @return a string representation of this port.
>>> */
>>> String toString();
>>>
>>> /**
>>> * Closes this communications port. Further methods on this
>>> object will
>>> * throw IllegalStateException. All PortOwnershipListeners
>>> will be
>>> * notified of this change of ownership.
>>> */
>>> void close();
>>> }
>>>
>>>
>>> Now, RXTXPort shows:
>>> public synchronized void close() {
>>>
>>> Is this consistent with the present specification, and if not, what
>>> should
>>> we do?
>>>
>>> Thanks!
>>> - Doug
>>>
>>>
>>>> I always appreciate a discussion based on good arguments;-)
>>>>
>>>> I know that Linus T. has argued several times that non-blocking
>>>> close
>>>> does not work. However in those cases where it does not work on
>>>> the
>>>> kernel level it can not be made to work on the application level
>>>> neither. In my experience it is impossible to explain to a user
>>>> why
>>>> an application "hangs" on close. However they do understand
>>>> that it
>>>> may hang on open.
>>>>
>>>>>> From my experience a usefull implementation for close is a
>>>>>> timeout
>>>>>> based synchronous close with a rather small timeout. If the
>>>>>> close
>>>>>> "works" (ie no kernel/driver problem preventing it's execution)
>>>>>> wait
>>>>>> until it has returned. If it blocks (ie more than 1..4 seconds)
>>>>>> return from the call. The trend for the kernel
>>>>>> implementation of
>>>>>> close is clearly "abort" r/w operations. It used to be wait for
>>>>>> the
>>>>>> end of r/w operations.
>>>>>
>>>>> Well, serial data can flow slowly compared to what a program can
>>>>> generate. So,
>>>>> it should be possible for an application to open a port at
>>>>> 300baud,
>>>>> send some
>>>>> large amount of data and block on the close, waiting for the
>>>>> send
>>>>> to complete.
>>>>
>>>> This is the one point where I really disagree. Port.close() should
>>>> never be used in the meaning of OutputStream.flush(). Port.close
>>>> () is
>>>> the only abort mechanism available. The native implementation
>>>> should
>>>> be free to throw away unsend data on close, abort reads and
>>>> release
>>>> the system resource as soon as possible.
>>>>
>>>>>> From a user perspective, closing a resource means "I lost all
>>>>>> interest in you". In the case where the (synchronous) close is
>>>>>> immediate that is no problem. However in the case where it
>>>>>> is not,
>>>>>> this creates big problems. The application by itself has
>>>>>> basicly no
>>>>>> means of dealing with a blocking close. As the user has lost
>>>>>> interest
>>>>>> in the port he will not understand any kind of dialog
>>>>>> regarding an
>>>>>> already closed port, he might have even decided to close the
>>>>>> application and see that it "hangs" on close.
>>>>>
>>>>> It is possible for a java application to do
>>>>>
>>>>> Thread th = new Thread() {
>>>>> public void run() {
>>>>> try {
>>>>> port.close();
>>>>> } catch( Exception ex ) {
>>>>> log.log( Level.WARNING, ex.toString(), ex );
>>>>> }
>>>>> }
>>>>> };
>>>>> th.setDaemon( true );
>>>>> th.run();
>>>>>
>>>>
>>>> 1st problem) This returns immediately (even in the case where
>>>> close
>>>> does not block and returns within a few seconds). Hence the
>>>> application has no idea of the progress of the close, it has to
>>>> join
>>>> the close thread or even have a callback interface to get status
>>>> updates. This is what I often do, but it is neither elegant nor
>>>> simple.
>>>> 2nd problem) The application has to keep track of ports which
>>>> are in
>>>> the state of beeing closed if it wants to reuse/reopen the same
>>>> ports
>>>> (race conditions).
>>>> 3rd problem) From what I have seen the java "process" will not
>>>> unwind/
>>>> return if a java thread hangs in a kernel call (daemon or not).
>>>> That
>>>> problem most likely affects my prefered solution as well, but the
>>>> above code "suggests" otherwise.
>>>>
>>>>> ...implementation detail about networking. Serial ports
>>>>> don't have
>>>>> the same
>>>>> negociated close. Close progresses, unimpeded, when the write
>>>>> buffer is empty.
>>>>
>>>> In my experience the kernel level close may block if the USB
>>>> driver
>>>> is trapped in a weird situation (ie a client device not
>>>> reacting). It
>>>> may block even if no data remains to be send, I consider this a
>>>> kernel/driver/device bug - but unfortunately changes to the
>>>> kernel/
>>>> driver/device are often impossible.
>>>>
>>>>> If remote flow control is asserted, there is no negociation
>>>>> for
>>>>> relief. This
>>>>> is why it seems interesting, to let the close happen
>>>>> asynchronously. In single
>>>>> threaded programming environments, it becomes very convenient
>>>>> to do
>>>>> this, but I
>>>>> don't think it's a correct behavior. When serial hardware/
>>>>> software
>>>>> is broken,
>>>>> and the flowcontrol never subsides, the application can hang
>>>>> forever.
>>>>
>>>> I have seen cases where unplugging a USB device at the "right
>>>> moment"
>>>> or a buggy device that stops responding leads to a close
>>>> blocking for
>>>> hours. This is nothing the application can influence. Arguing,
>>>> that
>>>> an application should be allowed to hang if the OS/driver/hardware
>>>> has bugs works in theory but not in (my) practice. I have seen
>>>> cases,
>>>> where (only) quitting the Java VM will unwind a hanging kernel
>>>> close.
>>>> I can not explain this behaviour, but I can reproduce it with a
>>>> buggy
>>>> USB device.
>>>>
>>>> Best regards,
>>>> Joachim
>>>>
>>>> _______________________________________________
>>>> Rxtx mailing list
>>>> Rxtx at qbang.org
>>>> http://mailman.qbang.org/mailman/listinfo/rxtx
>>>
>>>
>>> _______________________________________________
>>> Rxtx mailing list
>>> Rxtx at qbang.org
>>> http://mailman.qbang.org/mailman/listinfo/rxtx
>>
>>
>> _______________________________________________
>> Rxtx mailing list
>> Rxtx at qbang.org
>> http://mailman.qbang.org/mailman/listinfo/rxtx
>
>
> _______________________________________________
> Rxtx mailing list
> Rxtx at qbang.org
> http://mailman.qbang.org/mailman/listinfo/rxtx
More information about the Rxtx
mailing list