[GRIZZLY-597] OutputWriter#flushChannel() throws an exception inadequately on a load test Created: 12/May/09 Updated: 11/Nov/11 Resolved: 17/Dec/09 Status: Project: Component/s: Affects Version/s: Fix Version/s: Resolved grizzly http 1.9.22 Type: Reporter: Resolution: Labels: Remaining Estimate: Time Spent: Original Estimate: Environment: Bug carryel Fixed None Not Specified Attachments: outputwriter_patch.jar 597 Issuezilla Id: 1.9.16 Priority: Assignee: Votes: Major grizzly-issues 0 Not Specified Not Specified Operating System: All Platform: All Description When I sent many large packets to the remote server with OutputWriter#flushChannel(), some packets could be lost with receiving the following exception. In the sender side, java.io.IOException: Client disconnected at com.sun.grizzly.util.OutputWriter.flushChannel(OutputWriter.java:124) at GrizzlyTCPConnectorWrapper.send(GrizzlyTCPConnectorWrapper.java:61) at GrizzlyTCPConnectorWrapper.send(GrizzlyTCPConnectorWrapper.java:44) at GrizzlyNetworkManager.send(GrizzlyNetworkManager.java:218) at GrizzlyNetworkManagerSendTest$2.run (GrizzlyNetworkManagerSendTest.java:78) at java.lang.Thread.run(Thread.java:717) First, I thought that the server might close the client's connection because of overflow, and the exception log said "Client disconnected", so I tuned server's params. But this error could not be resolved unfortunately. When I tried to debug the server and client, I was very confused because the connection was still alive and had no problems. I used the OutputWrite for sending packets like this. long timeout = 30000; // 30sec ... OutputWriter.flushChannel( connectorHandler.getUnderlyingChannel(), message, timeout); ... Intuitively, I thought that flushChannel() would retry and wait for at least the timeout if the server and client were busy. But I saw the code, I could know that my thought was wrong. Here is OutputWriter#flushChannel(SelectableChannel channel, ByteBuffer bb, long writeTimeout)' code. public static long flushChannel(SelectableChannel channel, ByteBuffer bb, long writeTimeout) throw IOExceiton{ ... try{ WritableByteChannel writableChannel = (WritableByteChannel) channel; while( bb.hasRemaining() ) { len = writableChannel.write(bb); if( len>0 ) { attempts = 0; nWrite += len; } else { attempts++; ... if(writeSelector.select(writeTimeout) == 0) { ------ (1) if(attempts > 2) throw new IOException("Client disconnected"); ------ (2) } } } } catch( IOException ex ) { ... } finally { ... } return nWrite; } (1) When I tested, Selector#select() could return 0 though it was not timed out. Then OutputWriter#flushChannel()'s writeTimeout is perhaps meaningless. Actually above the wrong IOException will be thrown before OutputWriter#flushChannel() waits for writeTimeout and tries to send the message again. Of course, attempts has 2 limit value, but I think that it is not enough. (2) I think that "Client disconnected"'s message is wrong. If client disconnected, the following exception will usually be thrown rather than above (2). java.nio.channels.ClosedChannelException at sun.nio.ch.SocketChannelImpl.ensureWriteOpen (SocketChannelImpl.java:236) at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:434) at com.sun.grizzly.util.OutputWriter.flushChannel(OutputWriter.java:106) at GrizzlyTCPConnectorWrapper.send(GrizzlyTCPConnectorWrapper.java:61) at GrizzlyTCPConnectorWrapper.send(GrizzlyTCPConnectorWrapper.java:44) at GrizzlyNetworkManager.send(GrizzlyNetworkManager.java:218) at GrizzlyNetworkManagerSendTest$2.run (GrizzlyNetworkManagerSendTest.java:78) at java.lang.Thread.run(Thread.java:717) I think that "Client is busy or timed out"'s message is better. So, I tried to make the patch experimentally. Here is the diff Index: com/sun/grizzly/util/OutputWriter.java =================================================================== — com/sun/grizzly/util/OutputWriter.java (revision 3143) +++ com/sun/grizzly/util/OutputWriter.java (working copy) @@ -100,12 +100,14 @@ int attempts = 0; int nWrite = 0; int len = -1; + long elapsedTime = 0; try { WritableByteChannel writableChannel = (WritableByteChannel) channel; while ( bb.hasRemaining() ) { len = writableChannel.write(bb); if (len > 0) { attempts = 0; + elapsedTime = 0; nWrite += len; } else { attempts++; @@ -119,10 +121,12 @@ SelectionKey.OP_WRITE); } + long startTime = System.currentTimeMillis(); if (writeSelector.select(writeTimeout) == 0) { - if (attempts > 2) - throw new IOException("Client disconnected"); - } + elapsedTime += (System.currentTimeMillis() startTime); + if (attempts > 2 && ( writeTimeout > 0 && elapsedTime >= writeTimeout ) ) + throw new IOException("Client is busy or timed out"); +} } } } catch (IOException ex){ @@ -198,11 +202,13 @@ long nWrite = 0; long len = -1; + long elapsedTime = 0; try { while (nWrite < totalBytes ) { len = socketChannel.write(bb); if (len > 0){ attempts = 0;+ elapsedTime = 0; nWrite += len; } else { if ( writeSelector == null ){ @@ -215,10 +221,12 @@ key = socketChannel.register(writeSelector, SelectionKey.OP_WRITE); + + long startTime = System.currentTimeMillis(); if (writeSelector.select(writeTimeout) == 0) { - if (attempts > 2) - throw new IOException("Client disconnected"); + elapsedTime += (System.currentTimeMillis() - startTime); + if (attempts > 2 && ( writeTimeout > 0 && elapsedTime >= writeTimeout ) ) + throw new IOException("Client is busy or timed out"); } } } @@ -298,11 +306,13 @@ Selector writeSelector = null; int attempts = 0; int nWrite = 0; + long elapsedTime = 0; try { while ( bb.hasRemaining() ) { int len = datagramChannel.send(bb,socketAddress); if (len > 0){ attempts = 0; + elapsedTime = 0; nWrite += len; } else { if ( writeSelector == null ){ @@ -315,10 +325,12 @@ key = datagramChannel.register(writeSelector, SelectionKey.OP_WRITE); + + long startTime = System.currentTimeMillis(); if (writeSelector.select(writeTimeout) == 0) {- if (attempts > 2)- throw new IOException("Client disconnected");+ elapsedTime += (System.currentTimeMillis() startTime);+ if (attempts > 2 && ( writeTimeout > 0 && elapsedTime >= writeTimeout ) )+ throw new IOException("Client is busy or timed out"); } else { attempts--; } Comments Comment by carryel [ 12/May/09 ] Created an attachment (id=68) I attached the proposed patch and diff Comment by jfarcand [ 13/May/09 ] Patch applied. Thanks YOU! Sending java/com/sun/grizzly/util/OutputWriter.java Transmitting file data . Committed revision 3191. Generated at Sun Mar 06 08:33:03 UTC 2016 using JIRA 6.2.3#6260sha1:63ef1d6dac3f4f4d7db4c1effd405ba38ccdc558.