Previously if a client makes an unsupported operation,
like a POSIX rename, it would exit the server.
Both support POSIX rename, and do not abort the connection
if there is an unsupported operation is made by the client.
When fstat is called it now uses the handle to get the opened request,
pulls the filepath from that and creates a new request for the fstat
call. This eliminates the need to change the Method on the request
object and furthers the goal of eliminating post-creation mutation of
the request object to open up the possibility to eliminate or at least
reduce the use of the global request lock.
Just like with files, the call to the hander to create the Lister is
done when the OPENDIR packet is received. This eliminates the need for
the lock in the filelist() method and opens the path for eliminating
more locks.
Testing feasibility of creating the reader/writer when first receiving
the Open packet instead of waiting for the Get/Put.
This work is meant to make the Request locks easier to manage and
ultimately reducing/eliminating them as much as possible.
Previous code used the request ids to do ordering. This worked until a
client came along that used un-ordered request ids. This reworks the
ordering to use an internal counter (per session) to order all packets
ensuring that responses are sent in the same order as the requests were
received.
Fixes#260
Instead of accepting a more general type and then asserting it to the
proper type, just take the proper type as the argument.
Also clean up some of the use of it where it checked old direct sending
code's return error (error is now always nil).
PR #257 fixes the issue with running Stat and Readdir on the same
Request. Thus we no longer need to close and recreate the Request
when handling an Opendir call.
The initial Opendir packet is supposed to repond with an error status if
the directory wasn't found. It was just returning a handle without
checking, now it does a Stat on the path and only returns the handle if
the Stat is successful and it indicates it is a directory, otherwise it
returns an error.
Create a master Context in the per-session ReqeustServer.Serve method
that is then used as the parent for all Context's created in that
server. Its cancelFunc is then always called when Serve exits. This is
in line which out the http.serve code works.
Each Request gets a child WithCancel context created in the
requestFromPacket call. This Context is still closed on request.close().
Recent refactoring has eliminated the need for Fstat and Fsetstat (stat
operations on file handle instead of path)to be special cased. They will
now just work with the normal case for any packet with a file handle.
Fixes#217, where the path cleaning code was mangling the volume name on
windows; ie. "c:/some/path" was turned into "/c:/some/path". Using the
filepath.IsAbs() (vs path.IsAbs()) uses OS specific tests and fixes
this.
This goes against strict POSIX paths, but this used to work and fixing
it breaks nothing on other platforms. And we have still standardized on
POSIX style delimiters.
When the SSH_FXP_OPEN packet has the SSH_FXF_CREAT pflag set the file
should get created even if no data is sent. We do this by having the
Open method call through to the FilePut Handler with empty packet data.
Fixes#214
In #192 we normalized all Request use to have pointer receivers. This
simplified reasoning and allowed for a simpler Request struct. This
change completes that work by eliminating the need for the
state/stackLock pointers and also removes the need for a new Request
each packet, reducing the pressure on the GC.
The main bit is that the RequestServer.openRequests[] map now stores Request
pointers and it only creates new Requests (replacing the mapped entry) when the
Method changes. It never updates/modifies an existing Request and the packet to
Method mapping is now all in place place (requestMethod).
Fix#207
The primary issue of the resolved ticket was path cleaning code returned
the path of '/.' instead of '/'.
We also reviewed and clarified the use of filepath/path and how OS
specific paths are handled in the server code. All paths are converted
to POSIX paths as they come into the server. They are then POSIX paths
for all internal operations.
Request object had a packet channel where it stored packets for
read/write/readdir handling. It was required in an earlier version of
the code (that was refactored before merging) and never got cleaned up.
This removes that channel and just passes the packet through the methods
instead. This is much simpler and would have eliminated the chance of
issue #195. It also looks like it should allow for a few more
simplifications as well.
TLDR; have the Handler wrappers generate the response packets for the
errors instead of returning the errors.
Errors from the handers was returned up the stack to the request-server
handler call. In cases of errors the packet for the error was then
generated using the passed in packet (the incoming packet).
For file read/write/list operations the incoming packet data (includeing
ID) is put in a queue/channel so that it can be handled by the same file
handling code returned by the handler.
Due to gorouting scheduling, in some cases the packets in the channel
won't line up with the incoming packets. That is the worker that took an
incoming packet with one ID might respond with a packet with a different
ID. This doesn't matter as the reply order of packets doesn't matter.
BUT when this response packet that doesn't match IDs with the incoming
packet system returns an error, it uses the incoming packets ID to
generate the error packet. So the error response would use that ID and
the packet from the queue, once processed, would also use that ID
(because on success it uses the ID from the packet in the queue).
This patch fixes the issue by having the code that works with the
packets, either incoming or from the queue, generate the error packet.
So there is no chance of them getting out of sync.
Splitted cleanPath into cleanPacketPath and cleanPath for better handling of slashes in file paths
Added test for cleanPath func
Removed code duplication => filepath.ToSlash(filepath.Clean(...)) => cleanPath(...)
Fixed tests for runLs to match year or time
Renamed constants to fit hound rules
I didn't like how the initial fix for #184 required the handler author
to track the token/offset for the filelist. I came up with a way to do
it that mimic'd the FileReader handler interface using a method
something like ReaderAt except for file lists.
changed MaxFileinfoList to a var for testing and tweaking
Renaming FileInfoer interface to FileLister.
FileInfoer -> FileLister
Fileinfo -> Filelist
There is a data race with the waitgroup (wg) object used to synchronize
the workers with the server exit. The workers called wg.Add()
asynchronously and it was possible for the Wait() to get hit before any
of the Add() calls were made in certain conditions. I only ever saw this
sporatically in the travis tests.
This fixes it by making the wg.Add() calls synchronous.
Handle fsetstat in same was a fstat, by pulling path from stored file
info and processing in that form. Makes handlers simpler while
preserving functionality.
Support fstat by converting to a standard Stat call and processing that.
I didn't want to add another special case to the request backend, I
wanted to keep it only having to support Stat.
Server struct contains Mutex which might be copied in inconsistent
state. Avoid this by declaring methods on pointer receiver.
This calms down go race detector.
Signed-off-by: Pavel Borzenkov <pavel.borzenkov@gmail.com>
Nicola Murino
John Eikenberry
soopsio
Daniel Theophanes
Tommie Gannert
Allan Feid
Mayank Agarwal
sandreas
Pavel Borzenkov