Commit Graph

173 Commits

Author SHA1 Message Date
Cassondra Foesch 6181f5c673
Merge pull request #414 from greatroar/chmod-bits
Support os.Mode{Setuid,Setgid,Sticky} in Chmod
2021-03-15 09:58:20 +00:00
greatroar 2573693ec5 Support os.Mode{Setuid,Setgid,Sticky} in Chmod
Previously, these bits were ignored by Chmod, which sent the numerical
value of the mode argument as-is to the server. As a result, callers had
to supply POSIX values for setuid, setgid and sticky to Chmod.

The new version supports both the POSIX values and the Go values.

Also added a note to the docs to clarify that the umask is not
subtracted from the mode, and why that is. The only portable way to get
the umask is to set it, then reset it, but that's racy. On Linux, we
could parse /proc/self/status, but that doesn't work portably and will
fail where /proc is not available (some Docker containers, notably).
2021-03-12 16:17:56 +01:00
Cassondra Foesch c34ea374a2 A simpler bytes buffer pool 2021-03-12 15:14:27 +00:00
Nicola Murino e164c76a12
Merge pull request #413 from greatroar/mkdirall-slash
Check for '/', not os.IsPathSeparator, in MkdirAll
2021-03-11 21:39:17 +01:00
greatroar 9bd6912a11 Check for '/', not os.IsPathSeparator, in MkdirAll
The SFTP protocol uses '/' as the path separator, always.
2021-03-11 00:22:17 +01:00
Nicola Murino 23462f6cd7 client fix potential crash if we receive a short packet 2021-03-10 12:26:09 +01:00
Cassondra Foesch 714bd5db80 +godoc 2021-03-06 09:03:37 +00:00
Cassondra Foesch b83052b538 Export RealPath for potential client use 2021-03-06 08:48:23 +00:00
Nicola Murino c539fdb9b4 improve readAtSequential as for review 2021-03-05 17:10:39 +01:00
Nicola Murino e1e59da6e3 add readAtSequential: used if concurrent reads are disabled ...
... and the requested buffer is bigger than maxPacket
2021-03-05 15:04:17 +01:00
Nicola Murino 5f2c008b8e add an option to disable concurrent reads
Fixs #345
2021-02-23 19:13:26 +01:00
Cassondra Foesch 861a8eaf5c pointer receivers and statusFromError(uint32, error) 2021-02-22 12:11:42 +00:00
Cassondra Foesch c6f90f0596 polishing done? 2021-02-22 12:04:43 +00:00
Cassondra Foesch fc15699691 fixed concurrent writes, mid-polish 2021-02-22 12:04:43 +00:00
Cassondra Foesch 0be6950c0e fix comments and variable names to reflect Write instead of Read 2021-02-22 12:04:43 +00:00
Cassondra Foesch 64bc1f82e3 WriteTo better, but not best, version 2021-02-22 12:04:43 +00:00
Cassondra Foesch cdedb55a3b implement ReadFrom, normalize code patterns 2021-02-22 12:04:43 +00:00
Cassondra Foesch 89afa8096c more concurrency, more overhead, but sometimes more throughput? 2021-02-22 12:04:43 +00:00
Cassondra Foesch ac6027de63 numerous subtle race conditions resolved in clientConn 2021-02-22 12:04:43 +00:00
Nicola Murino 14bb577288 client statVFS: normalize the returned error
added a test case for the request server to check os.IsNotExist error
2021-02-10 19:35:30 +01:00
Nicola Murino c811ca3a25 Remove: permission denied is now os.ErrPermission
fix TestClientRemoveDir test case on macOS
2020-12-08 14:43:19 +01:00
Nicola Murino f5fd2fb058 normalise permission denied error as file not found error 2020-12-08 12:14:21 +01:00
Cassondra Foesch 3c22ebff33
Merge pull request #385 from greatroar/list-extensions
New method Client.Extensions to list server extensions
2020-10-23 19:04:51 +00:00
greatroar d352a1d176 Add Client.Sync method
This uses the fsync@openssh.com extension:
https://github.com/openssh/openssh-portable/blob/master/PROTOCOL, §3.6.
2020-10-23 13:09:54 +02:00
greatroar 265b8168fd New method Client.Extensions to list server extensions 2020-10-23 11:16:44 +02:00
Nicola Murino 3177348ca5 pack structures so they have less size
This is a micro optimization, it fixes warnings like this one:

packet.go:707:24: struct of size 32 bytes could be of size 24 bytes
2020-09-08 10:18:15 +02:00
Nicola Murino b6bdd772b5 truncate should not change offset
improved Fsetstat test case
2020-08-25 07:46:42 +02:00
Nicola Murino a011842257 request server: add support for SSH_FXP_FSETSTAT
we need to add a case for this packet inside the packet worker otherwise
it will be handled in hasHandle case and it will become a "Put" request.

Client side if a Truncate request is called on the open file we should
send a FSETSTAT packet, the request is on the handle, and not a SETSTAT
packet that should be used for paths and not for handle.
2020-08-24 15:00:36 +02:00
Cassondra Foesch 4550e71d6e Add a mutex to protect f.offset in new File.Read 2020-08-19 11:09:42 +00:00
Cassondra Foesch 3ee8d0ba91
Merge pull request #285 from polygon-io/master
Add io.ReaderAt interface compatibility
2020-08-19 11:07:14 +00:00
Nicola Murino 1f178f9671 the allocator can now be enabled per request
Other minor changes as per review comments
2020-03-18 09:36:07 +01:00
Nicola Murino 3f969fcd59 add optional AllocationModeOptimized
after processing a packet we keep in memory the allocated slices and we reuse
them for new packets.
Slices are allocated in:

- recvPacket
- when we receive a sshFxpReadPacket (downloads)

The allocated slices have a fixed size = maxMsgLength.

Allocated slices are referenced to the request order id and are marked for reuse
after a request is served in maybeSendPackets.

The allocator is added to the packetManager struct and it is cleaned at the end
of the Serve() function.

This allocation mode is optional and disabled by default
2020-03-14 19:42:19 +01:00
Eran Sandler 77a4076c35 fixes based on comments + minor test added 2020-01-06 14:54:28 -08:00
Eran Sandler 09f03e7bff Added a ClientOption to determine whetehr to use Fstat or Stat when File.WriteTo is being called to support strange behaviour on some servers 2020-01-03 11:29:41 -08:00
John Eikenberry 8067637962 add note in Create() docs about AWS + O_RDWR 2019-10-13 19:18:30 -07:00
Nicola Murino b4ea0fd6f6 fix lint issues
These lint issues remain:

- request-errors.go, aliases for new error types
- request-attrs.go, UidGid. Changing this will break compatibility
2019-08-30 17:04:37 +02:00
Tommie Gannert 2c24eaad1c Implement the hardlink@openssh.com extension.
Both client and server. This is documented in

  https://github.com/openssh/openssh-portable/blob/master/PROTOCOL

Draft 7 of SFTP added support for SSH_FXP_LINK which supports both
symlinks and hardlinks, but unfortunately OpenSSH doesn't support
that:

  https://tools.ietf.org/html/draft-ietf-secsh-filexfer-07#section-7.7

Adding support for this as an option would be a nice extension to
this.
2019-08-25 20:16:37 -07:00
Nick Craig-Wood 48c87f62c9 Use Stat rather than Fstat in File.WriteTo to work around server bugs
Before this change in File.WriteTo() we used Fstat to discover the
length of the file being transferred.  It appears that some SFTP
servers do not implement this properly perhaps because it is a seldom
used call.

After this change we replace the Fstat on the file handle with a Stat
of the path.  Stat is commonly used function and implemented correctly
in both the servers that had the problem with Fstat, thus working
around the problem.

The code before and after uses the same number of SFTP roundtrips so
performance should be identical.

Fixes #288
2019-05-21 12:02:15 +01:00
Quinton Pike 00faa91b63 :| 2019-04-11 16:56:38 -04:00
Quinton Pike beae100160 Spacing 2019-04-11 16:46:08 -04:00
Quinton Pike a004247d1f Add io.ReaderAt interface compatibility
Read is now a proxy for ReadAt. Read uses the bytes read to set the new offset for the file, since ReadAt does not alter the offset.
2019-04-11 16:45:25 -04:00
Taizeng Wu 5cd7f324f9 Use channel to implement a simple way to wait 2018-12-05 16:30:09 +08:00
Taizeng Wu 3a53acc96b Reimplement Wait method to make it can be called concurrently from multiple goroutines 2018-12-05 14:47:03 +08:00
Taizeng Wu fbf9e05c66 Add Wait method to detect underlying SFTP connection closed
Closes #278
2018-12-03 17:46:45 +08:00
gm42 f9330fc78c Allow to configure maxConcurrentRequests 2018-05-28 09:44:19 +02:00
Nick Craig-Wood 0fc9a33d9a Document that ReadFrom/WriteTo are best for high throughput - fixes #158 2018-05-24 16:41:25 +01:00
Urjit Singh Bhatia 831654e4aa Client.MkdirAll - mimic os.MkdirAll code for consistency & perf 2018-04-25 08:13:19 -07:00
Urjit Singh Bhatia a03704e892 Add mkdirall implementation 2018-04-24 23:58:10 -07:00
John Eikenberry 49488377fa fix client deadlock on server death during writes
Fixes #234

Similar to #181 (which was about read methods), the client deadlocks on
server drop on write methods (Write/ReadFrom). This is another case of
broadcastErr() and dispatchRequest() deadlocking.

The fix is to bump up the channel used to communicate about inflight
packets to maxConcurrentRequests+1 (+1 is new). It seems that it can
have the full set of packets going and hit the error (triggering
broadcastErr) yet it still tries to call dispatchRequest again. More
details in that ticket.

I'm also bumping up the chan buffer in the read methods to keep them
consistent. I can't reproduce the problem, but it looks like it should
have it. So it might just be a much harder race to trigger.

This also includes 2 tests which reprocuded the issue for ReadFrom and
Write.
2018-03-15 13:53:30 -07:00
John Eikenberry 51703e3702
Merge pull request #220 from CodyDWJones/master
Expanded limits to the MaxPacket option.
2018-03-04 17:16:37 -08:00