root
/
sftp
mirror of https://github.com/pkg/sftp.git
1
0
Fork 0
Code Issues Actions Packages Projects Releases Wiki Activity
1,184 Commits 14 Branches 34 Tags 4 MiB
Commit Graph

1184 Commits

Author SHA1 Message Date
Tobias Klauser 3cda2a1c6b Exclude test/example code in non-test builds 
The fakeFileInfoSys and testOsSys funcs are only used in tests, so
rename the files they are defined in to request_*_test.go to exclude
them from non-test builds.

Also rename request-example.go to request_example_test.go to exclude it
as well, since it uses said funcs in example code.
 2020-08-10 14:14:59 +02:00
Cassondra Foesch b508b936be
Merge pull request #363 from pkg/patch/RequestServer-Serve-bugs 
RequestServer.Serve bugs found looking at PR-361
 2020-07-31 12:49:47 +00:00
Cassondra Foesch eace420102
Merge branch 'master' into patch/RequestServer-Serve-bugs 2020-07-25 10:50:34 +00:00
Cassondra Foesch f129610309
Merge pull request #366 from drakkan/simplify 
remove redundant types declaration
 2020-07-25 10:47:09 +00:00
Cassondra Foesch dce5f46339
Merge pull request #365 from drakkan/deps_update 
update deps
 2020-07-25 10:46:05 +00:00
Nicola Murino 1acb74f50a remove redundant types declaration 2020-07-25 10:17:41 +02:00
Nicola Murino 124fca874e update deps 2020-07-24 21:02:01 +02:00
Cassondra Foesch c276d4b423 reorder writers over readers for Close() and transmitError() 2020-07-24 07:23:55 +00:00
Cassondra Foesch 161b43cdfc this makes more sense as cleanup of serveLoop() 2020-07-18 20:02:25 +00:00
Cassondra Foesch e716c21f78 go mod tidy 2020-07-18 19:45:44 +00:00
Cassondra Foesch e56b4ff6ad Additional bug-fixes found looking at PR-361 2020-07-18 19:45:27 +00:00
Cassondra Foesch 97b9df616e
Merge pull request #355 from drakkan/path 
cleanPath: use path.IsAbs after converting ToSlash
 2020-07-16 19:17:56 +00:00
Cassondra Foesch 0e3fa91d83
Merge pull request #356 from 178inaba/patch-1 
Fix example error
 2020-07-16 19:15:20 +00:00
Cassondra Foesch 95fa324c0c
Merge pull request #344 from drakkan/allocator_pr 
add optional allocator
 2020-07-16 17:25:45 +00:00
Nicola Murino 118ca57204 cleanPath: use path.IsAbs after converting ToSlash 
we need a POSIX path filepath.IsAbs can give unexpected results on Windows
 2020-07-05 22:18:13 +02:00
Masahiro Furudate 2c5b78c2ef
Fix example error 
Replace from `sftp.Close()` to `client.Close()`.
 2020-06-30 12:01:49 +09:00
Nicola Murino abc70d5950 improve test cases for cleanPath 2020-06-29 18:27:11 +02:00
Cassondra Foesch 7d72039736
Merge pull request #354 from eleijonmarck/patch-1 
Update example_test.go
 2020-06-11 20:13:51 +00:00
Nicola Murino 166a37e8c0 cleanPath: use path.IsAbs after converting ToSlash 
we need a POSIX path filepath.IsAbs can give unexpected results on Windows
 2020-06-06 19:26:12 +02:00
Eric Leijonmarck b6c57af363
Update example_test.go 2020-06-05 11:48:26 +02:00
Eric Leijonmarck e38ca124c6
Update example_test.go 
Add Closing of the file. Had trouble finding the error in which a sftp did not catch my file through this example.

Need to close the file in order for it completely see the newly created file.
 2020-06-02 09:31:56 +02:00
Cassondra Foesch 54c2491980
Merge pull request #352 from eleijonmarck/patch-1 
Update example_test.go
 2020-05-30 13:57:37 +00:00
Eric Leijonmarck 0142f240ba
Update example_test.go 
Overwriting the package name of the sftp.

Should use client in the example instead
 2020-05-28 14:42:41 +02:00
Nicola Murino 2fc68482d2 remove NewRequestServerWithOptions 
we can keep compatibility removing the error return value from
RequestServerOption
 2020-03-19 13:20:22 +01:00
Nicola Murino 1f178f9671 the allocator can now be enabled per request 
Other minor changes as per review comments
 2020-03-18 09:36:07 +01:00
Nicola Murino 7168541b0e travis: add go 1.14 and remove 1.12 2020-03-14 19:53:41 +01:00
Nicola Murino 3f969fcd59 add optional AllocationModeOptimized 
after processing a packet we keep in memory the allocated slices and we reuse
them for new packets.
Slices are allocated in:

- recvPacket
- when we receive a sshFxpReadPacket (downloads)

The allocated slices have a fixed size = maxMsgLength.

Allocated slices are referenced to the request order id and are marked for reuse
after a request is served in maybeSendPackets.

The allocator is added to the packetManager struct and it is cleaned at the end
of the Serve() function.

This allocation mode is optional and disabled by default
 2020-03-14 19:42:19 +01:00
John Eikenberry 18dc4db7a4
Merge pull request #343 from drakkan/allocations 
fileget: allocate a slice with enough capacity
 2020-03-10 15:48:33 -07:00
Nicola Murino 0a45bc4b6d sshFxpDataPacket: document the required capacity to avoid a new allocation 2020-03-10 22:56:41 +01:00
Nicola Murino 0f0e40a3a6 minor changes as requested in the review 2020-03-10 15:35:56 +01:00
Nicola Murino 3ea9f24c15 server: remove now unused maxTxPacket 
maxTxPacket was only used to get the size for the read packet it is not
needed anymore
 2020-03-10 11:48:52 +01:00
Nicola Murino eeafeeff60 FxpReadPacket: add an helper method for slice allocation 
This way we can use the same method in both server and request-server
 2020-03-10 11:46:46 +01:00
Nicola Murino 44e44d716f fileget: allocate a slice with enough capacity 
so a new allocation is not needed in MarshalBinary and sendPacket.

Here are some profiling results while downloading a file (file size is about 1GB),

before this patch:

1254.24MB 55.18% 55.18%  1254.24MB 55.18%  github.com/pkg/sftp.sshFxpDataPacket.MarshalBinary
  991.81MB 43.63% 98.81%   991.81MB 43.63%  github.com/pkg/sftp.fileget
       1MB 0.044% 98.86%  1255.24MB 55.22%  github.com/pkg/sftp.(*packetManager).maybeSendPackets
    0.50MB 0.022% 98.88%  1260.24MB 55.44%  github.com/pkg/sftp.(*packetManager).controller
         0     0% 98.88%   991.81MB 43.63%  github.com/pkg/sftp.(*Request).call
         0     0% 98.88%   993.31MB 43.70%  github.com/pkg/sftp.(*RequestServer).Serve.func1.1
         0     0% 98.88%   993.31MB 43.70%  github.com/pkg/sftp.(*RequestServer).packetWorker
         0     0% 98.88%  1254.24MB 55.18%  github.com/pkg/sftp.(*conn).sendPacket
         0     0% 98.88%  1254.24MB 55.18%  github.com/pkg/sftp.sendPacket

with this patch:

1209.48MB 98.46% 98.46%  1209.48MB 98.46%  github.com/pkg/sftp.fileget
       2MB  0.16% 98.63%     7.50MB  0.61%  github.com/pkg/sftp.recvPacket
         0     0% 98.63%        8MB  0.65%  github.com/drakkan/sftpgo/sftpd.Configuration.handleSftpConnection
         0     0% 98.63%  1209.48MB 98.46%  github.com/pkg/sftp.(*Request).call
         0     0% 98.63%        8MB  0.65%  github.com/pkg/sftp.(*RequestServer).Serve
         0     0% 98.63%  1209.98MB 98.50%  github.com/pkg/sftp.(*RequestServer).Serve.func1.1
         0     0% 98.63%  1209.98MB 98.50%  github.com/pkg/sftp.(*RequestServer).packetWorker
         0     0% 98.63%     7.50MB  0.61%  github.com/pkg/sftp.(*conn).recvPacket (inline)
 2020-03-09 19:22:48 +01:00
John Eikenberry 0e1232311c
Merge pull request #342 from pkg/issue-333-conn-close-race 
fix connection close race
 2020-03-08 14:49:16 -07:00
John Eikenberry c31b3c3b22 fix connection close race 
Need to prevent Close from being called at the same time. The lock
already existed, but was only used in one place where it should be
applied generally.
 2020-03-07 15:05:46 -08:00
John Eikenberry 8f77623878
Merge pull request #338 from drakkan/copy 
data packet unmarshal binary: remove uneeded copy
 2020-03-01 15:36:14 -08:00
Cassondra Foesch 21ab0e5222
Merge pull request #340 from twmb/twmb/update_ssh 
bump ssh dep to fix vuln
 2020-02-27 08:00:34 +00:00
Nicola Murino 01ec2e2e14 write packet UnmarshalBinary: reuse already allocated data 2020-02-26 12:04:13 +01:00
Travis Bischel 26f1844016 bump ssh dep to fix vuln 
https://groups.google.com/forum/#!topic/golang-dev/bhd3Qn9_Q1k
 2020-02-25 12:06:54 -07:00
Nicola Murino 4eda1f42bb data packet unmarshal binary: remove uneeded copy 2020-02-16 21:52:13 +01:00
John Eikenberry f1f62a1b95
Merge pull request #337 from drakkan/servertest 
server integration test: add /usr/lib/ssh/sftp-server to search paths
 2020-02-16 00:12:50 +00:00
Nicola Murino 07905cb843 server integration test: add /usr/lib/ssh/sftp-server to search paths 
os some distro, for example ArchLinux, the sftp-server CLI is in this
path
 2020-02-14 17:27:18 +01:00
John Eikenberry 5616182052
Merge pull request #332 from peachfinance/Fstat-stat-options 
Added a ClientOption to determine whetehr to use Fstat or Stat when File.WriteTo is being called to support strange behaviour on some servers
 2020-01-11 00:44:58 +00:00
John Eikenberry a5f5a4cb21
Merge pull request #330 from pkg/issue-295-corrupt-resumes 
Issue 295 corrupt resumes
 2020-01-11 00:44:14 +00:00
John Eikenberry e6f382575a
Merge pull request #329 from pkg/issue-291-ifmt-platform-fix 
fix S_IFMT value on windows, plan9 and js/wasm
 2020-01-11 00:43:47 +00:00
Eran Sandler 77a4076c35 fixes based on comments + minor test added 2020-01-06 14:54:28 -08:00
John Eikenberry 1bc3ea14d1 fix issue with file put resume/append 
Fixes issue with append uploads. Was opening the file with O_APPEND, but
it uses WriteAt() to write the data which doesn't work with a file
opened in append mode. Removing the append flag fixes the issue as the
client is sending the offsets anyways.

Also added a note to the Request server's FileWriter interface on
handling append flags.
 2020-01-05 14:23:49 -08:00
John Eikenberry 3c6b3a4ff1 add test to exercise file upload resume 
Simulates resuming a file upload where more data is appended to the same
file (eg. network error, uploading log files, etc). Currently broken in
the server as it uses O_APPEND + WriteAt() which don't work together.
 2020-01-03 12:44:31 -08:00
Eran Sandler 09f03e7bff Added a ClientOption to determine whetehr to use Fstat or Stat when File.WriteTo is being called to support strange behaviour on some servers 2020-01-03 11:29:41 -08:00
John Eikenberry f3d13089ef add makefile to simplify running integration tests 2020-01-02 14:49:23 -08:00