root
/
minio
mirror of https://github.com/minio/minio.git
1
0
Fork 0
Code Issues Actions 1 Packages Projects Releases Wiki Activity
minio/cmd/api-router.go

324 lines
18 KiB
Go
Raw Normal View History

routers: Move API and Web routers into their own files. This is done to ensure we have a clean way to add new routers such as - diskRouter - configRouter - lockRouter
2016-03-28 03:37:21 +08:00
/*
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
2020-03-25 03:43:40 +08:00
* MinIO Cloud Storage, (C) 2016-2020 MinIO, Inc.
routers: Move API and Web routers into their own files. This is done to ensure we have a clean way to add new routers such as - diskRouter - configRouter - lockRouter
2016-03-28 03:37:21 +08:00
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
server: Move all the top level files into cmd folder. (#2490) This change brings a change which was done for the 'mc' package to allow for clean repo and have a cleaner github drop in experience.
2016-08-19 07:23:42 +08:00
package cmd
routers: Move API and Web routers into their own files. This is done to ensure we have a clean way to add new routers such as - diskRouter - configRouter - lockRouter
2016-03-28 03:37:21 +08:00
Add disk based edge caching support. (#5182) This PR adds disk based edge caching support for minio server. Cache settings can be configured in config.json to take list of disk drives, cache expiry in days and file patterns to exclude from cache or via environment variables MINIO_CACHE_DRIVES, MINIO_CACHE_EXCLUDE and MINIO_CACHE_EXPIRY Design assumes that Atime support is enabled and the list of cache drives is fixed. - Objects are cached on both GET and PUT/POST operations. - Expiry is used as hint to evict older entries from cache, or if 80% of cache capacity is filled. - When object storage backend is down, GET, LIST and HEAD operations fetch object seamlessly from cache. Current Limitations - Bucket policies are not cached, so anonymous operations are not supported in offline mode. - Objects are distributed using deterministic hashing among list of cache drives specified.If one or more drives go offline, or cache drive configuration is altered - performance could degrade to linear lookup. Fixes #4026
2018-03-29 05:14:06 +08:00
import (
"net/http"
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
2020-03-25 03:43:40 +08:00
"strconv"
"time"
Add disk based edge caching support. (#5182) This PR adds disk based edge caching support for minio server. Cache settings can be configured in config.json to take list of disk drives, cache expiry in days and file patterns to exclude from cache or via environment variables MINIO_CACHE_DRIVES, MINIO_CACHE_EXCLUDE and MINIO_CACHE_EXPIRY Design assumes that Atime support is enabled and the list of cache drives is fixed. - Objects are cached on both GET and PUT/POST operations. - Expiry is used as hint to evict older entries from cache, or if 80% of cache capacity is filled. - When object storage backend is down, GET, LIST and HEAD operations fetch object seamlessly from cache. Current Limitations - Bucket policies are not cached, so anonymous operations are not supported in offline mode. - Objects are distributed using deterministic hashing among list of cache drives specified.If one or more drives go offline, or cache drive configuration is altered - performance could degrade to linear lookup. Fixes #4026
2018-03-29 05:14:06 +08:00
use package name correctly (#5827)
2018-04-22 10:23:54 +08:00
"github.com/gorilla/mux"
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
2020-03-25 03:43:40 +08:00
"github.com/minio/minio/cmd/config"
Ensure that we use constants everywhere (#7845) This allows for canonicalization of the strings throughout our code and provides a common space for all these constants to reside. This list is rather non-exhaustive but captures all the headers used in AWS S3 API operations
2019-07-03 13:34:32 +08:00
xhttp "github.com/minio/minio/cmd/http"
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
2020-03-25 03:43:40 +08:00
"github.com/minio/minio/cmd/logger"
"github.com/minio/minio/pkg/env"
Add disk based edge caching support. (#5182) This PR adds disk based edge caching support for minio server. Cache settings can be configured in config.json to take list of disk drives, cache expiry in days and file patterns to exclude from cache or via environment variables MINIO_CACHE_DRIVES, MINIO_CACHE_EXCLUDE and MINIO_CACHE_EXPIRY Design assumes that Atime support is enabled and the list of cache drives is fixed. - Objects are cached on both GET and PUT/POST operations. - Expiry is used as hint to evict older entries from cache, or if 80% of cache capacity is filled. - When object storage backend is down, GET, LIST and HEAD operations fetch object seamlessly from cache. Current Limitations - Bucket policies are not cached, so anonymous operations are not supported in offline mode. - Objects are distributed using deterministic hashing among list of cache drives specified.If one or more drives go offline, or cache drive configuration is altered - performance could degrade to linear lookup. Fixes #4026
2018-03-29 05:14:06 +08:00
)
routers: Move API and Web routers into their own files. This is done to ensure we have a clean way to add new routers such as - diskRouter - configRouter - lockRouter
2016-03-28 03:37:21 +08:00
Enable build verification with race (#8583)
2019-12-03 07:54:26 +08:00
func newHTTPServerFn() *xhttp.Server {
globalObjLayerMutex.Lock()
defer globalObjLayerMutex.Unlock()
return globalHTTPServer
}
Bring in safe mode support (#8478) This PR refactors object layer handling such that upon failure in sub-system initialization server reaches a stage of safe-mode operation wherein only certain API operations are enabled and available. This allows for fixing many scenarios such as - incorrect configuration in vault, etcd, notification targets - missing files, incomplete config migrations unable to read encrypted content etc - any other issues related to notification, policies, lifecycle etc
2019-11-10 01:27:23 +08:00
func newObjectLayerWithoutSafeModeFn() ObjectLayer {
globalObjLayerMutex.Lock()
defer globalObjLayerMutex.Unlock()
return globalObjectAPI
}
func newObjectLayerFn() ObjectLayer {
globalObjLayerMutex.Lock()
defer globalObjLayerMutex.Unlock()
if globalSafeMode {
return nil
}
return globalObjectAPI
}
func newCachedObjectLayerFn() CacheObjectLayer {
globalObjLayerMutex.Lock()
defer globalObjLayerMutex.Unlock()
if globalSafeMode {
return nil
}
return globalCacheObjectAPI
}
storage/server/client: Enable storage server, enable client storage.
2016-04-13 03:45:15 +08:00
// objectAPIHandler implements and provides http handlers for S3 API.
type objectAPIHandlers struct {
distribute: Make server work with multiple remote disks This change initializes rpc servers associated with disks that are local. It makes object layer initialization on demand, namely on the first request to the object layer. Also adds lock RPC service vendorized minio/dsync
2016-08-01 05:11:14 +08:00
ObjectAPI func() ObjectLayer
Add disk based edge caching support. (#5182) This PR adds disk based edge caching support for minio server. Cache settings can be configured in config.json to take list of disk drives, cache expiry in days and file patterns to exclude from cache or via environment variables MINIO_CACHE_DRIVES, MINIO_CACHE_EXCLUDE and MINIO_CACHE_EXPIRY Design assumes that Atime support is enabled and the list of cache drives is fixed. - Objects are cached on both GET and PUT/POST operations. - Expiry is used as hint to evict older entries from cache, or if 80% of cache capacity is filled. - When object storage backend is down, GET, LIST and HEAD operations fetch object seamlessly from cache. Current Limitations - Bucket policies are not cached, so anonymous operations are not supported in offline mode. - Objects are distributed using deterministic hashing among list of cache drives specified.If one or more drives go offline, or cache drive configuration is altered - performance could degrade to linear lookup. Fixes #4026
2018-03-29 05:14:06 +08:00
CacheAPI func() CacheObjectLayer
Add double encryption at S3 gateway. (#6423) This PR adds pass-through, single encryption at gateway and double encryption support (gateway encryption with pass through of SSE headers to backend). If KMS is set up (either with Vault as KMS or using MINIO_SSE_MASTER_KEY),gateway will automatically perform single encryption. If MINIO_GATEWAY_SSE is set up in addition to Vault KMS, double encryption is performed.When neither KMS nor MINIO_GATEWAY_SSE is set, do a pass through to backend. When double encryption is specified, MINIO_GATEWAY_SSE can be set to "C" for SSE-C encryption at gateway and backend, "S3" for SSE-S3 encryption at gateway/backend or both to support more than one option. Fixes #6323, #6696
2019-01-06 06:16:43 +08:00
// Returns true of handlers should interpret encryption.
EncryptionEnabled func() bool
enable SSE-KMS pass-through on S3 gateway (#7788) This commit relaxes the restriction that the MinIO gateway does not accept SSE-KMS headers. Now, the S3 gateway allows SSE-KMS headers for PUT and MULTIPART PUT requests and forwards them to the S3 gateway backend (AWS). This is considered SSE pass-through mode. Fixes #7753
2019-06-20 08:37:08 +08:00
// Returns true if handlers allow SSE-KMS encryption headers.
AllowSSEKMS func() bool
routers: Move API and Web routers into their own files. This is done to ensure we have a clean way to add new routers such as - diskRouter - configRouter - lockRouter
2016-03-28 03:37:21 +08:00
}
// registerAPIRouter - registers S3 compatible APIs.
enable SSE-KMS pass-through on S3 gateway (#7788) This commit relaxes the restriction that the MinIO gateway does not accept SSE-KMS headers. Now, the S3 gateway allows SSE-KMS headers for PUT and MULTIPART PUT requests and forwards them to the S3 gateway backend (AWS). This is considered SSE pass-through mode. Fixes #7753
2019-06-20 08:37:08 +08:00
func registerAPIRouter(router *mux.Router, encryptionEnabled, allowSSEKMS bool) {
server: Add more elaborate startup messages. (#2731) These messages based on our prep stage during XL and prints more informative message regarding drive information. This change also does a much needed refactoring.
2016-10-06 03:48:07 +08:00
// Initialize API.
api := objectAPIHandlers{
Bring in safe mode support (#8478) This PR refactors object layer handling such that upon failure in sub-system initialization server reaches a stage of safe-mode operation wherein only certain API operations are enabled and available. This allows for fixing many scenarios such as - incorrect configuration in vault, etcd, notification targets - missing files, incomplete config migrations unable to read encrypted content etc - any other issues related to notification, policies, lifecycle etc
2019-11-10 01:27:23 +08:00
ObjectAPI: newObjectLayerFn,
CacheAPI: newCachedObjectLayerFn,
Add double encryption at S3 gateway. (#6423) This PR adds pass-through, single encryption at gateway and double encryption support (gateway encryption with pass through of SSE headers to backend). If KMS is set up (either with Vault as KMS or using MINIO_SSE_MASTER_KEY),gateway will automatically perform single encryption. If MINIO_GATEWAY_SSE is set up in addition to Vault KMS, double encryption is performed.When neither KMS nor MINIO_GATEWAY_SSE is set, do a pass through to backend. When double encryption is specified, MINIO_GATEWAY_SSE can be set to "C" for SSE-C encryption at gateway and backend, "S3" for SSE-S3 encryption at gateway/backend or both to support more than one option. Fixes #6323, #6696
2019-01-06 06:16:43 +08:00
EncryptionEnabled: func() bool {
return encryptionEnabled
},
enable SSE-KMS pass-through on S3 gateway (#7788) This commit relaxes the restriction that the MinIO gateway does not accept SSE-KMS headers. Now, the S3 gateway allows SSE-KMS headers for PUT and MULTIPART PUT requests and forwards them to the S3 gateway backend (AWS). This is considered SSE pass-through mode. Fixes #7753
2019-06-20 08:37:08 +08:00
AllowSSEKMS: func() bool {
return allowSSEKMS
},
server: Add more elaborate startup messages. (#2731) These messages based on our prep stage during XL and prints more informative message regarding drive information. This change also does a much needed refactoring.
2016-10-06 03:48:07 +08:00
}
routers: Move API and Web routers into their own files. This is done to ensure we have a clean way to add new routers such as - diskRouter - configRouter - lockRouter
2016-03-28 03:37:21 +08:00
// API Router
Use const slashSeparator instead of "/" everywhere (#8028)
2019-08-07 03:08:58 +08:00
apiRouter := router.PathPrefix(SlashSeparator).Subrouter()
use package name correctly (#5827)
2018-04-22 10:23:54 +08:00
var routers []*mux.Router
Support multiple-domains in MINIO_DOMAIN (#7274) Fixes #7173
2019-02-23 11:18:01 +08:00
for _, domainName := range globalDomainNames {
Fix: Handle regression caused by gorilla mux v1.7.0 (#7625) PR #7595 fixed part of the regression, but did not handle the scenario, where in docker, the internal port is different from the port on the host. This PR modifies the regular expression such that all the scenarios are handled. Fixes #7619
2019-05-08 01:36:00 +08:00
routers = append(routers, apiRouter.Host("{bucket:.+}."+domainName).Subrouter())
routers = append(routers, apiRouter.Host("{bucket:.+}."+domainName+":{port:.*}").Subrouter())
Virtual host style S3 requests (#5095)
2017-11-15 08:56:24 +08:00
}
routers = append(routers, apiRouter.PathPrefix("/{bucket}").Subrouter())
routers: Move API and Web routers into their own files. This is done to ensure we have a clean way to add new routers such as - diskRouter - configRouter - lockRouter
2016-03-28 03:37:21 +08:00
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
2020-03-25 03:43:40 +08:00
requestsMax, _ := strconv.Atoi(env.Get(config.EnvAPIRequestsMax, "0"))
if requestsMax < 0 {
requestsMax = 0 // 0 means unlimited
}
var (
enabled bool
requestsMaxPerServer int
requestsDeadline time.Duration
)
if len(globalEndpoints.Hosts()) == 0 {
requestsMaxPerServer = requestsMax
} else {
requestsMaxPerServer = requestsMax / len(globalEndpoints.Hosts())
}
enabled = requestsMaxPerServer > 0
if enabled {
var err error
requestsDeadline, err = time.ParseDuration(env.Get(config.EnvAPIRequestsDeadline, "10s"))
if err != nil {
logger.Info("(%s) parsing environment value MINIO_API_REQUESTS_DEADLINE, defaulting to 10 seconds", err)
requestsDeadline = 10 * time.Second
}
}
requestsMaxCh := make(chan struct{}, requestsMaxPerServer)
Virtual host style S3 requests (#5095)
2017-11-15 08:56:24 +08:00
for _, bucket := range routers {
// Object operations
// HeadObject
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
2020-03-25 03:43:40 +08:00
bucket.Methods(http.MethodHead).Path("/{object:.+}").HandlerFunc(
maxClients(collectAPIStats("headobject", httpTraceAll(api.HeadObjectHandler)), enabled, requestsMaxCh, requestsDeadline))
Virtual host style S3 requests (#5095)
2017-11-15 08:56:24 +08:00
// CopyObjectPart
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
2020-03-25 03:43:40 +08:00
bucket.Methods(http.MethodPut).Path("/{object:.+}").HeadersRegexp(xhttp.AmzCopySource, ".*?(\\/|%2F).*?").HandlerFunc(maxClients(collectAPIStats("copyobjectpart", httpTraceAll(api.CopyObjectPartHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("partNumber", "{partNumber:[0-9]+}", "uploadId", "{uploadId:.*}")
Virtual host style S3 requests (#5095)
2017-11-15 08:56:24 +08:00
// PutObjectPart
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
2020-03-25 03:43:40 +08:00
bucket.Methods(http.MethodPut).Path("/{object:.+}").HandlerFunc(
maxClients(collectAPIStats("putobjectpart", httpTraceHdrs(api.PutObjectPartHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("partNumber", "{partNumber:[0-9]+}", "uploadId", "{uploadId:.*}")
Add listBucketObjectsVersions implementation (#8093) This API implementation simply behaves like listObjects() but returns back single version for each object, this implementation should be considered dummy it is only meant for some applications which rely on this.
2019-08-20 05:02:54 +08:00
// ListObjectParts
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
2020-03-25 03:43:40 +08:00
bucket.Methods(http.MethodGet).Path("/{object:.+}").HandlerFunc(
maxClients(collectAPIStats("listobjectparts", httpTraceAll(api.ListObjectPartsHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("uploadId", "{uploadId:.*}")
Virtual host style S3 requests (#5095)
2017-11-15 08:56:24 +08:00
// CompleteMultipartUpload
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
2020-03-25 03:43:40 +08:00
bucket.Methods(http.MethodPost).Path("/{object:.+}").HandlerFunc(
maxClients(collectAPIStats("completemutipartupload", httpTraceAll(api.CompleteMultipartUploadHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("uploadId", "{uploadId:.*}")
Virtual host style S3 requests (#5095)
2017-11-15 08:56:24 +08:00
// NewMultipartUpload
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
2020-03-25 03:43:40 +08:00
bucket.Methods(http.MethodPost).Path("/{object:.+}").HandlerFunc(
maxClients(collectAPIStats("newmultipartupload", httpTraceAll(api.NewMultipartUploadHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("uploads", "")
Virtual host style S3 requests (#5095)
2017-11-15 08:56:24 +08:00
// AbortMultipartUpload
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
2020-03-25 03:43:40 +08:00
bucket.Methods(http.MethodDelete).Path("/{object:.+}").HandlerFunc(
maxClients(collectAPIStats("abortmultipartupload", httpTraceAll(api.AbortMultipartUploadHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("uploadId", "{uploadId:.*}")
Fix s3cmd issue with ACL handling (#5999) With the implementation of dummy GET ACL handlers, tools like s3cmd perform few operations which causes the ACL call to be invoked. Make sure that in our router configuration GET?acl comes before actual GET call to facilitate this dummy call.
2018-06-01 10:43:50 +08:00
// GetObjectACL - this is a dummy call.
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
2020-03-25 03:43:40 +08:00
bucket.Methods(http.MethodGet).Path("/{object:.+}").HandlerFunc(
maxClients(collectAPIStats("getobjectacl", httpTraceHdrs(api.GetObjectACLHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("acl", "")
acl: Support PUT calls with success for 'private' ACL's (#9000) Add dummy calls which respond success when ACL's are set to be private and fails, if user tries to change them from their default 'private' Some applications such as nuxeo may have an unnecessary requirement for this operation, we support this anyways such that don't have to fully implement the functionality just that we can respond with success for default ACLs
2020-02-16 14:07:52 +08:00
// PutObjectACL - this is a dummy call.
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
2020-03-25 03:43:40 +08:00
bucket.Methods(http.MethodPut).Path("/{object:.+}").HandlerFunc(
maxClients(collectAPIStats("putobjectacl", httpTraceHdrs(api.PutObjectACLHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("acl", "")
Add ObjectTagging Support (#8754) This PR adds support for AWS S3 ObjectTagging API as explained here https://docs.aws.amazon.com/AmazonS3/latest/dev/object-tagging.html
2020-01-21 00:45:59 +08:00
// GetObjectTagging
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
2020-03-25 03:43:40 +08:00
bucket.Methods(http.MethodGet).Path("/{object:.+}").HandlerFunc(
maxClients(collectAPIStats("getobjecttagging", httpTraceHdrs(api.GetObjectTaggingHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("tagging", "")
Add ObjectTagging Support (#8754) This PR adds support for AWS S3 ObjectTagging API as explained here https://docs.aws.amazon.com/AmazonS3/latest/dev/object-tagging.html
2020-01-21 00:45:59 +08:00
// PutObjectTagging
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
2020-03-25 03:43:40 +08:00
bucket.Methods(http.MethodPut).Path("/{object:.+}").HandlerFunc(
maxClients(collectAPIStats("putobjecttagging", httpTraceHdrs(api.PutObjectTaggingHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("tagging", "")
Add ObjectTagging Support (#8754) This PR adds support for AWS S3 ObjectTagging API as explained here https://docs.aws.amazon.com/AmazonS3/latest/dev/object-tagging.html
2020-01-21 00:45:59 +08:00
// DeleteObjectTagging
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
2020-03-25 03:43:40 +08:00
bucket.Methods(http.MethodDelete).Path("/{object:.+}").HandlerFunc(
maxClients(collectAPIStats("deleteobjecttagging", httpTraceHdrs(api.DeleteObjectTaggingHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("tagging", "")
S3 Select API Support for CSV (#6127) Add support for trivial where clause cases
2018-08-15 18:30:19 +08:00
// SelectObjectContent
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
2020-03-25 03:43:40 +08:00
bucket.Methods(http.MethodPost).Path("/{object:.+}").HandlerFunc(
maxClients(collectAPIStats("selectobjectcontent", httpTraceHdrs(api.SelectObjectContentHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("select", "").Queries("select-type", "2")
Add object retention at the per object (#8528) level - this PR builds on #8120 which added PutBucketObjectLockConfiguration and GetBucketObjectLockConfiguration APIS This PR implements PutObjectRetention, GetObjectRetention API and enhances PUT and GET API operations to display governance metadata if permissions allow.
2019-11-21 05:18:09 +08:00
// GetObjectRetention
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
2020-03-25 03:43:40 +08:00
bucket.Methods(http.MethodGet).Path("/{object:.+}").HandlerFunc(
maxClients(collectAPIStats("getobjectretention", httpTraceAll(api.GetObjectRetentionHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("retention", "")
Add support for object locking with legal hold. (#8634)
2020-01-17 07:41:56 +08:00
// GetObjectLegalHold
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
2020-03-25 03:43:40 +08:00
bucket.Methods(http.MethodGet).Path("/{object:.+}").HandlerFunc(
maxClients(collectAPIStats("getobjectlegalhold", httpTraceAll(api.GetObjectLegalHoldHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("legal-hold", "")
Virtual host style S3 requests (#5095)
2017-11-15 08:56:24 +08:00
// GetObject
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
2020-03-25 03:43:40 +08:00
bucket.Methods(http.MethodGet).Path("/{object:.+}").HandlerFunc(
maxClients(collectAPIStats("getobject", httpTraceHdrs(api.GetObjectHandler)), enabled, requestsMaxCh, requestsDeadline))
Virtual host style S3 requests (#5095)
2017-11-15 08:56:24 +08:00
// CopyObject
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
2020-03-25 03:43:40 +08:00
bucket.Methods(http.MethodPut).Path("/{object:.+}").HeadersRegexp(xhttp.AmzCopySource, ".*?(\\/|%2F).*?").HandlerFunc(maxClients(collectAPIStats("copyobject", httpTraceAll(api.CopyObjectHandler)), enabled, requestsMaxCh, requestsDeadline))
Add object retention at the per object (#8528) level - this PR builds on #8120 which added PutBucketObjectLockConfiguration and GetBucketObjectLockConfiguration APIS This PR implements PutObjectRetention, GetObjectRetention API and enhances PUT and GET API operations to display governance metadata if permissions allow.
2019-11-21 05:18:09 +08:00
// PutObjectRetention
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
2020-03-25 03:43:40 +08:00
bucket.Methods(http.MethodPut).Path("/{object:.+}").HandlerFunc(
maxClients(collectAPIStats("putobjectretention", httpTraceAll(api.PutObjectRetentionHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("retention", "")
Add support for object locking with legal hold. (#8634)
2020-01-17 07:41:56 +08:00
// PutObjectLegalHold
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
2020-03-25 03:43:40 +08:00
bucket.Methods(http.MethodPut).Path("/{object:.+}").HandlerFunc(
maxClients(collectAPIStats("putobjectlegalhold", httpTraceAll(api.PutObjectLegalHoldHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("legal-hold", "")
Add support for object locking with legal hold. (#8634)
2020-01-17 07:41:56 +08:00
Virtual host style S3 requests (#5095)
2017-11-15 08:56:24 +08:00
// PutObject
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
2020-03-25 03:43:40 +08:00
bucket.Methods(http.MethodPut).Path("/{object:.+}").HandlerFunc(
maxClients(collectAPIStats("putobject", httpTraceHdrs(api.PutObjectHandler)), enabled, requestsMaxCh, requestsDeadline))
Virtual host style S3 requests (#5095)
2017-11-15 08:56:24 +08:00
// DeleteObject
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
2020-03-25 03:43:40 +08:00
bucket.Methods(http.MethodDelete).Path("/{object:.+}").HandlerFunc(
maxClients(collectAPIStats("deleteobject", httpTraceAll(api.DeleteObjectHandler)), enabled, requestsMaxCh, requestsDeadline))
routers: Move API and Web routers into their own files. This is done to ensure we have a clean way to add new routers such as - diskRouter - configRouter - lockRouter
2016-03-28 03:37:21 +08:00
Virtual host style S3 requests (#5095)
2017-11-15 08:56:24 +08:00
/// Bucket operations
// GetBucketLocation
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
2020-03-25 03:43:40 +08:00
bucket.Methods(http.MethodGet).HandlerFunc(
maxClients(collectAPIStats("getbucketlocation", httpTraceAll(api.GetBucketLocationHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("location", "")
Virtual host style S3 requests (#5095)
2017-11-15 08:56:24 +08:00
// GetBucketPolicy
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
2020-03-25 03:43:40 +08:00
bucket.Methods(http.MethodGet).HandlerFunc(
maxClients(collectAPIStats("getbucketpolicy", httpTraceAll(api.GetBucketPolicyHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("policy", "")
Add initial support for bucket lifecycle (#7563) This PR is based off @sinhaashish's PR for object lifecycle management, which includes support only for, - Expiration of object - Filter using object prefix (_not_ object tags) N B the code for actual expiration of objects will be included in a subsequent PR.
2019-07-20 04:20:33 +08:00
// GetBucketLifecycle
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
2020-03-25 03:43:40 +08:00
bucket.Methods(http.MethodGet).HandlerFunc(
maxClients(collectAPIStats("getbucketlifecycle", httpTraceAll(api.GetBucketLifecycleHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("lifecycle", "")
Add support for bucket encryption feature (#8890) - pkg/bucket/encryption provides support for handling bucket encryption configuration - changes under cmd/ provide support for AES256 algorithm only Co-Authored-By: Poorna <poornas@users.noreply.github.com> Co-authored-by: Harshavardhana <harsha@minio.io>
2020-02-05 17:42:34 +08:00
// GetBucketEncryption
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
2020-03-25 03:43:40 +08:00
bucket.Methods(http.MethodGet).HandlerFunc(
maxClients(collectAPIStats("getbucketencryption", httpTraceAll(api.GetBucketEncryptionHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("encryption", "")
Implement a dummy GetBucketACL for legacy reasons (#5906) GetBucketACL call returns empty for all GET in ACL requests, the primary purpose of this PR is to provide legacy API support for legacy applications. Fixes #5706
2018-05-10 12:02:26 +08:00
Implements dummy tagging handlers for Terraform (#7040)
2019-02-09 08:18:13 +08:00
// Dummy Bucket Calls
Implement a dummy GetBucketACL for legacy reasons (#5906) GetBucketACL call returns empty for all GET in ACL requests, the primary purpose of this PR is to provide legacy API support for legacy applications. Fixes #5706
2018-05-10 12:02:26 +08:00
// GetBucketACL -- this is a dummy call.
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
2020-03-25 03:43:40 +08:00
bucket.Methods(http.MethodGet).HandlerFunc(
maxClients(collectAPIStats("getbucketacl", httpTraceAll(api.GetBucketACLHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("acl", "")
acl: Support PUT calls with success for 'private' ACL's (#9000) Add dummy calls which respond success when ACL's are set to be private and fails, if user tries to change them from their default 'private' Some applications such as nuxeo may have an unnecessary requirement for this operation, we support this anyways such that don't have to fully implement the functionality just that we can respond with success for default ACLs
2020-02-16 14:07:52 +08:00
// PutBucketACL -- this is a dummy call.
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
2020-03-25 03:43:40 +08:00
bucket.Methods(http.MethodPut).HandlerFunc(
maxClients(collectAPIStats("putbucketacl", httpTraceAll(api.PutBucketACLHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("acl", "")
Implements dummy tagging handlers for Terraform (#7040)
2019-02-09 08:18:13 +08:00
// GetBucketCors - this is a dummy call.
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
2020-03-25 03:43:40 +08:00
bucket.Methods(http.MethodGet).HandlerFunc(
maxClients(collectAPIStats("getbucketcors", httpTraceAll(api.GetBucketCorsHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("cors", "")
Implements dummy tagging handlers for Terraform (#7040)
2019-02-09 08:18:13 +08:00
// GetBucketWebsiteHandler - this is a dummy call.
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
2020-03-25 03:43:40 +08:00
bucket.Methods(http.MethodGet).HandlerFunc(
maxClients(collectAPIStats("getbucketwebsite", httpTraceAll(api.GetBucketWebsiteHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("website", "")
Implements dummy tagging handlers for Terraform (#7040)
2019-02-09 08:18:13 +08:00
// GetBucketAccelerateHandler - this is a dummy call.
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
2020-03-25 03:43:40 +08:00
bucket.Methods(http.MethodGet).HandlerFunc(
maxClients(collectAPIStats("getbucketaccelerate", httpTraceAll(api.GetBucketAccelerateHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("accelerate", "")
Implements dummy tagging handlers for Terraform (#7040)
2019-02-09 08:18:13 +08:00
// GetBucketRequestPaymentHandler - this is a dummy call.
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
2020-03-25 03:43:40 +08:00
bucket.Methods(http.MethodGet).HandlerFunc(
maxClients(collectAPIStats("getbucketrequestpayment", httpTraceAll(api.GetBucketRequestPaymentHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("requestPayment", "")
Implements dummy tagging handlers for Terraform (#7040)
2019-02-09 08:18:13 +08:00
// GetBucketLoggingHandler - this is a dummy call.
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
2020-03-25 03:43:40 +08:00
bucket.Methods(http.MethodGet).HandlerFunc(
maxClients(collectAPIStats("getbucketlogging", httpTraceAll(api.GetBucketLoggingHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("logging", "")
Implements dummy tagging handlers for Terraform (#7040)
2019-02-09 08:18:13 +08:00
// GetBucketLifecycleHandler - this is a dummy call.
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
2020-03-25 03:43:40 +08:00
bucket.Methods(http.MethodGet).HandlerFunc(
maxClients(collectAPIStats("getbucketlifecycle", httpTraceAll(api.GetBucketLifecycleHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("lifecycle", "")
Implements dummy tagging handlers for Terraform (#7040)
2019-02-09 08:18:13 +08:00
// GetBucketReplicationHandler - this is a dummy call.
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
2020-03-25 03:43:40 +08:00
bucket.Methods(http.MethodGet).HandlerFunc(
maxClients(collectAPIStats("getbucketreplication", httpTraceAll(api.GetBucketReplicationHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("replication", "")
Implements dummy tagging handlers for Terraform (#7040)
2019-02-09 08:18:13 +08:00
// GetBucketTaggingHandler - this is a dummy call.
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
2020-03-25 03:43:40 +08:00
bucket.Methods(http.MethodGet).HandlerFunc(
maxClients(collectAPIStats("getbuckettagging", httpTraceAll(api.GetBucketTaggingHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("tagging", "")
Implements dummy tagging handlers for Terraform (#7040)
2019-02-09 08:18:13 +08:00
//DeleteBucketWebsiteHandler
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
2020-03-25 03:43:40 +08:00
bucket.Methods(http.MethodDelete).HandlerFunc(
maxClients(collectAPIStats("deletebucketwebsite", httpTraceAll(api.DeleteBucketWebsiteHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("website", "")
Implements dummy tagging handlers for Terraform (#7040)
2019-02-09 08:18:13 +08:00
// DeleteBucketTaggingHandler
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
2020-03-25 03:43:40 +08:00
bucket.Methods(http.MethodDelete).HandlerFunc(
maxClients(collectAPIStats("deletebuckettagging", httpTraceAll(api.DeleteBucketTaggingHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("tagging", "")
Implement a dummy GetBucketACL for legacy reasons (#5906) GetBucketACL call returns empty for all GET in ACL requests, the primary purpose of this PR is to provide legacy API support for legacy applications. Fixes #5706
2018-05-10 12:02:26 +08:00
Add Get/Put Bucket Lock Configuration API support (#8120) This feature implements [PUT Bucket object lock configuration][1] and [GET Bucket object lock configuration][2]. After object lock configuration is set, existing and new objects are set to WORM for specified duration. Currently Governance mode works exactly like Compliance mode. Fixes #8101 [1] https://docs.aws.amazon.com/AmazonS3/latest/API/RESTBucketPUTObjectLockConfiguration.html [2] https://docs.aws.amazon.com/AmazonS3/latest/API/RESTBucketGETObjectLockConfiguration.html
2019-11-13 06:50:18 +08:00
// GetBucketObjectLockConfig
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
2020-03-25 03:43:40 +08:00
bucket.Methods(http.MethodGet).HandlerFunc(
maxClients(collectAPIStats("getbucketobjectlockconfiguration", httpTraceAll(api.GetBucketObjectLockConfigHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("object-lock", "")
Add Get/Put Bucket Lock Configuration API support (#8120) This feature implements [PUT Bucket object lock configuration][1] and [GET Bucket object lock configuration][2]. After object lock configuration is set, existing and new objects are set to WORM for specified duration. Currently Governance mode works exactly like Compliance mode. Fixes #8101 [1] https://docs.aws.amazon.com/AmazonS3/latest/API/RESTBucketPUTObjectLockConfiguration.html [2] https://docs.aws.amazon.com/AmazonS3/latest/API/RESTBucketGETObjectLockConfiguration.html
2019-11-13 06:50:18 +08:00
// GetBucketVersioning
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
2020-03-25 03:43:40 +08:00
bucket.Methods(http.MethodGet).HandlerFunc(
maxClients(collectAPIStats("getbucketversioning", httpTraceAll(api.GetBucketVersioningHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("versioning", "")
Virtual host style S3 requests (#5095)
2017-11-15 08:56:24 +08:00
// GetBucketNotification
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
2020-03-25 03:43:40 +08:00
bucket.Methods(http.MethodGet).HandlerFunc(
maxClients(collectAPIStats("getbucketnotification", httpTraceAll(api.GetBucketNotificationHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("notification", "")
Virtual host style S3 requests (#5095)
2017-11-15 08:56:24 +08:00
// ListenBucketNotification
The prometheus metrics refractoring (#8003) The measures are consolidated to the following metrics - `disk_storage_used` : Disk space used by the disk. - `disk_storage_available`: Available disk space left on the disk. - `disk_storage_total`: Total disk space on the disk. - `disks_offline`: Total number of offline disks in current MinIO instance. - `disks_total`: Total number of disks in current MinIO instance. - `s3_requests_total`: Total number of s3 requests in current MinIO instance. - `s3_errors_total`: Total number of errors in s3 requests in current MinIO instance. - `s3_requests_current`: Total number of active s3 requests in current MinIO instance. - `internode_rx_bytes_total`: Total number of internode bytes received by current MinIO server instance. - `internode_tx_bytes_total`: Total number of bytes sent to the other nodes by current MinIO server instance. - `s3_rx_bytes_total`: Total number of s3 bytes received by current MinIO server instance. - `s3_tx_bytes_total`: Total number of s3 bytes sent by current MinIO server instance. - `minio_version_info`: Current MinIO version with commit-id. - `s3_ttfb_seconds_bucket`: Histogram that holds the latency information of the requests. And this PR also modifies the current StorageInfo queries - Decouples StorageInfo from ServerInfo . - StorageInfo is enhanced to give endpoint information. NOTE: ADMIN API VERSION IS BUMPED UP IN THIS PR Fixes #7873
2019-10-23 12:01:14 +08:00
bucket.Methods(http.MethodGet).HandlerFunc(collectAPIStats("listenbucketnotification", httpTraceAll(api.ListenBucketNotificationHandler))).Queries("events", "{events:.*}")
Virtual host style S3 requests (#5095)
2017-11-15 08:56:24 +08:00
// ListMultipartUploads
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
2020-03-25 03:43:40 +08:00
bucket.Methods(http.MethodGet).HandlerFunc(
maxClients(collectAPIStats("listmultipartuploads", httpTraceAll(api.ListMultipartUploadsHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("uploads", "")
Enhance ListObjectsV2 API to return UserDefined metadata (#8539)
2019-11-21 17:54:49 +08:00
// ListObjectsV2M
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
2020-03-25 03:43:40 +08:00
bucket.Methods(http.MethodGet).HandlerFunc(
maxClients(collectAPIStats("listobjectsv2M", httpTraceAll(api.ListObjectsV2MHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("list-type", "2", "metadata", "true")
Virtual host style S3 requests (#5095)
2017-11-15 08:56:24 +08:00
// ListObjectsV2
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
2020-03-25 03:43:40 +08:00
bucket.Methods(http.MethodGet).HandlerFunc(
maxClients(collectAPIStats("listobjectsv2", httpTraceAll(api.ListObjectsV2Handler)), enabled, requestsMaxCh, requestsDeadline)).Queries("list-type", "2")
Add listBucketObjectsVersions implementation (#8093) This API implementation simply behaves like listObjects() but returns back single version for each object, this implementation should be considered dummy it is only meant for some applications which rely on this.
2019-08-20 05:02:54 +08:00
// ListBucketVersions
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
2020-03-25 03:43:40 +08:00
bucket.Methods(http.MethodGet).HandlerFunc(
maxClients(collectAPIStats("listbucketversions", httpTraceAll(api.ListBucketObjectVersionsHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("versions", "")
Virtual host style S3 requests (#5095)
2017-11-15 08:56:24 +08:00
// ListObjectsV1 (Legacy)
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
2020-03-25 03:43:40 +08:00
bucket.Methods(http.MethodGet).HandlerFunc(
maxClients(collectAPIStats("listobjectsv1", httpTraceAll(api.ListObjectsV1Handler)), enabled, requestsMaxCh, requestsDeadline))
Add initial support for bucket lifecycle (#7563) This PR is based off @sinhaashish's PR for object lifecycle management, which includes support only for, - Expiration of object - Filter using object prefix (_not_ object tags) N B the code for actual expiration of objects will be included in a subsequent PR.
2019-07-20 04:20:33 +08:00
// PutBucketLifecycle
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
2020-03-25 03:43:40 +08:00
bucket.Methods(http.MethodPut).HandlerFunc(
maxClients(collectAPIStats("putbucketlifecycle", httpTraceAll(api.PutBucketLifecycleHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("lifecycle", "")
Add support for bucket encryption feature (#8890) - pkg/bucket/encryption provides support for handling bucket encryption configuration - changes under cmd/ provide support for AES256 algorithm only Co-Authored-By: Poorna <poornas@users.noreply.github.com> Co-authored-by: Harshavardhana <harsha@minio.io>
2020-02-05 17:42:34 +08:00
// PutBucketEncryption
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
2020-03-25 03:43:40 +08:00
bucket.Methods(http.MethodPut).HandlerFunc(
maxClients(collectAPIStats("putbucketencryption", httpTraceAll(api.PutBucketEncryptionHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("encryption", "")
Add support for bucket encryption feature (#8890) - pkg/bucket/encryption provides support for handling bucket encryption configuration - changes under cmd/ provide support for AES256 algorithm only Co-Authored-By: Poorna <poornas@users.noreply.github.com> Co-authored-by: Harshavardhana <harsha@minio.io>
2020-02-05 17:42:34 +08:00
Virtual host style S3 requests (#5095)
2017-11-15 08:56:24 +08:00
// PutBucketPolicy
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
2020-03-25 03:43:40 +08:00
bucket.Methods(http.MethodPut).HandlerFunc(
maxClients(collectAPIStats("putbucketpolicy", httpTraceAll(api.PutBucketPolicyHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("policy", "")
Add initial support for bucket lifecycle (#7563) This PR is based off @sinhaashish's PR for object lifecycle management, which includes support only for, - Expiration of object - Filter using object prefix (_not_ object tags) N B the code for actual expiration of objects will be included in a subsequent PR.
2019-07-20 04:20:33 +08:00
Add Get/Put Bucket Lock Configuration API support (#8120) This feature implements [PUT Bucket object lock configuration][1] and [GET Bucket object lock configuration][2]. After object lock configuration is set, existing and new objects are set to WORM for specified duration. Currently Governance mode works exactly like Compliance mode. Fixes #8101 [1] https://docs.aws.amazon.com/AmazonS3/latest/API/RESTBucketPUTObjectLockConfiguration.html [2] https://docs.aws.amazon.com/AmazonS3/latest/API/RESTBucketGETObjectLockConfiguration.html
2019-11-13 06:50:18 +08:00
// PutBucketObjectLockConfig
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
2020-03-25 03:43:40 +08:00
bucket.Methods(http.MethodPut).HandlerFunc(
maxClients(collectAPIStats("putbucketobjectlockconfig", httpTraceAll(api.PutBucketObjectLockConfigHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("object-lock", "")
Add Get/Put Bucket Lock Configuration API support (#8120) This feature implements [PUT Bucket object lock configuration][1] and [GET Bucket object lock configuration][2]. After object lock configuration is set, existing and new objects are set to WORM for specified duration. Currently Governance mode works exactly like Compliance mode. Fixes #8101 [1] https://docs.aws.amazon.com/AmazonS3/latest/API/RESTBucketPUTObjectLockConfiguration.html [2] https://docs.aws.amazon.com/AmazonS3/latest/API/RESTBucketGETObjectLockConfiguration.html
2019-11-13 06:50:18 +08:00
// PutBucketVersioning
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
2020-03-25 03:43:40 +08:00
bucket.Methods(http.MethodPut).HandlerFunc(
maxClients(collectAPIStats("putbucketversioning", httpTraceAll(api.PutBucketVersioningHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("versioning", "")
Virtual host style S3 requests (#5095)
2017-11-15 08:56:24 +08:00
// PutBucketNotification
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
2020-03-25 03:43:40 +08:00
bucket.Methods(http.MethodPut).HandlerFunc(
maxClients(collectAPIStats("putbucketnotification", httpTraceAll(api.PutBucketNotificationHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("notification", "")
Virtual host style S3 requests (#5095)
2017-11-15 08:56:24 +08:00
// PutBucket
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
2020-03-25 03:43:40 +08:00
bucket.Methods(http.MethodPut).HandlerFunc(
maxClients(collectAPIStats("putbucket", httpTraceAll(api.PutBucketHandler)), enabled, requestsMaxCh, requestsDeadline))
Virtual host style S3 requests (#5095)
2017-11-15 08:56:24 +08:00
// HeadBucket
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
2020-03-25 03:43:40 +08:00
bucket.Methods(http.MethodHead).HandlerFunc(
maxClients(collectAPIStats("headbucket", httpTraceAll(api.HeadBucketHandler)), enabled, requestsMaxCh, requestsDeadline))
Virtual host style S3 requests (#5095)
2017-11-15 08:56:24 +08:00
// PostPolicy
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
2020-03-25 03:43:40 +08:00
bucket.Methods(http.MethodPost).HeadersRegexp(xhttp.ContentType, "multipart/form-data*").HandlerFunc(
maxClients(collectAPIStats("postpolicybucket", httpTraceHdrs(api.PostPolicyBucketHandler)), enabled, requestsMaxCh, requestsDeadline))
Virtual host style S3 requests (#5095)
2017-11-15 08:56:24 +08:00
// DeleteMultipleObjects
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
2020-03-25 03:43:40 +08:00
bucket.Methods(http.MethodPost).HandlerFunc(
maxClients(collectAPIStats("deletemultipleobjects", httpTraceAll(api.DeleteMultipleObjectsHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("delete", "")
Virtual host style S3 requests (#5095)
2017-11-15 08:56:24 +08:00
// DeleteBucketPolicy
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
2020-03-25 03:43:40 +08:00
bucket.Methods(http.MethodDelete).HandlerFunc(
maxClients(collectAPIStats("deletebucketpolicy", httpTraceAll(api.DeleteBucketPolicyHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("policy", "")
Add initial support for bucket lifecycle (#7563) This PR is based off @sinhaashish's PR for object lifecycle management, which includes support only for, - Expiration of object - Filter using object prefix (_not_ object tags) N B the code for actual expiration of objects will be included in a subsequent PR.
2019-07-20 04:20:33 +08:00
// DeleteBucketLifecycle
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
2020-03-25 03:43:40 +08:00
bucket.Methods(http.MethodDelete).HandlerFunc(
maxClients(collectAPIStats("deletebucketlifecycle", httpTraceAll(api.DeleteBucketLifecycleHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("lifecycle", "")
Add support for bucket encryption feature (#8890) - pkg/bucket/encryption provides support for handling bucket encryption configuration - changes under cmd/ provide support for AES256 algorithm only Co-Authored-By: Poorna <poornas@users.noreply.github.com> Co-authored-by: Harshavardhana <harsha@minio.io>
2020-02-05 17:42:34 +08:00
// DeleteBucketEncryption
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
2020-03-25 03:43:40 +08:00
bucket.Methods(http.MethodDelete).HandlerFunc(
maxClients(collectAPIStats("deletebucketencryption", httpTraceAll(api.DeleteBucketEncryptionHandler)), enabled, requestsMaxCh, requestsDeadline)).Queries("encryption", "")
Virtual host style S3 requests (#5095)
2017-11-15 08:56:24 +08:00
// DeleteBucket
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
2020-03-25 03:43:40 +08:00
bucket.Methods(http.MethodDelete).HandlerFunc(
maxClients(collectAPIStats("deletebucket", httpTraceAll(api.DeleteBucketHandler)), enabled, requestsMaxCh, requestsDeadline))
Virtual host style S3 requests (#5095)
2017-11-15 08:56:24 +08:00
}
routers: Move API and Web routers into their own files. This is done to ensure we have a clean way to add new routers such as - diskRouter - configRouter - lockRouter
2016-03-28 03:37:21 +08:00
/// Root operation
// ListBuckets
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
2020-03-25 03:43:40 +08:00
apiRouter.Methods(http.MethodGet).Path(SlashSeparator).HandlerFunc(
maxClients(collectAPIStats("listbuckets", httpTraceAll(api.ListBucketsHandler)), enabled, requestsMaxCh, requestsDeadline))
Virtual host style S3 requests (#5095)
2017-11-15 08:56:24 +08:00
Return appropriate errors API versions changes across REST APIs (#8480) This PR adds code to appropriately handle versioning issues that come up quite constantly across our API changes. Currently we were also routing our requests wrong which sort of made it harder to write a consistent error handling code to appropriately reject or honor requests. This PR potentially fixes issues - old mc is used against new minio release which is incompatible returns an appropriate for client action. - any older servers talking to each other, report appropriate error - incompatible peer servers should report error and reject the calls with appropriate error
2019-11-05 01:30:59 +08:00
// If none of the routes match add default error handler routes
apiRouter.NotFoundHandler = http.HandlerFunc(collectAPIStats("notfound", httpTraceAll(errorResponseHandler)))
apiRouter.MethodNotAllowedHandler = http.HandlerFunc(collectAPIStats("methodnotallowed", httpTraceAll(errorResponseHandler)))
routers: Move API and Web routers into their own files. This is done to ensure we have a clean way to add new routers such as - diskRouter - configRouter - lockRouter
2016-03-28 03:37:21 +08:00
}