root
/
minio
mirror of https://github.com/minio/minio.git
1
0
Fork 0
Code Issues Actions 1 Packages Projects Releases Wiki Activity
minio/cmd/api-router.go

539 lines
24 KiB
Go
Raw Normal View History

update license change for MinIO Signed-off-by: Harshavardhana <harsha@minio.io>
2021-04-19 03:41:13 +08:00
// Copyright (c) 2015-2021 MinIO, Inc.
//
// This file is part of MinIO Object Storage stack
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// This program is distributed in the hope that it will be useful
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU Affero General Public License for more details.
//
// You should have received a copy of the GNU Affero General Public License
// along with this program. If not, see <http://www.gnu.org/licenses/>.
routers: Move API and Web routers into their own files. This is done to ensure we have a clean way to add new routers such as - diskRouter - configRouter - lockRouter
2016-03-28 03:37:21 +08:00
server: Move all the top level files into cmd folder. (#2490) This change brings a change which was done for the 'mc' package to allow for clean repo and have a cleaner github drop in experience.
2016-08-19 07:23:42 +08:00
package cmd
routers: Move API and Web routers into their own files. This is done to ensure we have a clean way to add new routers such as - diskRouter - configRouter - lockRouter
2016-03-28 03:37:21 +08:00
Add disk based edge caching support. (#5182) This PR adds disk based edge caching support for minio server. Cache settings can be configured in config.json to take list of disk drives, cache expiry in days and file patterns to exclude from cache or via environment variables MINIO_CACHE_DRIVES, MINIO_CACHE_EXCLUDE and MINIO_CACHE_EXPIRY Design assumes that Atime support is enabled and the list of cache drives is fixed. - Objects are cached on both GET and PUT/POST operations. - Expiry is used as hint to evict older entries from cache, or if 80% of cache capacity is filled. - When object storage backend is down, GET, LIST and HEAD operations fetch object seamlessly from cache. Current Limitations - Bucket policies are not cached, so anonymous operations are not supported in offline mode. - Objects are distributed using deterministic hashing among list of cache drives specified.If one or more drives go offline, or cache drive configuration is altered - performance could degrade to linear lookup. Fixes #4026
2018-03-29 05:14:06 +08:00
import (
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-13 01:49:50 +08:00
"compress/gzip"
skip subdomain from bucket DNS which start with `minio.domain` (#10390) extend host matcher to reject the host match
2020-09-10 00:57:37 +08:00
"net"
Add disk based edge caching support. (#5182) This PR adds disk based edge caching support for minio server. Cache settings can be configured in config.json to take list of disk drives, cache expiry in days and file patterns to exclude from cache or via environment variables MINIO_CACHE_DRIVES, MINIO_CACHE_EXCLUDE and MINIO_CACHE_EXPIRY Design assumes that Atime support is enabled and the list of cache drives is fixed. - Objects are cached on both GET and PUT/POST operations. - Expiry is used as hint to evict older entries from cache, or if 80% of cache capacity is filled. - When object storage backend is down, GET, LIST and HEAD operations fetch object seamlessly from cache. Current Limitations - Bucket policies are not cached, so anonymous operations are not supported in offline mode. - Objects are distributed using deterministic hashing among list of cache drives specified.If one or more drives go offline, or cache drive configuration is altered - performance could degrade to linear lookup. Fixes #4026
2018-03-29 05:14:06 +08:00
"net/http"
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-13 01:49:50 +08:00
"github.com/klauspost/compress/gzhttp"
do not flush if Write() failed (#13597) - Go might reset the internal http.ResponseWriter() to `nil` after Write() failure if the go-routine has returned, do not flush() such scenarios and avoid spurious flushes() as returning handlers always flush. - fix some racy tests with the console - avoid ticker leaks in certain situations
2021-11-19 09:19:58 +08:00
"github.com/minio/console/restapi"
rename all remaining packages to internal/ (#12418) This is to ensure that there are no projects that try to import `minio/minio/pkg` into their own repo. Any such common packages should go to `https://github.com/minio/pkg`
2021-06-02 05:59:40 +08:00
xhttp "github.com/minio/minio/internal/http"
Simplify gzhttp wrapper (#12912) The wrapper now accepts interfaces so we don't need a wrapper for that any more.
2021-08-10 03:45:59 +08:00
"github.com/minio/minio/internal/logger"
migrate to minio/mux from gorilla/mux (#16456)
2023-01-23 19:12:47 +08:00
"github.com/minio/mux"
Update to minio/pkg/v2 (#17967)
2023-09-05 03:57:37 +08:00
"github.com/minio/pkg/v2/wildcard"
fix: cors handling after gorilla mux update (#9980) fixes #9979
2020-07-07 11:55:19 +08:00
"github.com/rs/cors"
Add disk based edge caching support. (#5182) This PR adds disk based edge caching support for minio server. Cache settings can be configured in config.json to take list of disk drives, cache expiry in days and file patterns to exclude from cache or via environment variables MINIO_CACHE_DRIVES, MINIO_CACHE_EXCLUDE and MINIO_CACHE_EXPIRY Design assumes that Atime support is enabled and the list of cache drives is fixed. - Objects are cached on both GET and PUT/POST operations. - Expiry is used as hint to evict older entries from cache, or if 80% of cache capacity is filled. - When object storage backend is down, GET, LIST and HEAD operations fetch object seamlessly from cache. Current Limitations - Bucket policies are not cached, so anonymous operations are not supported in offline mode. - Objects are distributed using deterministic hashing among list of cache drives specified.If one or more drives go offline, or cache drive configuration is altered - performance could degrade to linear lookup. Fixes #4026
2018-03-29 05:14:06 +08:00
)
routers: Move API and Web routers into their own files. This is done to ensure we have a clean way to add new routers such as - diskRouter - configRouter - lockRouter
2016-03-28 03:37:21 +08:00
Enable build verification with race (#8583)
2019-12-03 07:54:26 +08:00
func newHTTPServerFn() *xhttp.Server {
feat: Small object optimization read data in single bulk call (#11207)
2021-01-04 03:27:57 +08:00
globalObjLayerMutex.RLock()
defer globalObjLayerMutex.RUnlock()
Enable build verification with race (#8583)
2019-12-03 07:54:26 +08:00
return globalHTTPServer
}
feat: Small object optimization read data in single bulk call (#11207)
2021-01-04 03:27:57 +08:00
func setHTTPServer(h *xhttp.Server) {
Bring in safe mode support (#8478) This PR refactors object layer handling such that upon failure in sub-system initialization server reaches a stage of safe-mode operation wherein only certain API operations are enabled and available. This allows for fixing many scenarios such as - incorrect configuration in vault, etcd, notification targets - missing files, incomplete config migrations unable to read encrypted content etc - any other issues related to notification, policies, lifecycle etc
2019-11-10 01:27:23 +08:00
globalObjLayerMutex.Lock()
feat: Small object optimization read data in single bulk call (#11207)
2021-01-04 03:27:57 +08:00
globalHTTPServer = h
globalObjLayerMutex.Unlock()
}
do not flush if Write() failed (#13597) - Go might reset the internal http.ResponseWriter() to `nil` after Write() failure if the go-routine has returned, do not flush() such scenarios and avoid spurious flushes() as returning handlers always flush. - fix some racy tests with the console - avoid ticker leaks in certain situations
2021-11-19 09:19:58 +08:00
func newConsoleServerFn() *restapi.Server {
globalObjLayerMutex.RLock()
defer globalObjLayerMutex.RUnlock()
return globalConsoleSrv
}
func setConsoleSrv(srv *restapi.Server) {
globalObjLayerMutex.Lock()
globalConsoleSrv = srv
globalObjLayerMutex.Unlock()
}
feat: Small object optimization read data in single bulk call (#11207)
2021-01-04 03:27:57 +08:00
func newObjectLayerFn() ObjectLayer {
globalObjLayerMutex.RLock()
defer globalObjLayerMutex.RUnlock()
Bring in safe mode support (#8478) This PR refactors object layer handling such that upon failure in sub-system initialization server reaches a stage of safe-mode operation wherein only certain API operations are enabled and available. This allows for fixing many scenarios such as - incorrect configuration in vault, etcd, notification targets - missing files, incomplete config migrations unable to read encrypted content etc - any other issues related to notification, policies, lifecycle etc
2019-11-10 01:27:23 +08:00
return globalObjectAPI
}
ListObjects Metadata Caching (#10648) Design: https://gist.github.com/klauspost/025c09b48ed4a1293c917cecfabdf21c Gist of improvements: * Cross-server caching and listing will use the same data across servers and requests. * Lists can be arbitrarily resumed at a constant speed. * Metadata for all files scanned is stored for streaming retrieval. * The existing bloom filters controlled by the crawler is used for validating caches. * Concurrent requests for the same data (or parts of it) will not spawn additional walkers. * Listing a subdirectory of an existing recursive cache will use the cache. * All listing operations are fully streamable so the number of objects in a bucket no longer dictates the amount of memory. * Listings can be handled by any server within the cluster. * Caches are cleaned up when out of date or superseded by a more recent one.
2020-10-29 00:18:35 +08:00
func setObjectLayer(o ObjectLayer) {
globalObjLayerMutex.Lock()
globalObjectAPI = o
globalObjLayerMutex.Unlock()
}
storage/server/client: Enable storage server, enable client storage.
2016-04-13 03:45:15 +08:00
// objectAPIHandler implements and provides http handlers for S3 API.
type objectAPIHandlers struct {
distribute: Make server work with multiple remote disks This change initializes rpc servers associated with disks that are local. It makes object layer initialization on demand, namely on the first request to the object layer. Also adds lock RPC service vendorized minio/dsync
2016-08-01 05:11:14 +08:00
ObjectAPI func() ObjectLayer
routers: Move API and Web routers into their own files. This is done to ensure we have a clean way to add new routers such as - diskRouter - configRouter - lockRouter
2016-03-28 03:37:21 +08:00
}
skip subdomain from bucket DNS which start with `minio.domain` (#10390) extend host matcher to reject the host match
2020-09-10 00:57:37 +08:00
// getHost tries its best to return the request host.
// According to section 14.23 of RFC 2616 the Host header
// can include the port number if the default value of 80 is not used.
func getHost(r *http.Request) string {
if r.URL.IsAbs() {
return r.URL.Host
}
return r.Host
}
fix: handle unsupported APIs more granularly (#11674)
2021-03-31 14:19:36 +08:00
func notImplementedHandler(w http.ResponseWriter, r *http.Request) {
feat: Deprecate embedded browser and import console (#12460) This feature also changes the default port where the browser is running, now the port has moved to 9001 and it can be configured with ``` --console-address ":9001" ```
2021-06-18 11:27:04 +08:00
writeErrorResponse(r.Context(), w, errorCodes.ToAPIErr(ErrNotImplemented), r.URL)
fix: handle unsupported APIs more granularly (#11674)
2021-03-31 14:19:36 +08:00
}
type rejectedAPI struct {
api string
methods []string
queries []string
path string
}
fix: muxing order for rejected APIs (#12321)
2021-05-20 00:21:34 +08:00
var rejectedObjAPIs = []rejectedAPI{
{
api: "torrent",
methods: []string{http.MethodPut, http.MethodDelete, http.MethodGet},
queries: []string{"torrent", ""},
path: "/{object:.+}",
},
{
api: "acl",
methods: []string{http.MethodDelete},
queries: []string{"acl", ""},
path: "/{object:.+}",
},
}
var rejectedBucketAPIs = []rejectedAPI{
fix: handle unsupported APIs more granularly (#11674)
2021-03-31 14:19:36 +08:00
{
api: "inventory",
methods: []string{http.MethodGet, http.MethodPut, http.MethodDelete},
queries: []string{"inventory", ""},
},
{
api: "cors",
methods: []string{http.MethodPut, http.MethodDelete},
queries: []string{"cors", ""},
},
{
api: "metrics",
methods: []string{http.MethodGet, http.MethodPut, http.MethodDelete},
queries: []string{"metrics", ""},
},
{
api: "website",
methods: []string{http.MethodPut},
queries: []string{"website", ""},
},
{
api: "logging",
methods: []string{http.MethodPut, http.MethodDelete},
queries: []string{"logging", ""},
},
{
api: "accelerate",
methods: []string{http.MethodPut, http.MethodDelete},
queries: []string{"accelerate", ""},
},
{
api: "requestPayment",
methods: []string{http.MethodPut, http.MethodDelete},
queries: []string{"requestPayment", ""},
},
{
api: "acl",
methods: []string{http.MethodDelete, http.MethodPut, http.MethodHead},
queries: []string{"acl", ""},
},
{
api: "publicAccessBlock",
methods: []string{http.MethodDelete, http.MethodPut, http.MethodGet},
queries: []string{"publicAccessBlock", ""},
},
{
api: "ownershipControls",
methods: []string{http.MethodDelete, http.MethodPut, http.MethodGet},
queries: []string{"ownershipControls", ""},
},
{
api: "intelligent-tiering",
methods: []string{http.MethodDelete, http.MethodPut, http.MethodGet},
queries: []string{"intelligent-tiering", ""},
},
{
api: "analytics",
methods: []string{http.MethodDelete, http.MethodPut, http.MethodGet},
queries: []string{"analytics", ""},
},
}
routers: Move API and Web routers into their own files. This is done to ensure we have a clean way to add new routers such as - diskRouter - configRouter - lockRouter
2016-03-28 03:37:21 +08:00
// registerAPIRouter - registers S3 compatible APIs.
fix: S3 gateway doesn't support full passthrough for encryption (#10484) The entire encryption layer is dependent on the fact that KMS should be configured for S3 encryption to work properly and we only support passing the headers as is to the backend for encryption only if KMS is configured. Make sure that this predictability is maintained, currently the code was allowing encryption to go through and fail at later to indicate that KMS was not configured. We should simply reply "NotImplemented" if KMS is not configured, this allows clients to simply proceed with their tests.
2020-09-16 04:57:15 +08:00
func registerAPIRouter(router *mux.Router) {
server: Add more elaborate startup messages. (#2731) These messages based on our prep stage during XL and prints more informative message regarding drive information. This change also does a much needed refactoring.
2016-10-06 03:48:07 +08:00
// Initialize API.
api := objectAPIHandlers{
Bring in safe mode support (#8478) This PR refactors object layer handling such that upon failure in sub-system initialization server reaches a stage of safe-mode operation wherein only certain API operations are enabled and available. This allows for fixing many scenarios such as - incorrect configuration in vault, etcd, notification targets - missing files, incomplete config migrations unable to read encrypted content etc - any other issues related to notification, policies, lifecycle etc
2019-11-10 01:27:23 +08:00
ObjectAPI: newObjectLayerFn,
server: Add more elaborate startup messages. (#2731) These messages based on our prep stage during XL and prints more informative message regarding drive information. This change also does a much needed refactoring.
2016-10-06 03:48:07 +08:00
}
routers: Move API and Web routers into their own files. This is done to ensure we have a clean way to add new routers such as - diskRouter - configRouter - lockRouter
2016-03-28 03:37:21 +08:00
// API Router
Use const slashSeparator instead of "/" everywhere (#8028)
2019-08-07 03:08:58 +08:00
apiRouter := router.PathPrefix(SlashSeparator).Subrouter()
skip subdomain from bucket DNS which start with `minio.domain` (#10390) extend host matcher to reject the host match
2020-09-10 00:57:37 +08:00
use package name correctly (#5827)
2018-04-22 10:23:54 +08:00
var routers []*mux.Router
Support multiple-domains in MINIO_DOMAIN (#7274) Fixes #7173
2019-02-23 11:18:01 +08:00
for _, domainName := range globalDomainNames {
skip subdomain from bucket DNS which start with `minio.domain` (#10390) extend host matcher to reject the host match
2020-09-10 00:57:37 +08:00
if IsKubernetes() {
routers = append(routers, apiRouter.MatcherFunc(func(r *http.Request, match *mux.RouteMatch) bool {
fix: host extraction without port
2020-09-24 03:10:09 +08:00
host, _, err := net.SplitHostPort(getHost(r))
if err != nil {
host = r.Host
}
skip subdomain from bucket DNS which start with `minio.domain` (#10390) extend host matcher to reject the host match
2020-09-10 00:57:37 +08:00
// Make sure to skip matching minio.<domain>` this is
// specifically meant for operator/k8s deployment
// The reason we need to skip this is for a special
// usecase where we need to make sure that
// minio.<namespace>.svc.<cluster_domain> is ignored
// by the bucketDNS style to ensure that path style
// is available and honored at this domain.
//
// All other `<bucket>.<namespace>.svc.<cluster_domain>`
// makes sure that buckets are routed through this matcher
// to match for `<bucket>`
return host != minioReservedBucket+"."+domainName
}).Host("{bucket:.+}."+domainName).Subrouter())
} else {
routers = append(routers, apiRouter.Host("{bucket:.+}."+domainName).Subrouter())
}
Virtual host style S3 requests (#5095)
2017-11-15 08:56:24 +08:00
}
routers = append(routers, apiRouter.PathPrefix("/{bucket}").Subrouter())
routers: Move API and Web routers into their own files. This is done to ensure we have a clean way to add new routers such as - diskRouter - configRouter - lockRouter
2016-03-28 03:37:21 +08:00
Simplify gzhttp wrapper (#12912) The wrapper now accepts interfaces so we don't need a wrapper for that any more.
2021-08-10 03:45:59 +08:00
gz, err := gzhttp.NewWrapper(gzhttp.MinSize(1000), gzhttp.CompressionLevel(gzip.BestSpeed))
if err != nil {
// Static params, so this is very unlikely.
logger.Fatal(err, "Unable to initialize server")
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-13 01:49:50 +08:00
}
fix: handle unsupported APIs more granularly (#11674)
2021-03-31 14:19:36 +08:00
for _, router := range routers {
fix: muxing order for rejected APIs (#12321)
2021-05-20 00:21:34 +08:00
// Register all rejected object APIs
for _, r := range rejectedObjAPIs {
t := router.Methods(r.methods...).
HandlerFunc(collectAPIStats(r.api, httpTraceAll(notImplementedHandler))).
Queries(r.queries...)
t.Path(r.path)
}
Virtual host style S3 requests (#5095)
2017-11-15 08:56:24 +08:00
// Object operations
// HeadObject
fix: handle unsupported APIs more granularly (#11674)
2021-03-31 14:19:36 +08:00
router.Methods(http.MethodHead).Path("/{object:.+}").HandlerFunc(
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-13 01:49:50 +08:00
collectAPIStats("headobject", maxClients(gz(httpTraceAll(api.HeadObjectHandler)))))
Virtual host style S3 requests (#5095)
2017-11-15 08:56:24 +08:00
// CopyObjectPart
fix: handle unsupported APIs more granularly (#11674)
2021-03-31 14:19:36 +08:00
router.Methods(http.MethodPut).Path("/{object:.+}").
skip subdomain from bucket DNS which start with `minio.domain` (#10390) extend host matcher to reject the host match
2020-09-10 00:57:37 +08:00
HeadersRegexp(xhttp.AmzCopySource, ".*?(\\/|%2F).*?").
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-13 01:49:50 +08:00
HandlerFunc(collectAPIStats("copyobjectpart", maxClients(gz(httpTraceAll(api.CopyObjectPartHandler))))).
Stricter partNumber checks (#17270) Fixes #17269
2023-05-24 23:00:47 +08:00
Queries("partNumber", "{partNumber:.*}", "uploadId", "{uploadId:.*}")
Virtual host style S3 requests (#5095)
2017-11-15 08:56:24 +08:00
// PutObjectPart
fix: handle unsupported APIs more granularly (#11674)
2021-03-31 14:19:36 +08:00
router.Methods(http.MethodPut).Path("/{object:.+}").HandlerFunc(
Stricter partNumber checks (#17270) Fixes #17269
2023-05-24 23:00:47 +08:00
collectAPIStats("putobjectpart", maxClients(gz(httpTraceHdrs(api.PutObjectPartHandler))))).Queries("partNumber", "{partNumber:.*}", "uploadId", "{uploadId:.*}")
Add listBucketObjectsVersions implementation (#8093) This API implementation simply behaves like listObjects() but returns back single version for each object, this implementation should be considered dummy it is only meant for some applications which rely on this.
2019-08-20 05:02:54 +08:00
// ListObjectParts
fix: handle unsupported APIs more granularly (#11674)
2021-03-31 14:19:36 +08:00
router.Methods(http.MethodGet).Path("/{object:.+}").HandlerFunc(
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-13 01:49:50 +08:00
collectAPIStats("listobjectparts", maxClients(gz(httpTraceAll(api.ListObjectPartsHandler))))).Queries("uploadId", "{uploadId:.*}")
Virtual host style S3 requests (#5095)
2017-11-15 08:56:24 +08:00
// CompleteMultipartUpload
fix: handle unsupported APIs more granularly (#11674)
2021-03-31 14:19:36 +08:00
router.Methods(http.MethodPost).Path("/{object:.+}").HandlerFunc(
fix(Router): typo: completemutipartupload to completemultipartupload (#13051)
2021-08-24 22:14:34 +08:00
collectAPIStats("completemultipartupload", maxClients(gz(httpTraceAll(api.CompleteMultipartUploadHandler))))).Queries("uploadId", "{uploadId:.*}")
Virtual host style S3 requests (#5095)
2017-11-15 08:56:24 +08:00
// NewMultipartUpload
fix: handle unsupported APIs more granularly (#11674)
2021-03-31 14:19:36 +08:00
router.Methods(http.MethodPost).Path("/{object:.+}").HandlerFunc(
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-13 01:49:50 +08:00
collectAPIStats("newmultipartupload", maxClients(gz(httpTraceAll(api.NewMultipartUploadHandler))))).Queries("uploads", "")
Virtual host style S3 requests (#5095)
2017-11-15 08:56:24 +08:00
// AbortMultipartUpload
fix: handle unsupported APIs more granularly (#11674)
2021-03-31 14:19:36 +08:00
router.Methods(http.MethodDelete).Path("/{object:.+}").HandlerFunc(
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-13 01:49:50 +08:00
collectAPIStats("abortmultipartupload", maxClients(gz(httpTraceAll(api.AbortMultipartUploadHandler))))).Queries("uploadId", "{uploadId:.*}")
Fix s3cmd issue with ACL handling (#5999) With the implementation of dummy GET ACL handlers, tools like s3cmd perform few operations which causes the ACL call to be invoked. Make sure that in our router configuration GET?acl comes before actual GET call to facilitate this dummy call.
2018-06-01 10:43:50 +08:00
// GetObjectACL - this is a dummy call.
fix: handle unsupported APIs more granularly (#11674)
2021-03-31 14:19:36 +08:00
router.Methods(http.MethodGet).Path("/{object:.+}").HandlerFunc(
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-13 01:49:50 +08:00
collectAPIStats("getobjectacl", maxClients(gz(httpTraceHdrs(api.GetObjectACLHandler))))).Queries("acl", "")
acl: Support PUT calls with success for 'private' ACL's (#9000) Add dummy calls which respond success when ACL's are set to be private and fails, if user tries to change them from their default 'private' Some applications such as nuxeo may have an unnecessary requirement for this operation, we support this anyways such that don't have to fully implement the functionality just that we can respond with success for default ACLs
2020-02-16 14:07:52 +08:00
// PutObjectACL - this is a dummy call.
fix: handle unsupported APIs more granularly (#11674)
2021-03-31 14:19:36 +08:00
router.Methods(http.MethodPut).Path("/{object:.+}").HandlerFunc(
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-13 01:49:50 +08:00
collectAPIStats("putobjectacl", maxClients(gz(httpTraceHdrs(api.PutObjectACLHandler))))).Queries("acl", "")
Add ObjectTagging Support (#8754) This PR adds support for AWS S3 ObjectTagging API as explained here https://docs.aws.amazon.com/AmazonS3/latest/dev/object-tagging.html
2020-01-21 00:45:59 +08:00
// GetObjectTagging
fix: handle unsupported APIs more granularly (#11674)
2021-03-31 14:19:36 +08:00
router.Methods(http.MethodGet).Path("/{object:.+}").HandlerFunc(
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-13 01:49:50 +08:00
collectAPIStats("getobjecttagging", maxClients(gz(httpTraceHdrs(api.GetObjectTaggingHandler))))).Queries("tagging", "")
Add ObjectTagging Support (#8754) This PR adds support for AWS S3 ObjectTagging API as explained here https://docs.aws.amazon.com/AmazonS3/latest/dev/object-tagging.html
2020-01-21 00:45:59 +08:00
// PutObjectTagging
fix: handle unsupported APIs more granularly (#11674)
2021-03-31 14:19:36 +08:00
router.Methods(http.MethodPut).Path("/{object:.+}").HandlerFunc(
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-13 01:49:50 +08:00
collectAPIStats("putobjecttagging", maxClients(gz(httpTraceHdrs(api.PutObjectTaggingHandler))))).Queries("tagging", "")
Add ObjectTagging Support (#8754) This PR adds support for AWS S3 ObjectTagging API as explained here https://docs.aws.amazon.com/AmazonS3/latest/dev/object-tagging.html
2020-01-21 00:45:59 +08:00
// DeleteObjectTagging
fix: handle unsupported APIs more granularly (#11674)
2021-03-31 14:19:36 +08:00
router.Methods(http.MethodDelete).Path("/{object:.+}").HandlerFunc(
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-13 01:49:50 +08:00
collectAPIStats("deleteobjecttagging", maxClients(gz(httpTraceHdrs(api.DeleteObjectTaggingHandler))))).Queries("tagging", "")
S3 Select API Support for CSV (#6127) Add support for trivial where clause cases
2018-08-15 18:30:19 +08:00
// SelectObjectContent
fix: handle unsupported APIs more granularly (#11674)
2021-03-31 14:19:36 +08:00
router.Methods(http.MethodPost).Path("/{object:.+}").HandlerFunc(
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-13 01:49:50 +08:00
collectAPIStats("selectobjectcontent", maxClients(gz(httpTraceHdrs(api.SelectObjectContentHandler))))).Queries("select", "").Queries("select-type", "2")
Add object retention at the per object (#8528) level - this PR builds on #8120 which added PutBucketObjectLockConfiguration and GetBucketObjectLockConfiguration APIS This PR implements PutObjectRetention, GetObjectRetention API and enhances PUT and GET API operations to display governance metadata if permissions allow.
2019-11-21 05:18:09 +08:00
// GetObjectRetention
fix: handle unsupported APIs more granularly (#11674)
2021-03-31 14:19:36 +08:00
router.Methods(http.MethodGet).Path("/{object:.+}").HandlerFunc(
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-13 01:49:50 +08:00
collectAPIStats("getobjectretention", maxClients(gz(httpTraceAll(api.GetObjectRetentionHandler))))).Queries("retention", "")
Add support for object locking with legal hold. (#8634)
2020-01-17 07:41:56 +08:00
// GetObjectLegalHold
fix: handle unsupported APIs more granularly (#11674)
2021-03-31 14:19:36 +08:00
router.Methods(http.MethodGet).Path("/{object:.+}").HandlerFunc(
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-13 01:49:50 +08:00
collectAPIStats("getobjectlegalhold", maxClients(gz(httpTraceAll(api.GetObjectLegalHoldHandler))))).Queries("legal-hold", "")
feat: add lambda transformation functions target (#16507)
2023-03-08 00:12:41 +08:00
// GetObject with lambda ARNs
router.Methods(http.MethodGet).Path("/{object:.+}").HandlerFunc(
collectAPIStats("getobject", maxClients(gz(httpTraceHdrs(api.GetObjectLambdaHandler))))).Queries("lambdaArn", "{lambdaArn:.*}")
// GetObject
fix: handle unsupported APIs more granularly (#11674)
2021-03-31 14:19:36 +08:00
router.Methods(http.MethodGet).Path("/{object:.+}").HandlerFunc(
Add GetObject gzip option (#14226) Enabled with `mc admin config set alias/ api gzip_objects=on` Standard filtering applies (1K response minimum, not compressed content type, not range request, gzip accepted by client).
2022-02-15 01:19:01 +08:00
collectAPIStats("getobject", maxClients(gz(httpTraceHdrs(api.GetObjectHandler)))))
Virtual host style S3 requests (#5095)
2017-11-15 08:56:24 +08:00
// CopyObject
fix: handle unsupported APIs more granularly (#11674)
2021-03-31 14:19:36 +08:00
router.Methods(http.MethodPut).Path("/{object:.+}").HeadersRegexp(xhttp.AmzCopySource, ".*?(\\/|%2F).*?").HandlerFunc(
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-13 01:49:50 +08:00
collectAPIStats("copyobject", maxClients(gz(httpTraceAll(api.CopyObjectHandler)))))
Add object retention at the per object (#8528) level - this PR builds on #8120 which added PutBucketObjectLockConfiguration and GetBucketObjectLockConfiguration APIS This PR implements PutObjectRetention, GetObjectRetention API and enhances PUT and GET API operations to display governance metadata if permissions allow.
2019-11-21 05:18:09 +08:00
// PutObjectRetention
fix: handle unsupported APIs more granularly (#11674)
2021-03-31 14:19:36 +08:00
router.Methods(http.MethodPut).Path("/{object:.+}").HandlerFunc(
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-13 01:49:50 +08:00
collectAPIStats("putobjectretention", maxClients(gz(httpTraceAll(api.PutObjectRetentionHandler))))).Queries("retention", "")
Add support for object locking with legal hold. (#8634)
2020-01-17 07:41:56 +08:00
// PutObjectLegalHold
fix: handle unsupported APIs more granularly (#11674)
2021-03-31 14:19:36 +08:00
router.Methods(http.MethodPut).Path("/{object:.+}").HandlerFunc(
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-13 01:49:50 +08:00
collectAPIStats("putobjectlegalhold", maxClients(gz(httpTraceAll(api.PutObjectLegalHoldHandler))))).Queries("legal-hold", "")
Add support for object locking with legal hold. (#8634)
2020-01-17 07:41:56 +08:00
[feat] Add targz transparent extract support (#11849) This feature brings in support for auto extraction of objects onto MinIO's namespace from an incoming tar gzipped stream, the only expected metadata sent by the client is to set `snowball-auto-extract`. All the contents from the tar stream are saved as folders and objects on the namespace. fixes #8715
2021-03-27 08:15:09 +08:00
// PutObject with auto-extract support for zip
fix: handle unsupported APIs more granularly (#11674)
2021-03-31 14:19:36 +08:00
router.Methods(http.MethodPut).Path("/{object:.+}").HeadersRegexp(xhttp.AmzSnowballExtract, "true").HandlerFunc(
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-13 01:49:50 +08:00
collectAPIStats("putobject", maxClients(gz(httpTraceHdrs(api.PutObjectExtractHandler)))))
[feat] Add targz transparent extract support (#11849) This feature brings in support for auto extraction of objects onto MinIO's namespace from an incoming tar gzipped stream, the only expected metadata sent by the client is to set `snowball-auto-extract`. All the contents from the tar stream are saved as folders and objects on the namespace. fixes #8715
2021-03-27 08:15:09 +08:00
Virtual host style S3 requests (#5095)
2017-11-15 08:56:24 +08:00
// PutObject
fix: handle unsupported APIs more granularly (#11674)
2021-03-31 14:19:36 +08:00
router.Methods(http.MethodPut).Path("/{object:.+}").HandlerFunc(
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-13 01:49:50 +08:00
collectAPIStats("putobject", maxClients(gz(httpTraceHdrs(api.PutObjectHandler)))))
[feat] Add targz transparent extract support (#11849) This feature brings in support for auto extraction of objects onto MinIO's namespace from an incoming tar gzipped stream, the only expected metadata sent by the client is to set `snowball-auto-extract`. All the contents from the tar stream are saved as folders and objects on the namespace. fixes #8715
2021-03-27 08:15:09 +08:00
Virtual host style S3 requests (#5095)
2017-11-15 08:56:24 +08:00
// DeleteObject
fix: handle unsupported APIs more granularly (#11674)
2021-03-31 14:19:36 +08:00
router.Methods(http.MethodDelete).Path("/{object:.+}").HandlerFunc(
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-13 01:49:50 +08:00
collectAPIStats("deleteobject", maxClients(gz(httpTraceAll(api.DeleteObjectHandler)))))
routers: Move API and Web routers into their own files. This is done to ensure we have a clean way to add new routers such as - diskRouter - configRouter - lockRouter
2016-03-28 03:37:21 +08:00
[feat]: implement GetBucketPolicyStatus API (#11673) additionally also add more APIs in notImplemented list, adjust routing rules appropriately
2021-03-02 15:10:33 +08:00
// PostRestoreObject
fix: handle unsupported APIs more granularly (#11674)
2021-03-31 14:19:36 +08:00
router.Methods(http.MethodPost).Path("/{object:.+}").HandlerFunc(
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-13 01:49:50 +08:00
collectAPIStats("restoreobject", maxClients(gz(httpTraceAll(api.PostRestoreObjectHandler))))).Queries("restore", "")
[feat]: implement GetBucketPolicyStatus API (#11673) additionally also add more APIs in notImplemented list, adjust routing rules appropriately
2021-03-02 15:10:33 +08:00
add gocritic/ruleguard checks back again, cleanup code. (#13665) - remove some duplicated code - reported a bug, separately fixed in #13664 - using strings.ReplaceAll() when needed - using filepath.ToSlash() use when needed - remove all non-Go style comments from the codebase Co-authored-by: Aditya Manthramurthy <donatello@users.noreply.github.com>
2021-11-17 01:28:29 +08:00
// Bucket operations
Virtual host style S3 requests (#5095)
2017-11-15 08:56:24 +08:00
// GetBucketLocation
fix: handle unsupported APIs more granularly (#11674)
2021-03-31 14:19:36 +08:00
router.Methods(http.MethodGet).HandlerFunc(
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-13 01:49:50 +08:00
collectAPIStats("getbucketlocation", maxClients(gz(httpTraceAll(api.GetBucketLocationHandler))))).Queries("location", "")
Virtual host style S3 requests (#5095)
2017-11-15 08:56:24 +08:00
// GetBucketPolicy
fix: handle unsupported APIs more granularly (#11674)
2021-03-31 14:19:36 +08:00
router.Methods(http.MethodGet).HandlerFunc(
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-13 01:49:50 +08:00
collectAPIStats("getbucketpolicy", maxClients(gz(httpTraceAll(api.GetBucketPolicyHandler))))).Queries("policy", "")
Add initial support for bucket lifecycle (#7563) This PR is based off @sinhaashish's PR for object lifecycle management, which includes support only for, - Expiration of object - Filter using object prefix (_not_ object tags) N B the code for actual expiration of objects will be included in a subsequent PR.
2019-07-20 04:20:33 +08:00
// GetBucketLifecycle
fix: handle unsupported APIs more granularly (#11674)
2021-03-31 14:19:36 +08:00
router.Methods(http.MethodGet).HandlerFunc(
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-13 01:49:50 +08:00
collectAPIStats("getbucketlifecycle", maxClients(gz(httpTraceAll(api.GetBucketLifecycleHandler))))).Queries("lifecycle", "")
Add support for bucket encryption feature (#8890) - pkg/bucket/encryption provides support for handling bucket encryption configuration - changes under cmd/ provide support for AES256 algorithm only Co-Authored-By: Poorna <poornas@users.noreply.github.com> Co-authored-by: Harshavardhana <harsha@minio.io>
2020-02-05 17:42:34 +08:00
// GetBucketEncryption
fix: handle unsupported APIs more granularly (#11674)
2021-03-31 14:19:36 +08:00
router.Methods(http.MethodGet).HandlerFunc(
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-13 01:49:50 +08:00
collectAPIStats("getbucketencryption", maxClients(gz(httpTraceAll(api.GetBucketEncryptionHandler))))).Queries("encryption", "")
Add support for server side bucket replication (#9882)
2020-07-22 08:49:56 +08:00
// GetBucketObjectLockConfig
fix: handle unsupported APIs more granularly (#11674)
2021-03-31 14:19:36 +08:00
router.Methods(http.MethodGet).HandlerFunc(
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-13 01:49:50 +08:00
collectAPIStats("getbucketobjectlockconfiguration", maxClients(gz(httpTraceAll(api.GetBucketObjectLockConfigHandler))))).Queries("object-lock", "")
Add support for server side bucket replication (#9882)
2020-07-22 08:49:56 +08:00
// GetBucketReplicationConfig
fix: handle unsupported APIs more granularly (#11674)
2021-03-31 14:19:36 +08:00
router.Methods(http.MethodGet).HandlerFunc(
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-13 01:49:50 +08:00
collectAPIStats("getbucketreplicationconfiguration", maxClients(gz(httpTraceAll(api.GetBucketReplicationConfigHandler))))).Queries("replication", "")
Add support for server side bucket replication (#9882)
2020-07-22 08:49:56 +08:00
// GetBucketVersioning
fix: handle unsupported APIs more granularly (#11674)
2021-03-31 14:19:36 +08:00
router.Methods(http.MethodGet).HandlerFunc(
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-13 01:49:50 +08:00
collectAPIStats("getbucketversioning", maxClients(gz(httpTraceAll(api.GetBucketVersioningHandler))))).Queries("versioning", "")
Add support for server side bucket replication (#9882)
2020-07-22 08:49:56 +08:00
// GetBucketNotification
fix: handle unsupported APIs more granularly (#11674)
2021-03-31 14:19:36 +08:00
router.Methods(http.MethodGet).HandlerFunc(
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-13 01:49:50 +08:00
collectAPIStats("getbucketnotification", maxClients(gz(httpTraceAll(api.GetBucketNotificationHandler))))).Queries("notification", "")
Add support for server side bucket replication (#9882)
2020-07-22 08:49:56 +08:00
// ListenNotification
fix: handle unsupported APIs more granularly (#11674)
2021-03-31 14:19:36 +08:00
router.Methods(http.MethodGet).HandlerFunc(
avoid limits on the number of parallel trace/bucket notifications listeners (#14799) Simplifies overall limits on the incoming listeners for notifications. Fixes #14566
2022-06-06 05:29:12 +08:00
collectAPIStats("listennotification", gz(httpTraceAll(api.ListenNotificationHandler)))).Queries("events", "{events:.*}")
Refactor replication resync to be an active process (#14266) When resync is triggered, walk the bucket namespace and resync objects that are unreplicated. This PR also adds an API to report resync progress.
2022-02-11 02:16:52 +08:00
// ResetBucketReplicationStatus - MinIO extension API
router.Methods(http.MethodGet).HandlerFunc(
collectAPIStats("resetbucketreplicationstatus", maxClients(gz(httpTraceAll(api.ResetBucketReplicationStatusHandler))))).Queries("replication-reset-status", "")
Implement a dummy GetBucketACL for legacy reasons (#5906) GetBucketACL call returns empty for all GET in ACL requests, the primary purpose of this PR is to provide legacy API support for legacy applications. Fixes #5706
2018-05-10 12:02:26 +08:00
Implements dummy tagging handlers for Terraform (#7040)
2019-02-09 08:18:13 +08:00
// Dummy Bucket Calls
Implement a dummy GetBucketACL for legacy reasons (#5906) GetBucketACL call returns empty for all GET in ACL requests, the primary purpose of this PR is to provide legacy API support for legacy applications. Fixes #5706
2018-05-10 12:02:26 +08:00
// GetBucketACL -- this is a dummy call.
fix: handle unsupported APIs more granularly (#11674)
2021-03-31 14:19:36 +08:00
router.Methods(http.MethodGet).HandlerFunc(
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-13 01:49:50 +08:00
collectAPIStats("getbucketacl", maxClients(gz(httpTraceAll(api.GetBucketACLHandler))))).Queries("acl", "")
acl: Support PUT calls with success for 'private' ACL's (#9000) Add dummy calls which respond success when ACL's are set to be private and fails, if user tries to change them from their default 'private' Some applications such as nuxeo may have an unnecessary requirement for this operation, we support this anyways such that don't have to fully implement the functionality just that we can respond with success for default ACLs
2020-02-16 14:07:52 +08:00
// PutBucketACL -- this is a dummy call.
fix: handle unsupported APIs more granularly (#11674)
2021-03-31 14:19:36 +08:00
router.Methods(http.MethodPut).HandlerFunc(
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-13 01:49:50 +08:00
collectAPIStats("putbucketacl", maxClients(gz(httpTraceAll(api.PutBucketACLHandler))))).Queries("acl", "")
Implements dummy tagging handlers for Terraform (#7040)
2019-02-09 08:18:13 +08:00
// GetBucketCors - this is a dummy call.
fix: handle unsupported APIs more granularly (#11674)
2021-03-31 14:19:36 +08:00
router.Methods(http.MethodGet).HandlerFunc(
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-13 01:49:50 +08:00
collectAPIStats("getbucketcors", maxClients(gz(httpTraceAll(api.GetBucketCorsHandler))))).Queries("cors", "")
Implements dummy tagging handlers for Terraform (#7040)
2019-02-09 08:18:13 +08:00
// GetBucketWebsiteHandler - this is a dummy call.
fix: handle unsupported APIs more granularly (#11674)
2021-03-31 14:19:36 +08:00
router.Methods(http.MethodGet).HandlerFunc(
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-13 01:49:50 +08:00
collectAPIStats("getbucketwebsite", maxClients(gz(httpTraceAll(api.GetBucketWebsiteHandler))))).Queries("website", "")
Implements dummy tagging handlers for Terraform (#7040)
2019-02-09 08:18:13 +08:00
// GetBucketAccelerateHandler - this is a dummy call.
fix: handle unsupported APIs more granularly (#11674)
2021-03-31 14:19:36 +08:00
router.Methods(http.MethodGet).HandlerFunc(
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-13 01:49:50 +08:00
collectAPIStats("getbucketaccelerate", maxClients(gz(httpTraceAll(api.GetBucketAccelerateHandler))))).Queries("accelerate", "")
Implements dummy tagging handlers for Terraform (#7040)
2019-02-09 08:18:13 +08:00
// GetBucketRequestPaymentHandler - this is a dummy call.
fix: handle unsupported APIs more granularly (#11674)
2021-03-31 14:19:36 +08:00
router.Methods(http.MethodGet).HandlerFunc(
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-13 01:49:50 +08:00
collectAPIStats("getbucketrequestpayment", maxClients(gz(httpTraceAll(api.GetBucketRequestPaymentHandler))))).Queries("requestPayment", "")
Implements dummy tagging handlers for Terraform (#7040)
2019-02-09 08:18:13 +08:00
// GetBucketLoggingHandler - this is a dummy call.
fix: handle unsupported APIs more granularly (#11674)
2021-03-31 14:19:36 +08:00
router.Methods(http.MethodGet).HandlerFunc(
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-13 01:49:50 +08:00
collectAPIStats("getbucketlogging", maxClients(gz(httpTraceAll(api.GetBucketLoggingHandler))))).Queries("logging", "")
add bucket tagging support (#9389) This patch also simplifies object tagging support
2020-05-06 05:18:13 +08:00
// GetBucketTaggingHandler
fix: handle unsupported APIs more granularly (#11674)
2021-03-31 14:19:36 +08:00
router.Methods(http.MethodGet).HandlerFunc(
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-13 01:49:50 +08:00
collectAPIStats("getbuckettagging", maxClients(gz(httpTraceAll(api.GetBucketTaggingHandler))))).Queries("tagging", "")
add gocritic/ruleguard checks back again, cleanup code. (#13665) - remove some duplicated code - reported a bug, separately fixed in #13664 - using strings.ReplaceAll() when needed - using filepath.ToSlash() use when needed - remove all non-Go style comments from the codebase Co-authored-by: Aditya Manthramurthy <donatello@users.noreply.github.com>
2021-11-17 01:28:29 +08:00
// DeleteBucketWebsiteHandler
fix: handle unsupported APIs more granularly (#11674)
2021-03-31 14:19:36 +08:00
router.Methods(http.MethodDelete).HandlerFunc(
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-13 01:49:50 +08:00
collectAPIStats("deletebucketwebsite", maxClients(gz(httpTraceAll(api.DeleteBucketWebsiteHandler))))).Queries("website", "")
Implements dummy tagging handlers for Terraform (#7040)
2019-02-09 08:18:13 +08:00
// DeleteBucketTaggingHandler
fix: handle unsupported APIs more granularly (#11674)
2021-03-31 14:19:36 +08:00
router.Methods(http.MethodDelete).HandlerFunc(
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-13 01:49:50 +08:00
collectAPIStats("deletebuckettagging", maxClients(gz(httpTraceAll(api.DeleteBucketTaggingHandler))))).Queries("tagging", "")
Implement a dummy GetBucketACL for legacy reasons (#5906) GetBucketACL call returns empty for all GET in ACL requests, the primary purpose of this PR is to provide legacy API support for legacy applications. Fixes #5706
2018-05-10 12:02:26 +08:00
Virtual host style S3 requests (#5095)
2017-11-15 08:56:24 +08:00
// ListMultipartUploads
fix: handle unsupported APIs more granularly (#11674)
2021-03-31 14:19:36 +08:00
router.Methods(http.MethodGet).HandlerFunc(
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-13 01:49:50 +08:00
collectAPIStats("listmultipartuploads", maxClients(gz(httpTraceAll(api.ListMultipartUploadsHandler))))).Queries("uploads", "")
Enhance ListObjectsV2 API to return UserDefined metadata (#8539)
2019-11-21 17:54:49 +08:00
// ListObjectsV2M
fix: handle unsupported APIs more granularly (#11674)
2021-03-31 14:19:36 +08:00
router.Methods(http.MethodGet).HandlerFunc(
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-13 01:49:50 +08:00
collectAPIStats("listobjectsv2M", maxClients(gz(httpTraceAll(api.ListObjectsV2MHandler))))).Queries("list-type", "2", "metadata", "true")
Virtual host style S3 requests (#5095)
2017-11-15 08:56:24 +08:00
// ListObjectsV2
fix: handle unsupported APIs more granularly (#11674)
2021-03-31 14:19:36 +08:00
router.Methods(http.MethodGet).HandlerFunc(
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-13 01:49:50 +08:00
collectAPIStats("listobjectsv2", maxClients(gz(httpTraceAll(api.ListObjectsV2Handler))))).Queries("list-type", "2")
Support bucket versioning (#9377) - Implement a new xl.json 2.0.0 format to support, this moves the entire marshaling logic to POSIX layer, top layer always consumes a common FileInfo construct which simplifies the metadata reads. - Implement list object versions - Migrate to siphash from crchash for new deployments for object placements. Fixes #2111
2020-06-13 11:04:01 +08:00
// ListObjectVersions
Add metadata extension to ListObjectVersions (#16930)
2023-03-31 03:20:42 +08:00
router.Methods(http.MethodGet).HandlerFunc(
collectAPIStats("listobjectversions", maxClients(gz(httpTraceAll(api.ListObjectVersionsMHandler))))).Queries("versions", "", "metadata", "true")
// ListObjectVersions
fix: handle unsupported APIs more granularly (#11674)
2021-03-31 14:19:36 +08:00
router.Methods(http.MethodGet).HandlerFunc(
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-13 01:49:50 +08:00
collectAPIStats("listobjectversions", maxClients(gz(httpTraceAll(api.ListObjectVersionsHandler))))).Queries("versions", "")
[feat]: implement GetBucketPolicyStatus API (#11673) additionally also add more APIs in notImplemented list, adjust routing rules appropriately
2021-03-02 15:10:33 +08:00
// GetBucketPolicyStatus
fix: handle unsupported APIs more granularly (#11674)
2021-03-31 14:19:36 +08:00
router.Methods(http.MethodGet).HandlerFunc(
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-13 01:49:50 +08:00
collectAPIStats("getpolicystatus", maxClients(gz(httpTraceAll(api.GetBucketPolicyStatusHandler))))).Queries("policyStatus", "")
Add initial support for bucket lifecycle (#7563) This PR is based off @sinhaashish's PR for object lifecycle management, which includes support only for, - Expiration of object - Filter using object prefix (_not_ object tags) N B the code for actual expiration of objects will be included in a subsequent PR.
2019-07-20 04:20:33 +08:00
// PutBucketLifecycle
fix: handle unsupported APIs more granularly (#11674)
2021-03-31 14:19:36 +08:00
router.Methods(http.MethodPut).HandlerFunc(
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-13 01:49:50 +08:00
collectAPIStats("putbucketlifecycle", maxClients(gz(httpTraceAll(api.PutBucketLifecycleHandler))))).Queries("lifecycle", "")
Add support for server side bucket replication (#9882)
2020-07-22 08:49:56 +08:00
// PutBucketReplicationConfig
fix: handle unsupported APIs more granularly (#11674)
2021-03-31 14:19:36 +08:00
router.Methods(http.MethodPut).HandlerFunc(
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-13 01:49:50 +08:00
collectAPIStats("putbucketreplicationconfiguration", maxClients(gz(httpTraceAll(api.PutBucketReplicationConfigHandler))))).Queries("replication", "")
Add support for bucket encryption feature (#8890) - pkg/bucket/encryption provides support for handling bucket encryption configuration - changes under cmd/ provide support for AES256 algorithm only Co-Authored-By: Poorna <poornas@users.noreply.github.com> Co-authored-by: Harshavardhana <harsha@minio.io>
2020-02-05 17:42:34 +08:00
// PutBucketEncryption
fix: handle unsupported APIs more granularly (#11674)
2021-03-31 14:19:36 +08:00
router.Methods(http.MethodPut).HandlerFunc(
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-13 01:49:50 +08:00
collectAPIStats("putbucketencryption", maxClients(gz(httpTraceAll(api.PutBucketEncryptionHandler))))).Queries("encryption", "")
Add support for bucket encryption feature (#8890) - pkg/bucket/encryption provides support for handling bucket encryption configuration - changes under cmd/ provide support for AES256 algorithm only Co-Authored-By: Poorna <poornas@users.noreply.github.com> Co-authored-by: Harshavardhana <harsha@minio.io>
2020-02-05 17:42:34 +08:00
Virtual host style S3 requests (#5095)
2017-11-15 08:56:24 +08:00
// PutBucketPolicy
fix: handle unsupported APIs more granularly (#11674)
2021-03-31 14:19:36 +08:00
router.Methods(http.MethodPut).HandlerFunc(
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-13 01:49:50 +08:00
collectAPIStats("putbucketpolicy", maxClients(gz(httpTraceAll(api.PutBucketPolicyHandler))))).Queries("policy", "")
Add initial support for bucket lifecycle (#7563) This PR is based off @sinhaashish's PR for object lifecycle management, which includes support only for, - Expiration of object - Filter using object prefix (_not_ object tags) N B the code for actual expiration of objects will be included in a subsequent PR.
2019-07-20 04:20:33 +08:00
Add Get/Put Bucket Lock Configuration API support (#8120) This feature implements [PUT Bucket object lock configuration][1] and [GET Bucket object lock configuration][2]. After object lock configuration is set, existing and new objects are set to WORM for specified duration. Currently Governance mode works exactly like Compliance mode. Fixes #8101 [1] https://docs.aws.amazon.com/AmazonS3/latest/API/RESTBucketPUTObjectLockConfiguration.html [2] https://docs.aws.amazon.com/AmazonS3/latest/API/RESTBucketGETObjectLockConfiguration.html
2019-11-13 06:50:18 +08:00
// PutBucketObjectLockConfig
fix: handle unsupported APIs more granularly (#11674)
2021-03-31 14:19:36 +08:00
router.Methods(http.MethodPut).HandlerFunc(
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-13 01:49:50 +08:00
collectAPIStats("putbucketobjectlockconfig", maxClients(gz(httpTraceAll(api.PutBucketObjectLockConfigHandler))))).Queries("object-lock", "")
add bucket tagging support (#9389) This patch also simplifies object tagging support
2020-05-06 05:18:13 +08:00
// PutBucketTaggingHandler
fix: handle unsupported APIs more granularly (#11674)
2021-03-31 14:19:36 +08:00
router.Methods(http.MethodPut).HandlerFunc(
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-13 01:49:50 +08:00
collectAPIStats("putbuckettagging", maxClients(gz(httpTraceAll(api.PutBucketTaggingHandler))))).Queries("tagging", "")
Add Get/Put Bucket Lock Configuration API support (#8120) This feature implements [PUT Bucket object lock configuration][1] and [GET Bucket object lock configuration][2]. After object lock configuration is set, existing and new objects are set to WORM for specified duration. Currently Governance mode works exactly like Compliance mode. Fixes #8101 [1] https://docs.aws.amazon.com/AmazonS3/latest/API/RESTBucketPUTObjectLockConfiguration.html [2] https://docs.aws.amazon.com/AmazonS3/latest/API/RESTBucketGETObjectLockConfiguration.html
2019-11-13 06:50:18 +08:00
// PutBucketVersioning
fix: handle unsupported APIs more granularly (#11674)
2021-03-31 14:19:36 +08:00
router.Methods(http.MethodPut).HandlerFunc(
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-13 01:49:50 +08:00
collectAPIStats("putbucketversioning", maxClients(gz(httpTraceAll(api.PutBucketVersioningHandler))))).Queries("versioning", "")
Virtual host style S3 requests (#5095)
2017-11-15 08:56:24 +08:00
// PutBucketNotification
fix: handle unsupported APIs more granularly (#11674)
2021-03-31 14:19:36 +08:00
router.Methods(http.MethodPut).HandlerFunc(
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-13 01:49:50 +08:00
collectAPIStats("putbucketnotification", maxClients(gz(httpTraceAll(api.PutBucketNotificationHandler))))).Queries("notification", "")
Refactor replication resync to be an active process (#14266) When resync is triggered, walk the bucket namespace and resync objects that are unreplicated. This PR also adds an API to report resync progress.
2022-02-11 02:16:52 +08:00
// ResetBucketReplicationStart - MinIO extension API
Add support for existing object replication. (#12109) Also adding an API to allow resyncing replication when existing object replication is enabled and the remote target is entirely lost. With the `mc replicate reset` command, the objects that are eligible for replication as per the replication config will be resynced to target if existing object replication is enabled on the rule.
2021-06-02 10:59:11 +08:00
router.Methods(http.MethodPut).HandlerFunc(
Refactor replication resync to be an active process (#14266) When resync is triggered, walk the bucket namespace and resync objects that are unreplicated. This PR also adds an API to report resync progress.
2022-02-11 02:16:52 +08:00
collectAPIStats("resetbucketreplicationstart", maxClients(gz(httpTraceAll(api.ResetBucketReplicationStartHandler))))).Queries("replication-reset", "")
Virtual host style S3 requests (#5095)
2017-11-15 08:56:24 +08:00
// PutBucket
fix: handle unsupported APIs more granularly (#11674)
2021-03-31 14:19:36 +08:00
router.Methods(http.MethodPut).HandlerFunc(
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-13 01:49:50 +08:00
collectAPIStats("putbucket", maxClients(gz(httpTraceAll(api.PutBucketHandler)))))
Virtual host style S3 requests (#5095)
2017-11-15 08:56:24 +08:00
// HeadBucket
fix: handle unsupported APIs more granularly (#11674)
2021-03-31 14:19:36 +08:00
router.Methods(http.MethodHead).HandlerFunc(
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-13 01:49:50 +08:00
collectAPIStats("headbucket", maxClients(gz(httpTraceAll(api.HeadBucketHandler)))))
Virtual host style S3 requests (#5095)
2017-11-15 08:56:24 +08:00
// PostPolicy
fix: post policy request security bypass (#16849)
2023-03-20 12:15:20 +08:00
router.Methods(http.MethodPost).MatcherFunc(func(r *http.Request, _ *mux.RouteMatch) bool {
return isRequestPostPolicySignatureV4(r)
}).HandlerFunc(collectAPIStats("postpolicybucket", maxClients(gz(httpTraceHdrs(api.PostPolicyBucketHandler)))))
Virtual host style S3 requests (#5095)
2017-11-15 08:56:24 +08:00
// DeleteMultipleObjects
fix: handle unsupported APIs more granularly (#11674)
2021-03-31 14:19:36 +08:00
router.Methods(http.MethodPost).HandlerFunc(
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-13 01:49:50 +08:00
collectAPIStats("deletemultipleobjects", maxClients(gz(httpTraceAll(api.DeleteMultipleObjectsHandler))))).Queries("delete", "")
Virtual host style S3 requests (#5095)
2017-11-15 08:56:24 +08:00
// DeleteBucketPolicy
fix: handle unsupported APIs more granularly (#11674)
2021-03-31 14:19:36 +08:00
router.Methods(http.MethodDelete).HandlerFunc(
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-13 01:49:50 +08:00
collectAPIStats("deletebucketpolicy", maxClients(gz(httpTraceAll(api.DeleteBucketPolicyHandler))))).Queries("policy", "")
Add support for server side bucket replication (#9882)
2020-07-22 08:49:56 +08:00
// DeleteBucketReplication
fix: handle unsupported APIs more granularly (#11674)
2021-03-31 14:19:36 +08:00
router.Methods(http.MethodDelete).HandlerFunc(
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-13 01:49:50 +08:00
collectAPIStats("deletebucketreplicationconfiguration", maxClients(gz(httpTraceAll(api.DeleteBucketReplicationConfigHandler))))).Queries("replication", "")
Add initial support for bucket lifecycle (#7563) This PR is based off @sinhaashish's PR for object lifecycle management, which includes support only for, - Expiration of object - Filter using object prefix (_not_ object tags) N B the code for actual expiration of objects will be included in a subsequent PR.
2019-07-20 04:20:33 +08:00
// DeleteBucketLifecycle
fix: handle unsupported APIs more granularly (#11674)
2021-03-31 14:19:36 +08:00
router.Methods(http.MethodDelete).HandlerFunc(
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-13 01:49:50 +08:00
collectAPIStats("deletebucketlifecycle", maxClients(gz(httpTraceAll(api.DeleteBucketLifecycleHandler))))).Queries("lifecycle", "")
Add support for bucket encryption feature (#8890) - pkg/bucket/encryption provides support for handling bucket encryption configuration - changes under cmd/ provide support for AES256 algorithm only Co-Authored-By: Poorna <poornas@users.noreply.github.com> Co-authored-by: Harshavardhana <harsha@minio.io>
2020-02-05 17:42:34 +08:00
// DeleteBucketEncryption
fix: handle unsupported APIs more granularly (#11674)
2021-03-31 14:19:36 +08:00
router.Methods(http.MethodDelete).HandlerFunc(
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-13 01:49:50 +08:00
collectAPIStats("deletebucketencryption", maxClients(gz(httpTraceAll(api.DeleteBucketEncryptionHandler))))).Queries("encryption", "")
Virtual host style S3 requests (#5095)
2017-11-15 08:56:24 +08:00
// DeleteBucket
fix: handle unsupported APIs more granularly (#11674)
2021-03-31 14:19:36 +08:00
router.Methods(http.MethodDelete).HandlerFunc(
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-13 01:49:50 +08:00
collectAPIStats("deletebucket", maxClients(gz(httpTraceAll(api.DeleteBucketHandler)))))
fix: muxing order for rejected APIs (#12321)
2021-05-20 00:21:34 +08:00
Various improvements in replication (#11949) - collect real time replication metrics for prometheus. - add pending_count, failed_count metric for total pending/failed replication operations. - add API to get replication metrics - add MRF worker to handle spill-over replication operations - multiple issues found with replication - fixes an issue when client sends a bucket name with `/` at the end from SetRemoteTarget API call make sure to trim the bucket name to avoid any extra `/`. - hold write locks in GetObjectNInfo during replication to ensure that object version stack is not overwritten while reading the content. - add additional protection during WriteMetadata() to ensure that we always write a valid FileInfo{} and avoid ever writing empty FileInfo{} to the lowest layers. Co-authored-by: Poorna Krishnamoorthy <poorna@minio.io> Co-authored-by: Harshavardhana <harsha@minio.io>
2021-04-04 00:03:42 +08:00
// MinIO extension API for replication.
//
Add additional info for replication metrics API (#17293) to track the replication transfer rate across different nodes, number of active workers in use and in-queue stats to get an idea of the current workload. This PR also adds replication metrics to the site replication status API. For site replication, prometheus metrics are no longer at the bucket level - but at the cluster level. Add prometheus metric to track credential errors since uptime
2023-08-30 16:00:59 +08:00
router.Methods(http.MethodGet).HandlerFunc(
collectAPIStats("getbucketreplicationmetrics", maxClients(gz(httpTraceAll(api.GetBucketReplicationMetricsV2Handler))))).Queries("replication-metrics", "2")
// deprecated handler
Various improvements in replication (#11949) - collect real time replication metrics for prometheus. - add pending_count, failed_count metric for total pending/failed replication operations. - add API to get replication metrics - add MRF worker to handle spill-over replication operations - multiple issues found with replication - fixes an issue when client sends a bucket name with `/` at the end from SetRemoteTarget API call make sure to trim the bucket name to avoid any extra `/`. - hold write locks in GetObjectNInfo during replication to ensure that object version stack is not overwritten while reading the content. - add additional protection during WriteMetadata() to ensure that we always write a valid FileInfo{} and avoid ever writing empty FileInfo{} to the lowest layers. Co-authored-by: Poorna Krishnamoorthy <poorna@minio.io> Co-authored-by: Harshavardhana <harsha@minio.io>
2021-04-04 00:03:42 +08:00
router.Methods(http.MethodGet).HandlerFunc(
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-13 01:49:50 +08:00
collectAPIStats("getbucketreplicationmetrics", maxClients(gz(httpTraceAll(api.GetBucketReplicationMetricsHandler))))).Queries("replication-metrics", "")
Add additional info for replication metrics API (#17293) to track the replication transfer rate across different nodes, number of active workers in use and in-queue stats to get an idea of the current workload. This PR also adds replication metrics to the site replication status API. For site replication, prometheus metrics are no longer at the bucket level - but at the cluster level. Add prometheus metric to track credential errors since uptime
2023-08-30 16:00:59 +08:00
replication: add validation API (#17520) To check if replication is set up properly on a bucket.
2023-07-11 11:09:20 +08:00
// ValidateBucketReplicationCreds
router.Methods(http.MethodGet).HandlerFunc(
collectAPIStats("checkbucketreplicationconfiguration", maxClients(gz(httpTraceAll(api.ValidateBucketReplicationCredsHandler))))).Queries("replication-check", "")
fix: route for replication-metrics API (#11968)
2021-04-06 04:36:39 +08:00
fix: muxing order for rejected APIs (#12321)
2021-05-20 00:21:34 +08:00
// Register rejected bucket APIs
for _, r := range rejectedBucketAPIs {
router.Methods(r.methods...).
HandlerFunc(collectAPIStats(r.api, httpTraceAll(notImplementedHandler))).
Queries(r.queries...)
}
fix: route for replication-metrics API (#11968)
2021-04-06 04:36:39 +08:00
// S3 ListObjectsV1 (Legacy)
router.Methods(http.MethodGet).HandlerFunc(
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-13 01:49:50 +08:00
collectAPIStats("listobjectsv1", maxClients(gz(httpTraceAll(api.ListObjectsV1Handler)))))
Virtual host style S3 requests (#5095)
2017-11-15 08:56:24 +08:00
}
routers: Move API and Web routers into their own files. This is done to ensure we have a clean way to add new routers such as - diskRouter - configRouter - lockRouter
2016-03-28 03:37:21 +08:00
add gocritic/ruleguard checks back again, cleanup code. (#13665) - remove some duplicated code - reported a bug, separately fixed in #13664 - using strings.ReplaceAll() when needed - using filepath.ToSlash() use when needed - remove all non-Go style comments from the codebase Co-authored-by: Aditya Manthramurthy <donatello@users.noreply.github.com>
2021-11-17 01:28:29 +08:00
// Root operation
routers: Move API and Web routers into their own files. This is done to ensure we have a clean way to add new routers such as - diskRouter - configRouter - lockRouter
2016-03-28 03:37:21 +08:00
feat: Add notification support for bucketCreates and removal (#10075)
2020-07-21 03:52:49 +08:00
// ListenNotification
apiRouter.Methods(http.MethodGet).Path(SlashSeparator).HandlerFunc(
avoid limits on the number of parallel trace/bucket notifications listeners (#14799) Simplifies overall limits on the incoming listeners for notifications. Fixes #14566
2022-06-06 05:29:12 +08:00
collectAPIStats("listennotification", gz(httpTraceAll(api.ListenNotificationHandler)))).Queries("events", "{events:.*}")
feat: Add notification support for bucketCreates and removal (#10075)
2020-07-21 03:52:49 +08:00
routers: Move API and Web routers into their own files. This is done to ensure we have a clean way to add new routers such as - diskRouter - configRouter - lockRouter
2016-03-28 03:37:21 +08:00
// ListBuckets
Add rate limiter for S3 API layer (#9196) - total number of S3 API calls per server - maximum wait duration for any S3 API call This implementation is primarily meant for situations where HDDs are not capable enough to handle the incoming workload and there is no way to throttle the client. This feature allows MinIO server to throttle itself such that we do not overwhelm the HDDs.
2020-03-25 03:43:40 +08:00
apiRouter.Methods(http.MethodGet).Path(SlashSeparator).HandlerFunc(
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-13 01:49:50 +08:00
collectAPIStats("listbuckets", maxClients(gz(httpTraceAll(api.ListBucketsHandler)))))
Virtual host style S3 requests (#5095)
2017-11-15 08:56:24 +08:00
add missing signature v2 query params (#9670)
2020-05-22 09:51:23 +08:00
// S3 browser with signature v4 adds '//' for ListBuckets request, so rather
// than failing with UnknownAPIRequest we simply handle it for now.
apiRouter.Methods(http.MethodGet).Path(SlashSeparator + SlashSeparator).HandlerFunc(
feat: gzip S3/Admin API responses whenever applicable (#12631) Gzip responses if appropriate, except GetObject requests. List reponses has an almost 10:1 compression ratio with no measurable slowdown (in fact it seems a bit faster).
2021-07-13 01:49:50 +08:00
collectAPIStats("listbuckets", maxClients(gz(httpTraceAll(api.ListBucketsHandler)))))
add missing signature v2 query params (#9670)
2020-05-22 09:51:23 +08:00
Return appropriate errors API versions changes across REST APIs (#8480) This PR adds code to appropriately handle versioning issues that come up quite constantly across our API changes. Currently we were also routing our requests wrong which sort of made it harder to write a consistent error handling code to appropriately reject or honor requests. This PR potentially fixes issues - old mc is used against new minio release which is incompatible returns an appropriate for client action. - any older servers talking to each other, report appropriate error - incompatible peer servers should report error and reject the calls with appropriate error
2019-11-05 01:30:59 +08:00
// If none of the routes match add default error handler routes
Remove redundant http.HandlerFunc type conversion. (#10576)
2020-09-29 04:33:49 +08:00
apiRouter.NotFoundHandler = collectAPIStats("notfound", httpTraceAll(errorResponseHandler))
ListObjects Metadata Caching (#10648) Design: https://gist.github.com/klauspost/025c09b48ed4a1293c917cecfabdf21c Gist of improvements: * Cross-server caching and listing will use the same data across servers and requests. * Lists can be arbitrarily resumed at a constant speed. * Metadata for all files scanned is stored for streaming retrieval. * The existing bloom filters controlled by the crawler is used for validating caches. * Concurrent requests for the same data (or parts of it) will not spawn additional walkers. * Listing a subdirectory of an existing recursive cache will use the cache. * All listing operations are fully streamable so the number of objects in a bucket no longer dictates the amount of memory. * Listings can be handled by any server within the cluster. * Caches are cleaned up when out of date or superseded by a more recent one.
2020-10-29 00:18:35 +08:00
apiRouter.MethodNotAllowedHandler = collectAPIStats("methodnotallowed", httpTraceAll(methodNotAllowedHandler("S3")))
routers: Move API and Web routers into their own files. This is done to ensure we have a clean way to add new routers such as - diskRouter - configRouter - lockRouter
2016-03-28 03:37:21 +08:00
}
fix: cors handling after gorilla mux update (#9980) fixes #9979
2020-07-07 11:55:19 +08:00
fix: cors handling again for not just OPTIONS request (#10025) CORS is notorious requires specific headers to be handled appropriately in request and response, using cors package as part of handlerFunc() for options method lacks the necessary control this package needs to add headers.
2020-07-13 01:56:57 +08:00
// corsHandler handler for CORS (Cross Origin Resource Sharing)
func corsHandler(handler http.Handler) http.Handler {
fix: cors handling after gorilla mux update (#9980) fixes #9979
2020-07-07 11:55:19 +08:00
commonS3Headers := []string{
xhttp.Date,
xhttp.ETag,
xhttp.ServerInfo,
xhttp.Connection,
xhttp.AcceptRanges,
xhttp.ContentRange,
xhttp.ContentEncoding,
xhttp.ContentLength,
xhttp.ContentType,
fix: add missing content-disposition from CORS handler (#10137)
2020-07-28 00:03:38 +08:00
xhttp.ContentDisposition,
fix: add missing response headers to CORS handler (#10124)
2020-07-24 15:46:51 +08:00
xhttp.LastModified,
xhttp.ContentLanguage,
xhttp.CacheControl,
xhttp.RetryAfter,
xhttp.AmzBucketRegion,
xhttp.Expires,
fix: cors handling after gorilla mux update (#9980) fixes #9979
2020-07-07 11:55:19 +08:00
"X-Amz*",
"x-amz*",
"*",
}
fix: when Origin: null is set return back '*' for allow origins (#17651)
2023-07-16 03:15:06 +08:00
opts := cors.Options{
fix: cors handling after gorilla mux update (#9980) fixes #9979
2020-07-07 11:55:19 +08:00
AllowOriginFunc: func(origin string) bool {
fix: when Origin: null is set return back '*' for allow origins (#17651)
2023-07-16 03:15:06 +08:00
for _, allowedOrigin := range globalAPIConfig.getCorsAllowOrigins() {
fix: cors handling after gorilla mux update (#9980) fixes #9979
2020-07-07 11:55:19 +08:00
if wildcard.MatchSimple(allowedOrigin, origin) {
return true
}
}
return false
},
AllowedMethods: []string{
http.MethodGet,
http.MethodPut,
http.MethodHead,
http.MethodPost,
http.MethodDelete,
http.MethodOptions,
http.MethodPatch,
},
AllowedHeaders: commonS3Headers,
ExposedHeaders: commonS3Headers,
AllowCredentials: true,
fix: when Origin: null is set return back '*' for allow origins (#17651)
2023-07-16 03:15:06 +08:00
}
return cors.New(opts).Handler(handler)
fix: cors handling after gorilla mux update (#9980) fixes #9979
2020-07-07 11:55:19 +08:00
}