root
/
minio
mirror of https://github.com/minio/minio.git
1
0
Fork 0
Code Issues Actions 1 Packages Projects Releases Wiki Activity
minio/cmd/http-tracer.go

191 lines
5.4 KiB
Go
Raw Normal View History

log: Store http request/responses in a log file (#4804) When MINIO_TRACE_DIR is provided, create a new log file and store all HTTP requests + responses data, body are excluded to reduce memory consumption. MINIO_HTTP_TRACE=1 enables logging. Use non mem consuming http req/resp recorders, the maximum is about 32k per request. This logs to STDOUT, body logging is disabled for PutObject PutObjectPart GetObject.
2017-10-25 10:04:51 +08:00
/*
Replace Minio refs in docs with MinIO and links (#7494)
2019-04-10 02:39:42 +08:00
* MinIO Cloud Storage, (C) 2017 MinIO, Inc.
log: Store http request/responses in a log file (#4804) When MINIO_TRACE_DIR is provided, create a new log file and store all HTTP requests + responses data, body are excluded to reduce memory consumption. MINIO_HTTP_TRACE=1 enables logging. Use non mem consuming http req/resp recorders, the maximum is about 32k per request. This logs to STDOUT, body logging is disabled for PutObject PutObjectPart GetObject.
2017-10-25 10:04:51 +08:00
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
Add admin API to send trace notifications to registered (#7128) Remove current functionality to log trace to file using MINIO_HTTP_TRACE env, and replace it with mc admin trace command on mc client.
2019-06-09 06:54:41 +08:00
package cmd
log: Store http request/responses in a log file (#4804) When MINIO_TRACE_DIR is provided, create a new log file and store all HTTP requests + responses data, body are excluded to reduce memory consumption. MINIO_HTTP_TRACE=1 enables logging. Use non mem consuming http req/resp recorders, the maximum is about 32k per request. This logs to STDOUT, body logging is disabled for PutObject PutObjectPart GetObject.
2017-10-25 10:04:51 +08:00
import (
"bytes"
"fmt"
"io"
"io/ioutil"
"net/http"
"reflect"
"runtime"
Add admin API to send trace notifications to registered (#7128) Remove current functionality to log trace to file using MINIO_HTTP_TRACE env, and replace it with mc admin trace command on mc client.
2019-06-09 06:54:41 +08:00
"strconv"
log: Store http request/responses in a log file (#4804) When MINIO_TRACE_DIR is provided, create a new log file and store all HTTP requests + responses data, body are excluded to reduce memory consumption. MINIO_HTTP_TRACE=1 enables logging. Use non mem consuming http req/resp recorders, the maximum is about 32k per request. This logs to STDOUT, body logging is disabled for PutObject PutObjectPart GetObject.
2017-10-25 10:04:51 +08:00
"strings"
"time"
Add admin API to send trace notifications to registered (#7128) Remove current functionality to log trace to file using MINIO_HTTP_TRACE env, and replace it with mc admin trace command on mc client.
2019-06-09 06:54:41 +08:00
xnet "github.com/minio/minio/pkg/net"
trace "github.com/minio/minio/pkg/trace"
log: Store http request/responses in a log file (#4804) When MINIO_TRACE_DIR is provided, create a new log file and store all HTTP requests + responses data, body are excluded to reduce memory consumption. MINIO_HTTP_TRACE=1 enables logging. Use non mem consuming http req/resp recorders, the maximum is about 32k per request. This logs to STDOUT, body logging is disabled for PutObject PutObjectPart GetObject.
2017-10-25 10:04:51 +08:00
)
// recordRequest - records the first recLen bytes
// of a given io.Reader
type recordRequest struct {
// Data source to record
io.Reader
// Response body should be logged
logBody bool
// Internal recording buffer
buf bytes.Buffer
}
func (r *recordRequest) Read(p []byte) (n int, err error) {
n, err = r.Reader.Read(p)
if r.logBody {
r.buf.Write(p[:n])
}
if err != nil {
return n, err
}
return n, err
}
// Return the bytes that were recorded.
func (r *recordRequest) Data() []byte {
return r.buf.Bytes()
}
// recordResponseWriter - records the first recLen bytes
// of a given http.ResponseWriter
type recordResponseWriter struct {
// Data source to record
http.ResponseWriter
// Response body should be logged
logBody bool
// Internal recording buffer
headers bytes.Buffer
body bytes.Buffer
// The status code of the current HTTP request
statusCode int
// Indicate if headers are written in the log
headersLogged bool
}
// Write the headers into the given buffer
func writeHeaders(w io.Writer, statusCode int, headers http.Header) {
fmt.Fprintf(w, "%d %s\n", statusCode, http.StatusText(statusCode))
for k, v := range headers {
fmt.Fprintf(w, "%s: %s\n", k, v[0])
}
}
// Record the headers.
func (r *recordResponseWriter) WriteHeader(i int) {
r.statusCode = i
if !r.headersLogged {
writeHeaders(&r.headers, i, r.ResponseWriter.Header())
r.headersLogged = true
}
r.ResponseWriter.WriteHeader(i)
}
func (r *recordResponseWriter) Write(p []byte) (n int, err error) {
n, err = r.ResponseWriter.Write(p)
if !r.headersLogged {
Avoid logging the body of the http 206 response (#6258) When an S3 client issues a GET request with range specified, Minio server returns some partial data with 206 http code. The latter is sent in MINIO_HTTP_TRACE output which is incorrect. This PR fixes the issue.
2018-08-09 03:34:42 +08:00
// We assume the response code to be '200 OK' when WriteHeader() is not called,
// that way following Golang HTTP response behavior.
log: Store http request/responses in a log file (#4804) When MINIO_TRACE_DIR is provided, create a new log file and store all HTTP requests + responses data, body are excluded to reduce memory consumption. MINIO_HTTP_TRACE=1 enables logging. Use non mem consuming http req/resp recorders, the maximum is about 32k per request. This logs to STDOUT, body logging is disabled for PutObject PutObjectPart GetObject.
2017-10-25 10:04:51 +08:00
writeHeaders(&r.headers, http.StatusOK, r.ResponseWriter.Header())
r.headersLogged = true
}
Avoid logging the body of the http 206 response (#6258) When an S3 client issues a GET request with range specified, Minio server returns some partial data with 206 http code. The latter is sent in MINIO_HTTP_TRACE output which is incorrect. This PR fixes the issue.
2018-08-09 03:34:42 +08:00
if (r.statusCode != http.StatusOK && r.statusCode != http.StatusPartialContent && r.statusCode != 0) || r.logBody {
// Always logging error responses.
log: Store http request/responses in a log file (#4804) When MINIO_TRACE_DIR is provided, create a new log file and store all HTTP requests + responses data, body are excluded to reduce memory consumption. MINIO_HTTP_TRACE=1 enables logging. Use non mem consuming http req/resp recorders, the maximum is about 32k per request. This logs to STDOUT, body logging is disabled for PutObject PutObjectPart GetObject.
2017-10-25 10:04:51 +08:00
r.body.Write(p)
}
return n, err
}
// Calls the underlying Flush.
func (r *recordResponseWriter) Flush() {
r.ResponseWriter.(http.Flusher).Flush()
}
// Return response body.
func (r *recordResponseWriter) Body() []byte {
return r.body.Bytes()
}
Trace: Replace function name with API prefix (#7794) This change is required for `Admin Trace`
2019-06-19 04:55:13 +08:00
// getOpName sanitizes the operation name for mc
func getOpName(name string) (op string) {
op = strings.TrimPrefix(name, "github.com/minio/minio/cmd.")
op = strings.TrimSuffix(op, "Handler-fm")
op = strings.Replace(op, "objectAPIHandlers", "s3", 1)
op = strings.Replace(op, "webAPIHandlers", "s3", 1)
op = strings.Replace(op, "adminAPIHandlers", "admin", 1)
op = strings.Replace(op, "(*storageRESTServer)", "internal", 1)
op = strings.Replace(op, "(*peerRESTServer)", "internal", 1)
op = strings.Replace(op, "(*lockRESTServer)", "internal", 1)
op = strings.Replace(op, "stsAPIHandlers", "sts", 1)
op = strings.Replace(op, "LivenessCheckHandler", "healthcheck", 1)
op = strings.Replace(op, "ReadinessCheckHandler", "healthcheck", 1)
return op
}
Add admin API to send trace notifications to registered (#7128) Remove current functionality to log trace to file using MINIO_HTTP_TRACE env, and replace it with mc admin trace command on mc client.
2019-06-09 06:54:41 +08:00
// Trace gets trace of http request
func Trace(f http.HandlerFunc, logBody bool, w http.ResponseWriter, r *http.Request) trace.Info {
Trace: Replace function name with API prefix (#7794) This change is required for `Admin Trace`
2019-06-19 04:55:13 +08:00
name := getOpName(runtime.FuncForPC(reflect.ValueOf(f).Pointer()).Name())
Add admin API to send trace notifications to registered (#7128) Remove current functionality to log trace to file using MINIO_HTTP_TRACE env, and replace it with mc admin trace command on mc client.
2019-06-09 06:54:41 +08:00
log: Store http request/responses in a log file (#4804) When MINIO_TRACE_DIR is provided, create a new log file and store all HTTP requests + responses data, body are excluded to reduce memory consumption. MINIO_HTTP_TRACE=1 enables logging. Use non mem consuming http req/resp recorders, the maximum is about 32k per request. This logs to STDOUT, body logging is disabled for PutObject PutObjectPart GetObject.
2017-10-25 10:04:51 +08:00
bodyPlaceHolder := []byte("<BODY>")
Add admin API to send trace notifications to registered (#7128) Remove current functionality to log trace to file using MINIO_HTTP_TRACE env, and replace it with mc admin trace command on mc client.
2019-06-09 06:54:41 +08:00
var reqBodyRecorder *recordRequest
t := trace.Info{FuncName: name}
reqBodyRecorder = &recordRequest{Reader: r.Body, logBody: logBody}
r.Body = ioutil.NopCloser(reqBodyRecorder)
log: Store http request/responses in a log file (#4804) When MINIO_TRACE_DIR is provided, create a new log file and store all HTTP requests + responses data, body are excluded to reduce memory consumption. MINIO_HTTP_TRACE=1 enables logging. Use non mem consuming http req/resp recorders, the maximum is about 32k per request. This logs to STDOUT, body logging is disabled for PutObject PutObjectPart GetObject.
2017-10-25 10:04:51 +08:00
Add admin API to send trace notifications to registered (#7128) Remove current functionality to log trace to file using MINIO_HTTP_TRACE env, and replace it with mc admin trace command on mc client.
2019-06-09 06:54:41 +08:00
host, err := xnet.ParseHost(GetLocalPeer(globalEndpoints))
if err == nil {
t.NodeName = host.Name
}
Trace: Replace function name with API prefix (#7794) This change is required for `Admin Trace`
2019-06-19 04:55:13 +08:00
rq := trace.RequestInfo{Time: time.Now().UTC(), Method: r.Method, Path: r.URL.Path, RawQuery: r.URL.RawQuery, Client: r.RemoteAddr}
Add admin API to send trace notifications to registered (#7128) Remove current functionality to log trace to file using MINIO_HTTP_TRACE env, and replace it with mc admin trace command on mc client.
2019-06-09 06:54:41 +08:00
rq.Headers = cloneHeader(r.Header)
rq.Headers.Set("Content-Length", strconv.Itoa(int(r.ContentLength)))
rq.Headers.Set("Host", r.Host)
for _, enc := range r.TransferEncoding {
rq.Headers.Add("Transfer-Encoding", enc)
}
if logBody {
// If body logging is disabled then we print <BODY> as a placeholder
// for the actual body.
rq.Body = reqBodyRecorder.Data()
} else {
rq.Body = bodyPlaceHolder
}
// Setup a http response body recorder
respBodyRecorder := &recordResponseWriter{ResponseWriter: w, logBody: logBody}
f(respBodyRecorder, r)
rs := trace.ResponseInfo{Time: time.Now().UTC()}
rs.Headers = cloneHeader(respBodyRecorder.Header())
rs.StatusCode = respBodyRecorder.statusCode
if rs.StatusCode == 0 {
rs.StatusCode = http.StatusOK
}
bodyContents := respBodyRecorder.Body()
if bodyContents != nil {
rs.Body = bodyContents
}
if !logBody {
// If there was no error response and body logging is disabled
// then we print <BODY> as a placeholder for the actual body.
rs.Body = bodyPlaceHolder
log: Store http request/responses in a log file (#4804) When MINIO_TRACE_DIR is provided, create a new log file and store all HTTP requests + responses data, body are excluded to reduce memory consumption. MINIO_HTTP_TRACE=1 enables logging. Use non mem consuming http req/resp recorders, the maximum is about 32k per request. This logs to STDOUT, body logging is disabled for PutObject PutObjectPart GetObject.
2017-10-25 10:04:51 +08:00
}
Add admin API to send trace notifications to registered (#7128) Remove current functionality to log trace to file using MINIO_HTTP_TRACE env, and replace it with mc admin trace command on mc client.
2019-06-09 06:54:41 +08:00
t.ReqInfo = rq
t.RespInfo = rs
return t
log: Store http request/responses in a log file (#4804) When MINIO_TRACE_DIR is provided, create a new log file and store all HTTP requests + responses data, body are excluded to reduce memory consumption. MINIO_HTTP_TRACE=1 enables logging. Use non mem consuming http req/resp recorders, the maximum is about 32k per request. This logs to STDOUT, body logging is disabled for PutObject PutObjectPart GetObject.
2017-10-25 10:04:51 +08:00
}