Commit Graph

12554 Commits

Author SHA1 Message Date
Aditya Manthramurthy 72a0d14195
fix: avoid useless `expires` value in listing meta (#20584)
When listing objects with metadata, avoid returning an "expires" time
metadata value when its value is the zero time as this means that no
expires value is set on the object.
2024-10-24 19:13:19 -07:00
Klaus Post 6abe4128d7
Fix ILM expire workers exiting (#20578)
Fix expire workers exiting

Under 2 conditions ILM expire workers would exit, eventually causing all workers to terminate.
2024-10-23 08:35:37 -07:00
Klaus Post ed5ed7e490
Trace ILM errors (#20576)
Some paths would attempt transitions but in case of failures 
no traces would be emitted.

Add traces (with errors) when transition operations fail.
2024-10-22 14:10:34 -07:00
Klaus Post 51410c9023
Clear omitted fields (#20575)
Searched `msg:"[a-zA-Z0-9]*,omitempty` through the codebase.

Uses latest tinylib master.
2024-10-22 08:30:50 -07:00
Shubhendu 96ca402dcd
Correct the date filter check for batch replication (#20569)
The condition were incorrect as we were comparing the filter
value against the modification time object.

For example if created after filter date is after modification
time of object, that means object was created before the filter
time and should be skipped while replication because as per the
filter we need only the objects created after the filter date.

Signed-off-by: Shubhendu Ram Tripathi <shubhendu@minio.io>
2024-10-18 08:32:09 -07:00
Anis Eleuch 3da7c9cce3 repl: Fix removal of replicator svc when keycloak is configured (#120)
When Keycloak vendor is set, the code will start to clean up service
accounts that parents do not exist anymore. However, the code will also
look for the parent user of site-replicator-0, MINIO_ROOT_USER, which
obviously does not exist in Keycloak. Therefore, the site-replicator-0
will be removed automatically.

This commit will avoid cleaning up service accounts generated from
the root user.
2024-10-14 09:35:37 -07:00
Harshavardhana a14e19ec54 remove support for s390x 2024-10-13 07:06:17 -07:00
Minio Trusted e091dde041 Update yaml files to latest version RELEASE.2024-10-13T13-34-11Z 2024-10-13 14:05:41 +00:00
Harshavardhana d10bb7e1b6 remove reference to s390x binary, we wont publish them anymore 2024-10-13 06:34:11 -07:00
Anis Eleuch 7ebceacac6 heal: Fix deep scan failing to heal objects (#117)
The verify file handler response format was changed from gob to msgp
since two months but we forgot updating the verify handler client.

VerifyFile is only called during a heal deep scan (bitrot check).
HealObject() will fail in that case and will mark all disks corrupted and
will return early (as unrecoverable object but it will also not be
removed)

It is a bit rare for HealObject to be called with a deep scan flag. It
is called when a HealObject with a normal scan (e.g. new drive healing)
detects a bitrot corruption, therefore healing objects with a detected
bitrot corruption will fail.
2024-10-13 06:07:21 -07:00
Harshavardhana 1593cb615d
avoid unnecessary logging for KMS secret key mismatch (#20549) 2024-10-13 06:06:08 -07:00
Yannis Mazzer 86a41d1631
fix(helm) removing clusterDomain from startup command to avoid local … (#20547) 2024-10-11 05:21:05 -07:00
Taran Pelkey d4157b819c
Allow LDAP DNs with slashes to be loaded from object store (#20541) 2024-10-10 16:40:37 -07:00
Yannis Mazzer e0aceca1b7
feat(helm) making securityContext consistent (#20546) 2024-10-10 08:48:31 -07:00
Harrison Brown 87804624fe
Helm: Add extraVolumes and extraVolumeMounts to the customCommandJob section (#19988)
added extraVolumes and extraVolumeMounts to the customCommandJob section
2024-10-10 08:48:16 -07:00
Poorna e029f8a9d7
set kms keyid in replication opts (#20542) 2024-10-09 23:49:55 -07:00
Poorna 1bc6681176
fix tagging overwrite during resync (#20525) 2024-10-04 22:16:15 -07:00
Poorna 28322124e2
remove replication stats from data usage cache (#20524)
This is no longer needed since historical stats are not maintained anymore.
2024-10-04 15:23:33 -07:00
Harshavardhana cbfe9de3e7
do not download binary before verifying the version (#20523)
fixes https://github.com/minio/mc/issues/4980
2024-10-04 04:32:32 -07:00
Harshavardhana dc86b8d9d4
fix: when readQuorum, inconsistent metadata return 404 (#20522)
in cases where we cannot possibly know a way to read and 
construct the object,  it is impossible to achieve any form of 
quorum via xl.meta while we have sufficient responses from 
all the drives, we should return object not found.
2024-10-04 00:13:14 -07:00
Taran Pelkey ba70118e2b
Add root user to `ListAccessKeysBulk` (#20517) 2024-10-03 16:11:02 -07:00
Minio Trusted cb1d3e50f7 Update yaml files to latest version RELEASE.2024-10-02T17-50-41Z 2024-10-02 21:26:16 +00:00
Harshavardhana ded0b19d97
avoid audit logs with unexpected errors (#20516)
fixes #20513
2024-10-02 10:50:41 -07:00
Poorna d0bb3dd136
list all batch job types (#20510)
continues #20480
2024-10-01 23:38:17 -07:00
Harshavardhana ab7714b01e
upgrade relevant dependencies (#20507) 2024-10-01 23:37:55 -07:00
Ramon de Klein e5b18df6db
Fix checksum error during startup when `minio` is loaded via `PATH` environment variable (#20509) 2024-10-01 15:13:18 -07:00
Anis Eleuch 0abfd1bcb1
heal: Use etag as quorum when none found for modtime (#20500) 2024-10-01 08:19:10 -07:00
Harshavardhana 6186d11761
handle the locks properly for multi-pool callers (#20495)
- PutObjectMetadata()
- PutObjectTags()
- DeleteObjectTags()
- TransitionObject()
- RestoreTransitionObject()

Also improve the behavior of multipart code across
pool locks, hold locks only once per upload ID for

- CompleteMultipartUpload()
- AbortMultipartUpload()
- ListObjectParts() (read-lock)
- GetMultipartInfo() (read-lock)
- PutObjectPart() (read-lock)

This avoids lock attempts across pools for no
reason, this increases O(n) when there are n-pools.
2024-09-29 15:40:36 -07:00
Poorna e8b457e8a6
Change delete marker proxy test to use distributed setup (#20494) 2024-09-27 18:02:26 -07:00
Harshavardhana afea40cc0f fix: keep locks based on the first pool, first EC set (#93)
multi-object deletion may or may not compete with locks
granted for other callers, causing concurrent operations
to succeed on each other.

A continuation of the PR https://github.com/minio/minio/pull/20356
2024-09-27 03:41:37 -07:00
Aditya Manthramurthy 402b798f1b
fix: allow all console actions with custom authZ (#20489)
When custom authorization via plugin is enabled, the console will now
render the UI as if all actions are allowed. Since server cannot
determine the exact policy allowed for a user via the plugin, this is
acceptable to do. If a particular action is actually not allowed by the
plugin the call will result in an error.

Previously the server was evaluating a policy when custom authZ is
enabled - this is fixed now.
2024-09-26 23:44:44 -07:00
Klaus Post 4759532e90
Fix PPC cgroup memory limit (#20488)
The "unlimited" value on PPC wasn't exactly the same as amd64.

Instead compare against an "unreasonably big value".

Would cause OOM in anything using the concurrent request limit.
2024-09-26 10:07:10 -07:00
Harshavardhana 7f1e1713ab
use absolute path for binary checksum verification (#20487) 2024-09-26 08:03:08 -07:00
Poorna b2c5819dbc
hold on to batch job stats till cleanup (#20480)
This PR also fixes job stats not available after restart
2024-09-24 14:50:11 -07:00
Anis Eleuch 2b0156b1fc Add TTFB to all APIs and enable for responses without body (#20479)
Add TTFB for all requests in metrics-v3 in addition to the existing
GetObject. Also for the requests that do not return a body in the
response, calculate TTFB as the HTTP status code and the headers are
sent.
2024-09-24 10:13:00 -07:00
Harshavardhana f6f0807c86
cleanup existing part.N's before renamePart() (#20466)
this is a safety-net to avoid any unexpected parts to show up.
2024-09-24 04:26:41 -07:00
Harshavardhana 0c53d86017
remove the list from 'mc stat' from testing via '--no-list' (#20468)
avoid 'listing' as it may get incorrect results for
these tests, we are only interested in 'mc stat' as
in HEAD object here.
2024-09-24 01:03:58 -07:00
Minio Trusted 6a6ee46d76 Update yaml files to latest version RELEASE.2024-09-22T00-33-43Z 2024-09-23 19:42:46 +00:00
Klaus Post 974cbb3bb7
Limit jstream parse depth (#20474)
Add https://github.com/bcicen/jstream/pull/15 by vendoring the package.

Sets JSON depth limit to 100 entries in S3 Select.
2024-09-23 12:35:41 -07:00
Harshavardhana 03e996320e
upgrade deps pkg/v3, madmin-go/v3 and lz4/v4 (#20467) 2024-09-21 17:33:43 -07:00
Taran Pelkey 78fcb76294
Add `ListAccessKeysBulk` API for builtin user access keys (#20381) 2024-09-21 04:35:40 -07:00
Ramon de Klein 3d152015eb
Use MinIO console v1.7.1 (#20465) 2024-09-20 18:18:54 -07:00
jiuker ade8925155
fix: add default http timeout for audit webhook (#20460) 2024-09-20 09:02:50 -07:00
Klaus Post 05a6c170bf
Fix PutObject Trailing checksum (#20456)
PutObject would verify trailing checksums, but not store them.

Fixes #20455
2024-09-19 05:59:07 -07:00
Ramon de Klein e1c2344591
Log an error when calculating the binary checksum failed (#20454) 2024-09-18 20:48:32 -07:00
Ramon de Klein 48a591e9b4
Ensure proper `stale_uploads_cleanup_interval` is used at all times (#20451) 2024-09-18 10:59:26 -07:00
Anis Eleuch fa5d9c02ef
batch: Set a default retry attempts and a prefix (#20452)
A batch job will fail if the retry attempt is not provided. The reason
is that the code mistakenly gets the retry attempts from the job status
rather than the job yaml file.

This will also set a default empty prefix for batch expiration.

Also this will avoid trimming the prefix since the yaml decoder already
does that if no quotes were provided, and we should not trim if quotes
were provided and the user provided a leading or a trailing space.
2024-09-18 10:59:03 -07:00
Shubhendu 5bd27346ac
Added iam import tests for openid (#20432)
Tests if imported service accounts have 
required access to buckets and objects.

Signed-off-by: Shubhendu Ram Tripathi <shubhendu@minio.io>

Co-authored-by: Harshavardhana <harsha@minio.io>
2024-09-17 09:45:46 -07:00
Taran Pelkey 3c82cf9327
Fix behavior of `AddServiceAccountLDAP` for non-admin users (#20442) 2024-09-16 16:04:51 -07:00
Harshavardhana 70d40083e9
remove windows CI/CD for now (#20441)
windows has decided to be a community support
only and source compile-friendly.
2024-09-16 13:46:53 -07:00