redis/tests/unit/moduleapi
Raz Monsonego 04589f90d7
Add internal connection and command mechanism (#13740)
# PR: Add Mechanism for Internal Commands and Connections in Redis

This PR introduces a mechanism to handle **internal commands and
connections** in Redis. It includes enhancements for command
registration, internal authentication, and observability.

## Key Features

1. **Internal Command Flag**:
   - Introduced a new **module command registration flag**: `internal`.
- Commands marked with `internal` can only be executed by **internal
connections**, AOF loading flows, and master-replica connections.
- For any other connection, these commands will appear as non-existent.

2. **Support for internal authentication added to `AUTH`**:
- Used by depicting the special username `internal connection` with the
right internal password, i.e.,: `AUTH "internal connection"
<internal_secret>`.
- No user-defined ACL username can have this name, since spaces are not
aloud in the ACL parser.
   - Allows connections to authenticate as **internal connections**.
- Authenticated internal connections can execute internal commands
successfully.

4. **Module API for Internal Secret**:
- Added the `RedisModule_GetInternalSecret()` API, that exposes the
internal secret that should be used as the password for the new `AUTH
"internal connection" <password>` command.
- This API enables the modules to authenticate against other shards as
local connections.

## Notes on Behavior

- **ACL validation**:
- Commands dispatched by internal connections bypass ACL validation, to
give the caller full access regardless of the user with which it is
connected.

- **Command Visibility**:
- Internal commands **do not appear** in `COMMAND <subcommand>` and
`MONITOR` for non-internal connections.
- Internal commands **are logged** in the slow log, latency report and
commands' statistics to maintain observability.

- **`RM_Call()` Updates**:
  - **Non-internal connections**:
- Cannot execute internal commands when the command is sent with the `C`
flag (otherwise can).
- Internal connections bypass ACL validations (i.e., run as the
unrestricted user).

- **Internal commands' success**:
- Internal commands succeed upon being sent from either an internal
connection (i.e., authenticated via the new `AUTH "internal connection"
<internal_secret>` API), an AOF loading process, or from a master via
the replication link.
Any other connections that attempt to execute an internal command fail
with the `unknown command` error message raised.

- **`CLIENT LIST` flags**:
  - Added the `I` flag, to indicate that the connection is internal.

- **Lua Scripts**:
   - Prevented internal commands from being executed via Lua scripts.

---------

Co-authored-by: Meir Shpilraien <meir@redis.com>
2025-02-05 11:48:08 +02:00
..
aclcheck.tcl modules API: Add test for ACL check of empty prefix (#13678) 2024-12-10 09:16:30 +02:00
async_rm_call.tcl Before evicted and before expired server events are not executed inside an execution unit. (#12733) 2023-11-08 09:28:22 +02:00
auth.tcl Fix module redact test for valgrind (#10432) 2022-03-16 08:53:57 +02:00
basics.tcl Add flag for ability of a module context to execute debug commands (#13774) 2025-02-03 09:52:41 +02:00
blockedclient.tcl Async IO Threads (#13695) 2024-12-23 14:16:40 +08:00
blockonbackground.tcl Fix external test hang in redis-cli test when run in a certain order (#13423) 2024-07-17 15:42:28 +03:00
blockonkeys.tcl Modules: Unblock from within a timer coverage (#12337) 2023-06-22 23:15:16 +03:00
cluster.tcl Fix delKeysInSlot server events are not executed inside an execution unit (#12745) 2023-12-11 20:15:19 +02:00
cmdintrospection.tcl Add reply_schema to command json files (internal for now) (#10273) 2023-03-11 10:14:16 +02:00
commandfilter.tcl Un-register notification and server event when RedisModule_OnLoad fails (#12809) 2023-11-27 17:26:33 +02:00
crash.tcl Print command tokens on a crash when hide-user-data-from-log is enabled (#13639) 2024-11-11 09:34:18 +03:00
datatype.tcl Gradually reduce defrag CPU usage when defragmentation is ineffective (#13752) 2025-01-24 11:35:32 +08:00
datatype2.tcl Tests: don't rely on the response of MEMORY USAGE when mem_allocator is not jemalloc (#10010) 2021-12-27 21:37:21 +02:00
defrag.tcl Added new defrag API to allocate and free raw memory. (#13509) 2024-09-03 15:03:19 +03:00
eventloop.tcl sub-command support for ACL CAT and COMMAND LIST. redisCommand always stores fullname (#10127) 2022-01-23 10:05:06 +02:00
fork.tcl Fix async safety in signal handlers (#12658) 2023-11-23 13:22:20 +02:00
getchannels.tcl Implemented module getchannels api and renamed channel keyspec (#10299) 2022-02-22 11:00:03 +02:00
getkeys.tcl Unify ACL failure error messaging. (#11160) 2022-10-16 09:01:37 +03:00
hash.tcl Modules API: new HashFieldMinExpire(). Add flag REDISMODULE_HASH_EXPIRE_TIME to HashGet(). (#13676) 2024-12-05 11:14:52 +02:00
hooks.tcl Un-register notification and server event when RedisModule_OnLoad fails (#12809) 2023-11-27 17:26:33 +02:00
infotest.tcl Fix missing sections for INFO ALL with module (#11291) 2022-09-21 08:10:03 +03:00
infra.tcl Build TLS as a loadable module 2022-08-23 12:37:56 +03:00
internalsecret.tcl Add internal connection and command mechanism (#13740) 2025-02-05 11:48:08 +02:00
keyspace_events.tcl Un-register notification and server event when RedisModule_OnLoad fails (#12809) 2023-11-27 17:26:33 +02:00
keyspecs.tcl Unify ACL failure error messaging. (#11160) 2022-10-16 09:01:37 +03:00
list.tcl Add listpack encoding for list (#11303) 2022-11-16 20:29:46 +02:00
mallocsize.tcl Add RM_MallocSizeString, RM_MallocSizeDict (#10542) 2022-04-17 08:31:57 +03:00
misc.tcl Add API RedisModule_ClusterKeySlot and RedisModule_ClusterCanonicalKeyNameInSlot (#13069) 2024-03-12 09:26:12 -07:00
moduleauth.tcl Custom authentication for Modules (#11659) 2023-03-15 15:18:42 -07:00
moduleconfigs.tcl modules API: Support register unprefixed config parameters (#13656) 2024-11-21 09:55:02 +02:00
postnotifications.tcl Before evicted and before expired server events are not executed inside an execution unit. (#12733) 2023-11-08 09:28:22 +02:00
propagate.tcl Don't keep global replication buffer reference for replicas marked CLIENT_CLOSE_ASAP (#13363) 2024-06-26 08:26:23 +08:00
publish.tcl Fix broken protocol when PUBLISH emits local push inside MULTI (#12326) 2023-06-20 20:41:41 +03:00
rdbloadsave.tcl Fix RM_RdbLoad() to enable AOF after loading is completed (#13510) 2024-09-04 11:11:04 +03:00
reply.tcl Align RM_ReplyWithErrorFormat with RM_ReplyWithError (#12321) 2023-06-20 20:44:43 +03:00
scan.tcl Fix some test failures caused by key being deleted due to premature expiration (#13453) 2024-07-31 08:15:39 +08:00
stream.tcl Delete empty key if fails after moduleCreateEmptyKey() in module (#12129) 2023-05-07 10:13:19 +03:00
subcommands.tcl Fix regression not aborting transaction on error, and re-edit some error responses (#10612) 2022-04-25 13:08:13 +03:00
test_lazyfree.tcl Sort out mess around propagation and MULTI/EXEC (#9890) 2021-12-23 00:03:48 +02:00
testrdb.tcl Tests: Do not save an RDB by default and add a SIGTERM default AOFRW test (#12064) 2023-04-18 16:14:26 +03:00
timer.tcl forbid module to unload when it holds ongoing timer (#10187) 2022-02-01 14:54:11 +02:00
usercall.tcl WAITAOF: Update fsynced_reploff_pending even if there's nothing to fsync (#12622) 2023-09-28 17:19:20 +03:00
zset.tcl Delete empty key if fails after moduleCreateEmptyKey() in module (#12129) 2023-05-07 10:13:19 +03:00