Commit Graph

4036 Commits

Author SHA1 Message Date
kjnilsson 52f745dcde Update rabbitmq-components.mk
use v1.x branch of ra
2021-03-18 15:14:40 +00:00
Loïc Hoguin d5e3bdd623
Add ADDITIONAL_PLUGINS variable
This allows including additional applications or third party
plugins when creating a release, running the broker locally,
or just building from the top-level Makefile.

To include Looking Glass in a release, for example:

$ make package-generic-unix ADDITIONAL_PLUGINS="looking_glass"

A Docker image can then be built using this release and will
contain Looking Glass:

$ make docker-image

Beware macOS users! Applications such as Looking Glass include
NIFs. NIFs must be compiled in the right environment. If you
are building a Docker image then make sure to build the NIF
on Linux! In the two steps above, this corresponds to Step 1.

To run the broker with Looking Glass available:

$ make run-broker ADDITIONAL_PLUGINS="looking_glass"

This commit also moves Looking Glass dependency information
into rabbitmq-components.mk so it is available at all times.
2021-03-12 12:29:28 +01:00
Michael Klishin 97ff62d3b2
Drop trailing newlines from logged messages where possible
Lager strips trailing newline characters but OTP logger with the default
formatter adds a newline at the end. To avoid unintentional multi-line log
messages we have to revisit most messages logged.

Some log entries are intentionally multiline, others
are printed to stdout directly: newlines are required there
for sensible formatting.
2021-03-11 15:17:37 +01:00
Jean-Sébastien Pédron cdcf602749
Switch from Lager to the new Erlang Logger API for logging
The configuration remains the same for the end-user. The only exception
is the log root directory: it is now set through the `log_root`
application env. variable in `rabbit`. People using the Cuttlefish-based
configuration file are not affected by this exception.

The main change is how the logging facility is configured. It now
happens in `rabbit_prelaunch_logging`. The `rabbit_lager` module is
removed.

The supported outputs remain the same: the console, text files, the
`amq.rabbitmq.log` exchange and syslog.

The message text format slightly changed: the timestamp is more precise
(now to the microsecond) and the level can be abbreviated to always be
4-character long to align all messages and improve readability. Here is
an example:

    2021-03-03 10:22:30.377392+01:00 [dbug] <0.229.0> == Prelaunch DONE ==
    2021-03-03 10:22:30.377860+01:00 [info] <0.229.0>
    2021-03-03 10:22:30.377860+01:00 [info] <0.229.0>  Starting RabbitMQ 3.8.10+115.g071f3fb on Erlang 23.2.5
    2021-03-03 10:22:30.377860+01:00 [info] <0.229.0>  Licensed under the MPL 2.0. Website: https://rabbitmq.com

The example above also shows that multiline messages are supported and
each line is prepended with the same prefix (the timestamp, the level
and the Erlang process PID).

JSON is also supported as a message format and now for any outputs.
Indeed, it is possible to use it with e.g. syslog or the exchange. Here
is an example of a JSON-formatted message sent to syslog:

    Mar  3 11:23:06 localhost rabbitmq-server[27908] <0.229.0> - {"time":"2021-03-03T11:23:06.998466+01:00","level":"notice","msg":"Logging: configured log handlers are now ACTIVE","meta":{"domain":"rabbitmq.prelaunch","file":"src/rabbit_prelaunch_logging.erl","gl":"<0.228.0>","line":311,"mfa":["rabbit_prelaunch_logging","configure_logger",1],"pid":"<0.229.0>"}}

For quick testing, the values accepted by the `$RABBITMQ_LOGS`
environment variables were extended:
  * `-` still means stdout
  * `-stderr` means stderr
  * `syslog:` means syslog on localhost
  * `exchange:` means logging to `amq.rabbitmq.log`

`$RABBITMQ_LOG` was also extended. It now accepts a `+json` modifier (in
addition to the existing `+color` one). With that modifier, messages are
formatted as JSON intead of plain text.

The `rabbitmqctl rotate_logs` command is deprecated. The reason is
Logger does not expose a function to force log rotation. However, it
will detect when a file was rotated by an external tool.

From a developer point of view, the old `rabbit_log*` API remains
supported, though it is now deprecated. It is implemented as regular
modules: there is no `parse_transform` involved anymore.

In the code, it is recommended to use the new Logger macros. For
instance, `?LOG_INFO(Format, Args)`. If possible, messages should be
augmented with some metadata. For instance (note the map after the
message):

    ?LOG_NOTICE("Logging: switching to configured handler(s); following "
                "messages may not be visible in this log output",
                #{domain => ?RMQLOG_DOMAIN_PRELAUNCH}),

Domains in Erlang Logger parlance are the way to categorize messages.
Some predefined domains, matching previous categories, are currently
defined in `rabbit_common/include/logging.hrl` or headers in the
relevant plugins for plugin-specific categories.

At this point, very few messages have been converted from the old
`rabbit_log*` API to the new macros. It can be done gradually when
working on a particular module or logging.

The Erlang builtin console/file handler, `logger_std_h`, has been forked
because it lacks date-based file rotation. The configuration of
date-based rotation is identical to Lager. Once the dust has settled for
this feature, the goal is to submit it upstream for inclusion in Erlang.
The forked module is calld `rabbit_logger_std_h` and is based
`logger_std_h` in Erlang 23.0.
2021-03-11 15:17:36 +01:00
dcorbacho 61f7b2a723 Update to ranch 2.0 2021-03-08 23:11:05 +01:00
Michael Klishin b6c4831e75
Bump Lager to 3.9.1 2021-03-04 04:36:39 +03:00
Michael Klishin ad47eba2fa
pg2 => pg for OTP 24 compatibility
there is still one failing queue federation test.
2021-03-04 04:06:40 +03:00
Loïc Hoguin 66ac1bf5e9
Bump observer_cli to 1.6.1
More responsive when the system is overloaded with file calls.
2021-03-01 21:55:27 +03:00
Michael Klishin b981cead37
Correct a child spec typo 2021-03-01 21:55:26 +03:00
Michael Klishin 8cbf5a37c3
Regular supervisor is sufficient here 2021-03-01 21:55:25 +03:00
Michael Klishin c01706a758
Extract a constant 2021-03-01 21:55:25 +03:00
Michael Klishin 99ab1eda31
OTP 24 compat: switch management and management_agent to use pg
since pg2 was removed in OTP 24.

The only decision worth mentioning here is that both plugins share
a pg scope, which is started in rabbitmq_management_agent supervision
tree and idempotently started in rabbitmq_management without
attaching the scope pid to its tree.

Per discussion with @lhoguin.
2021-03-01 21:55:25 +03:00
Michael Klishin 8fe3df9343
Upgrade Lager to 3.9.0 for OTP 24 compatibility
`lager_util:expand_path/1` use changes are
due to erlang-lager/lager#540
2021-02-26 00:52:15 +03:00
Michael Klishin 17b082abeb
Merge pull request #2843 from rabbitmq/consumer-capacity
Rename consumer_utilisation to consumer_capacity
2021-02-25 16:17:09 +03:00
Michael Klishin b04d1cc3db
Improve UI help hint for consumer capacity 2021-02-24 21:27:04 +03:00
dcorbacho 930c78795c Rename consumer_utilisation to consumer_capacity
Capacity is 100% when there are online consumers and no messages
2021-02-24 16:20:52 +01:00
Michael Klishin f73e851f9c
Bump observer_cli to 1.6.0 2021-02-24 12:53:55 +03:00
Michael Klishin a5098b28a7
Bump Lager to 3.8.2 for OTP 24 compatibility 2021-02-24 12:53:30 +03:00
Michael Klishin a71771b8c1
Correct a health check unit validation message 2021-02-18 13:23:12 +03:00
Lajos Gerecs 8d1afe42e1 fix search field for values containing spaces
The value of the input field was not surrounded by quotes properly,
which made values containing spaces not show up correctly.
2021-02-17 13:30:38 +01:00
Michael Klishin b11a79cccf
Bump (c) year in header files 2021-02-04 07:04:58 +03:00
Michal Kuratczyk 8b9c1ba297
Update link to the reference docs 2021-02-01 19:55:54 +01:00
Arnaud Cogoluègnes b921ac11a8
Merge pull request #2712 from rabbitmq/rabbitmq-stream-prometheus
Add stream prometheus plugin
2021-01-27 16:46:37 +01:00
Arnaud Cogoluègnes 268bc7257c
Merge pull request #2729 from rabbitmq/rabbitmq-stream-consumers-in-queue-details
Report stream consumers as regular consumers in management
2021-01-27 16:06:59 +01:00
Michael Klishin 52479099ec
Bump (c) year 2021-01-22 09:00:14 +03:00
Arnaud Cogoluègnes 1e583abd43
Add help tooltip for consumers channel column
To explain the channel can actually be a stream connection.
2021-01-21 09:29:36 +01:00
Arnaud Cogoluègnes b88ad3d265
Adapt consumer link in queue details page
Depending on whether the "owner" of the consumer is a channel (for AMQP
consumers) or a connection (for stream consumers) and whether the stream
management plugin is enabled or not. The link will lead to the channel
page, the connection page, or the stream connection page.
2021-01-20 17:01:31 +01:00
Arnaud Cogoluègnes 78b723d7c3
Report stream consumers as regular consumers in management
This helps to see that a queue is consumed from.

WIP
2021-01-20 15:36:16 +01:00
Arnaud Cogoluègnes b5315c0166
Merge branch 'master' into rabbitmq-stream-prometheus 2021-01-18 11:26:06 +01:00
Michael Klishin 07a0dd7438
Make sure legacy rabbit_management_load_definitions step does not block boot
Follow-up to #2656, #2384.
2021-01-15 04:40:32 +05:00
Michael Klishin a28c554a89
Merge pull request #2680 from rabbitmq/rabbitmq-managenet-2615
HTTP API reference: add documentation for pagination parameters
2021-01-13 20:52:23 +03:00
Michael Klishin ad4cbad789
Don't unintentionally gitignore HTTP API doc reference 2021-01-13 20:51:17 +03:00
Michael Klishin 2ade2bf67f
HTTP API doc reference: wording, markup 2021-01-13 20:50:22 +03:00
Arnaud Cogoluègnes bf72683eb2
Add stream prometheus plugin 2021-01-11 16:49:56 +01:00
Arnaud Cogoluègnes cbd3c8dfdd
Merge branch 'master' into rabbitmq-stream-management 2021-01-04 09:50:47 +01:00
Gabriele Santomaggio c2cc4b75de Add documentation for the pagination
fixes: https://github.com/rabbitmq/rabbitmq-server/issues/2615
2020-12-14 15:30:15 +01:00
Arnaud Cogoluègnes 335c7b4270
Do not split user tags in management UI
As they are no longer a comma-separated list in a string.

References #2667
2020-12-11 16:57:33 +01:00
Michael Klishin 10ced3cbd4
Adapt HTTP API test suite expectations 2020-12-10 15:27:17 +03:00
Arnaud Cogoluègnes 7bbbe3bb12
Add proper title to stream connections page 2020-12-09 09:29:47 +01:00
Arnaud Cogoluègnes 3f1d010ecd
Add stream connection consumers REST endpoint 2020-12-07 15:56:23 +01:00
Arnaud Cogoluègnes 41160210ab
Start stream consumer REST endpoint 2020-12-01 18:23:22 +01:00
Arnaud Cogoluègnes ed12ae0bbf
Add permission tests for stream management 2020-11-23 16:32:14 +01:00
Arnaud Cogoluègnes 23d7e8114c
Introduce stream management plugin 2020-11-19 14:48:25 +01:00
Michael Klishin 142c9e3cc7
Merge pull request #2627 from rabbitmq/stream-purge
Hide Purge option for stream queues
2020-11-17 16:56:31 +03:00
dcorbacho a1f2284479 Hide Purge option for stream queues 2020-11-17 13:26:06 +00:00
Jean-Sébastien Pédron 47686ee1f0
Remove unused .github directories
They were valid until the switch to the "monorepository" when everything
was merged into a single Git repository.
2020-11-17 13:33:16 +01:00
Michael Klishin dda288a8c3 Merge pull request #864 from rabbitmq/lrb-confirm-timeout-is-seconds
wait_for_confirms timeout is in seconds
2020-11-03 17:48:29 +03:00
Arnaud Cogoluègnes 8e11b894b7 Update rabbitmq-components.mk 2020-11-03 14:27:41 +01:00
Luke Bakken 55c3f3670f wait_for_confirms timeout is in seconds
References rabbitmq/rabbitmq-erlang-client#138

cc @dumbbell
2020-11-02 11:20:15 -08:00
Arnaud Cogoluègnes 27df24170a Add gen_server:call to PID on closing connection request
This allows to send a closing message to non-direct, non-network
connection processes.

The stream plugin will make use of it.
2020-10-23 11:16:07 +02:00
Jean-Sébastien Pédron ac303a2c74 rabbit_mgmt_http_health_checks_SUITE: Remove trailing whitespaces 2020-10-22 15:00:04 +02:00
Jean-Sébastien Pédron 6864929fe8 rabbit_mgmt_http_health_checks_SUITE: Don't import unused functions 2020-10-22 14:59:41 +02:00
Jean-Sébastien Pédron d39b691216 rabbit_mgmt_http_health_checks_SUITE: Pay attention to rabbit_ct_broker_helpers:enable_feature_flag() return value
If it returns `{skip, _}`, we must skip the test group.
2020-10-22 14:58:47 +02:00
Arnaud Cogoluègnes 675375928d Merge pull request #860 from rabbitmq/stream-queue-cluster-args
Add missing queue arguments for stream and quorum queues
2020-10-21 16:45:01 +02:00
Arnaud Cogoluègnes 2479628d05 Arrange queue arguments in UI
References rabbitmq/rabbitmq-server#2471
2020-10-21 16:41:54 +02:00
Michael Klishin 0d993d6983 Update rabbitmq-components.mk 2020-10-21 12:55:37 +03:00
dcorbacho e395a5223c Add missing queue arguments for stream and quorum queues 2020-10-20 13:33:55 +01:00
Michael Klishin 22f447157f Include top-level port property for TLS listeners
This value is used at listener registration and will show up in
`rabbitmq-diagnostics listeners' output.

Closes #857
2020-10-19 13:48:21 +03:00
Michael Klishin c2350b1f59 Update tests to not use foobar x-arguments
Queues now validate x-arguments in addition to policy definitions.
2020-10-19 12:58:38 +03:00
Michael Klishin 9ce361813d Merge pull request #858 from rabbitmq/document-health-checks
Document new health check endpoints
2020-10-19 08:25:25 +03:00
Michael Klishin 4fd2ec4acf Clarify 2020-10-19 08:25:06 +03:00
Michael Klishin 9f4cea0f98 Further updates to health check endpoint docs
* Remove documentation to the now deprecated One True Health Check™.
   It is too intrusive and too prone to false positives to recommend.
 * Aliveness test is also a health check, and will respond with a 200
   or 503 as of #856
2020-10-19 08:13:54 +03:00
Michael Klishin c1a1bc9393 Simpler wording for new health check endpoint docs 2020-10-19 07:57:15 +03:00
dcorbacho ea37d7dd67 Document new health check endpoints 2020-10-16 11:48:56 +01:00
Michael Klishin 07ec5b6fc3 Merge pull request #856 from rabbitmq/rabbitmq-management-855
Take error scenarios into account in aliveness test
2020-10-16 05:43:17 +03:00
Michael Klishin 5377d5e7fb /auth-attempts => /auth/attempts in one more place 2020-10-14 20:27:07 +03:00
Michael Klishin b6f775fa24 /api/auth-attempts/ => /api/auth/attempts 2020-10-14 08:03:17 +03:00
dcorbacho 2947906180 Document auth attempts endpoints 2020-10-14 05:18:49 +03:00
dcorbacho db637b6d56 Split get auth attempts API into global counters and detailed by source 2020-10-14 05:18:49 +03:00
dcorbacho 86bc48fc81 Query and rest auth attempt metrics 2020-10-14 05:18:48 +03:00
Luke Bakken 795f255873 Update Travis CI config 2020-10-13 16:35:19 -07:00
Luke Bakken a2b0df6524 Use -include directive when appropriate 2020-10-13 12:00:33 -07:00
Luke Bakken d236a5139c erlfmt 2020-10-13 09:58:59 -07:00
Luke Bakken 0e9564882b Take error scenarios into account
For instance, if `message-ttl` is set to `0`, there will never be a
message in the queue for the check. In this case, return `503`

Fixes #855
2020-10-13 09:57:31 -07:00
Michael Klishin f3b4caa996 Finish cluster-wide and local alarm health check tests 2020-10-07 22:14:49 +03:00
dcorbacho 9d04d95250 Test health checks 2020-10-07 15:27:00 +01:00
Michael Klishin c2e7d0e04b HTTP API health check test suite WIP 2020-10-07 12:40:08 +03:00
Michael Klishin 0e8798c6a3 Strip trailing whitespace 2020-10-06 14:16:18 +03:00
dcorbacho ad805eb091 Introduce health checks
GET /api/health/checks/certificate-expiration
GET /api/health/checks/port-listener
GET /api/health/checks/protocol-listener
GET /api/health/checks/virtual-hosts
GET /api/health/checks/node-is-mirror-sync-critical
GET /api/health/checks/node-is-quorum-critical
2020-10-06 12:11:23 +01:00
Michael Klishin d7ee7fb48b Introduce GET /api/health/checks/local-alarms 2020-10-05 20:49:17 +03:00
Michael Klishin ec368d9248 Add alarm details to GET /api/health/checks/alarms 2020-10-05 17:12:20 +03:00
Michael Klishin 1cbd84fa92 Return info message as a binary 2020-10-05 17:12:20 +03:00
Michael Klishin f619703c3d Introduce rabbit_alarms:get_alarms/1 2020-10-05 17:12:20 +03:00
Luke Bakken b24ed793a8 Use newer Elixir for testing
See rabbitmq/rabbitmq-ci#32
2020-10-01 13:04:21 -07:00
kjnilsson a8947ac6f0 Uncomment some tests 2020-09-30 14:29:01 +01:00
dcorbacho e11f517a87 Add stream policies 2020-09-30 14:29:01 +01:00
dcorbacho 3cca5fc685 Stream queues memory breakdown 2020-09-30 14:29:01 +01:00
dcorbacho 95ec943ced Hide 'Get messages' drop down for stream queues
basic.get operation is not supported
2020-09-30 14:29:01 +01:00
dcorbacho 0686190f15 Stream queue
[#171206871]
2020-09-30 14:29:01 +01:00
kjnilsson 323834a7fd Restore the pre 23.1 behavour of lists:sublist/3
As we depend on it to detect an out of range request and return an HTTP
400 response.
2020-09-30 14:03:28 +01:00
Michael Klishin 44c34a4b21 Update GH Actions badges 2020-09-25 20:27:50 +03:00
Michael Klishin fa37a6b6de Exclude dialyzer job from dependencies 2020-09-25 20:20:42 +03:00
Michael Klishin 5a10fd7d3c Define a (no-op in this repo) dialyzer job for GH Actions 2020-09-25 20:19:42 +03:00
Michael Klishin 70be13b90d Merge pull request #849 from rabbitmq/github-actions-otp-22.3-and-23.0
GitHub Actions: test on OTP 22.3 and 23
2020-09-25 19:56:34 +03:00
Michael Klishin 9cf5af5904 Commit Dialyzer action override 2020-09-25 19:14:50 +03:00
Michael Klishin 4fc36acce7 Disable Dialyzer, use Elixir 1.8.0 2020-09-25 19:14:32 +03:00
Michael Klishin cda4ab91f5 Correct more Dialyzer errors 2020-09-25 02:17:23 +03:00
Michael Klishin 2d14393963 Simplify 2020-09-25 02:11:29 +03:00
Michael Klishin 3a84afbced Correct more Dialyzer errors 2020-09-25 02:05:55 +03:00
Michael Klishin 6f89596c4a Some Dialyzer error fixes 2020-09-24 23:31:55 +03:00