Commit Graph

7 Commits

Author SHA1 Message Date
Michael Klishin f7d32d69f8 Introduce a new CLI tool (scope), rabbitmq-tanzu
For Tanzu (commercial) plugins to attach their commands to instead of
polluting rabbitmqctl.

Pair: @pjk25
(cherry picked from commit 6e0f2436fa)
2021-11-30 14:54:09 +00:00
Michael Klishin 040f8cc912
Replace a few more leftover MPLv1.1 license headers
Most files have been using the MPLv2 headers for months now.
These were detected by the OSL process.
2021-05-19 21:20:47 +03:00
Jean-Sébastien Pédron cdcf602749
Switch from Lager to the new Erlang Logger API for logging
The configuration remains the same for the end-user. The only exception
is the log root directory: it is now set through the `log_root`
application env. variable in `rabbit`. People using the Cuttlefish-based
configuration file are not affected by this exception.

The main change is how the logging facility is configured. It now
happens in `rabbit_prelaunch_logging`. The `rabbit_lager` module is
removed.

The supported outputs remain the same: the console, text files, the
`amq.rabbitmq.log` exchange and syslog.

The message text format slightly changed: the timestamp is more precise
(now to the microsecond) and the level can be abbreviated to always be
4-character long to align all messages and improve readability. Here is
an example:

    2021-03-03 10:22:30.377392+01:00 [dbug] <0.229.0> == Prelaunch DONE ==
    2021-03-03 10:22:30.377860+01:00 [info] <0.229.0>
    2021-03-03 10:22:30.377860+01:00 [info] <0.229.0>  Starting RabbitMQ 3.8.10+115.g071f3fb on Erlang 23.2.5
    2021-03-03 10:22:30.377860+01:00 [info] <0.229.0>  Licensed under the MPL 2.0. Website: https://rabbitmq.com

The example above also shows that multiline messages are supported and
each line is prepended with the same prefix (the timestamp, the level
and the Erlang process PID).

JSON is also supported as a message format and now for any outputs.
Indeed, it is possible to use it with e.g. syslog or the exchange. Here
is an example of a JSON-formatted message sent to syslog:

    Mar  3 11:23:06 localhost rabbitmq-server[27908] <0.229.0> - {"time":"2021-03-03T11:23:06.998466+01:00","level":"notice","msg":"Logging: configured log handlers are now ACTIVE","meta":{"domain":"rabbitmq.prelaunch","file":"src/rabbit_prelaunch_logging.erl","gl":"<0.228.0>","line":311,"mfa":["rabbit_prelaunch_logging","configure_logger",1],"pid":"<0.229.0>"}}

For quick testing, the values accepted by the `$RABBITMQ_LOGS`
environment variables were extended:
  * `-` still means stdout
  * `-stderr` means stderr
  * `syslog:` means syslog on localhost
  * `exchange:` means logging to `amq.rabbitmq.log`

`$RABBITMQ_LOG` was also extended. It now accepts a `+json` modifier (in
addition to the existing `+color` one). With that modifier, messages are
formatted as JSON intead of plain text.

The `rabbitmqctl rotate_logs` command is deprecated. The reason is
Logger does not expose a function to force log rotation. However, it
will detect when a file was rotated by an external tool.

From a developer point of view, the old `rabbit_log*` API remains
supported, though it is now deprecated. It is implemented as regular
modules: there is no `parse_transform` involved anymore.

In the code, it is recommended to use the new Logger macros. For
instance, `?LOG_INFO(Format, Args)`. If possible, messages should be
augmented with some metadata. For instance (note the map after the
message):

    ?LOG_NOTICE("Logging: switching to configured handler(s); following "
                "messages may not be visible in this log output",
                #{domain => ?RMQLOG_DOMAIN_PRELAUNCH}),

Domains in Erlang Logger parlance are the way to categorize messages.
Some predefined domains, matching previous categories, are currently
defined in `rabbit_common/include/logging.hrl` or headers in the
relevant plugins for plugin-specific categories.

At this point, very few messages have been converted from the old
`rabbit_log*` API to the new macros. It can be done gradually when
working on a particular module or logging.

The Erlang builtin console/file handler, `logger_std_h`, has been forked
because it lacks date-based file rotation. The configuration of
date-based rotation is identical to Lager. Once the dust has settled for
this feature, the goal is to submit it upstream for inclusion in Erlang.
The forked module is calld `rabbit_logger_std_h` and is based
`logger_std_h` in Erlang 23.0.
2021-03-11 15:17:36 +01:00
Carl Hörberg 413bfe7b37 Disable Erlang busy wait by default
By disabling Erlang busy wait threshold CPU usage with 5000 idle connection
drops from 110% to 14%. Throughput does not seem to be affected at all,
if any thing it actually goes up a bit when you have 5000 idle connections
(because less CPU cycles are wasted polling idle connections).

rabbitmq-perf-test-2.13.0/bin/runjava com.rabbitmq.perf.PerfTest -s 8000 -z 15

With default erlang busy wait threshold:
id: test-115706-497, sending rate avg: 39589 msg/s
id: test-115706-497, receiving rate avg: 39570 msg/s

With busy wait disabled:
id: test-115807-719, sending rate avg: 40340 msg/s
id: test-115807-719, receiving rate avg: 40301 msg/s

rabbitmq-diagnostics runtime_thread_stats output while running the
PerfTest:

with default busy wait threshold:

Stats per type:
         async    0.00%    0.00%    0.00%    0.00%    0.00%    0.00%  100.00%
           aux    0.01%    0.00%    0.00%    0.00%    0.00%    0.00%   99.98%
dirty_cpu_sche    0.00%    0.00%    0.00%    0.03%    0.05%    0.00%   99.92%
dirty_io_sched    0.00%    0.00%    0.00%    0.00%    0.01%    0.00%   99.99%
          poll    0.00%    0.67%    0.00%    0.00%    0.00%    0.00%   99.33%
     scheduler    0.69%    0.18%   28.41%    5.49%    9.50%    7.43%   48.29%

without busy wait threshold:

Stats per type:
         async    0.00%    0.00%    0.00%    0.00%    0.00%    0.00%  100.00%
           aux    0.01%    0.00%    0.00%    0.00%    0.01%    0.00%   99.98%
dirty_cpu_sche    0.00%    0.00%    0.00%    0.00%    0.00%    0.00%  100.00%
dirty_io_sched    0.00%    0.00%    0.00%    0.00%    0.00%    0.00%  100.00%
          poll    0.00%    0.77%    0.00%    0.00%    0.00%    0.00%   99.23%
     scheduler    0.70%    0.14%   28.29%    5.41%    0.86%    7.22%   57.38%
2021-02-10 12:35:12 +01:00
Luke Bakken 7a3bd539d3
Pass RABBITMQ_NODENAME via Windows service environment
Without this change using anything other than `rabbit` or the `rabbitmq-env-conf.bat` file will result in `erlang_dist_running_with_unexpected_nodename`

Follow-up to #2673

cc @dumbbell @michaelklishin
2020-12-07 12:03:06 -08:00
Lajos Gerecs 8fdbc222e3 allow configuring multiple rmq bases for multiple services
Currently RABBITMQ_BASE is always dynamically picked up from the
environment. This change would fix it at the time of configuration
of the service allowing multiple RabbitMQ services to be configured.
2020-12-05 22:13:00 +01:00
Philip Kuryloski a1fe3ab061 Change repo "root" to deps/rabbit
rabbit must not be the monorepo root application, as other applications depend on it
2020-11-13 14:34:42 +01:00