Commit Graph

36 Commits

Author SHA1 Message Date
Diana Parra Corbacho 4ec2f21dbe Shovel: more common testcases 2025-10-29 20:53:13 +01:00
Diana Parra Corbacho 059813a831 Shovel: more common testcases 2025-10-16 12:07:05 +02:00
Diana Parra Corbacho 629cf77743 Shovel: delete duplicated tests
Test cases covered by shovel_dynamic_SUITE
2025-10-16 12:07:05 +02:00
Diana Parra Corbacho c4138d5194 Shovel tests: refactor and new tests for all protocols 2025-10-08 17:43:53 +02:00
Diana Parra Corbacho 7f25c52eaf Shovel: tests delete after with queue rejections
Peer Discovery AWS Integration Test / Integration Test (push) Has been cancelled Details
2025-10-08 13:07:55 +02:00
Diana Parra Corbacho bc9b2f5a71 Shovel tests: code refactor 2025-10-07 12:28:44 +02:00
Michael Klishin 30fb9c1128
Re-arrange shovel test suites
* Use more descriptive names
 * Prefix unit test suites accordingly
 * Reuse await_credit/1
 * await_credit/1 in a flakey test
2025-08-29 18:34:51 -04:00
Diana Parra Corbacho b6d831b115 Shovel amqp1.0: fix delete after validation
Queue-length is not supported, so it should fail already during validation and
not shovel startup
2025-08-23 13:26:46 +02:00
Diana Parra Corbacho c8840785d9 AMQP10 shovel: make bare message inmutable 2025-08-11 21:37:04 +02:00
Diana Parra Corbacho 6e6a33ec0b Shovel: use message containers 2025-08-11 21:37:03 +02:00
Michal Kuratczyk c5271ea602
Some AMQP 0.9.1 headers, in particular x-death headers, cannot be set as
application properties. Before this change, trying to shovel
dead-lettered messages from an AMQP 0.9.1 source to AMQP 1.0 destination
would fail with:
```
reason: {badarg,
            [{unicode,characters_to_binary,
                 [[{table,
                       [{<<"count">>,long,1},
                        {<<"reason">>,longstr,<<"maxlen">>},
                        {<<"queue">>,longstr,<<"tmp">>},
                        {<<"time">>,timestamp,1745575728},
                        {<<"exchange">>,longstr,<<>>},
                        {<<"routing-keys">>,array,
                         [{longstr,<<"tmp">>}]}]}]],
                 [{file,"unicode.erl"},
                  {line,1219},
                  {error_info,#{module => erl_stdlib_errors}}]},
             {amqp10_client_types,utf8,1,
                 [{file,"amqp10_client_types.erl"},{line,99}]},
             {amqp10_msg,'-set_application_properties/2-fun-0-',3,
                 [{file,"amqp10_msg.erl"},{line,385}]},
             {maps,fold_1,4,[{file,"maps.erl"},{line,860}]},
             {amqp10_msg,set_application_properties,2,
                 [{file,"amqp10_msg.erl"},{line,384}]},
             {maps,fold_1,4,[{file,"maps.erl"},{line,860}]},
             {rabbit_amqp10_shovel,forward,4,
                 [{file,"rabbit_amqp10_shovel.erl"},{line,337}]},
             {rabbit_shovel_worker,handle_info,2,
                 [{file,"rabbit_shovel_worker.erl"},{line,104}]}]}
```
2025-04-25 15:46:45 +02:00
Michael Klishin 968eefa1bb
Bump (c) line year
There are no functional changes to this massive diff.
2025-01-01 17:54:10 -05:00
David Ansari b1eb354385 Strictly validate annotations 2024-09-18 12:42:27 +02:00
Michal Kuratczyk cfa3de4b2b
Remove unused imports (thanks elp!) 2024-05-23 16:36:08 +02:00
Lajos Gerecs 792e0a121b Remove delivery_mode header from amqp 1.0 shovel.
This header is not present if delivery_mode = 2, it does not really
makes sense to leave it when delivery_mode = 1.
The durability of the message is reflected in the durable flag of the
amqp 1.0 message.
2024-02-07 14:20:08 +01:00
Michael Klishin 01092ff31f
(c) year bumps 2024-01-01 22:02:20 -05:00
Michael Klishin 1b642353ca
Update (c) according to [1]
1. https://investors.broadcom.com/news-releases/news-release-details/broadcom-and-vmware-intend-close-transaction-november-22-2023
2023-11-21 23:18:22 -05:00
Michael Klishin ec4f1dba7d
(c) year bump: 2022 => 2023 2023-01-01 23:17:36 -05:00
Michael Klishin c38a3d697d
Bump (c) year 2022-03-21 01:21:56 +04:00
Jean-Sébastien Pédron cdcf602749
Switch from Lager to the new Erlang Logger API for logging
The configuration remains the same for the end-user. The only exception
is the log root directory: it is now set through the `log_root`
application env. variable in `rabbit`. People using the Cuttlefish-based
configuration file are not affected by this exception.

The main change is how the logging facility is configured. It now
happens in `rabbit_prelaunch_logging`. The `rabbit_lager` module is
removed.

The supported outputs remain the same: the console, text files, the
`amq.rabbitmq.log` exchange and syslog.

The message text format slightly changed: the timestamp is more precise
(now to the microsecond) and the level can be abbreviated to always be
4-character long to align all messages and improve readability. Here is
an example:

    2021-03-03 10:22:30.377392+01:00 [dbug] <0.229.0> == Prelaunch DONE ==
    2021-03-03 10:22:30.377860+01:00 [info] <0.229.0>
    2021-03-03 10:22:30.377860+01:00 [info] <0.229.0>  Starting RabbitMQ 3.8.10+115.g071f3fb on Erlang 23.2.5
    2021-03-03 10:22:30.377860+01:00 [info] <0.229.0>  Licensed under the MPL 2.0. Website: https://rabbitmq.com

The example above also shows that multiline messages are supported and
each line is prepended with the same prefix (the timestamp, the level
and the Erlang process PID).

JSON is also supported as a message format and now for any outputs.
Indeed, it is possible to use it with e.g. syslog or the exchange. Here
is an example of a JSON-formatted message sent to syslog:

    Mar  3 11:23:06 localhost rabbitmq-server[27908] <0.229.0> - {"time":"2021-03-03T11:23:06.998466+01:00","level":"notice","msg":"Logging: configured log handlers are now ACTIVE","meta":{"domain":"rabbitmq.prelaunch","file":"src/rabbit_prelaunch_logging.erl","gl":"<0.228.0>","line":311,"mfa":["rabbit_prelaunch_logging","configure_logger",1],"pid":"<0.229.0>"}}

For quick testing, the values accepted by the `$RABBITMQ_LOGS`
environment variables were extended:
  * `-` still means stdout
  * `-stderr` means stderr
  * `syslog:` means syslog on localhost
  * `exchange:` means logging to `amq.rabbitmq.log`

`$RABBITMQ_LOG` was also extended. It now accepts a `+json` modifier (in
addition to the existing `+color` one). With that modifier, messages are
formatted as JSON intead of plain text.

The `rabbitmqctl rotate_logs` command is deprecated. The reason is
Logger does not expose a function to force log rotation. However, it
will detect when a file was rotated by an external tool.

From a developer point of view, the old `rabbit_log*` API remains
supported, though it is now deprecated. It is implemented as regular
modules: there is no `parse_transform` involved anymore.

In the code, it is recommended to use the new Logger macros. For
instance, `?LOG_INFO(Format, Args)`. If possible, messages should be
augmented with some metadata. For instance (note the map after the
message):

    ?LOG_NOTICE("Logging: switching to configured handler(s); following "
                "messages may not be visible in this log output",
                #{domain => ?RMQLOG_DOMAIN_PRELAUNCH}),

Domains in Erlang Logger parlance are the way to categorize messages.
Some predefined domains, matching previous categories, are currently
defined in `rabbit_common/include/logging.hrl` or headers in the
relevant plugins for plugin-specific categories.

At this point, very few messages have been converted from the old
`rabbit_log*` API to the new macros. It can be done gradually when
working on a particular module or logging.

The Erlang builtin console/file handler, `logger_std_h`, has been forked
because it lacks date-based file rotation. The configuration of
date-based rotation is identical to Lager. Once the dust has settled for
this feature, the goal is to submit it upstream for inclusion in Erlang.
The forked module is calld `rabbit_logger_std_h` and is based
`logger_std_h` in Erlang 23.0.
2021-03-11 15:17:36 +01:00
Michael Klishin 52479099ec
Bump (c) year 2021-01-22 09:00:14 +03:00
Michael Klishin 45b6b223d8 Switch to MPL2 2020-07-14 21:39:08 +03:00
Jean-Sébastien Pédron bdccd8c21e Update copyright (year 2020) 2020-03-10 16:50:05 +01:00
Michael Klishin 2a0719db88 (c) bump 2019-12-29 05:50:35 +03:00
Spring Operator 668d714e3c URL Cleanup
This commit updates URLs to prefer the https protocol. Redirects are not followed to avoid accidentally expanding intentionally shortened URLs (i.e. if using a URL shortener).

# HTTP URLs that Could Not Be Fixed
These URLs were unable to be fixed. Please review them to see if they can be manually resolved.

* http://blog.listincomprehension.com/search/label/procket (200) with 1 occurrences could not be migrated:
   ([https](https://blog.listincomprehension.com/search/label/procket) result ClosedChannelException).
* http://dozzie.jarowit.net/trac/wiki/TOML (200) with 1 occurrences could not be migrated:
   ([https](https://dozzie.jarowit.net/trac/wiki/TOML) result SSLHandshakeException).
* http://dozzie.jarowit.net/trac/wiki/subproc (200) with 1 occurrences could not be migrated:
   ([https](https://dozzie.jarowit.net/trac/wiki/subproc) result SSLHandshakeException).
* http://e2project.org (200) with 1 occurrences could not be migrated:
   ([https](https://e2project.org) result AnnotatedConnectException).
* http://nitrogenproject.com/ (200) with 2 occurrences could not be migrated:
   ([https](https://nitrogenproject.com/) result ConnectTimeoutException).
* http://proper.softlab.ntua.gr (200) with 1 occurrences could not be migrated:
   ([https](https://proper.softlab.ntua.gr) result SSLHandshakeException).
* http://yaws.hyber.org (200) with 1 occurrences could not be migrated:
   ([https](https://yaws.hyber.org) result AnnotatedConnectException).
* http://choven.ca (503) with 1 occurrences could not be migrated:
   ([https](https://choven.ca) result ConnectTimeoutException).

# Fixed URLs

## Fixed But Review Recommended
These URLs were fixed, but the https status was not OK. However, the https status was the same as the http request or http redirected to an https URL, so they were migrated. Your review is recommended.

* http://fixprotocol.org/ (301) with 1 occurrences migrated to:
  https://fixtrading.org ([https](https://fixprotocol.org/) result SSLHandshakeException).
* http://erldb.org (UnknownHostException) with 1 occurrences migrated to:
  https://erldb.org ([https](https://erldb.org) result UnknownHostException).

## Fixed Success
These URLs were switched to an https URL with a 2xx status. While the status was successful, your review is still recommended.

* http://cloudi.org/ with 27 occurrences migrated to:
  https://cloudi.org/ ([https](https://cloudi.org/) result 200).
* http://erlware.org/ with 1 occurrences migrated to:
  https://erlware.org/ ([https](https://erlware.org/) result 200).
* http://inaka.github.io/cowboy-trails/ with 1 occurrences migrated to:
  https://inaka.github.io/cowboy-trails/ ([https](https://inaka.github.io/cowboy-trails/) result 200).
* http://ninenines.eu with 6 occurrences migrated to:
  https://ninenines.eu ([https](https://ninenines.eu) result 200).
* http://www.actordb.com/ with 2 occurrences migrated to:
  https://www.actordb.com/ ([https](https://www.actordb.com/) result 200).
* http://www.cs.kent.ac.uk/projects/wrangler/Home.html with 1 occurrences migrated to:
  https://www.cs.kent.ac.uk/projects/wrangler/Home.html ([https](https://www.cs.kent.ac.uk/projects/wrangler/Home.html) result 200).
* http://www.rabbitmq.com/shovel.html with 1 occurrences migrated to:
  https://www.rabbitmq.com/shovel.html ([https](https://www.rabbitmq.com/shovel.html) result 200).
* http://www.rebar3.org with 1 occurrences migrated to:
  https://www.rebar3.org ([https](https://www.rebar3.org) result 200).
* http://contributor-covenant.org with 1 occurrences migrated to:
  https://contributor-covenant.org ([https](https://contributor-covenant.org) result 301).
* http://contributor-covenant.org/version/1/3/0/ with 1 occurrences migrated to:
  https://contributor-covenant.org/version/1/3/0/ ([https](https://contributor-covenant.org/version/1/3/0/) result 301).
* http://inaka.github.com/apns4erl with 1 occurrences migrated to:
  https://inaka.github.com/apns4erl ([https](https://inaka.github.com/apns4erl) result 301).
* http://inaka.github.com/edis/ with 1 occurrences migrated to:
  https://inaka.github.com/edis/ ([https](https://inaka.github.com/edis/) result 301).
* http://lasp-lang.org/ with 1 occurrences migrated to:
  https://lasp-lang.org/ ([https](https://lasp-lang.org/) result 301).
* http://saleyn.github.com/erlexec with 1 occurrences migrated to:
  https://saleyn.github.com/erlexec ([https](https://saleyn.github.com/erlexec) result 301).
* http://www.mozilla.org/MPL/ with 25 occurrences migrated to:
  https://www.mozilla.org/MPL/ ([https](https://www.mozilla.org/MPL/) result 301).
* http://zhongwencool.github.io/observer_cli with 1 occurrences migrated to:
  https://zhongwencool.github.io/observer_cli ([https](https://zhongwencool.github.io/observer_cli) result 301).
2019-03-20 03:23:23 -05:00
Daniil Fedotov 3a0aaa6ad6 Add tests to for AMQP1.0 shovel message properties parsing. 2019-02-11 17:42:35 -05:00
Michael Klishin 6edda5381e Update (c) year 2019-01-30 04:32:59 +03:00
kjnilsson c352fe5b4e Fix off-by-one error in amqp10 shovel
This bug only manifested itself when the source used amqp10 and the
destination used amqp091 and the first publisher confirm was a multiple.
In this scenario the first message would not have been acked back to the
source.
2017-06-09 19:12:11 +01:00
Diana Corbacho d17ef783a1 Refactor of duplicated code 2017-05-31 14:49:18 +01:00
kjnilsson e95fd3e5bd Tidy up logging 2017-05-11 10:43:09 +01:00
kjnilsson bc10689e3c Refactor static configuration validation. 2017-05-10 12:42:24 +01:00
kjnilsson 1b7a72b64d Translate multi-acks into amqp 1.0 disposition
range.
2017-05-09 10:59:35 +01:00
kjnilsson 645b50a0d9 mixed dynamic tests 2017-05-08 17:16:22 +01:00
kjnilsson 0f6976994f amqp10 autodelete 2017-05-05 12:29:41 +01:00
kjnilsson f180ba2bf7 Move dynamic calls to the module defining the behaviour. 2017-05-04 11:07:27 +01:00
kjnilsson 9e03d3e7dc amqp10 dynamic tests 2017-04-28 11:13:22 +01:00