[#58]
Just a file change date can be not enough to detect the certificate
file change. Add a phash2 hash of a file content to check that certificate
should be reloaded.
[#58]
Directory change time is not reliable to detect updates in
certificate files. Time resolution is too big (1 second)
and directory modification time does not update when a file
contents is changed.
The file provider uses stat(2) which has a resolution of one second in
Erlang. Thus, the refresh may miss the new test certificates if the
creation happens in the same second after the previous refresh.
References #58.
With Erlang 18.3, there is a regression which causes the SSL connection
to crash with the following exception:
** {badarg,[{ets,update_counter,[1507362,#Ref<0.0.3.9>,-1],[]},
{ssl_pkix_db,ref_count,3,...
When this exception reaches the connection process before the expected
TLS error, amqp_connection:start() returns `{error, closed}` instead.
Now, testcases tolerate `{error, closed}` as a return value: we know
that the connection was indeed terminated.
Signed-off: Gerhard Lazu <gerhard@rabbitmq.com>
... instead of 2 seconds. Hopefully, this increases the chance of test
success.
The real fix would be to explicitely verify that the trust store was
refreshed.
Fixes#54
Provider will list certificates as JSON, requiring list of objects
with `id` and `url` fields.
JSON root is an object with a single `certificates` field, containing
a list of certificate objects. (some web services require json root to
be object)
Certificates are loaded as PEM encoded files.
This is the recommended way with Erlang.mk.
By default, the version is inherited from rabbitmq-server-release when
the source archive is created, or computed from git-describe(1) (see
`rabbitmq-components.mk`). One can override the version from the command
line by setting the `PROJECT_VERSION` variable.
[#130992027]
- use the private directory for the test run instead of TMPDIR
- use dynamic TCP port allocation
- get hostname from test configuration
- do not use hardcoded tcp port numbers
- don't test for the existence of the whitelist directory
- increase test timeout
- create test dir per test case
- use ct groups
- use a subdirectory of the certs directory for trust store certs
Fixes#29.
- improve naming
- remove trailing whitespace
- add parens to make logic more explicit
- shorten long lines
- switch to using a proplist getter instead of lists:keyfind
That is, SSL socket options that reside in the broker's OTP
environment. These are reverted in the plugin's `cleanup`
procedure. Indirect dependencies can then be restarted without their
SSL listeners interfacing with the trust-store.
TLS sockets (OTP's SSL) must be configured to interface with the
Trust-Store before they start listening. The boot-steps make sure that
the trust store makes the necessary amendments to the configuration
options in time. We put the initial SSL configuration options in the
OTP Application environment so that they can be restored if the plugin
is disabled.
Don't default to the (user's) home directory, this is far from ideal
on Linux systems, but worse with Windows because a home directory
doesn't exist there. Instead we dismantle the Mnesia directory and
root the default whitelist directory nearby. This is because Rabbit
doesn't use environment variables (e.g. RABBITMQ_HOME and
RABBITMQ_BASE) elsewhere in the source, as the definitive place to
root directories for application/pluging data.
NOTE: paths continue to append a "/" so commit is not Windows
friendly.
This fixes a bug when removing certificate details, which would cause
the trust-store server to crash, though the test set succeeded because
it got restarted by it's supervisor.
We need somewhere to write and remove certificate files. The test set
uses a directory `data` which is relative to it's current path. It's
an improvement over what was there before becuase this is more
contained and portable between OSs.
Refactor branching, procedure and case clauses, into simple matches
which'll fail on a bad result. The OTP Application no longer re-checks
preconditions, like the existance of the whitelist directory &
interface module/procedure, which are put in place at Rabbit Plugin
boot.
Fix buggy Application start when we get an empty list of SSL options.
A refresh interval of 0 SECONDS configures the trust-store for manual
whitelist refresh, i.e. to reflect the certificates currently in the
whitelist directory, via call to `rabbit_trust_store:refresh/0`. An
interval >= 1 SECONDS configures automatic refresh, as before, through
timers.
This fixes a related BUG: the degree of time accuracy was not
consistent between the application, server, and test set. The intended
unit is SECONDS, NOT milliseconds, to make configuration more
friendly. I.e. `60` seconds instead of `60 * 1000` milliseconds.
Store the filename along with certificate issuer name and serial
number, so as to perform a diff on the directory contents, then only
install and remove those entries which need it. We were deleting all
entries + reading in the entire (newer) contents of the directory when
directory modification time had changed.
Along the way it made more sense to optimise ETS for querying the
whitelist than it did to refresh it: the key is still the
unique/distinctive certificate value (issuer name & serial
number). While installing and removing certificates rely on a
`select`.
The client facing interface, `whitelisted/3`, ultimately makes a call
to the ETS table directly. That is, it no longer goes through the
`trust_store` process, which was unecessary.
The file system reports time to an accuracy of one second. So, unless
we wait for at least one second, we may see no change in modification
time. The trust-store relies on this to tell when the whitelist needs
to be refreshed.
* Make changing configuration of more than one option easier.
* Distinguish which whitelisted certificate comes from which test
by using differnt filenames.
* Clean-up after writing out certificate files.
This also introduces testing for a means by which to set an expiry
time, after which the trust-store will refresh its contents to reflect
the underlying directory, so as to keep it up-to-date.
Building the whitelist with a list is practicle initially but not
performant. Introduce a record `entry` which will contain a filename +
modification time in the whitelist.
This completes the client facing procedure `whitelisted/3` with which
SSL sockets effectively query the trust-store, introducing basic
functionality for the server internals, but simplifies matters by
using a list to store the whitelist information. Error logging for
debugging purposes is removed.
Some unecessary SSL client options meant that it will try to validate
the server certificate. We only want to test that the server tries to
validate the client certificate. These complicated the test set and
raised errors which were difficult to track down: the reason reported
for the client failing to authenticate the server were very similar to
those reported when the server tries to authenticate the client. In
the former case, the server will send a TLS alert to the client,
reporting an "unknown CA".
A `stop` procedure isn't necessary if the server will always find
itself in a supervision tree. We trap exits upon initialisation and
handle the reason `shutdown` in the procedure `terminate/2`, instead.
Clean the procedures `handle_call` and `handle_cast`.
* Give the interface procedure a meaningful name.
* Order the procedure's clauses by relevance.
* Log something informative for each clause.
* Dialyzer type & signature for the procedure.
This is a better choice than having the server retrieve the directory
name itself: (1) we can have guarentees earlier from the application,
and (2) it makes testing easier and clearer.
The test set would fail before it began because the application
couldn't retrieve a path to a whitlist directory from it's
configuration data. This change makes the application look for
whitelisted certificates in a default directory. The corresponding
test stops the application, changes the configuration, and starts it
to test with given configuration parameters.
A simple end-to-end test of the trust-store, in the sense that a .PEM
file is written to a directory from which Rabbit is configured to read
whitelisted certificates, then an AMQP client connects. Note: all
those variables which are necessary but not central to the test are
prefixed with an underscore to distinguish them.
Several changes:
* Test names tell us which protocol they use (AMQP). STOMP and MQTT to
follow.
* Give tests more time to complete as old value proved to be short
* Indicate variables which aren't really involved (but necessary) in a
given test with a leading underscore
* Format with more stringent guidlines in mind
Loïc Hoguin has done a lot of helpful work around the PKI + SLL
applications in OTP to build certificates, authorities, and more all
through Erlang, so no calls from the command-line necessary. At the
end of the day they'd use the same stuff behind the scenes: Open
SLL. But, these make it extreamly convenient to test changes in Rabbit
revolving around TLS + certificates.
Michael Klishin
Jean-Sébastien Pédron
Michael Klishin
Daniil Fedotov
Ayanda Dube
Michael Klishin
Gerhard Lazu
kjnilsson
Karl Nilsson
kjnilsson
Joseph Yiasemides