Note that the `Docker` version **must be >= 20.10.4**.
The prior Docker versions may cause permission errors when running the Kafka container, as they do not correctly set directory permissions when creating container paths like `/opt/kafka/config`.
If you are using the prior version, you may encounter the following error during container startup:
If no user provided configuration (file input or environment variables) is passed to the Docker container, the default KRaft configuration for single combined-mode node will be used.
This default configuration is packaged in the Kafka tarball.
- This method requires users to provide path to a local folder which contains the Kafka property files and mount it to Docker container using Docker volume.
- It replaces the default KRaft configuration file present in Docker container.
- The Command `docker run --volume /path/to/property/folder:/mnt/shared/config -p 9092:9092 apache/kafka:latest` can be used to mount the folder containing the property files.
- Property files will be only read by the Docker container.
When using the environment variables, you need to set all properties required to start the KRaft node.
Therefore, the recommended way to use environment variables is via Docker Compose, which allows users to set all the properties that are needed.
It is also possible to use the input file to have a common set of configurations, and then override specific node properties using the environment variables.
- Kafka property defined via environment variables will override the value of that property defined in the user provided property file.
- If properties are provided via environment variables only, all required properties must be specified.
- The following rules must be used to construct the environment variable key name:
- Recommended way to run in ssl mode is by mounting secrets on `/etc/kafka/secrets` in Docker container and providing configs following through environment variables (`KAFKA_SSL_KEYSTORE_FILENAME`, `KAFKA_SSL_KEYSTORE_CREDENTIALS`, `KAFKA_SSL_KEY_CREDENTIALS`, `KAFKA_SSL_TRUSTSTORE_FILENAME` and `KAFKA_SSL_TRUSTSTORE_CREDENTIALS`) to let the Docker image scripts extract passwords and populate correct paths in `server.properties`.
- Please ensure appropriate `KAFKA_ADVERTISED_LISTENERS` are provided through environment variables to enable SSL mode in Kafka server, i.e. it should contain an `SSL` listener.
- Alternatively property file input can be used to provide ssl properties.
- Make sure you set location of truststore and keystore correctly when using file input. See example for file input in `docker-compose-files/single-node/file-input` for better clarity.
- Note that advertised.listeners property needs to be provided along with SSL properties in file input and cannot be provided through environment variable separately.
- In conclusion, ssl properties with advertised.listeners should be treated as a group and provided in file input or environment variables in it's entirety.
- In case ssl properties are provided both through file input and environment variables, environment variable properties will override the file input properties, just as mentioned in the beginning of this section.
- We are using environment variables purely for providing configs.
-`KAFKA_LISTENERS` is getting supplied. But if it was not provided, defaulting would have kicked in and we would have used `KAFKA_ADVERTISED_LISTENERS` to generate `KAFKA_LISTENERS`, by replacing the host with `0.0.0.0`.
- Note that we have provided a `CLUSTER_ID`, but it's not mandatory as there is a default cluster id present in container.
- We had to provide `KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR` and set it explicitly to 1, because if we don't provide it default value provided by kafka will be taken which is 3.
- Note that here we are using environment variables to pass configs.
- Notice how secrets folder is mounted to docker container.
- In case of environment variable it is mandatory to keep the files in `/etc/kafka/secrets` folder in docker container, given that the path of the files will be derived from that, as we are just providing file names in other SSL configs.
- Each broker must expose a unique port to host machine.
- For example broker-1, broker2 and broker3 are listening on port 9092, they're exposing it to the host via ports 29092, 39092 and 49092 respectively.
- Here important thing to note is that to ensure that kafka brokers are accessible both to clients as well as to each other we have introduced an additional listener.
- PLAINTEXT is supposed to be listener accessible to other brokers.
- The inter broker listener advertised by the brokers is exposed on container's hostname. This is done so that brokers can find each other in Docker network.
- The port advertised for host machine is done on localhost, as this is the hostname (in this example) that client will use to connect with kafka running inside Docker container.
- Similar to Plaintext example, for inter broker communication in SSL mode, SSL-INTERNAL is required and for client to broker communication, SSL is required.
