| 
									
										
										
											
												KAFKA-7500: MirrorMaker 2.0 (KIP-382)
Implementation of [KIP-382 "MirrorMaker 2.0"](https://cwiki.apache.org/confluence/display/KAFKA/KIP-382%3A+MirrorMaker+2.0)
Author: Ryanne Dolan <ryannedolan@gmail.com>
Author: Arun Mathew <arunmathew88@gmail.com>
Author: In Park <inpark@cloudera.com>
Author: Andre Price <obsoleted@users.noreply.github.com>
Author: christian.hagel@rio.cloud <christian.hagel@rio.cloud>
Reviewers: Eno Thereska <eno.thereska@gmail.com>, William Hammond <william.t.hammond@gmail.com>, Viktor Somogyi <viktorsomogyi@gmail.com>, Jakub Korzeniowski, Tim Carey-Smith, Kamal Chandraprakash <kamal.chandraprakash@gmail.com>, Arun Mathew, Jeremy-l-ford, vpernin, Oleg Kasian <oleg.kasian@gmail.com>, Mickael Maison <mickael.maison@gmail.com>, Qihong Chen, Sriharsha Chintalapani <sriharsha@apache.org>, Jun Rao <junrao@gmail.com>, Randall Hauch <rhauch@gmail.com>, Manikumar Reddy <manikumar.reddy@gmail.com>, Ismael Juma <ismael@juma.me.uk>
Closes #6295 from ryannedolan/KIP-382
											
										 
											2019-10-07 16:27:54 +08:00
										 |  |  | # Licensed to the Apache Software Foundation (ASF) under A or more | 
					
						
							|  |  |  | # contributor license agreements.  See the NOTICE file distributed with | 
					
						
							|  |  |  | # this work for additional information regarding copyright ownership. | 
					
						
							|  |  |  | # The ASF licenses this file to You under the Apache License, Version 2.0 | 
					
						
							|  |  |  | # (the "License"); you may not use this file except in compliance with | 
					
						
							|  |  |  | # the License.  You may obtain a copy of the License at | 
					
						
							|  |  |  | #  | 
					
						
							|  |  |  | #    http://www.apache.org/licenses/LICENSE-2.0 | 
					
						
							|  |  |  | #  | 
					
						
							|  |  |  | # Unless required by applicable law or agreed to in writing, software | 
					
						
							|  |  |  | # distributed under the License is distributed on an "AS IS" BASIS, | 
					
						
							|  |  |  | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | 
					
						
							|  |  |  | # See the License for the specific language governing permissions and | 
					
						
							|  |  |  | # limitations under the License. | 
					
						
							|  |  |  | # see org.apache.kafka.clients.consumer.ConsumerConfig for more details | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | # Sample MirrorMaker 2.0 top-level configuration file | 
					
						
							|  |  |  | # Run with ./bin/connect-mirror-maker.sh connect-mirror-maker.properties  | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | # specify any number of cluster aliases | 
					
						
							| 
									
										
										
										
											2020-01-08 01:00:30 +08:00
										 |  |  | clusters = A, B | 
					
						
							| 
									
										
										
											
												KAFKA-7500: MirrorMaker 2.0 (KIP-382)
Implementation of [KIP-382 "MirrorMaker 2.0"](https://cwiki.apache.org/confluence/display/KAFKA/KIP-382%3A+MirrorMaker+2.0)
Author: Ryanne Dolan <ryannedolan@gmail.com>
Author: Arun Mathew <arunmathew88@gmail.com>
Author: In Park <inpark@cloudera.com>
Author: Andre Price <obsoleted@users.noreply.github.com>
Author: christian.hagel@rio.cloud <christian.hagel@rio.cloud>
Reviewers: Eno Thereska <eno.thereska@gmail.com>, William Hammond <william.t.hammond@gmail.com>, Viktor Somogyi <viktorsomogyi@gmail.com>, Jakub Korzeniowski, Tim Carey-Smith, Kamal Chandraprakash <kamal.chandraprakash@gmail.com>, Arun Mathew, Jeremy-l-ford, vpernin, Oleg Kasian <oleg.kasian@gmail.com>, Mickael Maison <mickael.maison@gmail.com>, Qihong Chen, Sriharsha Chintalapani <sriharsha@apache.org>, Jun Rao <junrao@gmail.com>, Randall Hauch <rhauch@gmail.com>, Manikumar Reddy <manikumar.reddy@gmail.com>, Ismael Juma <ismael@juma.me.uk>
Closes #6295 from ryannedolan/KIP-382
											
										 
											2019-10-07 16:27:54 +08:00
										 |  |  | 
 | 
					
						
							|  |  |  | # connection information for each cluster | 
					
						
							| 
									
										
										
										
											2020-01-08 01:00:30 +08:00
										 |  |  | # This is a comma separated host:port pairs for each cluster | 
					
						
							|  |  |  | # for e.g. "A_host1:9092, A_host2:9092, A_host3:9092" | 
					
						
							| 
									
										
										
											
												KAFKA-7500: MirrorMaker 2.0 (KIP-382)
Implementation of [KIP-382 "MirrorMaker 2.0"](https://cwiki.apache.org/confluence/display/KAFKA/KIP-382%3A+MirrorMaker+2.0)
Author: Ryanne Dolan <ryannedolan@gmail.com>
Author: Arun Mathew <arunmathew88@gmail.com>
Author: In Park <inpark@cloudera.com>
Author: Andre Price <obsoleted@users.noreply.github.com>
Author: christian.hagel@rio.cloud <christian.hagel@rio.cloud>
Reviewers: Eno Thereska <eno.thereska@gmail.com>, William Hammond <william.t.hammond@gmail.com>, Viktor Somogyi <viktorsomogyi@gmail.com>, Jakub Korzeniowski, Tim Carey-Smith, Kamal Chandraprakash <kamal.chandraprakash@gmail.com>, Arun Mathew, Jeremy-l-ford, vpernin, Oleg Kasian <oleg.kasian@gmail.com>, Mickael Maison <mickael.maison@gmail.com>, Qihong Chen, Sriharsha Chintalapani <sriharsha@apache.org>, Jun Rao <junrao@gmail.com>, Randall Hauch <rhauch@gmail.com>, Manikumar Reddy <manikumar.reddy@gmail.com>, Ismael Juma <ismael@juma.me.uk>
Closes #6295 from ryannedolan/KIP-382
											
										 
											2019-10-07 16:27:54 +08:00
										 |  |  | A.bootstrap.servers = A_host1:9092, A_host2:9092, A_host3:9092 | 
					
						
							|  |  |  | B.bootstrap.servers = B_host1:9092, B_host2:9092, B_host3:9092 | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | # enable and configure individual replication flows | 
					
						
							|  |  |  | A->B.enabled = true | 
					
						
							| 
									
										
										
										
											2020-01-08 01:00:30 +08:00
										 |  |  | 
 | 
					
						
							|  |  |  | # regex which defines which topics gets replicated. For eg "foo-.*" | 
					
						
							|  |  |  | A->B.topics = .* | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | B->A.enabled = true | 
					
						
							|  |  |  | B->A.topics = .* | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | # Setting replication factor of newly created remote topics | 
					
						
							|  |  |  | replication.factor=1 | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | ############################# Internal Topic Settings  ############################# | 
					
						
							|  |  |  | # The replication factor for mm2 internal topics "heartbeats", "B.checkpoints.internal" and | 
					
						
							|  |  |  | # "mm2-offset-syncs.B.internal" | 
					
						
							|  |  |  | # For anything other than development testing, a value greater than 1 is recommended to ensure availability such as 3. | 
					
						
							|  |  |  | checkpoints.topic.replication.factor=1 | 
					
						
							|  |  |  | heartbeats.topic.replication.factor=1 | 
					
						
							|  |  |  | offset-syncs.topic.replication.factor=1 | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | # The replication factor for connect internal topics "mm2-configs.B.internal", "mm2-offsets.B.internal" and | 
					
						
							|  |  |  | # "mm2-status.B.internal" | 
					
						
							|  |  |  | # For anything other than development testing, a value greater than 1 is recommended to ensure availability such as 3. | 
					
						
							|  |  |  | offset.storage.replication.factor=1 | 
					
						
							|  |  |  | status.storage.replication.factor=1 | 
					
						
							|  |  |  | config.storage.replication.factor=1 | 
					
						
							| 
									
										
										
											
												KAFKA-7500: MirrorMaker 2.0 (KIP-382)
Implementation of [KIP-382 "MirrorMaker 2.0"](https://cwiki.apache.org/confluence/display/KAFKA/KIP-382%3A+MirrorMaker+2.0)
Author: Ryanne Dolan <ryannedolan@gmail.com>
Author: Arun Mathew <arunmathew88@gmail.com>
Author: In Park <inpark@cloudera.com>
Author: Andre Price <obsoleted@users.noreply.github.com>
Author: christian.hagel@rio.cloud <christian.hagel@rio.cloud>
Reviewers: Eno Thereska <eno.thereska@gmail.com>, William Hammond <william.t.hammond@gmail.com>, Viktor Somogyi <viktorsomogyi@gmail.com>, Jakub Korzeniowski, Tim Carey-Smith, Kamal Chandraprakash <kamal.chandraprakash@gmail.com>, Arun Mathew, Jeremy-l-ford, vpernin, Oleg Kasian <oleg.kasian@gmail.com>, Mickael Maison <mickael.maison@gmail.com>, Qihong Chen, Sriharsha Chintalapani <sriharsha@apache.org>, Jun Rao <junrao@gmail.com>, Randall Hauch <rhauch@gmail.com>, Manikumar Reddy <manikumar.reddy@gmail.com>, Ismael Juma <ismael@juma.me.uk>
Closes #6295 from ryannedolan/KIP-382
											
										 
											2019-10-07 16:27:54 +08:00
										 |  |  | 
 | 
					
						
							|  |  |  | # customize as needed | 
					
						
							|  |  |  | # replication.policy.separator = _ | 
					
						
							|  |  |  | # sync.topic.acls.enabled = false | 
					
						
							|  |  |  | # emit.heartbeats.interval.seconds = 5 |