Preserving Kafka broker name on HDInsight teardown

%3CLINGO-SUB%20id%3D%22lingo-sub-140864%22%20slang%3D%22en-US%22%3EPreserving%20Kafka%20broker%20name%20on%20HDInsight%20teardown%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-140864%22%20slang%3D%22en-US%22%3E%3CP%3EHi%2C%3C%2FP%3E%0A%3CP%3EIs%20there%20any%20way%20to%20ensure%20that%20kafka%20broker%20name%20does%20not%20get%20changed%20as%20we%20tear%20down%20and%20spin%20up%20the%20clusters%3F%20To%20give%20a%20back%20ground%20of%20our%20application%20architecture%2C%20we%20have%26nbsp%3Benterprise%20applications%20which%20acts%20as%20producer%20for%20our%20kafka.%20These%20producers%20send%20the%20data%20to%20kafka%20as%20a%20nightly%20job.%20Since%20the%20kafka%20application%20is%20not%20used%20in%20the%20day%20time%2C%20we%20would%20like%20to%20shut%20it%20down%20to%20leverage%20the%20cost%20advantage.%20However%2C%20currently%20there%20is%20no%20way%20to%20preserve%20the%20kafka%20broker%20names%20on%20every%20start%20up%20and%20tear%20down.%26nbsp%3B%3C%2FP%3E%0A%3CP%3EIf%20the%20broker%20names%20gets%20changed%20on%20every%20restart%20%2C%20producer%20applications%20will%20fail.%26nbsp%3B%3C%2FP%3E%0A%3CP%3EIs%20there%20anyway%20to%20preserve%20the%20broker%20names%20on%20every%20shut%20down%20and%20re-start.%20If%20IP%20can%20remain%20constant%20on%20every%20and%20can%20be%20accessible%20over%20VPN%20that%20should%20also%20work%20fine.%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EPlease%20suggest.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EThanks%3C%2FP%3E%0A%3CP%3EHari%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-LABS%20id%3D%22lingo-labs-140864%22%20slang%3D%22en-US%22%3E%3CLINGO-LABEL%3EAzure%20HDInsight%3C%2FLINGO-LABEL%3E%3CLINGO-LABEL%3EHDInsight%3C%2FLINGO-LABEL%3E%3CLINGO-LABEL%3EKafka%20on%20HDInsght%3C%2FLINGO-LABEL%3E%3C%2FLINGO-LABS%3E
Occasional Visitor

Hi,

Is there any way to ensure that kafka broker name does not get changed as we tear down and spin up the clusters? To give a back ground of our application architecture, we have enterprise applications which acts as producer for our kafka. These producers send the data to kafka as a nightly job. Since the kafka application is not used in the day time, we would like to shut it down to leverage the cost advantage. However, currently there is no way to preserve the kafka broker names on every start up and tear down. 

If the broker names gets changed on every restart , producer applications will fail. 

Is there anyway to preserve the broker names on every shut down and re-start. If IP can remain constant on every and can be accessible over VPN that should also work fine. 

 

Please suggest.

 

Thanks

Hari

0 Replies