I commented out both #host.name, #advertised.host.nam
(new server.properties)
broker.id=11
port=9093
#host.name=n1.test.com
#advertised.host.name=192.168.0.11
allow.everyone.if.no.acl.found=true
super.users=User:CN=n1.test.com,OU=TEST,O=TEST,L=TEST,ST=TEST,C=TEST
listeners=SSL://n1.test.com:9093
advertised.listeners=SSL://n1.test.com:9093
ssl.client.auth=required
ssl.enabled.protocols=TLSv1.2,TLSv1.1,TLSv1
ssl.keystore.type=JKS
ssl.truststore.type=JKS
security.inter.broker.protocol=SSL
ssl.keystore.location=/home/kafka/kafka.server.keystore.jks
ssl.keystore.password=Test2017
ssl.key.password=Test2017
ssl.truststore.location=/home/kafka/kafka.server.truststore.jks
ssl.truststore.password=Test2017
authorizer.class.name=kafka.security.auth.SimpleAclAuthorizer
principal.builder.class=org.apache.kafka.common.security.auth.DefaultPrincipalBuilder
num.replica.fetchers=4
replica.fetch.max.bytes=1048576
replica.fetch.wait.max.ms=500
replica.high.watermark.checkpoint.interval.ms=5000
replica.socket.timeout.ms=30000
replica.socket.receive.buffer.bytes=65536
replica.lag.time.max.ms=10000
controller.socket.timeout.ms=30000
controller.message.queue.size=10
default.replication.factor=3
log.dirs=/usr/log/kafka
kafka.logs.dir=/usr/log/kafka
num.partitions=20
message.max.bytes=1000000
auto.create.topics.enable=true
log.index.interval.bytes=4096
log.index.size.max.bytes=10485760
log.retention.hours=720
log.flush.interval.ms=10000
log.flush.interval.messages=20000
log.flush.scheduler.interval.ms=2000
log.roll.hours=168
log.retention.check.interval.ms=300000
log.segment.bytes=1073741824
delete.topic.enable=true
socket.request.max.bytes=104857600
socket.receive.buffer.bytes=1048576
socket.send.buffer.bytes=1048576
num.io.threads=8
num.network.threads=8
queued.max.requests=16
fetch.purgatory.purge.interval.requests=100
producer.purgatory.purge.interval.requests=100
zookeeper.connect=n1:2181,n2:2181,n3:2181
zookeeper.connection.timeout.ms=2000
zookeeper.sync.time.ms=2000
(producer.properties)
bootstrap.servers=n1.test.com:9093
security.protocol=SSL
ssl.truststore.location=/home/kafka/kafka.client.truststore.jks
ssl.truststore.password=testkafka
ssl.keystore.location=/home/kafka/kafka.client.keystore.jks
ssl.keystore.password=testkafka
ssl.key.password=testkafka
(run producer)
./bin/kafka-console-producer.sh \
--broker-list n1:9093 \
--producer.config /home/kafka/config/producer.n1.properties \
--sync --topic test02
(got error)
[2017-08-10 07:10:31,881] ERROR Error when sending message to topic test02
with key: null, value: 0 bytes with error:
(org.apache.kafka.clients.producer.internals.ErrorLoggingCallback)
org.apache.kafka.common.errors.TimeoutException: Expiring 2 record(s) for
test02-0: 1518 ms has passed since batch creation plus linger time
[2017-08-10 07:10:32,230] ERROR Error when sending message to topic test02
with key: null, value: 0 bytes with error:
(org.apache.kafka.clients.producer.internals.ErrorLoggingCallback)
org.apache.kafka.common.errors.TimeoutException: Expiring 2 record(s) for
test02-1: 1543 ms has passed since batch creation plus linger time
By the way, where to set "-Djavax.security.debug=all" for Kafka?
Post by M. Mannaif you remove host.name, advertised.host.name and port from
server.properties, does it work for you?
I am using SSL without ACL. it seems to be working fine.
Post by M. Mannazookeeper-shell.sh localhost:2181
get /brokers/ids/11
zookeeper-shell.sh n1.test.com:2181
Connecting to n1.test.com:2181
Welcome to ZooKeeper!
JLine support is disabled
WatchedEvent state:SyncConnected type:None path:null
get /brokers/ids/11
WatchedEvent state:SaslAuthenticated type:None path:null
{"listener_security_protocol_map":{"SSL":"SSL"},"endpoints":["SSL://
n1.test.com:9093
"],"jmx_port":-1,"host":null,"timestamp":"1502310695312","
port":-1,"version":4}
cZxid = 0x40002787d
ctime = Thu Aug 10 04:31:37 HKT 2017
mZxid = 0x40002787d
mtime = Thu Aug 10 04:31:37 HKT 2017
pZxid = 0x40002787d
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x35d885c689c00a6
dataLength = 168
numChildren = 0
Post by Ascot MossAbout: zookeeper-shell.sh localhost:2181
get /brokers/ids/11
zookeeper-shell.sh n1.test.com:2181
Connecting to n1.test.com:2181
Welcome to ZooKeeper!
JLine support is disabled
WatchedEvent state:SyncConnected type:None path:null
WatchedEvent state:SaslAuthenticated type:None path:null
Post by Ascot MossFYI, about zookeeper, I used my existing zookeeper (as I have existing
zookeeper up and running, which is also used for hbase)
zookeeper versoom: 3.4.10
zoo.cfg
######
tickTime=2000
initLimit=10
syncLimit=5
dataDir=/usr/local/zookeeper/data
dataLogDir=/usr/local/zookeeper/datalog
clientPort=2181
maxClientCnxns=60
server.1=n1.test.com:2888:3888
server.2=n2.test.com:2888:3888
server.3=n3.test.com:2888:3888
authProvider.1=org.apache.zookeeper.server.auth.SASLAuthenti
cationProvider
jaasLoginRenew=3600000
requireClientAuthScheme=sasl
zookeeper.allowSaslFailedClients=false
kerberos.removeHostFromPrincipal=true
######
Post by Ascot Mossserver.properties
######
broker.id=11
port=9093
host.name=n1
advertised.host.name=192.168.0.11
allow.everyone.if.no.acl.found=true
super.users=User:CN=n1.test.com,OU=TEST,O=TEST,L=TEST,ST=TEST,C=TEST
listeners=SSL://n1.test.com:9093 <http://n1.test.com:9092/>
advertised.listeners=SSL://n1.test.com:9093 <
http://n1.test.com:9092/>
Post by M. MannaPost by Ascot MossPost by Ascot MossPost by Ascot Mossssl.client.auth=required
ssl.enabled.protocols=TLSv1.2,TLSv1.1,TLSv1
ssl.keystore.type=JKS
ssl.truststore.type=JKS
security.inter.broker.protocol=SSL
ssl.keystore.location=/home/kafka/kafka.server.keystore.jks
ssl.keystore.password=Test2017
ssl.key.password=Test2017
ssl.truststore.location=/home/kafka/kafka.server.truststore.jks
ssl.truststore.password=Test2017
authorizer.class.name=kafka.security.auth.SimpleAclAuthorizer
principal.builder.class=org.apache.kafka.common.security.aut
h.DefaultPrincipalBuilder
num.replica.fetchers=4
replica.fetch.max.bytes=1048576
replica.fetch.wait.max.ms=500
replica.high.watermark.checkpoint.interval.ms=5000
replica.socket.timeout.ms=30000
replica.socket.receive.buffer.bytes=65536
replica.lag.time.max.ms=10000
controller.socket.timeout.ms=30000
controller.message.queue.size=10
default.replication.factor=3
log.dirs=/usr/log/kafka
kafka.logs.dir=/usr/log/kafka
num.partitions=20
message.max.bytes=1000000
auto.create.topics.enable=true
log.index.interval.bytes=4096
log.index.size.max.bytes=10485760
log.retention.hours=720
log.flush.interval.ms=10000
log.flush.interval.messages=20000
log.flush.scheduler.interval.ms=2000
log.roll.hours=168
log.retention.check.interval.ms=300000
log.segment.bytes=1073741824
delete.topic.enable=true
socket.request.max.bytes=104857600
socket.receive.buffer.bytes=1048576
socket.send.buffer.bytes=1048576
num.io.threads=8
num.network.threads=8
queued.max.requests=16
fetch.purgatory.purge.interval.requests=100
producer.purgatory.purge.interval.requests=100
zookeeper.connect=n1:2181,n2:2181,n3:2181
zookeeper.connection.timeout.ms=2000
zookeeper.sync.time.ms=2000
######
producer.properties
######
bootstrap.servers=n1.test.com:9093 <http://n1.test.com:9092/>
security.protocol=SSL
ssl.truststore.location=/home/kafka/kafka.client.truststore.jks
ssl.truststore.password=testkafka
ssl.keystore.location=/home/kafka/kafka.client.keystore.jks
ssl.keystore.password=testkafka
ssl.key.password=testkafka
#####
(I had tried to switch to another port, 9093 is the correct port)
Post by M. MannaYour openssl test is showing connected with port 9092. but your
previous
Post by Ascot MossPost by Ascot MossPost by Ascot MossPost by M. Mannamessages show 9093 - is there some typo issues? Where is SSL running
Please share the following and don't leave any details out. This
will
Post by M. MannaPost by Ascot MossPost by Ascot MossPost by Ascot MossPost by M. Mannaonly
create more assumptions.
1) server.properties
2) Zookeeper.properties
Also, run the following command (when the cluster is running)
zookeeper-shell.sh localhost:2181
get /brokers/ids/11
Does it show that your broker #11 is connected?
Post by Ascot MossDear Manna,
What's the status of your SSL? Have you verified that the setup is
working?
Post by Ascot MossYes, I used "
openssl s_client -debug -connect n1.test.com:9092 -tls1
CONNECTED(00000003)
write to 0x853e70 [0x89fd43] (155 bytes => 155 (0x9B))
0000 - 16 03 01 00 96 01 00 00-92 03 01 59 8b 6d 0d b1
...........Y.m..
Post by Ascot Moss...
Server certificate
-----BEGIN CERTIFICATE-----
CwwCSEsxGT............
-----END CERTIFICATE-----
---
SSL handshake has read 2470 bytes and written 161 bytes
---
New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES256-SHA
PSK identity hint: None
Start Time: 1502309645
Timeout : 7200 (sec)
Verify return code: 19 (self signed certificate in certificate
chain)
Post by Ascot Moss---
Regards
Post by M. MannaHi,
What's the status of your SSL? Have you verified that the setup
is
info
Post by M. MannaPost by Ascot MossPost by Ascot MossPost by Ascot MossPost by M. Mannato
Post by Ascot Mosstrace
Post by M. Mannathings. Also, you can enable security logging by adding
-Djavax.security.debug=all
Please share your producer/broker configs with us.
Kindest Regards,
M. Manna
Post by Ascot MossHi,
I have setup Kafka 0.10.2.1 with SSL.
openssl s_client -debug -connect n1:9093 -tls1
New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES256-SHA
Protocol : TLSv1
PSK identity hint: None
Start Time: 1502285690
Timeout : 7200 (sec)
Verify return code: 19 (self signed certificate in
certificate
Post by Ascot MossPost by Ascot MossPost by Ascot MossPost by M. MannaPost by Ascot Mosschain)
Post by M. MannaPost by Ascot Mosskafka-topics.sh --create --zookeeper n1:2181,n2:2181,n3:2181
--replication-factor 3 --partitions 3 --topic test02
ERROR [ReplicaFetcherThread-2-111], Error for partition
[test02,2] to
rs.UnknownTopicOrPartitionExcepti
Post by Ascot MossPost by M. MannaPost by Ascot MossThis server does not host this topic-partition.
(kafka.server.ReplicaFetcherThread)
However, if I run describe topic, I can see it is created
kafka-topics.sh --zookeeper n1:2181,n2:2181,n3:2181 --describe
--topic
12,13,11
13,11,12
11,12,13
Post by Ascot MossPost by M. MannaPost by Ascot Mosskafka-console-consumer.sh --bootstrap-server n1:9093
--consumer.config
Post by Ascot MossPost by M. MannaPost by Ascot Moss/home/kafka/config/consumer.n1.properties --topic test02
--from-beginning
Post by Ascot Mosskafka-console-producer.sh --broker-list n1:9093
--producer.config
test02
null,