kfaker is a small project intended to generate a flow of streaming message in a kafka topic.
kfaker is provided as rpm packages (Sorry, only this packaging is currently provided. Contribution welcome), on the release pages.
Once installed, basic usage is the following:
# kfaker --brokers "brk1.my.domain.com:9092,brk2.my.domain.com:9092,brk3.my.domain.com:9092" --topic t1 --messon
This will send one message per second in the topic t1. Messages will be displayed.
Here all all the parameters
Option (* = required) Description
--------------------- -----------
* --brokers <br1:9092,br2:9092> Comma separated values of Target Kafka brokers
--burstCount <Integer: count> Burst count (default: 1)
--forceProperties Force unsafe properties
--initialCounter <Long: counter> Initial counter value (default: 0)
--messon Display all generated messages
--period <Long: period(ms)> Period between two bursts (ms) (default: 1000)
--property <prop=val> Producer property (May be specified several times)
--sender <sender name> Sender of the message (default: Chuck NORRIS)
* --topic <topic> Target topic
The generated messages value is a JSON string with the following format:
{
"body": "Chuck Norris breaks RSA 128-bit encrypted codes in milliseconds.",
"overallCounter": 3,
"recipient": "JADE",
"recipientCounter": 0,
"sender": "Chuck NORRIS",
"timestamp": 1487878257783
}
Here is some field description:
-
body
: A random text string. -
overallCounter
: The count of messages sent since the launch, added with the value provided by --initialCounter parameter. -
recipient
: A first name randomly choose in a list of 100. -
recipientCounter
: The count of message sent to this recipient since the launch -
sender
: A string provided by the --sender parameters. Useful when launching several kfaker process. -
timestamp
: The sending timestamp (Number of milliseconds since epoch).
The message key is the recipient. This will ensure all message to a given recipient will be pushed in the same partition.
If kerberos is activated, you will need to define a jaas configuration file as java option.
This can easily be achieved by uncomment this following line in /etc/kfaker/setenv.sh
JOPTS="$JOPTS -Djava.security.auth.login.config=/etc/kfaker/kafka_client_jaas.conf"
You can of course modify the kafka_client_jaas.conf
file to adjust to your needs or target another existing one.
But, keep in mind, you must also perform a kinit
command, with a principal granting write access to target topic
You will find an Ansible role at this location.
This role can be used as following;
- hosts: kafka_brokers
- hosts: cmd_node
vars:
kdescribe_rpm_url: https://github.com/Kappaware/kfaker/releases/download/v0.2.0/kfaker-0.2.0-1.noarch.rpm
kerberos: true
kafka_port: 9092
roles:
- kappatools/kfaker
Note
- hosts: kafka_brokers
at the beginning, which force ansible to grab info about the hosts in the[kafka_brokers]
group, to be able to fulfill this info into kfaker configuration. Of course, such a group must be defined in the inventory. In this case, brokers information will no longer be required on the command line.
Just clone this repository and then:
$ gradlew rpm
This should build everything. You should be able to find generated packages in build/distribution folder.
Copyright (C) 2017 BROADSoftware
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.