Days ago I was to get start with the newest Kafka document http://kafka.apache.org/documentation.html to learn about Kafka. But I meet some problem when I try to use the new Consumer API. I'd done the job with following steps:
1. Add a new dependency
<dependency>
<groupId>org.apache.kafka</groupId>
<artifactId>kafka-clients</artifactId>
<version>0.8.2.1</version>
</dependency>
2. Add configurations
Latest documents did not contains such configurable things, it took me a while to figure out how KafkaProducer
works, and I used some similar configurations to set the config map
Map<String, Object> config = new HashMap<String, Object>();
config.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG,
"host:9092");
config.put(ConsumerConfig.GROUP_ID_CONFIG, "test");
config.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG,
StringDeserializer.class.getName());
config.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG,
StringDeserializer.class.getName());
config.put(ConsumerConfig.PARTITION_ASSIGNMENT_STRATEGY, "range");
3. Use KafkaConsumer API
KafkaConsumer<String, String> consumer = new KafkaConsumer<String, String>(config);
consumer.subscribe("topic");
However, when I try to poll message from the broker, I got nothing but null:
Map<String, ConsumerRecords<String, String>> records = consumer.poll(0);
if (records != null)
process(records);
else
System.err.println("null");
And then I know what's wrong with the consumer after I checked the source code:
@Override
public Map<String, ConsumerRecords<K,V>> poll(long timeout) {
// TODO Auto-generated method stub
return null;
}
To make matters worse, I couldnot find any other useful information about the 0.8.2 API, since all usages about Kafka were not compatible with the latest version. I had to ask a question on SO, wishing to get some helpful advice.
Unfortunately, I could only get a bad news after ten days waiting:
--- @habsq ---
The new KafkaConsumer API will only be available in 0.8.3 cwiki.apache.org/confluence/display/KAFKA/Future+release+plan. Apparently there are some implementation in the trunk, although I have no clue about the state. For the time being I'm using the old consumer implementation.
Oops, documents could also lie, that's so sad (x﹏x)
Comments
comments powered by Disqus