Guaranteeing Message Processing

最新推荐文章于 2023-08-15 15:32:29 发布

cool_bomb

最新推荐文章于 2023-08-15 15:32:29 发布

阅读量764

点赞数 1

文章标签： storm

本文介绍了Apache Storm中消息处理的概念，包括消息如何被视为“完全处理”及其背后的工作原理。此外，还详细阐述了Storm可靠性API的使用方法，以及如何通过anchoring机制确保消息处理的可靠性。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

What does it mean for a message to be "fully processed"?

Storm considers a tuple coming off a spout "fully processed" when the tuple tree has been exhausted and every message in the tree has been processed.

：只有当tuple tree被耗尽，tree中的每个message都被处理时，来源于一个spot中的tuple就被完全处理。

What happens if a message is fully processed or fails to be fully processed?

If Storm detects that a tuple is fully processed, Storm will call the ack method on the originating Spout task with the message id that the Spout provided to Storm. Likewise, if the tuple times-out Storm will call the fail method on the Spout.

如果storm发现一个tuple被完全处理，storm会利用来自于spout 中的message id，调原始的spout中的ack方法。同样的，如果tuple处理超时，storm将会调用spout中的fail方法。

What is Storm's reliability API?

There's two things you have to do as a user to benefit from Storm's reliability capabilities. First, you need to tell Storm whenever you're creating a new link in the tree of tuples. Second, you need to tell Storm when you have finished processing an individual tuple.

为了使用storm的可靠性能力。第一，必须告诉storm什么时候在tuples tree中创建了一个新的链接。第二，告诉storm什么时候处理完一个tuple。

Specifying a link in the tuple tree is called anchoring. Anchoring is done at the same time you emit a new tuple.

在tuple tree中指定一个链接被称为anchoring。anchoring在你提交新的tuple的时候已经完成。

public class SplitSentence extends BaseRichBolt {
        OutputCollector _collector;

        public void prepare(Map conf, TopologyContext context, OutputCollector collector) {
            _collector = collector;
        }

        public void execute(Tuple tuple) {
            String sentence = tuple.getString(0);
            for(String word: sentence.split(" ")) {
                _collector.emit(tuple, new Values(word));
            }
            _collector.ack(tuple);
        }

        public void declareOutputFields(OutputFieldsDeclarer declarer) {
            declarer.declare(new Fields("word"));
        }        
    }

Since the word tuple is anchored, the spout tuple at the root of the tree will be replayed later on if the word tuple failed to be processed downstream.

如果word tuple被anchored了，word tuple在下游被处理失败， tree的根节点将会被重播。