How to choose the message queue ActiveMQ, RocketMQ, RabbitMQ and Kafka?
How to choose the message queue ActiveMQ, RocketMQ, RabbitMQ and Kafka?
1.ActiveMQ
advantage:
Single machine throughput: ten thousand
Timeliness: ms level
Availability: high, based on the master-slave architecture to achieve high availability
Message reliability: there is a low probability of losing data
Function support: extremely complete functions in the MQ field
shortcoming:
The official community now maintains less and less ActiveMQ 5.x, and it is seldom used in large-scale throughput scenarios.
2.Kafka
Known as the killer of big data, when it comes to message transmission in the field of big data, Kafka cannot be bypassed. This message middleware born for big data is famous for its million-level TPS throughput. It has quickly become the darling of the big data field, playing a pivotal role in the process of data collection, transmission, and storage.
Apache Kafka was originally implemented by LinkedIn as a distributed commit log system based on a unique design, and later became a part of the Apache project.
It has been adopted by big companies such as LinkedIn, Uber, Twitter, Netflix and so on.
advantage
The performance is excellent, and the single-machine write TPS is about one million entries per second. The biggest advantage is the high throughput.
Timeliness: ms level
Availability: Very high. Kafka is distributed, with multiple copies of one data, and a small number of machines are down, no data will be lost, and no unavailability will be caused
Consumers use the Pull method to obtain messages, and the messages are in order. Through control, it can be ensured that all messages are consumed and only consumed once;
There is an excellent third-party Kafka Web management interface Kafka-Manager;
It is relatively mature in the log field and is used by many companies and many open source projects;
Function support: The function is relatively simple, mainly supporting simple MQ functions, real-time calculation and log collection in the field of big data are used on a large scale
shortcoming:
Kafka single machine has more than 64 queues/partitions, and the load will increase significantly. The more queues, the higher the load, and the longer the response time for sending messages.
Use short polling mode, the real-time performance depends on the polling interval;
Retry is not supported for consumption failure;
Support message order, but when an agent goes down, the message will be out of order;
Community updates are slow;
3.RabbitMQ
RabbitMQ was released in 2007. It is a reusable enterprise messaging system based on AMQP (Advanced Message Queuing Protocol) and is currently one of the most mainstream messaging middleware.
Advantages of RabbitMQ:
Due to the characteristics of erlang language, mq has better performance and high concurrency;
Throughput reaches ten thousand, MQ function is relatively complete
Robust, stable, easy to use, cross-platform, support multiple languages, complete documentation;
The management interface provided by open source is great and easy to use
High community activity;
Disadvantages of RabbitMQ:
Erlang development is difficult to understand the source code. The basic functions depend on the rapid maintenance and bug fixes of the open source community, which is not conducive to secondary development and maintenance.
RabbitMQ does have lower throughput because of the heavier implementation mechanism.
Need to learn more complex interfaces and protocols, and the cost of learning and maintenance is high.
4.RocketMQ
RocketMQ comes from the open source product of Alibaba Company, implemented in Java language, referred to Kafka in the design, and made some improvements of its own.
RocketMQ is widely used in Alibaba Group in order, transaction, recharge, stream computing, message push, log stream processing, binglog distribution and other scenarios.
RocketMQ advantages:
Single machine throughput: 100,000 level
Availability: very high, distributed architecture
Message reliability: After parameter optimization configuration, the message can achieve zero loss
Function support: MQ function is relatively complete, or distributed, good scalability
Support 1 billion message accumulation, no performance degradation due to accumulation
The source code is java, we can read the source code by ourselves and customize our company's MQ
Disadvantages of RocketMQ:
There are not many client languages supported, currently java and c++, of which c++ is immature;
Community activity is average
There is no implementation of JMS and other interfaces in the mq core, some systems need to modify a lot of code to migrate