Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 7 additions & 7 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
We are always very happy to have contributions, whether for trivial cleanups or big new features.
We want to have high quality, well documented codes for each programming language, as well as the surrounding [ecosystem](https://github.com/apache/rocketmq-externals) of integration tools that people use with RocketMQ.

Nor is code the only way to contribute to the project. We strongly value documentation, integration with other project, and gladly accept improvements for these aspects.
Nor is code the only way to contribute to the project. We strongly value documentation, integration with other projects, and gladly accept improvements for these aspects.

Recommend reading:
* [Contributors Tech Guide](http://www.apache.org/dev/contributors)
Expand Down Expand Up @@ -34,15 +34,15 @@ More details of squash can be found at [stackoverflow](https://stackoverflow.com

We are always interested in adding new contributors. What we look for are series of contributions, good taste and ongoing interest in the project. If you are interested in becoming a committer, please let one of the existing committers know and they can help you walk through the process.

Nowadays,we have several important contribution points:
Nowadays, we have several important contribution points:
#### Wiki & JavaDoc
#### RocketMQ SDK(C++\.Net\Php\Python\Go\Node.js)
#### RocketMQ Connectors

##### Prerequisite
If you want to contribute the above listing points, you must abide our some prerequisites:
##### Prerequisites
If you want to contribute to the above listed points, you must abide by the following prerequisites:

###### Readability - API must have Javadoc, some very important methods also must have javadoc
###### Testability - 80% above unit test coverage about main process
###### Maintainability - Comply with our [checkstyle spec](style/rmq_checkstyle.xml), and at least 3 month update frequency
###### Readability - API must have Javadoc, and some very important methods must also have Javadoc
###### Testability - Above 80% unit test coverage for the main process
###### Maintainability - Comply with our [checkstyle spec](style/rmq_checkstyle.xml), and at least a 3-month update frequency
###### Deployability - We encourage you to deploy into [maven repository](http://search.maven.org/)
12 changes: 6 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,8 @@ It offers a variety of features:

* Messaging patterns including publish/subscribe, request/reply and streaming
* Financial grade transactional message
* Built-in fault tolerance and high availability configuration options base on [DLedger Controller](docs/en/controller/quick_start.md)
* Built-in message tracing capability, also support opentracing
* Built-in fault tolerance and high availability configuration options based on [DLedger Controller](docs/en/controller/quick_start.md)
* Built-in message tracing capability, also supports opentracing
* Versatile big-data and streaming ecosystem integration
* Message retroactivity by time or offset
* Reliable FIFO and strict ordered messaging in the same queue
Expand Down Expand Up @@ -80,7 +80,7 @@ $ tail -f ~/logs/rocketmqlogs/namesrv.log
The Name Server boot success...
```

For Windows users, you need set environment variables first:
For Windows users, you need to set environment variables first:
- From the desktop, right click the Computer icon.
- Choose Properties from the context menu.
- Click the Advanced system settings link.
Expand Down Expand Up @@ -139,7 +139,7 @@ Before your operations, make sure that `kubectl` and related kubeconfig file ins
$ git clone https://github.com/apache/rocketmq-operator
$ cd rocketmq-operator && make deploy

### check whether CRDs is successfully installed
### check whether CRDs are successfully installed
$ kubectl get crd | grep rocketmq.apache.org
brokers.rocketmq.apache.org 2022-05-12T09:23:18Z
consoles.rocketmq.apache.org 2022-05-12T09:23:19Z
Expand All @@ -156,7 +156,7 @@ rocketmq-operator-6f65c77c49-8hwmj 1/1 Running 0 93s
### create RocketMQ cluster resource
$ cd example && kubectl create -f rocketmq_v1alpha1_rocketmq_cluster.yaml

### check whether cluster resources is running
### check whether cluster resources are running
$ kubectl get sts
NAME READY AGE
broker-0-master 1/1 107m
Expand All @@ -182,7 +182,7 @@ name-service 1/1 107m
* [RocketMQ Dashboard](https://github.com/apache/rocketmq-dashboard): Operation and maintenance console of Apache RocketMQ.
* [RocketMQ Connect](https://github.com/apache/rocketmq-connect): A tool for scalably and reliably streaming data between Apache RocketMQ and other systems.
* [RocketMQ MQTT](https://github.com/apache/rocketmq-mqtt): A new MQTT protocol architecture model, based on which Apache RocketMQ can better support messages from terminals such as IoT devices and Mobile APP.
* [RocketMQ EventBridge](https://github.com/apache/rocketmq-eventbridge): EventBridge make it easier to build a event-driven application.
* [RocketMQ EventBridge](https://github.com/apache/rocketmq-eventbridge): EventBridge makes it easier to build an event-driven application.
* [RocketMQ Incubating Community Projects](https://github.com/apache/rocketmq-externals): Incubator community projects of Apache RocketMQ, including [logappender](https://github.com/apache/rocketmq-externals/tree/master/logappender), [rocketmq-ansible](https://github.com/apache/rocketmq-externals/tree/master/rocketmq-ansible), [rocketmq-beats-integration](https://github.com/apache/rocketmq-externals/tree/master/rocketmq-beats-integration), [rocketmq-cloudevents-binding](https://github.com/apache/rocketmq-externals/tree/master/rocketmq-cloudevents-binding), etc.
* [RocketMQ Site](https://github.com/apache/rocketmq-site): The repository for Apache RocketMQ website.
* [RocketMQ E2E](https://github.com/apache/rocketmq-e2e): A project for testing Apache RocketMQ, including end-to-end, performance, compatibility tests.
Expand Down
4 changes: 2 additions & 2 deletions distribution/bin/README.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
### Operating system tuning
Before deploying broker servers, it's highly recommended to run **os.sh**, which is to optimize your operating system for better performance.
Before deploying broker servers, it is highly recommended to run **os.sh**, which optimizes your operating system for better performance.

## Notice
### os.sh should be executed only once with root permission.
### os.sh parameter settings are for reference purpose only. You can tune them according to your target host configurations.
### os.sh parameter settings are for reference purposes only. You can tune them according to your target host configurations.


### Start broker
Expand Down
9 changes: 4 additions & 5 deletions docs/en/Concept.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,17 +18,16 @@ The Name Server serves as the provider of routing service. The producer or the c
## 7 Pull Consumer
A type of Consumer, the application pulls messages from brokers by actively invoking the consumer pull message method, and the application has the advantages of controlling the timing and frequency of pulling messages. Once the batch of messages is pulled, user application will initiate consuming process.
## 8 Push Consumer
A type of Consumer, the application do not invoke the consumer pull message method to pull messages, instead the client invoke pull message method itself. At the user level it seems like brokers
push to consumer when new messages arrived.
A type of Consumer, the application does not invoke the consumer pull message method to pull messages, instead the client invokes the pull message method itself. At the user level it seems like brokers push to the consumer when new messages arrive.
## 9 Producer Group
A collection of the same type of Producer, which sends the same type of messages with consistent logic. If a transaction message is sent and the original producer crashes after sending, the broker server will contact other producers in the same producer group to commit or rollback the transactional message.
## 10 Consumer Group
A collection of the same type of Consumer, which consume the same type of messages with consistent logic. The consumer group makes load-balance and fault-tolerance super easy in terms of message consuming.
Warning: consumer instances of one consumer group must have exactly the same topic subscription(s).

RocketMQ supports two types of consumption mode:Clustering and Broadcasting.
RocketMQ supports two types of consumption mode: Clustering and Broadcasting.
## 11 Consumption Mode - Clustering
Under the Clustering mode, all the messages from one topic will be delivered to all the consumers instances averagely as much as possible. That is, one message can be consumed by only one consumer instance.
Under the Clustering mode, all the messages from one topic will be delivered to all the consumer instances as evenly as possible. That is, one message can be consumed by only one consumer instance.
## 12 Consumption Mode - Broadcasting
Under the Broadcasting mode, each consumer instance of the same consumer group receives every message published to the corresponding topic.
## 13 Normal Ordered Message
Expand All @@ -39,4 +38,4 @@ Under the Strictly Ordered Message mode, all messages received by the consumers
The physical carrier of information transmitted by a messaging system, the smallest unit of production and consumption data, each message must belong to one topic.
Each Message in RocketMQ has a unique message id and can carry a key used to store business-related value. The system has the function to query messages by its id or key.
## 16 Tag
Flags set for messages to distinguish different types of messages under the same topic, functioning as a "sub-topic". Messages from the same business unit can set different tags under the same topic in terms of different business purposes. The tag can effectively maintain the clarity and consistency of the code and optimize the query system provided by RocketMQ. The consumer can realize different "sub-topic" by using tag in order to achieve better expandability.
Flags set for messages to distinguish different types of messages under the same topic, functioning as a "sub-topic". Messages from the same business unit can set different tags under the same topic for different business purposes. The tag can effectively maintain the clarity and consistency of the code and optimize the query system provided by RocketMQ. The consumer can realize different "sub-topics" by using tags in order to achieve better extensibility.
8 changes: 4 additions & 4 deletions docs/en/Configuration_TLS.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
# TLS Configuration
This section introduce TLS configuration in RocketMQ.
This section introduces TLS configuration in RocketMQ.

## 1 Generate Certification Files
User can generate certification files using OpenSSL. Suggested to generate files in Linux.
## 1 Generate Certificate Files
Users can generate certificate files using OpenSSL. It is suggested to generate files in Linux.

### 1.1 Generate ca.pem
```shell
Expand Down Expand Up @@ -107,7 +107,7 @@ Add following parameters in JVM. The value of "tls.config.file" needs to be repl
-Dtls.client.authServer=true -Dtls.enable=true -Dtls.test.mode.enable=false -Dtls.config.file=/opt/certs/tlsclient.properties
```

Enable TLS for client linke following:
Enable TLS for client like the following:
```Java
public class ExampleProducer {
public static void main(String[] args) throws Exception {
Expand Down
2 changes: 1 addition & 1 deletion docs/en/Deployment.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

### 1 Single Master mode

This is the simplest, but also the riskiest mode, that makes the entire service unavailable once the broker restarts or goes down. Production environments are not recommended, but can be used for local testing and development. Here are the steps to build.
This is the simplest, but also the riskiest mode, that makes the entire service unavailable once the broker restarts or goes down. Production environments are not recommended, but it can be used for local testing and development. Here are the steps to build.

**1)Start NameServer**

Expand Down
2 changes: 1 addition & 1 deletion docs/en/Design_LoadBlancing.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ Load balancing in RocketMQ is accomplished on Client side. Specifically, it can

### Producer Load Balancing
When the Producer sends a message, it will first find the specified TopicPublishInfo according to Topic. After getting the routing information of TopicPublishInfo, the RocketMQ client will select a queue (MessageQueue) from the messageQueue List in TopicPublishInfo to send the message by default.Specific fault-tolerant strategies are defined in the MQFaultStrategy class.
Here is a sendLatencyFaultEnable switch variable, which, if turned on, filters out the Broker agent of not available on the basis of randomly gradually increasing modular arithmetic selection. The so-called "latencyFault Tolerance" refers to a certain period of time to avoid previous failures. For example, if the latency of the last request exceeds 550 Lms, it will evade 30000 Lms; if it exceeds 1000L, it will evade 60000L; if it is closed, it will choose a queue (MessageQueue) to send messages by randomly gradually increasing modular arithmetic, and the latencyFault Tolerance mechanism is the key to achieve high availability of message sending.
Here is a sendLatencyFaultEnable switch variable, which, if turned on, filters out the Broker agents that are not available on the basis of randomly gradually increasing modular arithmetic selection. The so-called "latencyFault Tolerance" refers to a certain period of time to avoid previous failures. For example, if the latency of the last request exceeds 550 Lms, it will evade 30000 Lms; if it exceeds 1000L, it will evade 60000L; if it is closed, it will choose a queue (MessageQueue) to send messages by randomly gradually increasing modular arithmetic, and the latencyFault Tolerance mechanism is the key to achieve high availability of message sending.

### Consumer Load Balancing
In RocketMQ, the two consumption modes (Push/Pull) on the Consumer side are both based on the pull mode to get the message, while in the Push mode it is only a kind of encapsulation of the pull mode, which is essentially implemented as the message pulling thread after pulling a batch of messages from the server. After submitting to the message consuming thread pool, it continues to try again to pull the message to the server. If the message is not pulled, the pull is delayed and continues. In both pull mode based consumption patterns (Push/Pull), the Consumer needs to know which message queue - queue from the Broker side to get the message. Therefore, it is necessary to do load balancing on the Consumer side, that is, which Consumer consumption is allocated to the same ConsumerGroup by more than one MessageQueue on the Broker side.
Expand Down
2 changes: 1 addition & 1 deletion docs/en/Design_Trancation.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ The compensation phase is used to resolve the timeout or failure case of the mes
### 1.2 The design of RocketMQ Transaction Message
1 Transaction message is invisible to users in first phase(commit-request phase)

Upon on the main process of transaction message, the message of first phase is invisible to the user. This is also the biggest difference from normal message. So how do we write the message while making it invisible to the user? And below is the solution of RocketMQ: if the message is a Half message, the topic and queueId of the original message will be backed up, and then changes the topic to RMQ_SYS_TRANS_HALF_TOPIC. Since the consumer group does not subscribe to the topic, the consumer cannot consume the Half message. Then RocketMQ starts a timing task, pulls the message for RMQ_SYS_TRANS_HALF_TOPIC, obtains a channel according to producer group and sends a back-check to query local transaction status, and decide whether to submit or roll back the message according to the status.
In the main process of transaction message, the message of first phase is invisible to the user. This is also the biggest difference from normal message. So how do we write the message while making it invisible to the user? And below is the solution of RocketMQ: if the message is a Half message, the topic and queueId of the original message will be backed up, and then changes the topic to RMQ_SYS_TRANS_HALF_TOPIC. Since the consumer group does not subscribe to the topic, the consumer cannot consume the Half message. Then RocketMQ starts a timing task, pulls the message for RMQ_SYS_TRANS_HALF_TOPIC, obtains a channel according to producer group and sends a back-check to query local transaction status, and decide whether to submit or roll back the message according to the status.

In RocketMQ, the storage structure of the message in the broker is as follows. Each message has corresponding index information. The Consumer reads the content of the message through the secondary index of the ConsumeQueue. The flow is as follows:

Expand Down
6 changes: 3 additions & 3 deletions docs/en/FAQ.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ The following questions are frequently asked with regard to the RocketMQ project

Please refer to [Why RocketMQ](http://rocketmq.apache.org/docs/motivation)

2. Do I have to install other softeware, such as zookeeper, to use RocketMQ?
2. Do I have to install other software, such as zookeeper, to use RocketMQ?

No. RocketMQ can run independently.

Expand All @@ -24,9 +24,9 @@ The following questions are frequently asked with regard to the RocketMQ project

### 2. How to reconsume message when consumption fails?

 1) Cluster consumption pattern, The consumer business logic code returns Action.ReconsumerLater, NULL, or throws an exception, if a message failed to be consumed, it will retry for up to 16 times, after that, the message would be descarded.
 1) Cluster consumption pattern, The consumer business logic code returns Action.ReconsumerLater, NULL, or throws an exception, if a message failed to be consumed, it will retry for up to 16 times, after that, the message would be discarded.

 2) Broadcast consumption patternThe broadcaset consumption still ensures that a message is consumered at least once, but no resend option is provided.
 2) Broadcast consumption pattern. The broadcast consumption still ensures that a message is consumed at least once, but no resend option is provided.

### 3. How to query the failed message if there is a consumption failure?

Expand Down
2 changes: 1 addition & 1 deletion docs/en/Troubleshoopting.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@

Solution: Execute `export NAMESRV_ADDR=ip:9876` (ip refers to the address of NameServer deployed in the cluster) on the VM that deploys the RocketMQ cluster.Then you will execute commands of "mqadmin" successfully.

## 2 The inconsistent version of RocketMQ between the producer and consumer leads to the problem that message can't be consumed normally.
## 2 The inconsistent version of RocketMQ between the producer and consumer leads to the problem that messages can't be consumed normally.

> Problem: The same producer sends a message, consumer A can consume, but consumer B can't consume, and the RocketMQ Console appears:
>
Expand Down
Loading
Loading