По директориям раскидал
This commit is contained in:
224
Brokers/RabbitMQ_install.md
Normal file
224
Brokers/RabbitMQ_install.md
Normal file
@ -0,0 +1,224 @@
|
||||
****************************************************************************************************
|
||||
Подключаемся к нужной машине
|
||||
```sh
|
||||
ssh ivanov_i@10.101.1.3 -p 22
|
||||
```
|
||||
|
||||
Установка сервиса обмена сообщениями RabbitMQ – менеджер сообщений (message broker), написан на Erlang, ближайший аналог в AWS – SQS. По документации из: https://rtfm.co.ua/ru/rabbitmq-zapusk-opisanie-primery/
|
||||
```sh
|
||||
sudo apt-get update &&
|
||||
sudo apt-get dist-upgrade &&
|
||||
sudo apt install rabbitmq-server
|
||||
```
|
||||
|
||||
Создаём пользователя и задаём пароль
|
||||
```sh
|
||||
sudo rabbitmqctl list_users &&
|
||||
sudo rabbitmqctl add_user ivanov_i KGf4nxT8vxZWv3jqNasP &&
|
||||
sudo rabbitmqctl set_user_tags ivanov_i administrator &&
|
||||
sudo rabbitmqctl set_permissions -p / ivanov_i ".*" ".*" ".*" &&
|
||||
sudo rabbitmqctl change_password ivanov_i KGf4nxT8vxZWv3jqNasP
|
||||
```
|
||||
|
||||
Посмотреть список используемых плагинов:
|
||||
```sh
|
||||
sudo rabbitmq-plugins list
|
||||
```
|
||||
Активируем плагин rabbitmq_management для наблюдения за системой по HTTP https://www.rabbitmq.com/management.html https://thewebland.net/development/devops/chast-3-interfejs-upravleniya-rabbitmq/#:~:text=RabbitMQ%20Management%20–%20это%20удобный%20интерфейс,и%20отправлять%20%2F%20получать%20сообщения%20вручную.
|
||||
```sh
|
||||
sudo rabbitmq-plugins enable rabbitmq_management
|
||||
```
|
||||
rabbitmq_management активирует поддержку API на порту 15672: http://192.168.0.144:15672 http://10.1.7.70:15672 http://192.168.0.83:15672
|
||||
Но для активации нужно добавить пользователя, смотрим пользователей:
|
||||
```sh
|
||||
sudo rabbitmqctl list_users
|
||||
sudo rabbitmqctl add_user admin paRabbit! или test test для локального тестового сервера
|
||||
sudo rabbitmqctl set_user_tags admin administrator
|
||||
sudo rabbitmqctl set_permissions -p / admin ".*" ".*" ".*"
|
||||
```
|
||||
Также можно создать пользователя для публикации сообщений но уже это можно сделать через WEB интерфейс: producer lgGDWAJEwI для guest с паролем: efewG!4ev Александра: alex jefruf!43
|
||||
|
||||
Теперь можно логинется по test test поменять пароль можно: https://kb.vmware.com/s/article/323520#:~:text=Log%20in%20to%20the%20RabbitMQ,host%20where%20RabbitMQ%20is%20installed.
|
||||
Или так можно поменять пароль: sudo rabbitmqctl change_password zenoss New-Password
|
||||
producer: клиент, выполняющий отправку сообщения
|
||||
queue: собственно очередь сообщений
|
||||
consumer: клиент, получающий сообщения из очереди
|
||||
exchange: получает сообщения от producer, и отправялет их в очереди в соответствии с его типом (см. тут https://www.rabbitmq.com/tutorials/tutorial-three-python.html)
|
||||
|
||||
Дальше можно пробовать отправлять данные в очередь используя допустим java библиотеку урок: https://www.youtube.com/watch?v=6lPK_LgTZ9Y
|
||||
https://www.rabbitmq.com/devtools.html :
|
||||
Создаю Maven проект в консоли:
|
||||
```sh
|
||||
cd O:\projects\Workspace_Java\
|
||||
mvn archetype:generate -DgroupId=kz.istt.app -DartifactId=TransitToRabbitMQ -DarchetypeArtifactId=maven-archetype-quickstart -DarchetypeVersion=1.4 -DinteractiveMode=false
|
||||
```
|
||||
Открываю его в idea64 и добавляю в POM следующую зависемость из https://mvnrepository.com/artifact/com.rabbitmq/amqp-client:
|
||||
```xml
|
||||
<!-- https://mvnrepository.com/artifact/com.rabbitmq/amqp-client -->
|
||||
<dependency>
|
||||
<groupId>com.rabbitmq</groupId>
|
||||
<artifactId>amqp-client</artifactId>
|
||||
<version>5.14.2</version>
|
||||
</dependency>
|
||||
```
|
||||
Также поставил C:\Program Files\Java\jdk-14.0.2 а то не запускалось, также просит рут права для IntelliJ IDEA.
|
||||
|
||||
Урок создания maven проекта: https://devcolibri.com/unit/урок-1-подготовка-и-создание-maven-проекта/
|
||||
или https://www.jetbrains.com/idea/guide/tutorials/working-with-maven/creating-a-project/
|
||||
Добавил в POM файл:
|
||||
```xml
|
||||
<repositories>
|
||||
<repository>
|
||||
<name>RabbitMQ</name>
|
||||
<url>https://repo1.maven.org/maven2/com/rabbitmq/amqp-client/5.16.0</url>
|
||||
<id>com.rabbitmq</id>
|
||||
<releases>
|
||||
<enabled>false</enabled>
|
||||
</releases>
|
||||
<snapshots>
|
||||
<enabled>true</enabled>
|
||||
</snapshots>
|
||||
</repository>
|
||||
</repositories>
|
||||
|
||||
<dependencies>
|
||||
<dependency>
|
||||
<groupId>com.rabbitmq</groupId>
|
||||
<artifactId>amqp-client</artifactId>
|
||||
<version>5.16.0</version>
|
||||
</dependency>
|
||||
</dependencies>
|
||||
```
|
||||
|
||||
Перезагружаем:
|
||||
```sh
|
||||
sudo service rabbitmq-server restart
|
||||
```
|
||||
|
||||
****************************************************************************************************
|
||||
Установка Shovels Plugin для RabbitMQ (лопата) по мотивам из: https://www.rabbitmq.com/docs/shovel
|
||||
|
||||
Активирую плагин командой:
|
||||
```sh
|
||||
sudo rabbitmq-plugins enable rabbitmq_shovel &&
|
||||
sudo rabbitmq-plugins enable rabbitmq_shovel_management
|
||||
```
|
||||
|
||||
Настраиваю Shovels Dynamic плагин в RabbitMQ через командную строку:
|
||||
```sh
|
||||
sudo rabbitmqctl set_parameter shovel kaz_to_kaz '{
|
||||
"src-protocol": "amqp091",
|
||||
"src-uri": "amqp://admin:paRabbit!@127.0.0.1:20000/transportation",
|
||||
"src-queue": "TO_KAZ",
|
||||
"dest-protocol": "amqp091",
|
||||
"dest-uri": "amqp://admin:paRabbit!@10.101.1.11:20000/playground",
|
||||
"dest-exchange": "swap",
|
||||
"dest-exchange-key": ".KAZ.",
|
||||
"ack-mode": "on-confirm",
|
||||
"publish-properties": {
|
||||
"delivery_mode": 2
|
||||
}
|
||||
}'
|
||||
```
|
||||
|
||||
Для Росии такие настройки:
|
||||
```sh
|
||||
sudo rabbitmqctl set_parameter shovel kaz_to_rus '{
|
||||
"src-protocol": "amqp091",
|
||||
"src-uri": "amqp://admin:paRabbit!@127.0.0.1:20000/transportation_rus",
|
||||
"src-queue": "TO_RUS",
|
||||
"dest-protocol": "amqp091",
|
||||
"dest-uri": "amqp://iktt_kaz:jkrjHL7xj7PrW1D@192.168.89.133:20000/playground",
|
||||
"dest-exchange": "swap",
|
||||
"dest-exchange-key": "KZ",
|
||||
"ack-mode": "on-confirm",
|
||||
"publish-properties": {
|
||||
"delivery_mode": 2
|
||||
}
|
||||
}'
|
||||
```
|
||||
|
||||
Для Беларусии такие настройки:
|
||||
```sh
|
||||
sudo rabbitmqctl set_parameter shovel kaz_to_blr '{
|
||||
"src-protocol": "amqp091",
|
||||
"src-uri": "amqp://admin:paRabbit!@127.0.0.1:20000/transportation_blr",
|
||||
"src-queue": "TO_BLR",
|
||||
"dest-protocol": "amqp091",
|
||||
"dest-uri": "amqp://KZ_IKTT:DK34xDNlZQfQ551k@192.168.90.133:20000/plumber",
|
||||
"dest-exchange": "eec.swap",
|
||||
"dest-exchange-key": "KZ",
|
||||
"ack-mode": "on-confirm",
|
||||
"publish-properties": {
|
||||
"delivery_mode": 2
|
||||
}
|
||||
}'
|
||||
```
|
||||
|
||||
Для Армении такие настройки:
|
||||
```sh
|
||||
sudo rabbitmqctl set_parameter shovel kaz_to_arm '{
|
||||
"src-protocol": "amqp091",
|
||||
"src-uri": "amqp://admin:password@127.0.0.1:20000/transportation_arm",
|
||||
"src-queue": "TO_ARM",
|
||||
"dest-protocol": "amqp091",
|
||||
"dest-uri": "amqp://kz_istt:password@192.168.89.158:20000/transportation",
|
||||
"dest-exchange": "swap_kz",
|
||||
"dest-exchange-key": "KZ_ISTT",
|
||||
"ack-mode": "on-confirm",
|
||||
"publish-properties": {
|
||||
"delivery_mode": 2
|
||||
}
|
||||
}'
|
||||
```
|
||||
|
||||
Для Кыргызстана (таможенная инфраструктура)
|
||||
```sh
|
||||
sudo rabbitmqctl set_parameter shovel kaz_to_kgz '{
|
||||
"src-protocol": "amqp091",
|
||||
"src-uri": "amqp://admin:paRabbit!@127.0.0.1:20000/transportation_kgz",
|
||||
"src-queue": "TO_KGZ",
|
||||
"dest-protocol": "amqp091",
|
||||
"dest-uri": "amqp://kz_istt:uzZNYbVTElMDXnfwQx16@192.168.70.133:20000/transportation_kaz",
|
||||
"dest-exchange": "swap",
|
||||
"dest-exchange-key": "KAZ",
|
||||
"ack-mode": "on-confirm",
|
||||
"publish-properties": {
|
||||
"delivery_mode": 2
|
||||
}
|
||||
}'
|
||||
```
|
||||
Удалять так:
|
||||
```sh
|
||||
sudo rabbitmqctl clear_parameter shovel kaz_to_kgz
|
||||
```
|
||||
|
||||
Для отправки из Киргизии нам то такие настройки:
|
||||
```sh
|
||||
sudo rabbitmqctl set_parameter shovel kgz_to_kaz '{
|
||||
"src-protocol": "amqp091",
|
||||
"src-uri": "amqp://admin:pfta2OFt@127.0.0.1:20000/transportation_kaz",
|
||||
"src-queue": "TO_KAZ",
|
||||
"dest-protocol": "amqp091",
|
||||
"dest-uri": "amqp://kg_amap:qrd2fjEjkegdi7bfb@192.168.70.134:20000/transportation_kgz",
|
||||
"dest-exchange": "swap",
|
||||
"dest-exchange-key": "KGZ",
|
||||
"ack-mode": "on-confirm",
|
||||
"publish-properties": {
|
||||
"delivery_mode": 2
|
||||
}
|
||||
}'
|
||||
```
|
||||
|
||||
Удалять так:
|
||||
```sh
|
||||
sudo rabbitmqctl clear_parameter shovel kgz_to_kaz
|
||||
```
|
||||
|
||||
Чтобы проверить текущие параметры Shovel:
|
||||
```sh
|
||||
sudo rabbitmqctl list_parameters
|
||||
```
|
||||
А также можно взглянуть что создалось через WEB интерфейс
|
||||
|
||||
Не забывать что Exchange: swap должен быть не "direct" а как "topic" иначе маршрутизация не сработает.
|
||||
341
Brokers/rabbitmq_replication_setup_instructions.md
Normal file
341
Brokers/rabbitmq_replication_setup_instructions.md
Normal file
@ -0,0 +1,341 @@
|
||||
Here’s a step-by-step guide to setting up a **RabbitMQ cluster with replication** correctly:
|
||||
|
||||
---
|
||||
|
||||
## **Step 1: Prepare the Servers**
|
||||
|
||||
Ensure you have at least **three** nodes (recommended for HA) with:
|
||||
|
||||
- Ubuntu 22.04 (or a supported OS)
|
||||
- Sufficient CPU/RAM based on workload
|
||||
- Open necessary firewall ports (**5672, 15672, 25672**)
|
||||
|
||||
Set hostnames for clarity:
|
||||
|
||||
```bash
|
||||
sudo hostnamectl set-hostname rabbitmq-node1 # Change for each node
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## **Step 2: Install Erlang and RabbitMQ**
|
||||
|
||||
Run the following on **all nodes**:
|
||||
|
||||
### **1. Add the RabbitMQ Repository**
|
||||
|
||||
```bash
|
||||
sudo apt update
|
||||
sudo apt install -y curl gnupg
|
||||
curl -fsSL https://packagecloud.io/rabbitmq/rabbitmq-server/gpgkey | sudo tee /usr/share/keyrings/rabbitmq-key.asc
|
||||
echo "deb [signed-by=/usr/share/keyrings/rabbitmq-key.asc] https://packagecloud.io/rabbitmq/rabbitmq-server/ubuntu/ $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/rabbitmq.list
|
||||
```
|
||||
|
||||
### **2. Install Erlang and RabbitMQ**
|
||||
|
||||
```bash
|
||||
sudo apt update
|
||||
sudo apt install -y rabbitmq-server
|
||||
```
|
||||
|
||||
### **3. Enable RabbitMQ**
|
||||
|
||||
```bash
|
||||
sudo systemctl enable --now rabbitmq-server
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## **Step 3: Configure Clustering**
|
||||
|
||||
### **1. Stop RabbitMQ on All Nodes**
|
||||
|
||||
```bash
|
||||
sudo systemctl stop rabbitmq-server
|
||||
```
|
||||
|
||||
### **2. Configure Cookie for Clustering**
|
||||
|
||||
Run on **all nodes** (same cookie ensures clustering works):
|
||||
|
||||
```bash
|
||||
echo "MY_CLUSTER_COOKIE" | sudo tee /var/lib/rabbitmq/.erlang.cookie
|
||||
sudo chmod 600 /var/lib/rabbitmq/.erlang.cookie
|
||||
```
|
||||
|
||||
Replace `"MY_CLUSTER_COOKIE"` with a strong, identical value on all nodes.
|
||||
|
||||
---
|
||||
|
||||
## **Step 4: Join Nodes to the Cluster**
|
||||
|
||||
Perform this on **nodes 2 and 3**, joining them to **node1**:
|
||||
|
||||
```bash
|
||||
sudo rabbitmqctl stop_app
|
||||
sudo rabbitmqctl join_cluster rabbit@rabbitmq-node1
|
||||
sudo rabbitmqctl start_app
|
||||
```
|
||||
|
||||
Check cluster status:
|
||||
|
||||
```bash
|
||||
sudo rabbitmqctl cluster_status
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## **Step 5: Enable High Availability (HA) Mirroring**
|
||||
|
||||
To replicate all queues, run on **any one node**:
|
||||
|
||||
```bash
|
||||
rabbitmqctl set_policy ha-all "^.*" '{"ha-mode":"all","ha-sync-mode":"automatic"}'
|
||||
```
|
||||
|
||||
This ensures all queues are **replicated across all nodes**.
|
||||
|
||||
---
|
||||
|
||||
## **Step 6: Enable Management UI**
|
||||
|
||||
Run on **each node** to enable the web interface:
|
||||
|
||||
```bash
|
||||
sudo rabbitmq-plugins enable rabbitmq_management
|
||||
```
|
||||
|
||||
Access at:
|
||||
**http://[NODE_IP]:15672**
|
||||
(Default login: `guest/guest`, change this for security.)
|
||||
|
||||
---
|
||||
|
||||
## **Step 7: Test the Cluster**
|
||||
|
||||
Run on **each node**:
|
||||
|
||||
```bash
|
||||
rabbitmqctl list_queues
|
||||
```
|
||||
|
||||
Queues should be visible and synchronized.
|
||||
|
||||
---
|
||||
|
||||
## **Step 8: Enable Auto-Recovery** (Optional)
|
||||
|
||||
Edit `/etc/rabbitmq/rabbitmq.conf` on **each node** and add:
|
||||
|
||||
```ini
|
||||
cluster_formation.peer_discovery_backend = classic_config
|
||||
cluster_formation.classic_config.nodes.1 = rabbit@rabbitmq-node1
|
||||
cluster_formation.classic_config.nodes.2 = rabbit@rabbitmq-node2
|
||||
cluster_formation.classic_config.nodes.3 = rabbit@rabbitmq-node3
|
||||
```
|
||||
|
||||
Then restart RabbitMQ:
|
||||
|
||||
```bash
|
||||
sudo systemctl restart rabbitmq-server
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## **Step 9: Secure the Cluster** (Recommended)
|
||||
|
||||
### **1. Create an Admin User**
|
||||
|
||||
```bash
|
||||
rabbitmqctl add_user admin StrongPassword123!
|
||||
rabbitmqctl set_user_tags admin administrator
|
||||
rabbitmqctl set_permissions -p / admin ".*" ".*" ".*"
|
||||
```
|
||||
|
||||
Then **disable the guest account**:
|
||||
|
||||
```bash
|
||||
rabbitmqctl delete_user guest
|
||||
```
|
||||
|
||||
### **2. Enable TLS (Optional)**
|
||||
|
||||
For security, configure TLS in `/etc/rabbitmq/rabbitmq.conf`. Refer to RabbitMQ’s [TLS guide](https://www.rabbitmq.com/ssl.html).
|
||||
|
||||
---
|
||||
|
||||
## **Step 10: Setup Monitoring (Optional)**
|
||||
|
||||
Install **Prometheus & Grafana** or use **RabbitMQ Prometheus plugin**:
|
||||
|
||||
```bash
|
||||
sudo rabbitmq-plugins enable rabbitmq_prometheus
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
Now your RabbitMQ cluster is fully set up with **replication and high availability**! 🚀
|
||||
|
||||
---
|
||||
|
||||
<br>
|
||||
<br>
|
||||
<br>
|
||||
<br>
|
||||
|
||||
# Instructions to fix **unexpected** configuration errors
|
||||
|
||||
If you can't find the RabbitMQ config file under `/etc/rabbitmq`, it may not exist by default. You need to **create it manually**. Here's how:
|
||||
|
||||
## **1️⃣ Create the Configuration File**
|
||||
|
||||
RabbitMQ uses either a **`.conf` file** (modern format) or a **`.config` file** (legacy format). The recommended format is `.conf`.
|
||||
|
||||
Create the file if it doesn't exist:
|
||||
|
||||
```bash
|
||||
sudo nano /etc/rabbitmq/rabbitmq.conf
|
||||
```
|
||||
|
||||
Then add the following cluster configuration:
|
||||
|
||||
```
|
||||
cluster_formation.peer_discovery_backend = classic_config
|
||||
cluster_formation.classic_config.nodes.1 = rabbit@rabbitmq-main
|
||||
cluster_formation.classic_config.nodes.2 = rabbit@rabbitmq-replica
|
||||
```
|
||||
|
||||
Save and exit (`CTRL+X`, then `Y`, then `Enter`).
|
||||
|
||||
---
|
||||
|
||||
## **2️⃣ Set Correct Permissions**
|
||||
|
||||
Ensure the RabbitMQ user can read it:
|
||||
|
||||
```bash
|
||||
sudo chown rabbitmq:rabbitmq /etc/rabbitmq/rabbitmq.conf
|
||||
sudo chmod 644 /etc/rabbitmq/rabbitmq.conf
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## **3️⃣ Restart RabbitMQ**
|
||||
|
||||
After modifying the configuration, restart the RabbitMQ service:
|
||||
|
||||
```bash
|
||||
sudo systemctl restart rabbitmq-server
|
||||
```
|
||||
|
||||
Check the status:
|
||||
|
||||
```bash
|
||||
sudo systemctl status rabbitmq-server
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## **4️⃣ Verify the Cluster Configuration**
|
||||
|
||||
After restarting, verify that clustering is working:
|
||||
|
||||
```bash
|
||||
rabbitmqctl cluster_status
|
||||
```
|
||||
|
||||
If the nodes are listed correctly, your setup is working.
|
||||
|
||||
---
|
||||
|
||||
## **5️⃣ If Using the Legacy `.config` Format**
|
||||
|
||||
Some older installations use an **Erlang-based configuration file** (`rabbitmq.config`). If you prefer that, create:
|
||||
|
||||
```bash
|
||||
sudo nano /etc/rabbitmq/rabbitmq.config
|
||||
```
|
||||
|
||||
Add this:
|
||||
|
||||
```erlang
|
||||
[
|
||||
{rabbit, [
|
||||
{cluster_formation, [
|
||||
{peer_discovery_backend, classic_config},
|
||||
{classic_config, [
|
||||
{nodes, ['rabbit@rabbitmq-main', 'rabbit@rabbitmq-replica']}
|
||||
]}
|
||||
]}
|
||||
]}
|
||||
].
|
||||
```
|
||||
|
||||
Then restart RabbitMQ:
|
||||
|
||||
```bash
|
||||
sudo systemctl restart rabbitmq-server
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### **🔍 Troubleshooting**
|
||||
|
||||
❌ **RabbitMQ doesn't restart?**
|
||||
Check logs for errors:
|
||||
|
||||
```bash
|
||||
sudo journalctl -u rabbitmq-server --no-pager | tail -50
|
||||
```
|
||||
|
||||
❌ **Cluster not forming?**
|
||||
Try forcing a node to join manually:
|
||||
|
||||
```bash
|
||||
rabbitmqctl stop_app
|
||||
rabbitmqctl join_cluster rabbit@rabbitmq-main
|
||||
rabbitmqctl start_app
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
<br>
|
||||
<br>
|
||||
<br>
|
||||
<br>
|
||||
|
||||
# Instructions to fix **unexpected** management UI authentication errors
|
||||
|
||||
stackoverflow answer [link](https://stackoverflow.com/a/40845332/27251837)
|
||||
|
||||
## Answer
|
||||
|
||||
### ❌ **Cannot login with guest/guest credentials**
|
||||
|
||||
I had the same Problem..
|
||||
|
||||
I installed RabbitMQ and Enabled Web Interface also but still couldn't sign in with any user i newly created, this is because you need to be administrator to access this.
|
||||
|
||||
Do not create any config file and mess with it..
|
||||
|
||||
This is what i did then,
|
||||
|
||||
1. Add a new/fresh user, say user test and password test:
|
||||
|
||||
```bash
|
||||
rabbitmqctl add_user test test
|
||||
```
|
||||
|
||||
2. Give administrative access to the new user:
|
||||
|
||||
```bash
|
||||
rabbitmqctl set_user_tags test administrator
|
||||
```
|
||||
|
||||
3. Set permission to newly created user:
|
||||
|
||||
```bash
|
||||
rabbitmqctl set_permissions -p / test ".*" ".*" ".*"
|
||||
```
|
||||
|
||||
That's it, enjoy :)
|
||||
Reference in New Issue
Block a user