Compare commits

...

35 Commits

Author SHA1 Message Date
Brusk Awat
41074d617f
Merge pull request #15 from arikarim/multi-exchange-
fix: Enhance run_task method to handle existing RabbitMQ exchanges by…
2026-01-18 15:14:57 +03:00
Ari Karim
0bcaee2805
chore: Bump rabbit_carrots gem version to 1.1.1 in Gemfile.lock and version.rb. 2026-01-18 14:31:20 +03:00
Ari Karim
d553042d1c
fix: Enhance run_task method to handle existing RabbitMQ exchanges by attempting passive read before declaring new ones. 2026-01-18 14:30:07 +03:00
Brusk Awat
5d2e2df8d2
Merge pull request #14 from arikarim/multi-exchange-
feat: Add exchange_name parameter to run_task method and update Ruboc…
2026-01-18 12:04:10 +03:00
Ari Karim
8f8882c428
chore: Bump rabbit_carrots gem version to 1.1.0 in Gemfile.lock and version.rb. 2026-01-18 11:06:19 +03:00
Ari Karim
d7c7415438
feat: Add exchange_name parameter to run_task method and update Rubocop configuration for parameter lists. 2026-01-18 11:04:42 +03:00
Brusk Awat
4ac772d12c
Merge pull request #11 from muhammadnawzad/main
Update gem dependencies, refactor minor code patterns, and adjust Rubocop configuration.
2026-01-18 10:38:40 +03:00
Muhammad Nawzad
cdf3e42f35
chore: Update Ruby version to 4.0.1 in GitHub Actions workflows. 2026-01-18 09:52:07 +03:00
Muhammad Nawzad
c6e22d8cd1
Update gem dependencies, refactor minor code patterns, and adjust Rubocop configuration. 2026-01-18 09:50:02 +03:00
Brusk Awat
b526ef58ad
Merge pull request #10 from arikarim/orm-set
fix: Ensure message payloads are UTF-8 encoded before logging and upd…
2025-12-18 16:13:06 +03:00
Ari Karim
0a0ef429c0
fix: Ensure message payloads are UTF-8 encoded before logging and update gem version. 2025-12-18 15:47:52 +03:00
Brusk Awat
c5e10bd9e2
Merge pull request #9 from arikarim/orm-set
feat: Add setter for ORM configuration.
2025-12-18 14:10:06 +03:00
Ari Karim
c673d7e5a1
chore: Bump gem version to 1.0.5. 2025-12-18 14:08:31 +03:00
Ari Karim
7ec227174b
feat: Add setter for ORM configuration. 2025-12-18 14:08:01 +03:00
Brusk Awat
35586d81b4
Merge pull request #8 from arikarim/dev
Updates rabbit_carrots to version 1.0.4 and adds 'English' module req…
2025-05-25 15:15:59 +03:00
Ari Karim
ac06deb86a
Refactors process ID handling in Rabbit Carrots plugin to use $$ instead of $PROCESS_ID for improved compatibility 2025-05-25 15:14:30 +03:00
Ari Karim
cb41e257fc
Enhances logging functionality by adding additional log level methods (info, warn) to the adapter in Rabbit Carrots core module 2025-05-25 14:10:40 +03:00
Ari Karim
38ca7d3927
Updates rabbit_carrots to version 1.0.4 and adds 'English' module requirement in the Puma plugin 2025-05-25 14:08:25 +03:00
Brusk Awat
a0c5f80b58
Merge pull request #7 from arikarim/dev
Refines logging levels for shutdown, stop, and message handling in Rabbit Carrots service
2025-05-25 11:54:53 +03:00
Ari Karim
7cfe2a5a45
Updates dependencies: bumps rabbit_carrots to 1.0.3, amq-protocol to 2.3.4, bunny to 2.24.0, and set to 1.1.2 2025-05-25 11:44:21 +03:00
Ari Karim
076e1f52ce
Enhances logger initialization by introducing a logger adapter for improved compatibility and flexibility 2025-05-25 11:43:41 +03:00
Ari Karim
aee5a1e3cb
Refactors process ID handling in Rabbit Carrots plugin and enhances logging for shutdown scenarios 2025-05-25 10:56:35 +03:00
Ari Karim
f68ab83afc
Refines logging levels for shutdown, stop, and message handling in Rabbit Carrots service 2025-05-25 10:37:25 +03:00
59ad012e04
Updates README.md 2024-08-05 17:42:22 +03:00
3950ff5047
Improves error handling 2024-08-05 15:28:23 +03:00
Brusk Awat
edb1fffad1
Merge pull request #6 from ditkrg/rewrite
Rewrite
2024-08-05 01:07:48 +03:00
ef60e8342d
Updates README 2024-08-05 01:07:15 +03:00
a51eed8101
Adds support for puma 2024-08-05 01:03:10 +03:00
28b620b7b4
Adds automatic recovery and exit strategy 2024-08-05 00:54:19 +03:00
876a48a1ab
Improves error handling and shut down. Moves code for reusability 2024-08-05 00:54:02 +03:00
ff838c50ad
Merge pull request #4 from muhammadnawzad/main
Updates dependencies' versions
2023-12-14 11:00:44 +03:00
Muhammad Nawzad
7c4c7ea08f
Renames event_bus_exchange_name to rabbitmq_exchange_name for consistency 2023-12-06 14:26:16 +03:00
Muhammad Nawzad
6b6da5e5cb
Bumps gem version 2023-12-06 14:11:50 +03:00
Muhammad Nawzad
54025a5aa4
Fixes rubocop offenses 2023-12-06 14:11:35 +03:00
Muhammad Nawzad
8a95f69556
Updates dependencies versions 2023-12-06 14:11:14 +03:00
15 changed files with 403 additions and 154 deletions

View File

@ -14,7 +14,7 @@ jobs:
strategy: strategy:
matrix: matrix:
ruby: ruby:
- '3.1.2' - '4.0.1'
steps: steps:
- uses: actions/checkout@v3 - uses: actions/checkout@v3

View File

@ -14,7 +14,7 @@ jobs:
- uses: actions/checkout@v2 - uses: actions/checkout@v2
- uses: ruby/setup-ruby@v1 - uses: ruby/setup-ruby@v1
with: with:
ruby-version: 3.1.2 ruby-version: 4.0.1
- name: publish gem - name: publish gem
run: | run: |

View File

@ -1,4 +1,5 @@
require: rubocop-rails plugins:
- rubocop-rails
AllCops: AllCops:
NewCops: enable NewCops: enable
@ -111,3 +112,5 @@ Metrics/CyclomaticComplexity:
Max: 15 Max: 15
Metrics/PerceivedComplexity: Metrics/PerceivedComplexity:
Max: 15 Max: 15
Metrics/ParameterLists:
Max: 6

View File

@ -5,10 +5,10 @@ source 'https://rubygems.org'
# Specify your gem's dependencies in rabbit_carrots.gemspec # Specify your gem's dependencies in rabbit_carrots.gemspec
gemspec gemspec
gem 'rake', '~> 13.0' gem 'rake', '~> 13.3.1'
gem 'rspec', '~> 3.0' gem 'rspec', '~> 3.13.2'
gem 'rubocop', '~> 1.21' gem 'rubocop', '~> 1.82.1'
gem 'rubocop-rails', '~> 2.17' gem 'rubocop-rails', '~> 2.34.3'

View File

@ -1,86 +1,112 @@
PATH PATH
remote: . remote: .
specs: specs:
rabbit_carrots (0.1.19) rabbit_carrots (1.1.1)
bunny (>= 2.19.0) bunny (>= 2.22)
connection_pool (~> 2.3.0) connection_pool (>= 2.4)
GEM GEM
remote: https://rubygems.org/ remote: https://rubygems.org/
specs: specs:
activesupport (7.0.4) activesupport (8.1.2)
concurrent-ruby (~> 1.0, >= 1.0.2) base64
bigdecimal
concurrent-ruby (~> 1.0, >= 1.3.1)
connection_pool (>= 2.2.5)
drb
i18n (>= 1.6, < 2) i18n (>= 1.6, < 2)
json
logger (>= 1.4.2)
minitest (>= 5.1) minitest (>= 5.1)
tzinfo (~> 2.0) securerandom (>= 0.3)
amq-protocol (2.3.2) tzinfo (~> 2.0, >= 2.0.5)
ast (2.4.2) uri (>= 0.13.1)
bunny (2.22.0) amq-protocol (2.5.0)
amq-protocol (~> 2.3, >= 2.3.1) ast (2.4.3)
base64 (0.3.0)
bigdecimal (4.0.1)
bunny (2.24.0)
amq-protocol (~> 2.3)
sorted_set (~> 1, >= 1.0.2) sorted_set (~> 1, >= 1.0.2)
concurrent-ruby (1.1.10) concurrent-ruby (1.3.6)
connection_pool (2.3.0) connection_pool (3.0.2)
diff-lcs (1.5.0) diff-lcs (1.6.2)
i18n (1.12.0) drb (2.2.3)
i18n (1.14.8)
concurrent-ruby (~> 1.0) concurrent-ruby (~> 1.0)
json (2.6.2) json (2.18.0)
minitest (5.16.3) language_server-protocol (3.17.0.5)
parallel (1.22.1) lint_roller (1.1.0)
parser (3.1.2.1) logger (1.7.0)
minitest (6.0.1)
prism (~> 1.5)
parallel (1.27.0)
parser (3.3.10.1)
ast (~> 2.4.1) ast (~> 2.4.1)
rack (3.0.4.2) racc
prism (1.8.0)
racc (1.8.1)
rack (3.2.4)
rainbow (3.1.1) rainbow (3.1.1)
rake (13.0.6) rake (13.3.1)
rbtree (0.4.6) rbtree (0.4.6)
regexp_parser (2.6.1) regexp_parser (2.11.3)
rexml (3.2.5) rspec (3.13.2)
rspec (3.12.0) rspec-core (~> 3.13.0)
rspec-core (~> 3.12.0) rspec-expectations (~> 3.13.0)
rspec-expectations (~> 3.12.0) rspec-mocks (~> 3.13.0)
rspec-mocks (~> 3.12.0) rspec-core (3.13.6)
rspec-core (3.12.0) rspec-support (~> 3.13.0)
rspec-support (~> 3.12.0) rspec-expectations (3.13.5)
rspec-expectations (3.12.0)
diff-lcs (>= 1.2.0, < 2.0) diff-lcs (>= 1.2.0, < 2.0)
rspec-support (~> 3.12.0) rspec-support (~> 3.13.0)
rspec-mocks (3.12.0) rspec-mocks (3.13.7)
diff-lcs (>= 1.2.0, < 2.0) diff-lcs (>= 1.2.0, < 2.0)
rspec-support (~> 3.12.0) rspec-support (~> 3.13.0)
rspec-support (3.12.0) rspec-support (3.13.6)
rubocop (1.39.0) rubocop (1.82.1)
json (~> 2.3) json (~> 2.3)
language_server-protocol (~> 3.17.0.2)
lint_roller (~> 1.1.0)
parallel (~> 1.10) parallel (~> 1.10)
parser (>= 3.1.2.1) parser (>= 3.3.0.2)
rainbow (>= 2.2.2, < 4.0) rainbow (>= 2.2.2, < 4.0)
regexp_parser (>= 1.8, < 3.0) regexp_parser (>= 2.9.3, < 3.0)
rexml (>= 3.2.5, < 4.0) rubocop-ast (>= 1.48.0, < 2.0)
rubocop-ast (>= 1.23.0, < 2.0)
ruby-progressbar (~> 1.7) ruby-progressbar (~> 1.7)
unicode-display_width (>= 1.4.0, < 3.0) unicode-display_width (>= 2.4.0, < 4.0)
rubocop-ast (1.23.0) rubocop-ast (1.49.0)
parser (>= 3.1.1.0) parser (>= 3.3.7.2)
rubocop-rails (2.17.3) prism (~> 1.7)
rubocop-rails (2.34.3)
activesupport (>= 4.2.0) activesupport (>= 4.2.0)
lint_roller (~> 1.1)
rack (>= 1.1) rack (>= 1.1)
rubocop (>= 1.33.0, < 2.0) rubocop (>= 1.75.0, < 2.0)
ruby-progressbar (1.11.0) rubocop-ast (>= 1.44.0, < 2.0)
set (1.0.3) ruby-progressbar (1.13.0)
securerandom (0.4.1)
set (1.1.2)
sorted_set (1.0.3) sorted_set (1.0.3)
rbtree rbtree
set (~> 1.0) set (~> 1.0)
tzinfo (2.0.5) tzinfo (2.0.6)
concurrent-ruby (~> 1.0) concurrent-ruby (~> 1.0)
unicode-display_width (2.3.0) unicode-display_width (3.2.0)
unicode-emoji (~> 4.1)
unicode-emoji (4.2.0)
uri (1.1.1)
PLATFORMS PLATFORMS
arm64-darwin-25
x86_64-linux x86_64-linux
DEPENDENCIES DEPENDENCIES
rabbit_carrots! rabbit_carrots!
rake (~> 13.0) rake (~> 13.3.1)
rspec (~> 3.0) rspec (~> 3.13.2)
rubocop (~> 1.21) rubocop (~> 1.82.1)
rubocop-rails (~> 2.17) rubocop-rails (~> 2.34.3)
BUNDLED WITH BUNDLED WITH
2.3.26 2.3.26

View File

@ -32,7 +32,11 @@ RabbitCarrots.configure do |c|
c.rabbitmq_user = ENV.fetch('RABBITMQ__USER', nil) c.rabbitmq_user = ENV.fetch('RABBITMQ__USER', nil)
c.rabbitmq_password = ENV.fetch('RABBITMQ__PASSWORD', nil) c.rabbitmq_password = ENV.fetch('RABBITMQ__PASSWORD', nil)
c.rabbitmq_vhost = ENV.fetch('RABBITMQ__VHOST', nil) c.rabbitmq_vhost = ENV.fetch('RABBITMQ__VHOST', nil)
c.event_bus_exchange_name = ENV.fetch('EVENTBUS__EXCHANGE_NAME', nil) c.rabbitmq_exchange_name = ENV.fetch('RABBITMQ__EXCHANGE_NAME', nil)
c.automatically_recover = true
c.network_recovery_interval = 5
c.recovery_attempts = 5
c.orm = :activerecord || :mongoid
c.routing_key_mappings = [ c.routing_key_mappings = [
{ routing_keys: ['RK1', 'RK2'], queue: 'QUEUE_NAME', handler: 'CLASS HANDLER IN STRING' }, { routing_keys: ['RK1', 'RK2'], queue: 'QUEUE_NAME', handler: 'CLASS HANDLER IN STRING' },
{ routing_keys: ['RK1', 'RK2'], queue: 'QUEUE_NAME', handler: 'CLASS HANDLER IN STRING' } { routing_keys: ['RK1', 'RK2'], queue: 'QUEUE_NAME', handler: 'CLASS HANDLER IN STRING' }
@ -41,8 +45,6 @@ end
``` ```
Note that handler is a class that must implement a method named ```handle!``` that takes 4 parameters as follow: Note that handler is a class that must implement a method named ```handle!``` that takes 4 parameters as follow:
```ruby ```ruby
@ -53,8 +55,6 @@ class DummyEventHandler
end end
``` ```
Inside the handle message, you can NACK the message without re-queuing by raising ```RabbitCarrots::EventHandlers::Errors::NackMessage``` exception. Inside the handle message, you can NACK the message without re-queuing by raising ```RabbitCarrots::EventHandlers::Errors::NackMessage``` exception.
To NACK and re-queue, raise ```RabbitCarrots::EventHandlers::Errors::NackAndRequeueMessage``` exception. To NACK and re-queue, raise ```RabbitCarrots::EventHandlers::Errors::NackAndRequeueMessage``` exception.
@ -65,8 +65,16 @@ Note: Any other unrescued exception raised inside ```handle!``` the that is a su
### Running ### Running
Then run ```bundle exec rake rabbit_carrots:eat```. For better scalability and improved performance, you can run rabbit_carrots in standalone mode by invoking the following command:
```bundle exec rake rabbit_carrots:eat```.
#### Puma
For small and medium sized projects, you can delegate the management of the rabbit_carrots to the Puma web server. To achieve that, add the following line to your puma.rb
```plugin :rabbit_carrots```
This will make sure that Puma will manage rabbit carrots as a background service and will gracefully terminate if rabbit_carrots eventually loses connection after multiple automatic recovery.
## Development ## Development
After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment. After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.

View File

@ -0,0 +1,86 @@
# rabbit_carrots.rb
require 'English'
require 'puma/plugin'
require 'rabbit_carrots'
Puma::Plugin.create do
attr_reader :puma_pid, :rabbit_carrots_pid, :log_writer, :core_service
def start(launcher)
@log_writer = launcher.log_writer
@puma_pid = $PROCESS_ID
@core_service = RabbitCarrots::Core.new(logger: log_writer)
in_background do
monitor_rabbit_carrots
end
launcher.events.on_booted do
@rabbit_carrots_pid = fork do
Thread.new { monitor_puma }
start_rabbit_carrots_consumer
end
end
launcher.events.on_stopped { stop_rabbit_carrots }
launcher.events.on_restart { stop_rabbit_carrots }
end
private
def start_rabbit_carrots_consumer
core_service.start(kill_to_restart_on_standard_error: true)
rescue StandardError => e
Rails.logger.error "Error starting Rabbit Carrots: #{e.message}"
end
def stop_rabbit_carrots
return unless rabbit_carrots_pid
log 'Stopping Rabbit Carrots...'
core_service.request_shutdown
Process.kill('TERM', rabbit_carrots_pid)
Process.wait(rabbit_carrots_pid)
rescue Errno::ECHILD, Errno::ESRCH
log 'Rabbit Carrots already stopped'
end
def monitor_puma
monitor(:puma_dead?, 'Detected Puma has gone away, stopping Rabbit Carrots...')
end
def monitor_rabbit_carrots
monitor(:rabbit_carrots_dead?, 'Rabbits Carrot is dead, stopping Puma...')
end
def monitor(process_dead, message)
loop do
if send(process_dead)
log message
Process.kill('TERM', $PROCESS_ID)
break
end
sleep 2
end
end
def rabbit_carrots_dead?
Process.waitpid(rabbit_carrots_pid, Process::WNOHANG) if rabbit_carrots_started?
false
rescue Errno::ECHILD, Errno::ESRCH
true
end
def rabbit_carrots_started?
rabbit_carrots_pid.present?
end
def puma_dead?
Process.ppid != puma_pid
end
def log(...)
log_writer.log(...)
end
end

View File

@ -1,25 +1,11 @@
# frozen_string_literal: true # frozen_string_literal: true
require_relative 'rabbit_carrots/version' require_relative 'rabbit_carrots/version'
require 'rabbit_carrots/errors'
require 'rabbit_carrots/connection' require 'rabbit_carrots/connection'
require 'rabbit_carrots/configuration' require 'rabbit_carrots/configuration'
require 'rabbit_carrots/railtie' if defined?(Rails) require 'rabbit_carrots/railtie' if defined?(Rails)
require 'rabbit_carrots/core'
module RabbitCarrots module RabbitCarrots
class Error < StandardError; end
module EventHandlers
module Errors
class IrrelevantMessage < StandardError
end
class NackMessage < StandardError
end
class NackAndRequeueMessage < StandardError
end
class PlaceholderError < Error; end
end
end
end end

View File

@ -9,6 +9,21 @@ module RabbitCarrots
end end
class Configuration class Configuration
attr_accessor :rabbitmq_host, :rabbitmq_port, :rabbitmq_user, :rabbitmq_password, :rabbitmq_vhost, :routing_key_mappings, :event_bus_exchange_name attr_accessor :rabbitmq_host,
:rabbitmq_port,
:rabbitmq_user,
:rabbitmq_password,
:rabbitmq_vhost,
:routing_key_mappings,
:rabbitmq_exchange_name,
:automatically_recover,
:network_recovery_interval,
:recovery_attempts
def orm
@orm ||= :activerecord
end
attr_writer :orm
end end
end end

View File

@ -3,6 +3,7 @@ require 'singleton'
module RabbitCarrots module RabbitCarrots
class Connection class Connection
include ::Singleton include ::Singleton
attr_reader :connection attr_reader :connection
def initialize def initialize
@ -11,7 +12,11 @@ module RabbitCarrots
port: RabbitCarrots.configuration.rabbitmq_port, port: RabbitCarrots.configuration.rabbitmq_port,
user: RabbitCarrots.configuration.rabbitmq_user, user: RabbitCarrots.configuration.rabbitmq_user,
password: RabbitCarrots.configuration.rabbitmq_password, password: RabbitCarrots.configuration.rabbitmq_password,
vhost: RabbitCarrots.configuration.rabbitmq_vhost vhost: RabbitCarrots.configuration.rabbitmq_vhost,
automatically_recover: RabbitCarrots.configuration.automatically_recover || true,
network_recovery_interval: RabbitCarrots.configuration.network_recovery_interval || 5,
recovery_attempts: RabbitCarrots.configuration.recovery_attempts || 5,
recovery_attempts_exhausted: -> { Process.kill('TERM', Process.pid) }
) )
@connection.start @connection.start

164
lib/rabbit_carrots/core.rb Normal file
View File

@ -0,0 +1,164 @@
module RabbitCarrots
class Core # rubocop:disable Metrics/ClassLength
attr_reader :logger
@database_agnostic_not_null_violation = nil
@database_agnostic_connection_not_established = nil
@database_agnostic_record_invalid = nil
class << self
attr_accessor :database_agnostic_not_null_violation, :database_agnostic_connection_not_established, :database_agnostic_record_invalid
end
def initialize(logger: nil)
@logger = create_logger_adapter(logger || Logger.new(Rails.env.production? ? '/proc/self/fd/1' : $stdout))
@threads = []
@running = true
@shutdown_requested = false
end
def start(kill_to_restart_on_standard_error: false)
self.class.database_agnostic_not_null_violation = RabbitCarrots.configuration.orm == :activerecord ? ActiveRecord::NotNullViolation : RabbitCarrots::EventHandlers::Errors::PlaceholderError
self.class.database_agnostic_connection_not_established = RabbitCarrots.configuration.orm == :activerecord ? ActiveRecord::ConnectionNotEstablished : ::Mongo::Error::SocketError
self.class.database_agnostic_record_invalid = RabbitCarrots.configuration.orm == :activerecord ? ActiveRecord::RecordInvalid : ::Mongoid::Errors::Validations
channels = RabbitCarrots.configuration.routing_key_mappings.map do |mapping|
{ **mapping, handler: mapping[:handler].constantize }
end
channels.each do |channel|
handler_class = channel[:handler]
raise "#{handler_class.name} must respond to `handle!`" unless handler_class.respond_to?(:handle!)
@threads << Thread.new do
run_task(
queue_name: channel[:queue],
handler_class:,
routing_keys: channel[:routing_keys],
queue_arguments: channel[:arguments],
exchange_name: channel[:exchange_name],
kill_to_restart_on_standard_error:
)
end
end
Signal.trap('INT') { request_shutdown }
Signal.trap('TERM') { request_shutdown }
while @running
if @shutdown_requested
request_shutdown
sleep 1
break
end
sleep 1
end
@threads.each(&:join)
rescue StandardError => e
logger.error "Error starting Rabbit Carrots: #{e.message}"
end
def request_shutdown
# Workaround to a known issue with Signal Traps and logs
Thread.start do
logger.error 'Shutting down Rabbit Carrots service...'
end
@shutdown_requested = true
@threads.each(&:kill)
stop
end
def stop
# Workaround to a known issue with Signal Traps and logs
Thread.start do
logger.error 'Stoppig the Rabbit Carrots service...'
end
@running = false
end
def run_task(queue_name:, handler_class:, routing_keys:, queue_arguments: {}, exchange_name: nil, kill_to_restart_on_standard_error: false)
RabbitCarrots::Connection.instance.channel.with do |channel|
exchange_name ||= RabbitCarrots.configuration.rabbitmq_exchange_name
begin
# Try to passively read an existing exchange without declaring it
exchange = channel.topic(exchange_name, passive: true)
rescue Bunny::NotFound
# If the exchange does not exist, declare it
exchange = channel.topic(exchange_name, durable: true)
end
logger.info "Listening on QUEUE: #{queue_name} for ROUTING KEYS: #{routing_keys}"
queue = channel.queue(queue_name, durable: true, arguments: queue_arguments)
routing_keys.map(&:strip).each { |k| queue.bind(exchange, routing_key: k) }
queue.subscribe(block: false, manual_ack: true, prefetch: 10) do |delivery_info, properties, payload|
break if @shutdown_requested
logger.info "Received from queue: #{queue_name}, Routing Keys: #{routing_keys}"
handler_class.handle!(channel, delivery_info, properties, payload)
channel.ack(delivery_info.delivery_tag, false)
rescue RabbitCarrots::EventHandlers::Errors::NackMessage, JSON::ParserError => _e
payload = encode_payload(payload)
logger.warn "Nacked message: #{payload}"
channel.nack(delivery_info.delivery_tag, false, false)
rescue RabbitCarrots::EventHandlers::Errors::NackAndRequeueMessage => _e
payload = encode_payload(payload)
logger.warn "Nacked and Requeued message: #{payload}"
channel.nack(delivery_info.delivery_tag, false, true)
rescue self.class.database_agnostic_not_null_violation, self.class.database_agnostic_record_invalid => e
payload = encode_payload(payload)
logger.warn "Null constraint or Invalid violation: #{payload}. Error: #{e.message}"
channel.ack(delivery_info.delivery_tag, false)
rescue self.class.database_agnostic_connection_not_established => e
payload = encode_payload(payload)
logger.warn "Error connection not established to the database: #{payload}. Error: #{e.message}"
sleep 3
channel.nack(delivery_info.delivery_tag, false, true)
rescue StandardError => e
payload = encode_payload(payload)
logger.error "Error handling message: #{payload}. Error: #{e.message}"
sleep 3
channel.nack(delivery_info.delivery_tag, false, true)
Process.kill('SIGTERM', Process.pid) if kill_to_restart_on_standard_error
end
end
rescue StandardError => e
logger.error "Bunny session error: #{e.message}"
request_shutdown
end
private
def create_logger_adapter(logger)
return logger if logger.respond_to?(:info) && logger.respond_to?(:error) && logger.respond_to?(:warn)
adapter = Object.new
def adapter.info(msg)
@logger.write("[INFO] #{msg}\n")
end
def adapter.error(msg)
@logger.write("[ERROR] #{msg}\n")
end
def adapter.warn(msg)
@logger.write("[WARN] #{msg}\n")
end
adapter.instance_variable_set(:@logger, logger)
adapter
end
def encode_payload(payload)
payload.encode(
'UTF-8',
invalid: :replace,
undef: :replace,
replace: ''
)
end
end
end

View File

@ -0,0 +1,18 @@
module RabbitCarrots
class Error < StandardError; end
module EventHandlers
module Errors
class IrrelevantMessage < StandardError
end
class NackMessage < StandardError
end
class NackAndRequeueMessage < StandardError
end
class PlaceholderError < Error; end
end
end
end

View File

@ -1,75 +1,13 @@
require 'bunny'
namespace :rabbit_carrots do namespace :rabbit_carrots do
desc 'Listener for Queue' desc 'Rake task for standalone RabbitCarrots mode'
task eat: :environment do task eat: :environment do
Rails.application.eager_load! Rails.application.eager_load!
DatabaseAgonsticNotNullViolation = defined?(ActiveRecord) ? ActiveRecord::NotNullViolation : RabbitCarrots::EventHandlers::Errors::PlaceholderError logger = Logger.new(Rails.env.production? ? '/proc/self/fd/1' : $stdout)
DatabaseAgonsticConnectionNotEstablished = defined?(ActiveRecord) ? ActiveRecord::ConnectionNotEstablished : Mongo::Error::SocketError logger.level = Logger::INFO
DatabaseAgnosticRecordInvalid = defined?(ActiveRecord) ? ActiveRecord::RecordInvalid : Mongoid::Errors::Validations
channels = RabbitCarrots.configuration.routing_key_mappings.map do |mapping| core_service = RabbitCarrots::Core.new(logger:)
# This will be supplied in initializer. At that time, the Handler will not be available to be loaded and will throw Uninitialized Constant
{ **mapping, handler: mapping[:handler].constantize }
end
Rails.logger = Logger.new(Rails.env.production? ? '/proc/self/fd/1' : $stdout) core_service.start(kill_to_restart_on_standard_error: true)
# Run RMQ Subscriber for each channel
channels.each do |channel|
handler_class = channel[:handler]
raise "#{handler_class.name} must respond to `handle!`" unless handler_class.respond_to?(:handle!)
run_task(queue_name: channel[:queue], handler_class:, routing_keys: channel[:routing_keys], queue_arguments: channel[:arguments])
end
# Infinite loop to keep the process running
loop do
sleep 1
end
end
end
def run_task(queue_name:, queue_arguments: {}, handler_class:, routing_keys:)
RabbitCarrots::Connection.instance.channel.with do |channel|
exchange = channel.topic(RabbitCarrots.configuration.event_bus_exchange_name, durable: true)
Rails.logger.info "Listening on QUEUE: #{queue_name} for ROUTING KEYS: #{routing_keys}"
queue = channel.queue(queue_name, durable: true, arguments: queue_arguments)
routing_keys.map(&:strip).each { |k| queue.bind(exchange, routing_key: k) }
queue.subscribe(block: false, manual_ack: true, prefetch: 10) do |delivery_info, properties, payload|
Rails.logger.info "Received from queue: #{queue_name}, Routing Keys: #{routing_keys}"
handler_class.handle!(channel, delivery_info, properties, payload)
channel.ack(delivery_info.delivery_tag, false)
rescue RabbitCarrots::EventHandlers::Errors::NackMessage, JSON::ParserError => _e
Rails.logger.info "Nacked message: #{payload}"
channel.nack(delivery_info.delivery_tag, false, false)
rescue RabbitCarrots::EventHandlers::Errors::NackAndRequeueMessage => _e
Rails.logger.info "Nacked and Requeued message: #{payload}"
channel.nack(delivery_info.delivery_tag, false, true)
rescue DatabaseAgonsticNotNullViolation, DatabaseAgnosticRecordInvalid => e
# on null constraint violation, we want to ack the message
Rails.logger.error "Null constraint or Invalid violation: #{payload}. Error: #{e.message}"
channel.ack(delivery_info.delivery_tag, false)
rescue DatabaseAgonsticConnectionNotEstablished => e
# on connection not established, we want to requeue the message and sleep for 3 seconds
Rails.logger.error "Error connection not established to the database: #{payload}. Error: #{e.message}"
# delay for 3 seconds before requeuing
sleep 3
channel.nack(delivery_info.delivery_tag, false, true)
rescue StandardError => e
Rails.logger.error "Error handling message: #{payload}. Error: #{e.message}"
# requeue the message then kill the container
sleep 3
channel.nack(delivery_info.delivery_tag, false, true)
# kill the container with sigterm
Process.kill('SIGTERM', Process.pid)
end
Rails.logger.info 'RUN TASK ENDED'
end end
end end

View File

@ -1,5 +1,5 @@
# frozen_string_literal: true # frozen_string_literal: true
module RabbitCarrots module RabbitCarrots
VERSION = '0.1.19' VERSION = '1.1.1'
end end

View File

@ -32,8 +32,8 @@ Gem::Specification.new do |spec|
spec.require_paths = ['lib'] spec.require_paths = ['lib']
# Uncomment to register a new dependency of your gem # Uncomment to register a new dependency of your gem
spec.add_dependency 'bunny', '>= 2.19.0' spec.add_dependency 'bunny', '>= 2.22'
spec.add_dependency 'connection_pool', '~> 2.3.0' spec.add_dependency 'connection_pool', '>= 2.4'
# For more information and examples about making a new gem, check out our # For more information and examples about making a new gem, check out our
# guide at: https://bundler.io/guides/creating_gem.html # guide at: https://bundler.io/guides/creating_gem.html