MessageQueue

The Opsview message queue, based on Rabbit MQ and installed with the opsview-messagequeue package, use as a message queueing protocol to send and receive results and notifications between processes and different parts of Opsview.

Dependencies Copied

It uses our version of Erlang: opsview-erlang.

Installation Copied

Refer to Advanced Automated Installation.

Management Copied

Note

For hostname changes, RabbitMQ does not handle hostname changes well and reconfiguration or reinstallation of this component is required.

If you rename your messagequeue host after the opsview-messagequeue package has been installed and configured, the data store can no longer be found (as the data is stored in an area keys by the servers hostname) and the server process will not start correctly, effectively losing access to the data. The data store will need to be re-initialized by running the following commands:

# make sure messagequeue processes are not running.
# as root
/opt/opsview/watchdog/bin/opsview-monit stop opsview-messagequeue

# after waiting a moment, ensure the processes are stopped
pkill -f rabbitmq

# remove the old data store
rm -rf /opt/opsview/messagequeue/var/mnesia/*

# manually start the messagequeue process
sudo -iu opsview /opt/opsview/messagequeue/sbin/rabbitmq-server &

# create and configure the new data store
# obtain the opsview_messagequeue_password from the orchestrator by running
cat /opt/opsview/deploy/etc/user_secrets.yml | grep opsview_messagequeue_password | awk '{print $2}'

# Change line 11 in /opt/opsview/messagequeue/installer/setup-initial-users.sh to (substitute <opsview_messagequeue_password> )
/opt/opsview/messagequeue/sbin/rabbitmqctl add_user opsview <opsview_messagequeue_password> ||:
/opt/opsview/messagequeue/installer/setup-initial-users.sh

# revert the change back again
Change line 11 in /opt/opsview/messagequeue/installer/setup-initial-users.sh to the original:
  /opt/opsview/messagequeue/sbin/rabbitmqctl add_user opsview opsview ||:

# restart the messagequeue daemons
/opt/opsview/watchdog/bin/opsview-monit start opsview-messagequeue

If you need to rename the message server, please contact Support Portal for assistance.

Configuration Copied

DPKGs Copied

Watchdog service files are now managed by the package manager, removing the package would leave the watchdog service file behind with a .save extension. Purging the package will remove it. The package manager managed config file:

/opt/opsview/watchdog/etc/services/opsview-messagequeue.conf

RPMs Copied

Watchdog service files are now managed by the package manager. When removing the package, if the files listed have been modified by the user before; they will be left behind after rpmsave extension has been added to them. When upgrading the package, if the files listed have been modified by the user before; the modified file on the disk will remain and the new config file from the upgrading package will be written to the disk with the rpmnew extension. Users will have to:

1. Manually restore the file with the rpmsave extension.
2. Rename the rpmnew to the config file name if they want to use the new config for the package.

/opt/opsview/watchdog/etc/services/opsview-messagequeue.conf

There is one other config file which is managed by the package:

/opt/opsview/messagequeue/etc/rabbitmq.conf

Service Administration Copied

The default admin account created is opsview:opsview, which can also be used in the Management UI. To access Management UI enable remote access in Loadbalancer (usually /opt/opsview/loadbalancer/etc/messagequeue-management-balance.cfg) and point your browser to http://HOSTNAME:45672/.

Opsview message queue logs messages in the /opt/opsview/messagequeue/var/log/rabbit@node-name (normally the hostname of the device where the broker is running.)

As root, start, stop and restart the service using:

/opt/opsview/watchdog/bin/opsview-monit <start|stop|restart> opsview-messagequeue

Logging Copied

Opsview message queue logs messages in the /opt/opsview/messagequeue/var/log/rabbit@node-name (where node-name is the hostname of the device where the broker is running.)

Variables Copied

The related variable your Opsview Host may need is OPSVIEW_MESSAGEQUEUE_CREDENTIALS:

• Arguments one and four of this variable are populated by default, being “opsview” and 15672" = Arguments two and three may be obtained (if not already populated) by the below:

Argument 2: The messagequeue password Copied

grep opsview_messagequeue_password /opt/opsview/deploy/etc/user_secrets.yml

Argument 3: The NODENAME, which would be rabbit@hostname, where hostname is the full hostname, as from hostname -f. Copied

grep NODENAME /opt/opsview/messagequeue/etc/rabbitmq-env.conf

Troubleshooting Copied

If opsview-messagequeue fails to start on a host during an upgrade, this may be due to its internal data becoming corrupted through an improper shutdown. For example:

TASK [opsview_messagequeue : Wait for opsview-messagequeue application to start] ***
Thursday 25 March 2021  12:22:54 +0000 (0:00:07.474)       0:19:52.541 ******** 
ok: [infr-box2]
ok: [col1]
ok: [col2]
ok: [col3]
ok: [infr-box1]
fatal: [col4]: FAILED! => changed=false 
  elapsed: 300
  msg: Timeout when waiting for 127.0.0.1:5672

To resolve this you can perform the following steps on the affected host(s):

# 1. Uninstall component
# Debian/Ubuntu
apt purge opsview-messagequeue
# Centos/RHEL/OEL
yum remove opsview-messagequeue

# 2. Remove messagequeue directory
rm -rf /opt/opsview/messagequeue/

Then run deploy again.

cd /opt/opsview/deploy
bin/opsview-deploy lib/playbooks/setup-everything.yml