Web App
Opsview Web App provides the perl Catalyst web application server process, which includes the REST API functionality. It includes NGINX configuration to proxy HTTP requests to the application server.
This has a dependency on the opsview-web-app-ui package (front end application code), which in turn has a dependency on the opsview-web-app-ui-external package (third party front end libraries).
Altogether, this provides the Opsview web application.
This must be run on the Opsview Orchestrator server.
Dependencies Copied
Requires opsview-core-utils.
Installation Copied
This component is deployed on the Master server when installing Opsview Monitor and must not be moved to another server.
Configuration Copied
Opsview Web uses the file /opt/opsview/webapp/opsview_web_local.yml
as its main configuration file, with defaults provided in opsview_web.yml
(do not change this defaults file - it will be reset on every upgrade). The file opsview_web_local.yml
is managed by Opsview Deploy - if there are any configuration settings you want to persist, you must add to Deploy’s user_vars.yml
.
Note
The format of the
opsview_web_local.yml
file is very significant — you must use spaces and not tab characters for indentation. Incorrect changes in this file can leave your system broken.Changes to these files require a restart of Opsview Web.
Web authentication Copied
Opsview uses the Opsview Session Manager to authorize the use of the web application.
On an install, a randomly generated secret will be used and will be encrypted.
On an upgrade, if you have got the old default shared secret (shared-secret-please-change), a new secret will be generated and encrypted. Otherwise, no changes will occur.
Account lockout Copied
You can configure the throttling parameters for the lockout policy. Amend the opsview_core_web_local_overrides
parameter in /opt/opsview/deploy/etc/user_vars.yml
:
opsview_core_web_local_overrides:
Controller::Root:
login_throttling:
ip_limit: 50
ip_window: 3600
user_limit: 10
user_window: 3600
uid_only: 1
The parameters are:
ip_limit
— number of failed login attempts based on IP before blocking occurs, default: 50.ip_window
— amount of seconds backwards in time to check for failed IP attempts, default: 1 hour.user_limit
— number of failed login attempts for a user before blocking occurs, default: 10.user_window
— amount of seconds backwards in time to check for failed user attempts, default: 1 hour.uid_only
— if 1, will use only the userid to identify the user, else will use a userid-IP combination, default: 1 — this is the most secure option. Utilises user_limit and user_window parameters.
Preventing denial of service and preventing brute force attack Copied
You may decide to change uid_only
to 0 if you want to ensure that the legitimate users never get locked out of the system during a brute-force attack on their account, while still achieving some level of protection from primitive attacks. Since the legitimate user is connecting from a different IP to the attacker they can still login; this is less secure because a brute-force attack can still happen if the attacker has enough unique IP addresses attempting to authenticate. Regardless of this parameter, the blocking after 50 failed attempts from a single IP still happens according to the ip_limit and ip_window.
- errors received may be “IP address temporarily blocked. Try again later or contact your administrator”
Recovery from brute-force attack or mass-lockout Copied
If you need to reset the lockout controls in an emergency, you can clear all the stored attempted lockouts by running the following command within the opsview database:
delete from opsview.login_failures;
This will clear all the current login failures.
Starman Server Processes Copied
Opsview Web uses Starman via Catalyst to server dynamic pages. It is possible to increase the number of Starman server processes to improve web responses, at the cost of using more memory.
You can estimate the maximum amount of memory used by Starman with the following calculation: max_servers x 280MB
The default value is 10 for max_servers.
To increase the value, add to the opsview_web_local.yml
file:
ProductionEngine:
max_servers: 20
The smallest setting you can use is:
ProductionEngine:
max_servers: 3
min_servers: 2
max_spare_servers: 1
min_spare_servers: 1
Login timeout Copied
You can modify the following to change the time that the interface automatically logs you out:
session:
expires: 86400
Default thresholds for host interfaces Copied
You can add the following lines to change the default host interface thresholds:
host_interfaces:
default_throughput_critical: 55%
default_throughput_warning: ""
default_errors_critical: 20
default_discards_critical: 5
SNMPWalk timeouts Copied
Configure the number of seconds before SNMPWalk’s time out. The default is 1800 seconds (or 30 minutes).
snmpwalk_timeout: 1800
License Messages Copied
If your license for Opsview Monitor is coming up for renewal, there will be a banner that is displayed in the top of all Opsview screens.
You can change the access required to see this message by changing the configuration file to:
Controller::Root:
show_license_messages: ADMINACCESS
By default, show_license_messages
is set to 1 which means all users will see it. If set to 0, this disables all license messages. Otherwise, it expects a string to mean which particular access level is required to display the banner — this can be a comma-separated list and thus any one of those accesses will have the banner displayed.
Note
If you disable banner messages, ensure the Service Check Opsview License Checks is running and that notifications work, so that you still get warnings about an upcoming expiry.
Collector Host templates Copied
When a collector is registered, an Opsview host is created with a list of host templates. This list is defined in the yml file as:
Controller::REST::Config::Collector:
collector_registration_host_templates:
- Opsview - Component - Agent
- Opsview - Component - DataStore
- Opsview - Component - Executor
- Opsview - Component - Load Balancer
- Opsview - Component - Machine Stats
- Opsview - Component - MessageQueue
- Opsview - Component - Results Forwarder
- Opsview - Component - Results Sender
- Opsview - Component - Scheduler
- Opsview - Component - SNMP Traps
- Opsview - Component - SNMP Traps Collector
- Opsview - Component - Watchdog
- Network - Base
If you are changing this list, ensure the names are correct, otherwise an error will be raised when the host template is not found.
Minimum password strength Copied
The minimum password strength is configured in the yml file using:
# Valid values from 1 to 5
Controller::Settings:
min_password_strength: 3
# List of words not allowed in passwords
disallowed_passwords:
- opsview
# Number of historical passwords for validation
max_password_history: 4
The max_password_history
values are:
Setting | Meaning |
---|---|
0 | Turn off all historical password checking |
1 | Check the current password is not the same as the new password |
2 | Check the current password and one previous password are not the same as the new password |
3 | Check the current password and the two previous passwords are not the same as the new password |
4 | … etc … |
The min_password_strength
values are:
Level | Password Strength |
---|---|
1 | Very Weak |
2 | Weak |
3 | Medium |
4 | Strong |
5 | Very Strong |
If the new password does not match or improve upon this level then it cannot be saved.
Thesse settings are maintained by Opsview Deploy for future upgrades.
SSL certificate Copied
The SSL server certificate and key can be configured for the webserver. If they do not exist then a self-signed certificate will be automatically created.
By default the certificate and key are located at:
/opt/opsview/webapp/etc/ssl/server.crt
/opt/opsview/webapp/etc/ssl/server.key
The paths to the certificate and key can be overridden as the ‘root’ user in the /opt/opsview/deploy/etc/user_vars.yml
file:
opsview_webserver_ssl_cert: /opt/opsview/webapp/etc/ssl/server.crt
opsview_webserver_ssl_key: /opt/opsview/webapp/etc/ssl/server.key
After changing these variables you must run the following command:
/opt/opsview/deploy/bin/opsview-deploy /opt/opsview/deploy/lib/playbooks/orchestrator-install.yml
This will make the necessary adjustments to opsview-webserver automatically. If the certificate does not exist then a self-signed certificate will be created.
SSL Certificate (.pem) Copied
If your webserver certificate and key are actually combined into a single .pem
file instead of separate .crt
and .key
files, then you must point the webserver to it’s location by using the variables in the /opt/opsview/deploy/etc/user_vars.yml
file:
opsview_webserver_ssl_cert: /opt/opsview/webapp/etc/ssl/server.pem
opsview_webserver_ssl_key: /opt/opsview/webapp/etc/ssl/server.pem
After changing these variables you must run the following command:
/opt/opsview/deploy/bin/opsview-deploy /opt/opsview/deploy/lib/playbooks/orchestrator-install.yml
Note: Your certificate provider may provide the certificate in .pem
format, and the key in a separate file. The key will need to be added to the top of the .pem
file before adding this to Opsview, if this is the case.
Handling intermediate CA certificates in NGINX (SSL Certificates) Copied
If you find that your homepage/web-page isn’t loading up after upgrading from Apache you may find that you need to perform this configuration.
NGINX does not have an equivalent configuration to Apache’s “SSLCertificateChainFile” for specifying seperate intermediate CA files. Instead, if your Opsview webserver’s SSL certificate is signed by an intermediate CA, then you must include the intermediate CA certificate in the same file as the webserver’s SSL certificate.
You can roll up your intermediate CA .crt
s and webserver .crt
file in the following way:
cd /opt/opsview/webapp/etc/ssl/ #default location for server.crt. Your path may be different
cp -p server.crt server.crt.backup
# Replace filepaths below with paths to your certificates
cat /path/to/your/SERVER_SSL.crt /path/to/your/INTERMEDIATE_CA_SSL.crt [/path/to/your/INTERMEDIATE_CA_SSL_2.crt] ... > server.crt
# Restart the webserver to load the new certificates
/opt/opsview/watchdog/opsview-monit restart opsview-webserver
Switching to HTTP from HTTPS Copied
By default Opsview Monitor uses HTTPS for all connections. If you want to use HTTP then follow these steps:
As the ‘root’ user, update the /opt/opsview/deploy/etc/user_vars.yml
file and add or amend the following parameter:
opsview_webserver_use_ssl: False
Then run:
/opt/opsview/deploy/bin/opsview-deploy /opt/opsview/deploy/lib/playbooks/orchestrator-install.yml
This will make the necessary adjustments to opsview-webserver.
Be aware that Opsview Monitor now uses HSTS by default. If users have already connected to the web interface over HTTPS then they may have issues connecting using HTTP.
Login session limits Copied
By default the maximum number of concurrent sessions is set to 10; this limit applies to both the UI and REST API sessions combined.
To change the default value, you can add the following lines to /opt/opsview/deploy/etc/user_vars.yml
:
opsview_session_manager_config_overrides:
max_sessions: 30
and then run:
/opt/opsview/deploy/bin/opsview-deploy /opt/opsview/deploy/lib/playbooks/session-manager-install.yml
This will make the necessary adjustments to the configuration.
Webserver logging Copied
By default Opsview Monitor logs all web access to /var/log/opsview/web_access.log
and error messages are sent to /var/log/opsview/web_error.log
.
Log rotation Copied
As part of the opsview-webserver package, Opsview will configure logrotate by placing the configuration into /etc/logrotate.d/opsview-webserver
.
The default is to rotate logs every day for 7 days.
Note that as part of an upgrade, this configuration file will be overwritten.
Custom log files Copied
The default log files can be overridden as the root
user by adding logging configuration in a .conf
file in the /opt/opsview/webserver/etc/opsview-site-customisations.d/
directory. For example:
File: /opt/opsview/webserver/etc/opsview-site-customisations.d/50-logging.conf
error_log /var/log/mylogdir/error.log notice;
access_log /var/log/mylogdir/access.log combined;
Ensure that the logging directory exists, then restart the web server by running:
/opt/opsview/watchdog/bin/opsview-monit restart opsview-webserver
Note that these log files will grow while the webserver is running. It is advisable to add a policy to rotate the log files on a regular basis - see /etc/logrotate.d/opsview-webserver
for an example. Be aware that /etc/logrotate.d/opsview-webserver
will be overwritten during an upgrade.
Using syslog Copied
Opsview Monitor web accesses and errors can be sent to syslog, for example:
File: /opt/opsview/webserver/etc/opsview-site-customisations.d/50-logging.conf
error_log syslog:server=unix:/dev/log,facility=local6,tag=opsview_webserver notice;
access_log syslog:server=unix:/dev/log,facility=local6,tag=opsview_webserver combined;
Then restart the web server.
Disable logging Copied
Alternatively, to disable web logging in Opsview Monitor:
File: /opt/opsview/webserver/etc/opsview-site-customisations.d/50-logging.conf
error_log off;
access_log off;
Then restart the web server.
Further information Copied
See also the NGINX Docs : Configuring Logging
Re-homing Copied
See Customization
Customisations Copied
To extend the NGINX configuration, you have two possibilities:
- For server wide changes, drop your configuration into /opt/opsview/webserver/etc/conf.d/
- For the Opsview Monitor site changes, drop your configuration into
/opt/opsview/webserver/etc/opsview-site-customisations.d/
; Configuration files must have a file extension of “.conf”. Files are loaded in alphabetical order. The directory /opt/opsview/webserver/etc/opsview-site-customisations.d retain all the customisation and will not be touched (overwritten) upon a re-run of opsview-deploy. Therefore not being overwritten on any future upgrades.
Note
You should not change
/opt/opsview/webserver/etc/conf.d/opsview.conf
as this is managed by Opsview Deploy and will be changed on upgrades.Make sure your NGINX configuration is valid, especially when upgrading Opsview Monitor, as an invalid configuration will mean that opsview-webserver will not be able to restart and may cause an upgrade to fail.
Example of this would be adding the NGINX Status Page, shown below:
NGINX Status page Copied
There is no possibility to enable the status page through opsview-deploy ansible playbook, so creating a .conf file as mentioned above is the way to do this.
You may enable it manually by running the following commands:
vim /opt/opsview/webserver/etc/opsview-site-customisations.d/status.conf
location /nginx_status {
stub_status on;
access_log off;
}
Then restart the opsview-webserver (which is nginx):
/opt/opsview/watchdog/bin/opsview-monit restart opsview-webserver
Other configurable items Copied
The following settings can be modified within `/opt/opsview/deploy/etc/user_vars.yml for use with Deploy
#
# Session Management's ignore IP feature. Needed for double proxying
#
opsview_session_ignore_ip: False
#
# Disable configuration tasks for opsview-webserver.
#
# This is useful if you need to fine-tune the opsview-webserver configuration
# (e.g. SSO, alternative ports, SSL options, etc) with options not supported by
# opsview-deploy.
#
opsview_webserver_disable_config: False
#
# Stop and disable Apache2/HTTPD to prevent conflicts with opsview-webserver.
#
opsview_webserver_disable_apache: True
#
# Timeout for connections to upstream proxies.
#
opsview_webserver_proxy_connect_timeout: 30s
#
# Timeout for sending packets to upstream proxies.
#
opsview_webserver_proxy_send_timeout: 30s
#
# Timeout for reading packets from upstream proxies.
#
opsview_webserver_proxy_read_timeout: 86400s
When the changes have been made, run:
/opt/opsview/deploy/bin/opsview-deploy /opt/opsview/deploy/lib/playbooks/orchestrator-install.yml
This will make the necessary adjustments to opsview-webserver.
Troubleshooting Copied
opsview-webserver displays “Initializing” all the time Copied
Opsview Watchdog will try to restart NGINX if there are any issues, but if there is an invalid configuration, watchdog will continually retry and mark the service as Initializing. Check the syslog output. For example:
Jul 15 13:26:14 ubuntu18-nginx opsview-monit[32647]: 'opsview-webserver' failed to start (exit status 1) -- '/opt/opsview/webserver/bin/start.sh': nginx: [emerg] invalid number of arguments in "add_header" directive in /opt/opsview/webserver/etc/conf.d/opsview.conf:20
This means there is an invalid configuration in opsview.conf at line 20. Resolve the issue and check watchdog again.
Management Copied
Configuration Copied
DPKGs Copied
Watchdog service files are now managed by the package manager, removing the package would leave the watchdog service file behind with a .save extension. Purging the package will remove it. The package manager managed config file:
/opt/opsview/watchdog/etc/services/opsview-web.conf
RPMs Copied
Watchdog service files are now managed by the package manager. When removing the package, if the files listed have been modified by the user before; they will be left behind after rpmsave extension has been added to them. When upgrading the package, if the files listed have been modifed by the user before; the modified file on the disk will remain and the new config file from the upgrading package will be written to the disk with the rpmnew extension. Users will have to:
- Manually restore the file with the rpmsave extension.
- Rename the rpmnew to the config file name if they want to use the new config for the package:
/opt/opsview/watchdog/etc/services/opsview-web.conf`
Service administration Copied
Start, stop, and restart Opsview Web using Watchdog:
opsview_watchdog opsview-web <stop|start|restart>
Logging Copied
Logging is configured in /opt/opsview/webapp/etc/Log4perl.conf
, which by default will forward to syslog. This file is overwritten on an upgrade.
Double proxying Copied
You may want to have a second, and secured, proxy located in front of your Opsview Monitor server. See Customization.