Integrate with Icinga

Overview

What does the integration offer?

Jira Service Management’s Icinga plugin supports a bidirectional integration with Icinga. The integration leverages Jira Service Management’s Icinga-specific executable and JEC utility to automatically create rich alerts (status, alert histogram, trends, etc.) and synchronizes alert status between Icinga and Jira Service Management.

Set up the integration

The Icinga integration plugin utilizes the full capabilities of Jira Service Management and provides bi-directional integration with Icinga. The steps in the following procedure describe how to integrate Jira Service Management and Icinga by using the Icinga integration plugin. Note that slight alteration to these instructions may be necessary depending on the exact Linux distribution and your Icinga configuration.

Installation prerequisites

The installation packages support the following systems:

RedHat-based Linux distributions
Debian-based Linux distributions

Install the Jira Service Management plugin for Icinga

Jira Edge Connector (abbreviated as JEC) is a prerequisite for configuring the outgoing authentication of Icinga integration. You can combinedly use JEC and Icinga scripts to update alerts on Icinga. With this setup, you can deploy your own script, modify the ones provided, or run customized actions on Icinga. Download the latest version of the Icinga package from this repository.

Instructions for RedHat-based distributions

Run the following command:

rpm -i jsm-icinga-<your_version>.rpm

Before rpm upgrades, back up your configuration files.

The rpm package does not overwrite the existing configuration during upgrades. It saves the new default configuration file as integration.conf.rpmnew. To upgrade from version 201X-XX-XX to 2.X.X, add the --force parameter. For example: rpm -U --force jsm-icinga-<your_version>.rpm

Learn more about config file handling for rpm upgrades.

Instructions for Debian-based distributions

Run the following command:

dpkg -i jsm-icinga-<your_version>.deb

Add Icinga integration

If you're using the Free or Standard plan in Jira Service Management. To access the feature through Settings (gear icon) > Products (under JIRA SETTINGS) > OPERATIONS, you need to be on Premium or Enterprise plan.

Adding an integration from your team’s operations page makes your team the owner of the integration. This means Jira Service Management only assigns the alerts received through this integration to your team.

To add an Icinga integration in Jira Service Management:

Go to your team’s operations page.
On the left navigation panel, select Integrations and then Add integration.
Run a search and select “Icinga”.
On the next screen, enter a name for the integration.
Optional: Select a team in Assignee team if you want a specific team to receive alerts from the integration.
Select Continue.
The integration is saved at this point.
Expand the Steps to configure the integration section and copy the API key.
You will use this key while configuring the integration in Icinga later. Icinga uses the key to authenticate with Jira Service Management and specify the integration to be used to process Icinga alerts.
Select Turn on integration.
The rules you create for the integration will work only if you turn on the integration.

Configure the Jira Service Management plugin in Icinga

The plugin uses a golang-executable file (included in the plugin as send2jsm) to create, acknowledge, and close alerts in Jira Service Management. Configure Icinga to execute this file on events to create, acknowledge, and close alerts in Jira Service Management. The following table lists the parameters and states if they are mandatory.

Configuration parameter	Description	Mandatory?	Location
apiKey	Copy the URL from the integration configuration page in Jira Service Management. send2jsm uses this key to authenticate to Jira Service Management. API key is also used to identify the right integration configuration that should be used to process alerts.	Yes	/home/jsm/jec/conf/jec-config.json
baseUrl	Change this field according to your Jira Service Management environment (For example: EU, sandbox)	No	/home/jsm/jec/conf/jec-config.json
responders	The default responder. This field is used to specify which responders should be notified for Icinga alerts. You can modify it to route alerts to different teams or schedules in Jira Service Management. This field is required if you haven’t set responders in the integration configuration page.	No	/home/jsm/jec/conf/integration.conf
tags	Tags of the alert that created in Jira Service Management.	No	/home/jsm/jec/conf/integration.conf
icinga_server	The Icinga server in Jira Service Management and only required when there are multiple Icinga servers. This is used by Jira Service Management when sending actions run by users (acknowledge, close, etc.) back to your Icinga servers via JEC.	No	/home/jsm/jec/conf/integration.conf
logPath	The full path of the log file (Default: /var/log/jec/send2jsm.log)	No	/home/jsm/jec/conf/integration.conf
icinga2jsm.http.proxy.enabled	To enable or disable the external proxy configuration. Default: false	No	/home/jsm/jec/conf/integration.conf
icinga2jsm.http.proxy.host	Host of the proxy	No	/home/jsm/jec/conf/integration.conf
icinga2jsm.http.proxy.port	Port of the proxy	No	/home/jsm/jec/conf/integration.conf
icinga2jsm.http.proxy.scheme	The proxy connection protocol. It may be http or https, depending on your proxy servers. Default: http	No	/home/jsm/jec/conf/integration.conf
icinga2jsm.http.proxy.username	The username for proxy authentication	No	/home/jsm/jec/conf/integration.conf
icinga2jsm.http.proxy.password	The password for proxy authentication	No	/home/jsm/jec/conf/integration.conf

Configure the golang-executable file in any of the following three methods:

Method 1: Configure from conf file

Configure from the /home/jsm/jec/conf/integration.conf file. This overwrites any configuration you previously made in the script.

Method 2: Configure by using Golang flags

Configure by entering flags into the command of the notification created in Icinga, as described earlier in this section. Use -apiKey flag for your apiKey.

To send additional custom arguments, add them after the flags. For example, customArgName1 customArgValue1 customArgName2 customArgValue2. Parse custom arguments by adding {{_payload.customArgName}} to wherever is needed in the input fields. To learn more about using raw parameters, see dynamic fields.

Method 3: Configure from script

Configure apiKey send2jsm.go script. Build the script again and put the new executable into the /home/jsm/jec/scripts directory. Learn more about the location of the send2jsm.go and how to build a go script in the “Source and recompiling send2jsm” section in this article.

Define Icinga contacts

1. Copy the /home/jsm/jec/jsm-icinga/jsm.cfg file (configures a contact, its host, and service notification commands) in to /usr/local/icinga/etc/objects directory.

Shell

cp /home/jsm/jec/jsm-icinga/jsm.cfg /usr/local/icinga/etc/objects

2. Add the following line to the main Icinga configuration file (ICINGA_HOME/etc/icinga.cfg):

Shell

...
cfg_file=/usr/local/icinga/etc/objects/jsm.cfg
...

3. Add the contact “jsm" to the Icinga configuration’s main contact group in ICINGA_HOME/etc/objects/contacts.cfg file. If using the default configuration contacts.cfg, add "jsm" user to the "admins" contact group.

4. Restart Icinga.

If everything goes well, alerts are seen in Jira Service Management for every notification created in Icinga.

Configure Jira Service Management to update Icinga

This is an optional step.

Select the Send Alert Actions To Icinga checkbox on the integration configuration page. You can combinedly use JEC and Icinga scripts to update alerts on Icinga. With this setup, you can deploy your own script, modify the ones provided, or run customized actions on Icinga.

To run actions in Zenoss, JEC gets the configuration parameters from the configuration file, config.json (found at /home/jsm/jec/conf/jec-config.json).

Configuration parameters

alert_histogram_image_url: JEC retrieves histogram images from Icinga using this URL. Localhost should be replaced with the Icinga server address.
trends_image_url: JEC retrieves trends images from Icinga using this URL. Localhost should be replaced with the Icinga server address.
command_url: URL to update Icinga alerts when alerts get acknowledged, commented, etc.
user: Username to authenticate Icinga web server to get Icinga histogram and trends images.
password: Password to authenticate Icinga web server to get Icinga histogram and trends images.

The downloaded package includes the JEC utility (found in /usr/local/bin) and the script that JEC needs to run (found in /home/jsm/jec/scripts). Be sure to run JEC after configuring it. Learn more about running JEC documentation.

The Icinga integration package does not support SSL v1.0. If your Icinga Server has SSL v1.0, upgrade your SSL server.

Source and recompiling send2jsm

The source for the executable send2jsm is found in /usr/bin/ and send2jsm.go, in /home/jsm/jec/scripts respectively and is also available in this repository. To change the behavior of the executable, edit send2jsm.go and build it by using the following command: go build send2jsm.go

For installing go, refer to http://golang.org/doc/install. Note that the executable in the plugin is built for linux/386 systems.

FAQ and troubleshooting

If the integration is not working, review this section and follow the prescribed guidelines.

1. Icinga alerts are not getting created in Jira Service Management

Run the following test command from the shell and check if a test alert is created in Jira Service Management:

/home/jsm/jec/scripts/send2jsm -entityType=host -t=PROBLEM -hs=DOWN -hn=test_host

If you get a "Trace/breakpoint trap" error, the send2jsm plugin isn't compatible with the server distribution. Rebuild send2jsm.go according to the specific server environment as described in the “Source and recompiling send2jsm” section in this article.
If the alert is created in Jira Service Management, the integration is configured correctly. Icinga is probably not notifying the Jira Service Management contact for alerts. Check your Icinga alert notifications log.
If the alert is not created in Jira Service Management, check the logs at /var/log/jec/send2jsm.log.
Look for the following errors in the log file:
- If you see "RestException[Could not authenticate.]" in the logs, Jira Service Management couldn't identify the API key. Check if the API key is set correctly per the steps outlined in the “Configure the Jira Service Management plugin in Icinga” section of this article.
- If unsure of the problem, set the plugin's log level to debug and try again. Contact us and share the logs.
If there is no /var/log/jec/send2jsm.log file or there are no logs in it, check the following:
1. Check if the Icinga user has permission to write to /var/log/jec directory. The installation package should automatically do this for you. If you face issues, run the following command:
  chown -R icinga:jsm /var/log/jec
2. Check the Icinga server logs at /opt/icinga/log/zeneventd.log. See if there are error logs regarding send2jsm. Contact us with the logs as needed.

Setting send2jsm plugin's log level to DEBUG

Set the send2jsm plugin's log level to DEBUG. Open the /home/jsm/jec/conf/integration.conf file and change the line send2jsm.logger=warning to icinga2jsm.logger=debug.

2. The Icinga alert is not acknowledged when the alert is acknowledged in Jira Service Management

Check the alert logs.

If "Posted [Acknowledge] action to Icinga.." is not present in the log, Jira Service Management didn't send the Acknowledge action to Icinga. Check the integration configuration, it might not have a matching the alert action.
If only the "Posted [Acknowledge] action to Icinga.." log occurs followed by no related logs, it might mean JEC is having connection problems. Check the logs.

3. Could not open Icinga RPM package

If you figure out while installing the rpm package that the package is obsolete, use rpm -i jsm-icinga-1.0.4-rpm-x86-64.rpm --nodeps instead.
If you get "is already installed" error, use rpm -i jsm-icinga-1.0.4-rpm-x86-64.rpm --force instead.