Все статьиРазработка3 мин чтения

Why a monitoring service must not check internal addresses

Этот текст пока недоступен на вашем языке, поэтому он показан на английском.

A monitor calls addresses that customers enter, from our server. That is useful and dangerous at the same time. How Uptime protects itself against SSRF.

Server racks in a data centre

A monitoring service does something that looks like an attack at first glance: it calls addresses entered by a stranger, from its own server. For public websites that is exactly the point. For internal addresses it becomes a problem.

The problem

If someone enters an internal address instead of a website, for example the machine’s own localhost, a service in the data-centre network, or 169.254.169.254, where cloud providers serve their metadata, our server sends a request there. We show status, response time and error message. From those you can tell what is running behind the firewall, although the requester never had access to it. This attack is called Server-Side Request Forgery, SSRF for short, and has its own topic in the OWASP documentation.

While reviewing our checker we found exactly that: it checked every address that was entered, internal ones included. In the development environment that made it possible to query a service in the Docker network. We fixed this before the service launched.

What Uptime blocks now

  • Loopback addresses such as 127.0.0.1 and ::1.
  • Private networks (10/8, 172.16/12, 192.168/16) and their IPv6 counterparts.
  • Link-local addresses, including the cloud metadata address.
  • The carrier-grade NAT range (100.64/10), used by Tailscale for example.
  • Multicast, unspecified, reserved and some IPv6 transition ranges.

Why the check sits at the connection

The obvious way would be to check the address when it is saved. That is not enough, because a hostname can point somewhere else than when it was saved. Whoever controls the DNS records of their own domain can switch them to an internal address after the check (DNS rebinding). Redirects can lead to internal targets as well.

That is why our check sits where the connection is made:

  • The hostname is resolved and every address in the answer is checked. If the answer contains even one disallowed address, nothing is connected.
  • Only the checked numeric address is connected to. There is no second name lookup that an attacker could influence.
  • IPv4 addresses written in IPv6 notation are converted back first, so they cannot slip past the check.
  • If name resolution fails or returns no address, it stays an error. When in doubt, the protection opens nothing.
  • The same check applies to HTTP, TCP and ping.

For HTTP there is more: the checker does not use a proxy from the environment that could bypass the check. Host header and TLS name stay the original ones, so certificates are still verified properly. The checker does not follow redirects; a 3xx answer counts as up, as before. A redirect to an internal target therefore leads nowhere.

What else the review turned up

An independent review found two gaps: an old IPv6 range for local networks and a missing time limit for fallback addresses. Both were first reproduced with a failing test and then fixed. In addition the dialer gives itself an overall time limit, because Go does not pass the request deadline on at this step.

And development?

So that local test services can still be checked, there is an exception that applies only in the development environment. In every other environment the service refuses to start if it is set. The same protection applies to outgoing webhooks, because their addresses are customer input too.

What this means for you

Uptime can check services on the public internet, but not ones that are only reachable inside your private network. For internal systems you need a publicly reachable status endpoint that reveals only what is necessary. This is deliberate: a monitoring service that could look into your network could do the same for anybody else.