PSA: Potential security vulnerability in Elasticsearch and more via Apache Log4j (Log4Shell)

Verge

Supreme [H]ardness
Joined
May 27, 2001
Messages
7,416
Spent most of yesterday and part of today patching our systems at work. Some vendors don't even know if some of their systems are affected as of now. Been a shit show, but I think most companies will pull through.
Worked all weekend. Better safe than sorry approach.
 

Zarathustra[H]

Extremely [H]
Joined
Oct 29, 2000
Messages
34,254
I have been searching through all the servers and open source software I use.

Only place I have found it thus far is in the Unifi controller software for my AP's.

There was an update, I applied it, but it is unclear to me if this is a new update for log4j or something pre-existing I hadn't updated yet.

Edit:

Confirmed it from the release notes.
Code:
UniFi Network Application 6.5.54
Bugfixes
 - Fix a security vulnerability found in a 3rd party library (CVE-2021-44228).
 
Last edited:

Absalom

Gawd
Joined
Oct 3, 2007
Messages
1,019
A mitigation (or brute force maneuver) is to simply block all jndi:// urls - assuming you can write firewall rules that specific. All known exploits use the JNDI url format.

Unfortunately for NGINX this required pulling in a LUA plugin since NGINX's own syntax isn't that fancy.
 

Lakados

Supreme [H]ardness
Joined
Feb 3, 2014
Messages
5,779
Palo Alto pushed out an updated threat Library yesterday for that exploit, I’m glad they did but it caused me some fun unexpected surprises this morning when it was also blocking the BAcNet traffic coming too and from our boiler system causing them to not fire up on a morning of -20.
 

Zarathustra[H]

Extremely [H]
Joined
Oct 29, 2000
Messages
34,254
Palo Alto pushed out an updated threat Library yesterday for that exploit, I’m glad they did but it caused me some fun unexpected surprises this morning when it was also blocking the BAcNet traffic coming too and from our boiler system causing them to not fire up on a morning of -20.

Wow, where are you that you are having those temps? We have been having an unusually mild winter thus far here in New England.
 

Lakados

Supreme [H]ardness
Joined
Feb 3, 2014
Messages
5,779
Wow, where are you that you are having those temps? We have been having an unusually mild winter thus far here in New England.
Northern Canada. Either way PA fixed it once I let them know and they had it all squared up by their next virus definitions update some 8h later.
 

socK

Supreme [H]ardness
Joined
Jan 25, 2004
Messages
4,229
The design that even allowed for this in the first place is fucking insane.

Like seriously WTF.
 

Zarathustra[H]

Extremely [H]
Joined
Oct 29, 2000
Messages
34,254
The design that even allowed for this in the first place is fucking insane.

Like seriously WTF.

It reminds me of this XKCD comic:

1639590600405.png


Seemingly insignificant and small packages that everyone depends upon, that were originally written in a different era before everything was online and never brought up to modern standards.

There are probably plenty more log4j's hiding out there in plain sight, included with or depended on by all sorts of major projects.

Of course, the Apache Software Foundation is more than "some random person in Nebraska" but still... Things still fall through the cracks, especially when they are something seemingly trivial like a small log tracking module...
 
Last edited:

lopoetve

Fully [H]
Joined
Oct 11, 2001
Messages
32,611
It reminds me of this XKCD comic:

View attachment 422246

Seemingly insignificant and small packages that everyone depends upon, that were originally written in a different era before everything was online and never brought up to modern standards.

There are probably plenty more log4j's hiding out there in plain sight, included with or depended on all sorts of major projects.

Of course, the Apache Software Foundation is more than "some random person in Nebraska" but still... Things still fall through the cracks, especially when they are something seemingly trivial like a small log tracking module...
All the pieces, in isolation, make sense. Logging - cool. Parsing logs to trigger events - sane. Running code off of a log - great for testing (gather a crash dump, pull an analyzer, etc), but potentially risky.

Putting all that together in production? Terrifying. And disastrous.
 

Lakados

Supreme [H]ardness
Joined
Feb 3, 2014
Messages
5,779
It reminds me of this XKCD comic:

View attachment 422246

Seemingly insignificant and small packages that everyone depends upon, that were originally written in a different era before everything was online and never brought up to modern standards.

There are probably plenty more log4j's hiding out there in plain sight, included with or depended on by all sorts of major projects.

Of course, the Apache Software Foundation is more than "some random person in Nebraska" but still... Things still fall through the cracks, especially when they are something seemingly trivial like a small log tracking module...
I think it is a direct comment on the Heartbleed bug from OpenSSL, which was maintained by two guys in their spare time because nobody else was doing it. Funding for the project dried up in 2003, but so many sites were using it they maintained it for free in their spare time as their repeated requests for funding went unanswered, until Heartbleed happened and the world saw the situation they were in and went "oh shit!".
 

LodeRunner

Limp Gawd
Joined
Sep 8, 2006
Messages
141
A poster by the name of sarusa over at Ars T explained it like this, which I thought was amusing and terrifying:
It's a consequence of two things. First, log4j has a bunch of things you can include in the logging, of the format ${type:parameter}. You can think of the types as 'modules' and since everything enterprisey tends towards maximum bloat, there are dozens of modules. At some point someone added ${jndi:xxx} as a module. JNDI is a bloated enterprisey monstrosity that encompasses five major bits of network related functionality. As soon as you add the ${jndi:} bit you get access to parts of it that make no sense for logging like 'fetch and execute remote code'. It's like you needed a canoe to cross a pond, but you just dropped in a nuclear aircraft carrier because it was handy. What could go wrong?

The second part was the absolutely boneheaded decision by log4j to look for and execute the ${type:parameter} in everything being logged, including random user data. You never interpret user data, ever, unless you're explicitly doing user scripting. It should only apply in your log configuration strings. Now you've handed everyone access to jndi and the keys to your nuclear aircraft carrier.

So it's just a series of bad decisions that I'm sure seemed like okay ideas at the time because they made things more convenient and we always go for convenience over security.
 
Top