Package Instance

"Verify All" Returns Success Despite Validation Failures in Singularity
### Impact

The `--all / -a` option to `singularity verify` returns success even when some objects in a SIF container are not signed, or cannot be verified.

The SIF objects that are not verified are reported in `WARNING` log messages, but a `Container Verified` message and exit code of `0`  are returned.

Workflows that verify a container using `--all / -a` and use the exit code as an indicator of success are vulnerable to running SIF containers that have unsigned, or modified, objects that may be exploited to introduce malicious behavior.

```
$ singularity verify -a image.sif 
WARNING: Missing signature for SIF descriptor 2 (JSON.Generic)
WARNING: Missing signature for SIF descriptor 3 (FS)
Container is signed by 1 key(s):

Verifying partition: Def.FILE:
12045C8C0B1004D058DE4BEDA20C27EE7FF7BA84
[LOCAL]   Unit Test <unit@test.com>
[OK]      Data integrity verified

INFO:    Container verified: image.sif

$ echo $?
0
```


### Patches

Singularity 3.6.0 has a new implementation of sign/verify that fixes this issue.

All users are advised to upgrade to 3.6.0. Note that Singularity 3.6.0 uses a new signature format that is necessarily incompatible with Singularity < 3.6.0 - e.g. Singularity 3.5.3 cannot verify containers signed by 3.6.0.

Version 3.6.0 includes a `--legacy-insecure` flag for the `singularity verify` command, that will perform verification of the older, and insecure, legacy signatures for compatibility with existing containers. This does not guarantee that containers have not been modified since signing, due to other issues in the legacy signature format.

### Workarounds

If you are unable to update to 3.6.0 ensure that you do not rely on the return code of `singularity verify --all / -a` as an indicator of trust in a container.

Note that other issues in the sign/verify implementation in Singularity < 3.6.0 allow additional means to introduce malicious behavior to a signed container.


### For more information

General questions about the impact of the advisory / changes made in the 3.6.0 release can be asked in the:

* [Singularity Slack Channel](https://bit.ly/2m0g3lX)
* [Singularity Mailing List](https://groups.google.com/a/lbl.gov/forum/??sdf%7Csort:date#!forum/singularity)

Any sensitive security concerns should be directed to: security@sylabs.io

See our Security Policy here: https://sylabs.io/security-policy

Singularity insecure permissions
Insecure permissions (777) are set on `$HOME/.singularity` when it is newly created by Singularity (version from 3.3.0 to 3.5.1), which could lead to an information leak, and malicious redirection of operations performed against Sylabs cloud services.

Incorrect Permission Assignment for Critical Resource in Singularity
An issue was discovered in Singularity 3.1.0 to 3.2.0-rc2, a malicious user with local/network access to the host system (e.g. ssh) could exploit this vulnerability due to insecure permissions allowing a user to edit files within `/run/singularity/instances/sing/<user>/<instance>`. The manipulation of those files can change the behavior of the starter-suid program when instances are joined resulting in potential privilege escalation on the host.

Insecure permissions on user namespace / fakeroot temporary rootfs in Singularity
### Impact

Insecure permissions on temporary directories used in fakeroot or user namespace container execution.

When a Singularity action command (run, shell, exec) is run with the fakeroot or user namespace option, Singularity will extract a container image to a temporary sandbox directory. Due to insecure permissions on the temporary directory it is possible for any user with access to the system to read the contents of the image. Additionally, if the image contains a world-writable file or directory, it is possible for a user to inject arbitrary content into the running container.

### Patches

This issue is addressed in Singularity 3.6.3.

All users are advised to upgrade to 3.6.3.

### Workarounds

The issue is mitigated if `TMPDIR` is set to a location that is only accessible to the user, as any subdirectories directly under `TMPDIR` cannot then be accessed by others. However, this is difficult to enforce so it is not recommended to rely on this as a mitigation.

### For more information

General questions about the impact of the advisory / changes made in the 3.6.0 release can be asked in the:

* [Singularity Slack Channel](https://bit.ly/2m0g3lX)
* [Singularity Mailing List](https://groups.google.com/a/lbl.gov/forum/??sdf%7Csort:date#!forum/singularity)

Any sensitive security concerns should be directed to: security@sylabs.io

See our Security Policy here: https://sylabs.io/security-policy

containers/image library Insufficiently Protects Credentials
The containers/image library used by the container tools Podman, Buildah, and Skopeo in Red Hat Enterprise Linux version 8 and CRI-O in OpenShift Container Platform, does not enforce TLS connections to the container registry authorization service. An attacker could use this vulnerability to launch a MiTM attack and steal login credentials or bearer tokens.

scs-library-client may leak user credentials to third-party service via HTTP redirect
### Impact

When the scs-library-client is used to pull a container image, with authentication, the HTTP Authorization header sent by the client to the library service may be incorrectly leaked to an S3 backing storage provider. This occurs in a specific flow, where the library service redirects the client to a backing S3 storage server, to perform a multi-part concurrent download.

Depending on site configuration, the S3 service may be provided by a third party. An attacker with access to the S3 service may be able to extract user credentials, allowing them to impersonate the user.

The vulnerable multi-part concurrent download flow, with redirect to S3, is only used when communicating with a Singularity Enterprise 1.x installation, or third party server implementing this flow.

Interaction with Singularity Enterprise 2.x, and Singularity Container Services (cloud.sylabs.io), does not trigger the vulnerable flow.

We encourage all users to update. Users who interact with a Singularity Enterprise 1.x installation, using a 3rd party S3 storage service, are advised to revoke and recreate their authentication tokens within Singularity Enterprise.

### Patches

The security issue was identified after the integration of a bug-fix commit 68ac4ca into the previously released scs-library-client 1.3.4. This commit fixes the security issue in the 1.3 series.

scs-library-client 1.4.2 contains a fix for the same vulnerability in the 1.4 series, as commit eebd7ca.

### Workarounds

There is no workaround available at this time.

As above, access to Singularity Enterprise 2.x, or Singularity Container Services (cloud.sylabs.io), does not trigger the vulnerable flow.

### References

https://cwe.mitre.org/data/definitions/522.html

Singularity Incorrect Access Control
Singularity 2.3.0 through 2.5.1 is affected by an incorrect access control on systems supporting overlay file system. When using the overlay option, a malicious user may access sensitive information by exploiting a few specific Singularity features.

Insecure permissions on build temporary rootfs in Singularity
### Impact

Insecure permissions on temporary directories used in explicit and implicit container build operations.

When a Singularity command that results in a container build operation is executed, it is possible for a user with access to the system to read the contents of the image during the build. Additionally, if the image contains a world-writable file or directory, it is possible for a user to inject arbitrary content into the running build, which in certain circumstances may enable arbitrary code execution during the build and/or when the built container is run.

### Patches

This issue is addressed in Singularity 3.6.3.

All users are advised to upgrade to 3.6.3.

### Workarounds

The issue is mitigated if `TMPDIR` is set to a location that is only accessible to the user, as any subdirectories directly under `TMPDIR` cannot then be accessed by others. However, this is difficult to enforce so it is not recommended to rely on this as a mitigation.

### For more information

General questions about the impact of the advisory / changes made in the 3.6.0 release can be asked in the:

* [Singularity Slack Channel](https://bit.ly/2m0g3lX)
* [Singularity Mailing List](https://groups.google.com/a/lbl.gov/forum/??sdf%7Csort:date#!forum/singularity)

Any sensitive security concerns should be directed to: security@sylabs.io

See our Security Policy here: https://sylabs.io/security-policy

Unpatched extfs vulnerabilities are exploitable through suid-mode Apptainer
### Impact
There is an ext4 use-after-free flaw described in CVE-2022-1184 that is exploitable through versions of Apptainer < 1.1.0 and installations that include apptainer-suid < 1.1.8 on older operating systems where that CVE has not been patched.  That includes Red Hat Enterprise Linux 7, Debian 10 buster (unless the linux-5.10 package is installed), Ubuntu 18.04 bionic and Ubuntu 20.04 focal.  Use-after-free flaws in the kernel can be used to attack the kernel for denial of service and potentially for privilege escalation.

### Background
Historically there have been many CVEs published for extfs and a smaller number for squashfs, including serious use-after-free and buffer overrun vulnerabilities, that are scored as "Moderate" or "Low" impact only because unprivileged users were assumed to not have write access to the raw data.  Because of those ratings, vendors treat such CVEs as low urgency and either delay a patch until their next major release or never patch older but still supported operating systems at all.  Many Linux distributions automatically mount user-writable USB-drive volumes, but those are considered low risk because they require physical access to the machine.  However, since setuid-root installations of Apptainer by default allow all users to mount any extfs (specifically, ext3, which is implemented by the ext4 driver) and squashfs filesystem using kernel drivers even though the users have write access to the raw data, the setuid-root installations raise the severity of such unpatched CVEs.  

CVE-2022-1184 is currently such an unpatched CVE, at least on the above listed operating systems.  The descriptions from the operating system vendors about the CVE (referenced below) are incomplete, saying only that it allows a local attacker with user privilege to cause a denial of service.  Normally users would not be able to cause it because they cannot modify the filesystem image, and normally vulnerabilities that involve kernel memory corruption by unprivileged users are considered high severity even when there is not yet a known privilege escalation because someone with sufficient kernel knowledge can usually turn such a corruption into a privilege escalation. 

Red Hat did not list RHEL7 as vulnerable, but they also did not list it as unaffected, and testing confirmed that a filesystem image could be corrupted to get past the check inserted into the filesystem driver to fix the vulnerability (patches linked below).

All published squashfs CVEs have been patched in currently supported major operating systems.

### Patches
Apptainer 1.1.8 includes a patch that by default disables mounting of extfs filesystem types in setuid-root mode, while continuing to allow mounting of extfs filesystems in non-setuid "rootless" mode using fuse2fs.

### Workarounds
These workarounds are possible:
1. Either do not install apptainer-suid (for versions 1.1.0 through 1.1.7) or set `allow setuid = no` in apptainer.conf.  This requires having unprivileged user namespaces enabled and except for apptainer 1.1.x versions will disallow mounting of sif files, extfs files, and squashfs files in addition to other, less significant impacts.  (Encrypted sif files are also not supported unprivileged in apptainer 1.1.x.)
2. Alternatively, use the `limit containers` options in apptainer.conf/singularity.conf to limit sif files to trusted users, groups, and/or paths.  (The option `allow container extfs = no` disallows mounting extfs overlay files but does not disallow mounting of extfs overlay partitions inside SIF files, so it does not help work around the problem.)

### References
https://nvd.nist.gov/vuln/detail/CVE-2022-1184
https://access.redhat.com/security/cve/cve-2022-1184
https://security-tracker.debian.org/tracker/CVE-2022-1184
https://ubuntu.com/security/CVE-2022-1184
Kernel patches for CVE-2022-1184:
https://github.com/torvalds/linux/commit/65f8ea4cd57dbd46ea13b41dc8bac03176b04233
https://github.com/torvalds/linux/commit/61a1d87a324ad5e3ed27c6699dfc93218fcf3201

------

## Addendum 30 May 2023

New information has become available: many ext4 filesytem vulnerabilities similar to the one in CVE-2022-1184 continue to be found, and most of them do not ever have a CVE assigned.  The way to locate them is to search for "syzbot" in linux kernel commit messages under fs/ext4.  "syzbot" is a public automated system for finding kernel bugs.  Especially when syzbot reports are labeled "KASAN" (Kernel Address Sanitizer) and especially if they involve memory corruption including "use after free", "out of bounds", or "user-memory-access", they are vulnerabilities that can potentially be turned into privilege escalation when an unprivileged user has write access to the underlying data.

In particular there are two such commits from this month, May 2023, referenced below.  They both have commit messages describing a situation of write access to the underlying data while the kernel has that data mounted as a filesystem.  These commits have been backported to currently maintained kernel lines in versions 4.19.293 and 5.4.243, but since they are considered moderate severity, most common OS distributions do not include them immediately.  For example since RHEL9.2 was recently announced, they are not likely to be available in RHEL9 until 9.3 is released in about another 6 months.  Therefore if system administrators want to be protected against these vulnerabilities while still allowing ext4 filesystem mounts through setuid-root apptainer, they should watch for these types of commits and whenever such commits are released the administrators should update to the latest currently maintained kernel version (bypassing their OS vendor's distribution) and reboot.

https://github.com/torvalds/linux/commit/2220eaf90992c11d888fe771055d4de3303
https://github.com/torvalds/linux/commit/4f04351888a83e595571de672e0a4a8b74f
https://lwn.net/Articles/932137/
https://lwn.net/Articles/932136/