Bug#867997: jessie->stretch, all apt clients break with /var/lib/dpkg/status realpath error

[Alan Schwartz]

Package: upgrade-reports
Version: 9
Severity: grave

Followed the Debian 9 release notes to upgrade a jessie i386 (686) machine
to stretch. No apt sources except official debian ones. Following the apt-get
dist-upgrade step, attempts to use apt-get, apt-cache, aptitude
produce the following error:

# apt-get update
Hit:1 http://ftp.us.debian.org/debian stretch-updates InRelease
Ign:2 http://ftp.debian.org/debian stretch InRelease
Get:3 http://security.debian.org stretch/updates InRelease [62.9 kB]
Hit:4 http://ftp.debian.org/debian stretch Release
Fetched 62.9 kB in 1s (38.2 kB/s)
Reading package lists... Error!
E: flAbsPath on /var/lib/dpkg/status failed - realpath (22: Invalid
argument)
E: Could not open file - open (2: No such file or directory)
E: Problem opening
E: The package lists or status file could not be parsed or opened.


Error persists after reboot of the system.

However, dpkg -l and similar still work. The /var/lib/dpkg structure
appears to be intact:

$ ls -l /var/lib/dpkg
total 6904
drwxr-xr-x 2 root root 4096 Jul 7 16:39 alternatives
-rw-r--r-- 1 root root 1379057 Apr 28 2015 available
-rw-r--r-- 1 root root 1361618 Apr 28 2015 available-old
-rw-r--r-- 1 root root 8 Feb 20 2006 cmethopt
-rw-r--r-- 1 root root 953 Jul 7 15:54 diversions
-rw-r--r-- 1 root root 1034 Jul 7 15:54 diversions-old
drwxr-xr-x 2 root root 339968 Jul 7 16:03 info
-rw-r----- 1 root root 0 Jul 7 16:44 lock
-rw-r----- 1 root root 0 Feb 20 2006 methlock
drwxr-xr-x 7 root root 4096 Feb 7 2011 methods
drwxr-xr-x 2 root root 4096 May 26 2005 parts
-rw-r--r-- 1 root root 460 Jul 7 16:40 statoverride
-rw-r--r-- 1 root root 421 Jul 7 16:40 statoverride-old
-rw-r--r-- 1 root root 1961225 Jul 7 16:44 status
-rw-r--r-- 1 root root 1961225 Jul 7 16:44 status-old
drwxr-xr-x 2 root root 4096 Jul 7 16:43 triggers
drwxr-xr-x 2 root root 4096 Jul 7 16:44 updates


I am using Debian GNU/Linux 2.2, kernel 2.2.17-pre-patch-13 and libc6
2.1.3-10.

Earlier in the week, I did a similar upgrade on another system with same
hardware and didn't experience a problem.

[Niels Thykier]

Post by Alan Schwartz
Package: upgrade-reports
Version: 9
Severity: grave
Followed the Debian 9 release notes to upgrade a jessie i386 (686) machine
to stretch. No apt sources except official debian ones. Following the apt-get
dist-upgrade step, attempts to use apt-get, apt-cache, aptitude
# apt-get update
Hit:1 http://ftp.us.debian.org/debian stretch-updates InRelease
Ign:2 http://ftp.debian.org/debian stretch InRelease
Get:3 http://security.debian.org stretch/updates InRelease [62.9 kB]
Hit:4 http://ftp.debian.org/debian stretch Release
Fetched 62.9 kB in 1s (38.2 kB/s)
Reading package lists... Error!
E: flAbsPath on /var/lib/dpkg/status failed - realpath (22: Invalid
argument)
E: Could not open file - open (2: No such file or directory)
E: Problem opening
E: The package lists or status file could not be parsed or opened.
Error persists after reboot of the system.
[...]

Hi,

Sorry to hear you had issues with the upgrade.

The problem you describe appears to be related to the release notes item
[5.3.2] (5.3.2.1 actually, but I cannot find the direct link to that).
The exact case with /var/lib/dpkg/status is not mentioned, but it could
be that there are some permissions on your system that forbit "_apt" to
access /var/lib/dpkg/status.

* Could you please try to login in as the _apt user and check that it
can read /var/lib/dpkg/status?
- If it fails, please attempt to figure out where the permission
fails (e.g. [dir-test])

* If you have any policy framework enabled (SELinux or AppArmor),
please check if any of these are forbidding the access.

Thanks,
~Niels

[5.3.2]:
https://www.debian.org/releases/stretch/amd64/release-notes/ch-information.en.html#apt-issues

[dir-test]: Example:

ls -ld / /var /var/lib /var/lib/dpkg /var/lib/dpkg/status

[Alan Schwartz]

Post by Niels Thykier
Sorry to hear you had issues with the upgrade.
The problem you describe appears to be related to the release notes item
[5.3.2] (5.3.2.1 actually, but I cannot find the direct link to that).
The exact case with /var/lib/dpkg/status is not mentioned, but it could
be that there are some permissions on your system that forbit "_apt" to
access /var/lib/dpkg/status.
* Could you please try to login in as the _apt user and check that it
can read /var/lib/dpkg/status?

Due to the /nonexistent home directory and /bin/false login shell,
it does not appear to be possible to log in as _apt:

# su - _apt
No directory, logging in with HOME=/

If I change _apt's HOME to /tmp and login shell to /bin/bash:

# su - _apt
_apt$ apt-get check
E: Could not open lock file /var/lib/dpkg/lock - open (13: Permission denied)
E: Unable to lock the administration directory (/var/lib/dpkg/), are you root?

Post by Niels Thykier
- If it fails, please attempt to figure out where the permission
fails (e.g. [dir-test])

# ls -ld / /var /var/lib /var/lib/dpkg /var/lib/dpkg/status
drwxr-xr-x 21 root root 1024 Jul 10 11:21 /
drwxr-xr-x 13 root root 4096 May 7 2013 /var
drwxr-xr-x 74 root root 4096 Jul 7 16:01 /var/lib
drwxr-xr-x 8 root root 4096 Jul 11 08:48 /var/lib/dpkg
-rw-r--r-- 1 root root 1961225 Jul 11 08:48 /var/lib/dpkg/status

As you can see, none of these are writable by _apt; however,
these are the same permissions I see on the other (working)
Debian 9 system. lsattr shows no attributes on any /var/lib/dpkg/status
files.

Post by Niels Thykier
* If you have any policy framework enabled (SELinux or AppArmor),
please check if any of these are forbidding the access.

Neither of those are enabled.

[Niels Thykier]

Post by Alan Schwartz

Post by Niels Thykier
Sorry to hear you had issues with the upgrade.
The problem you describe appears to be related to the release notes item
[5.3.2] (5.3.2.1 actually, but I cannot find the direct link to that).
The exact case with /var/lib/dpkg/status is not mentioned, but it could
be that there are some permissions on your system that forbit "_apt" to
access /var/lib/dpkg/status.
* Could you please try to login in as the _apt user and check that it
can read /var/lib/dpkg/status?

Due to the /nonexistent home directory and /bin/false login shell,
[...]

For future reference, you can "by-pass" that by using:

su -s/bin/bash _apt

That will grant you a shell as _apt provided it is run as root
(regardless of _apt's login shell and $HOME).

Post by Alan Schwartz
_apt$ apt-get check
E: Could not open lock file /var/lib/dpkg/lock - open (13: Permission denied)
E: Unable to lock the administration directory (/var/lib/dpkg/), are you root?

FTR; I don't think that "apt-get check" is supposed to work as non-root.

Post by Alan Schwartz

Post by Niels Thykier
- If it fails, please attempt to figure out where the permission
fails (e.g. [dir-test])

# ls -ld / /var /var/lib /var/lib/dpkg /var/lib/dpkg/status
drwxr-xr-x 21 root root 1024 Jul 10 11:21 /
drwxr-xr-x 13 root root 4096 May 7 2013 /var
drwxr-xr-x 74 root root 4096 Jul 7 16:01 /var/lib
drwxr-xr-x 8 root root 4096 Jul 11 08:48 /var/lib/dpkg
-rw-r--r-- 1 root root 1961225 Jul 11 08:48 /var/lib/dpkg/status
As you can see, none of these are writable by _apt; however, these are
the same permissions I see on the other (working)

Indeed, and _apt does not need write access to the status file.

Does it work if you disable the sandbox user, e.g. by using:
apt-get -o APT::Sandbox::User=root update

Thanks,
~Niels

[Alan Schwartz]

Post by Niels Thykier
FTR; I don't think that "apt-get check" is supposed to work as non-root.

Post by Alan Schwartz

Post by Niels Thykier
- If it fails, please attempt to figure out where the permission
fails (e.g. [dir-test])

# ls -ld / /var /var/lib /var/lib/dpkg /var/lib/dpkg/status
drwxr-xr-x 21 root root 1024 Jul 10 11:21 /
drwxr-xr-x 13 root root 4096 May 7 2013 /var
drwxr-xr-x 74 root root 4096 Jul 7 16:01 /var/lib
drwxr-xr-x 8 root root 4096 Jul 11 08:48 /var/lib/dpkg
-rw-r--r-- 1 root root 1961225 Jul 11 08:48 /var/lib/dpkg/status
As you can see, none of these are writable by _apt; however, these are
the same permissions I see on the other (working)

Indeed, and _apt does not need write access to the status file.
apt-get -o APT::Sandbox::User=root update

No, same error:

# apt-get -o APT::Sandbox::User=root update
Get:1 http://security.debian.org stretch/updates InRelease [62.9 kB]
Get:2 http://ftp.us.debian.org/debian stretch-updates InRelease [88.5 kB]
Ign:3 http://ftp.debian.org/debian stretch InRelease
Hit:4 http://ftp.debian.org/debian stretch Release
Fetched 151 kB in 1s (81.1 kB/s)
Reading package lists... Error!
E: flAbsPath on /var/lib/dpkg/status failed - realpath (22: Invalid argument)
E: Could not open file - open (2: No such file or directory)
E: Problem opening
E: The package lists or status file could not be parsed or opened.

[Debian Bug Tracking System]

Your message dated Tue, 11 Jul 2017 20:27:00 +0000
with message-id <9743dd59-1e84-7908-66e1-***@thykier.net>
and subject line Re: Bug#867997: jessie->stretch, all apt clients break with /var/lib/dpkg/status realpath error
has caused the Debian Bug report #867997,
regarding jessie->stretch, all apt clients break with /var/lib/dpkg/status realpath error
to be marked as done.

This means that you claim that the problem has been dealt with.
If this is not the case it is now your responsibility to reopen the
Bug report if necessary, and/or fix the problem forthwith.

(NB: If you are a system administrator and have no idea what this
message is talking about, this may indicate a serious mail system
misconfiguration somewhere. Please contact ***@bugs.debian.org
immediately.)