I recently had a problem with my raspbian installation. Probably some upgrade messed up my SSD boot. After resolving it, I noticed that docker.service refuses to start. On the service’s log it says
Apr 23 19:24:08 raspberrypi systemd[1]: Starting Docker Application Container Engine...
Apr 23 19:24:08 raspberrypi dockerd[1629]: time="2020-04-23T19:24:08.702984211+01:00" level=info msg="Starting up"
Apr 23 19:24:08 raspberrypi dockerd[1629]: time="2020-04-23T19:24:08.706657044+01:00" level=info msg="parsed scheme: "unix"" module=grpc
Apr 23 19:24:08 raspberrypi dockerd[1629]: time="2020-04-23T19:24:08.707181266+01:00" level=info msg="scheme "unix" not registered, fallback to default scheme" module=grpc
Apr 23 19:24:08 raspberrypi dockerd[1629]: time="2020-04-23T19:24:08.707520970+01:00" level=info msg="ccResolverWrapper: sending update to cc: {[{unix:///run/containerd/containerd.sock 0 <
Apr 23 19:24:08 raspberrypi dockerd[1629]: time="2020-04-23T19:24:08.707855840+01:00" level=info msg="ClientConn switching balancer to "pick_first"" module=grpc
Apr 23 19:24:08 raspberrypi dockerd[1629]: time="2020-04-23T19:24:08.711273840+01:00" level=info msg="parsed scheme: "unix"" module=grpc
Apr 23 19:24:08 raspberrypi dockerd[1629]: time="2020-04-23T19:24:08.711450507+01:00" level=info msg="scheme "unix" not registered, fallback to default scheme" module=grpc
Apr 23 19:24:08 raspberrypi dockerd[1629]: time="2020-04-23T19:24:08.711545396+01:00" level=info msg="ccResolverWrapper: sending update to cc: {[{unix:///run/containerd/containerd.sock 0 <
Apr 23 19:24:08 raspberrypi dockerd[1629]: time="2020-04-23T19:24:08.711626674+01:00" level=info msg="ClientConn switching balancer to "pick_first"" module=grpc
Apr 23 19:24:08 raspberrypi dockerd[1629]: time="2020-04-23T19:24:08.721460748+01:00" level=error msg="failed to mount overlay: no such device" storage-driver=overlay2
Apr 23 19:24:08 raspberrypi dockerd[1629]: time="2020-04-23T19:24:08.721569822+01:00" level=error msg="[graphdriver] prior storage driver overlay2 failed: driver not supported"
Apr 23 19:24:08 raspberrypi dockerd[1629]: failed to start daemon: error initializing graphdriver: driver not supported
Apr 23 19:24:08 raspberrypi systemd[1]: docker.service: Main process exited, code=exited, status=1/FAILURE
Apr 23 19:24:08 raspberrypi systemd[1]: docker.service: Failed with result 'exit-code'.
Apr 23 19:24:08 raspberrypi systemd[1]: Failed to start Docker Application Container Engine.
I wasn’t able to find anything relevant online. Ideally I would like to avoid losing my docker images
Edit: The kernel version was 4.19.114-v7l+, however there were no modules installed for this kernel under /lib/module. Running a «sudo apt install —reinstall raspberrypi-bootloader raspberrypi-kernel» fixed it.
when I am using this command to start dockerd in Raspberry Pi 4, it shows me this error:
raspberrypi% sudo dockerd
INFO[2020-07-26T18:41:15.229201974+08:00] Starting up
INFO[2020-07-26T18:41:15.232284275+08:00] parsed scheme: "unix" module=grpc
INFO[2020-07-26T18:41:15.232347219+08:00] scheme "unix" not registered, fallback to default scheme module=grpc
INFO[2020-07-26T18:41:15.232413200+08:00] ccResolverWrapper: sending update to cc: {[{unix:///run/containerd/containerd.sock 0 <nil>}] <nil>} module=grpc
INFO[2020-07-26T18:41:15.232450033+08:00] ClientConn switching balancer to "pick_first" module=grpc
INFO[2020-07-26T18:41:15.234828378+08:00] parsed scheme: "unix" module=grpc
INFO[2020-07-26T18:41:15.234934914+08:00] scheme "unix" not registered, fallback to default scheme module=grpc
INFO[2020-07-26T18:41:15.234993914+08:00] ccResolverWrapper: sending update to cc: {[{unix:///run/containerd/containerd.sock 0 <nil>}] <nil>} module=grpc
INFO[2020-07-26T18:41:15.235048172+08:00] ClientConn switching balancer to "pick_first" module=grpc
ERRO[2020-07-26T18:41:15.243143273+08:00] failed to mount overlay: no such device storage-driver=overlay2
ERRO[2020-07-26T18:41:15.252902246+08:00] AUFS was not found in /proc/filesystems storage-driver=aufs
ERRO[2020-07-26T18:41:15.259121107+08:00] failed to mount overlay: no such device storage-driver=overlay
WARN[2020-07-26T18:41:15.275937636+08:00] Your kernel does not support swap memory limit
WARN[2020-07-26T18:41:15.275994209+08:00] Your kernel does not support cgroup cfs period
WARN[2020-07-26T18:41:15.276022875+08:00] Your kernel does not support cgroup cfs quotas
WARN[2020-07-26T18:41:15.276049431+08:00] Your kernel does not support cgroup rt period
WARN[2020-07-26T18:41:15.276075449+08:00] Your kernel does not support cgroup rt runtime
INFO[2020-07-26T18:41:15.276550759+08:00] Loading containers: start.
WARN[2020-07-26T18:41:15.287555903+08:00] Running modprobe bridge br_netfilter failed with message: modprobe: ERROR: ../libkmod/libkmod.c:586 kmod_search_moddep() could not open moddep file '/lib/modules/4.19.97-v7l+/modules.dep.bin'
modprobe: WARNING: Module bridge not found in directory /lib/modules/4.19.97-v7l+
modprobe: ERROR: ../libkmod/libkmod.c:586 kmod_search_moddep() could not open moddep file '/lib/modules/4.19.97-v7l+/modules.dep.bin'
modprobe: WARNING: Module br_netfilter not found in directory /lib/modules/4.19.97-v7l+
, error: exit status 1
WARN[2020-07-26T18:41:15.428797118+08:00] Running iptables --wait -t nat -L -n failed with message: `iptables v1.8.2 (nf_tables): CHAIN_ADD failed (No such file or directory): chain PREROUTING`, error: exit status 4
INFO[2020-07-26T18:41:15.981245605+08:00] stopping event stream following graceful shutdown error="<nil>" module=libcontainerd namespace=moby
failed to start daemon: Error initializing network controller: error obtaining controller instance: failed to create NAT chain DOCKER: iptables failed: iptables --wait -t nat -N DOCKER: iptables v1.8.2 (nf_tables): CHAIN_ADD failed (No such file or directory): chain PREROUTING
(exit status 4)
what should I do to fix this problem? This is my docker install commamd:
sudo apt-get update && sudo apt-get upgrade
curl -fsSL https://get.docker.com -o get-docker.sh
sudo sh get-docker.sh --mirror Aliyun
sudo apt install docker
sudo usermod -aG docker pi
this is my Raspberry Pi kernel version:
raspberrypi% uname -a
Linux raspberrypi 4.19.97-v7l+ #1294 SMP Thu Jan 30 13:21:14 GMT 2020 armv7l GNU/Linux
asked Jul 26, 2020 at 10:45
DolphinDolphin
1411 silver badge7 bronze badges
5
it needs to open cgroup in kernel level of Raspberry pi, I add cgroup config to the kernel by modify cmdline.txt:
cgroup_enable=memory cgroup_memory=1 swapaccount=1
and this is my full config:
raspberrypi% cat /boot/cmdline.txt
console=serial0,115200 console=tty1 root=PARTUUID=ea7d04d6-02 rootfstype=ext4 elevator=deadline fsck.repair=yes cgroup_enable=memory cgroup_memory=1 swapaccount=1 rootwait quiet splash plymouth.ignore-serial-consoles
this is the page I am conference.
answered Jul 26, 2020 at 11:50
DolphinDolphin
1411 silver badge7 bronze badges
4
I’m running into an error when running sudo systemctl start docker on a GoDaddy VPS running Ubuntu 16.4.
I installed docker following the steps for the repository: https://docs.docker.com/install/linux/docker-ce/ubuntu/
When I run sudo systemctl start docker I get the following:
Job for docker.service failed because the control process exited with error code. See "systemctl status docker.service" and "journalctl -xe" for details.
sudo systemctl status docker.service outputs the following:
● docker.service - Docker Application Container Engine
Loaded: loaded (/lib/systemd/system/docker.service; enabled; vendor preset: enabled)
Active: failed (Result: start-limit-hit) since Mon 2020-03-16 00:15:47 CDT; 44s ago
Docs: https://docs.docker.com
Process: 1411 ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock (code=exited, status=1/FAILURE)
Main PID: 1411 (code=exited, status=1/FAILURE)
sudo journalctl -xe outputs the following:
dockerd[1385]: time="2020-03-16T00:15:43.189085256-05:00" level=error msg="failed to mount overlay: no such device" storage-driver=overlay
dockerd[1385]: time="2020-03-16T00:15:43.191631124-05:00" level=warning msg="Your kernel does not support oom control"
dockerd[1385]: time="2020-03-16T00:15:43.191652077-05:00" level=warning msg="Your kernel does not support memory swappiness"
dockerd[1385]: time="2020-03-16T00:15:43.191662701-05:00" level=warning msg="Your kernel does not support kernel memory limit"
dockerd[1385]: time="2020-03-16T00:15:43.191672289-05:00" level=warning msg="Your kernel does not support kernel memory TCP limit"
dockerd[1385]: time="2020-03-16T00:15:43.191681175-05:00" level=warning msg="Unable to find cpu cgroup in mounts"
dockerd[1385]: time="2020-03-16T00:15:43.191690078-05:00" level=warning msg="Unable to find cpuset cgroup in mounts"
dockerd[1385]: time="2020-03-16T00:15:43.191698949-05:00" level=warning msg="mountpoint for pids not found"
dockerd[1385]: failed to start daemon: Devices cgroup isn't mounted
time="2020-03-16T00:15:45.425021456-05:00" level=info msg="Starting up"
time="2020-03-16T00:15:45.427712614-05:00" level=info msg="parsed scheme: "unix"" module=grpc
time="2020-03-16T00:15:45.427809136-05:00" level=info msg="scheme "unix" not registered, fallback to default scheme" module=grpc
time="2020-03-16T00:15:45.427939671-05:00" level=info msg="ccResolverWrapper: sending update to cc: {[{unix:///run/containerd/containerd.sock 0 <nil>}] <nil>}" module=grpc
time="2020-03-16T00:15:45.428015236-05:00" level=info msg="ClientConn switching balancer to "pick_first"" module=grpc
time="2020-03-16T00:15:45.429999144-05:00" level=info msg="parsed scheme: "unix"" module=grpc
time="2020-03-16T00:15:45.430026563-05:00" level=info msg="scheme "unix" not registered, fallback to default scheme" module=grpc
time="2020-03-16T00:15:45.430047585-05:00" level=info msg="ccResolverWrapper: sending update to cc: {[{unix:///run/containerd/containerd.sock 0 <nil>}] <nil>}" module=grpc
time="2020-03-16T00:15:45.430062970-05:00" level=info msg="ClientConn switching balancer to "pick_first"" module=grpc
time="2020-03-16T00:15:45.431676150-05:00" level=error msg="failed to mount overlay: no such device" storage-driver=overlay2
time="2020-03-16T00:15:45.433111110-05:00" level=error msg="AUFS was not found in /proc/filesystems" storage-driver=aufs
time="2020-03-16T00:15:45.433685934-05:00" level=error msg="failed to mount overlay: no such device" storage-driver=overlay
time="2020-03-16T00:15:45.436734982-05:00" level=warning msg="Your kernel does not support oom control"
time="2020-03-16T00:15:45.436773138-05:00" level=warning msg="Your kernel does not support memory swappiness"
time="2020-03-16T00:15:45.436791630-05:00" level=warning msg="Your kernel does not support kernel memory limit"
time="2020-03-16T00:15:45.436813539-05:00" level=warning msg="Your kernel does not support kernel memory TCP limit"
time="2020-03-16T00:15:45.436830639-05:00" level=warning msg="Unable to find cpu cgroup in mounts"
time="2020-03-16T00:15:45.436846725-05:00" level=warning msg="Unable to find cpuset cgroup in mounts"
time="2020-03-16T00:15:45.436861904-05:00" level=warning msg="mountpoint for pids not found"
failed to start daemon: Devices cgroup isn't mounted
-
Chura
- Posts: 8
- Joined: Sat May 30, 2020 8:16 pm
Docker not starting, missing overlay2 module
Hi,
Trying to run docker on recent RaspiOS (and USB Booted if it matters)
I fail on overlay2 module missing
Code: Select all
~ $ sudo /usr/bin/dockerd
INFO[2020-05-31T11:29:09.080481751+01:00] Starting up
INFO[2020-05-31T11:29:09.083369455+01:00] parsed scheme: "unix" module=grpc
INFO[2020-05-31T11:29:09.083433417+01:00] scheme "unix" not registered, fallback to default scheme module=grpc
INFO[2020-05-31T11:29:09.083499731+01:00] ccResolverWrapper: sending update to cc: {[{unix:///run/containerd/containerd.sock 0 <nil>}] <nil>} module=grpc
INFO[2020-05-31T11:29:09.083538286+01:00] ClientConn switching balancer to "pick_first" module=grpc
INFO[2020-05-31T11:29:09.086553970+01:00] parsed scheme: "unix" module=grpc
INFO[2020-05-31T11:29:09.086619099+01:00] scheme "unix" not registered, fallback to default scheme module=grpc
INFO[2020-05-31T11:29:09.086677320+01:00] ccResolverWrapper: sending update to cc: {[{unix:///run/containerd/containerd.sock 0 <nil>}] <nil>} module=grpc
INFO[2020-05-31T11:29:09.086715190+01:00] ClientConn switching balancer to "pick_first" module=grpc
ERRO[2020-05-31T11:29:09.095496005+01:00] failed to mount overlay: no such device storage-driver=overlay2
ERRO[2020-05-31T11:29:09.095571189+01:00] [graphdriver] prior storage driver overlay2 failed: driver not supported
failed to start daemon: error initializing graphdriver: driver not supported
Code: Select all
~ $ modprobe overlay2
modprobe: ERROR: ../libkmod/libkmod.c:586 kmod_search_moddep() could not open moddep file '/lib/modules/5.4.42-v7l+/modules.dep.bin'
modprobe: FATAL: Module overlay2 not found in directory /lib/modules/5.4.42-v7l+
I can’t even update the docker-ce 
Code: Select all
~ $ sudo apt upgrade
Reading package lists... Done
Building dependency tree
Reading state information... Done
Calculating upgrade... Done
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
1 not fully installed or removed.
After this operation, 0 B of additional disk space will be used.
Do you want to continue? [Y/n] y
Setting up docker-ce (5:19.03.10~3-0~raspbian-buster) ...
Job for docker.service failed because the control process exited with error code.
See "systemctl status docker.service" and "journalctl -xe" for details.
invoke-rc.d: initscript docker, action "start" failed.
● docker.service - Docker Application Container Engine
Loaded: loaded (/lib/systemd/system/docker.service; enabled; vendor preset: enabled)
Active: activating (auto-restart) (Result: exit-code) since Sun 2020-05-31 11:43:29 BST; 20ms ago
Docs: https://docs.docker.com
Process: 2198 ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock (code=exited, status=1/FAILURE)
Main PID: 2198 (code=exited, status=1/FAILURE)
dpkg: error processing package docker-ce (--configure):
installed docker-ce package post-installation script subprocess returned error exit status 1
Errors were encountered while processing:
docker-ce
E: Sub-process /usr/bin/dpkg returned an error code (1)
I can’t find anyone complain about this, its strange 
-
Chura
- Posts: 8
- Joined: Sat May 30, 2020 8:16 pm
Re: Docker not starting, missing overlay2 module
Sun May 31, 2020 12:54 pm
I just noticed, that the mod is actually empty.
Code: Select all
~ $ sudo lsmod
Module Size Used by
~ $
Maybe it is related to USB boot thing ? or using older version elf/dat ?
-
trejan
- Posts: 5371
- Joined: Tue Jul 02, 2019 2:28 pm
Re: Docker not starting, missing overlay2 module
Sun May 31, 2020 12:56 pm
Chura wrote: ↑
Sun May 31, 2020 12:54 pm
I just noticed, that the mod is actually empty.
Did you download files from github and manually copy them into /boot? If so then you most likely missed the modules that go into /lib/modules. They’re specific to the kernel version so the older ones won’t load.
As you’re on the 5.4 beta kernel, you should be able to just run rpi-update to fix it.
-
Chura
- Posts: 8
- Joined: Sat May 30, 2020 8:16 pm
Re: Docker not starting, missing overlay2 module
Sun May 31, 2020 1:13 pm
yeah i just noticed no 5.4 modules at /lib/modules.
Why am I on beta ? because I took the boot files from github ?
i do have beta firmware however for usb boot, can i use raspios original kernel to support that?
-
trejan
- Posts: 5371
- Joined: Tue Jul 02, 2019 2:28 pm
Re: Docker not starting, missing overlay2 module
Sun May 31, 2020 1:18 pm
Chura wrote: ↑
Sun May 31, 2020 1:13 pm
Why am I on beta ? I downloaded raspios from raspberry site.
i do have beta firmware however for usb boot
You ran rpi-update at some point. The USB boot beta instructions for «Create a bootable USB drive» mention using rpi-update but they say only copy the .elf and .dat files. Did you copy everything?
-
jamesh
- Raspberry Pi Engineer & Forum Moderator
- Posts: 32176
- Joined: Sat Jul 30, 2011 7:41 pm
Re: Docker not starting, missing overlay2 module
Sun May 31, 2020 1:25 pm
Chura wrote: ↑
Sun May 31, 2020 1:13 pm
yeah i just noticed no 5.4 modules at /lib/modules.Why am I on beta ? I downloaded raspios from raspberry site.
i do have beta firmware however for usb boot
If you are on a kenral version 5.4 then its beta software. Are you running the 64bit OS beta?
Principal Software Engineer at Raspberry Pi Ltd.
Working in the Applications Team.
Return to “Beginners”
I installed docker following the official website instructions, yet I get the following (from journalctl -xe):
Sep 12 11:15:16 myhostname dockerd[18504]: time="2021-09-12T11:15:16.614348477Z" level=info msg="Starting up"
Sep 12 11:15:16 myhostname dockerd[18504]: time="2021-09-12T11:15:16.646352167Z" level=info msg="parsed scheme: "unix"" module=grpc
Sep 12 11:15:16 myhostname dockerd[18504]: time="2021-09-12T11:15:16.646395968Z" level=info msg="scheme "unix" not registered, fallback to default scheme" module=grpc
Sep 12 11:15:16 myhostname dockerd[18504]: time="2021-09-12T11:15:16.646438441Z" level=info msg="ccResolverWrapper: sending update to cc: {[{unix:///run/containerd/containerd.sock <nil> 0 <nil>}] <nil> <nil>}" module=grpc
Sep 12 11:15:16 myhostname dockerd[18504]: time="2021-09-12T11:15:16.646482278Z" level=info msg="ClientConn switching balancer to "pick_first"" module=grpc
Sep 12 11:15:16 myhostname dockerd[18504]: time="2021-09-12T11:15:16.648745422Z" level=info msg="parsed scheme: "unix"" module=grpc
Sep 12 11:15:16 myhostname dockerd[18504]: time="2021-09-12T11:15:16.648780339Z" level=info msg="scheme "unix" not registered, fallback to default scheme" module=grpc
Sep 12 11:15:16 myhostname dockerd[18504]: time="2021-09-12T11:15:16.648824261Z" level=info msg="ccResolverWrapper: sending update to cc: {[{unix:///run/containerd/containerd.sock <nil> 0 <nil>}] <nil> <nil>}" module=grpc
Sep 12 11:15:16 myhostname dockerd[18504]: time="2021-09-12T11:15:16.648847918Z" level=info msg="ClientConn switching balancer to "pick_first"" module=grpc
Sep 12 11:15:16 myhostname dockerd[18504]: time="2021-09-12T11:15:16.650640781Z" level=error msg="failed to mount overlay: no such device" storage-driver=overlay2
Sep 12 11:15:16 myhostname dockerd[18504]: time="2021-09-12T11:15:16.652619063Z" level=error msg="AUFS was not found in /proc/filesystems" storage-driver=aufs
Sep 12 11:15:16 myhostname dockerd[18504]: time="2021-09-12T11:15:16.653285845Z" level=error msg="failed to mount overlay: no such device" storage-driver=overlay
Sep 12 11:15:16 myhostname dockerd[18504]: time="2021-09-12T11:15:16.678262705Z" level=warning msg="Your kernel does not support oom control"
Sep 12 11:15:16 myhostname dockerd[18504]: time="2021-09-12T11:15:16.678323247Z" level=warning msg="Your kernel does not support memory swappiness"
Sep 12 11:15:16 myhostname dockerd[18504]: time="2021-09-12T11:15:16.678339101Z" level=warning msg="Your kernel does not support kernel memory limit"
Sep 12 11:15:16 myhostname dockerd[18504]: time="2021-09-12T11:15:16.678352245Z" level=warning msg="Your kernel does not support kernel memory TCP limit"
Sep 12 11:15:16 myhostname dockerd[18504]: time="2021-09-12T11:15:16.678365163Z" level=warning msg="Unable to find cpu cgroup in mounts"
Sep 12 11:15:16 myhostname dockerd[18504]: time="2021-09-12T11:15:16.678378018Z" level=warning msg="Unable to find cpuset cgroup in mounts"
Sep 12 11:15:16 myhostname dockerd[18504]: time="2021-09-12T11:15:16.678391295Z" level=warning msg="Unable to find pids cgroup in mounts"
Sep 12 11:15:16 myhostname dockerd[18504]: failed to start daemon: Devices cgroup isn't mounted
System Information:
OS: Centos 7
Kernel Version: 3.10
The output of cat /etc/os-release is:
NAME="CentOS Linux"
VERSION="7 (Core)"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="7"
PRETTY_NAME="CentOS Linux 7 (Core)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:centos:centos:7"
HOME_URL="https://www.centos.org/"
BUG_REPORT_URL="https://bugs.centos.org/"
CENTOS_MANTISBT_PROJECT="CentOS-7"
CENTOS_MANTISBT_PROJECT_VERSION="7"
REDHAT_SUPPORT_PRODUCT="centos"
REDHAT_SUPPORT_PRODUCT_VERSION="7"
The output of uname -r is:
3.10.0-042stab141.3
While trying a rootless docker on my servers, I found a lot of issues. They reccomend to use an Ubuntu kernel, but I use CentOS 7.X. So I need to stick with it. The prerequisites are fine – I have newuidmap and newgidmap and enough subordinates. This is how it looks like when I run the rootless setup script as an user user.
user@server ~ $ > dockerd-rootless-setuptool.sh install
[ERROR] Missing system requirements.
[ERROR] Run the following commands to
[ERROR] install the requirements and run this tool again.########## BEGIN ##########
sudo sh -eux <<EOF
# Set user.max_user_namespaces
cat <<EOT > /etc/sysctl.d/51-rootless.conf
user.max_user_namespaces = 28633
EOT
sysctl --system
# Add subuid entry for user
echo "user:100000:65536" >> /etc/subuid
# Add subgid entry for user
echo "user:100000:65536" >> /etc/subgid
EOF
########## END ##########
We go as root and cut and copy the thing above between the #. This is the output, edited.
root@server ~ ## > cut-and-copy-of-the-thing-above
+ cat
+ sysctl --system
* Applying /usr/lib/sysctl.d/00-system.conf ...
net.bridge.bridge-nf-call-ip6tables = 0
net.bridge.bridge-nf-call-iptables = 0
net.bridge.bridge-nf-call-arptables = 0
* Applying /usr/lib/sysctl.d/10-default-yama-scope.conf ...
kernel.yama.ptrace_scope = 0
* Applying /usr/lib/sysctl.d/50-default.conf ...
kernel.sysrq = 16
kernel.core_uses_pid = 1
kernel.kptr_restrict = 1
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.all.rp_filter = 1
net.ipv4.conf.default.accept_source_route = 0
net.ipv4.conf.all.accept_source_route = 0
net.ipv4.conf.default.promote_secondaries = 1
net.ipv4.conf.all.promote_secondaries = 1
fs.protected_hardlinks = 1
fs.protected_symlinks = 1
* Applying /etc/sysctl.d/51-rootless.conf ...
user.max_user_namespaces = 28633
* Applying /etc/sysctl.d/99-sysctl.conf ...
* Applying /etc/sysctl.conf ...
+ echo user:100000:65536
+ echo user:100000:65536
root@server ~ ## > ########## END ##########
Time to try again. The result gives no error but is not like in the tutorial. Here you have it.
user@server ~ $ > dockerd-rootless-setuptool.sh install
[INFO] systemd not detected, dockerd-rootless.sh
needs to be started manually:
PATH=/usr/bin:/sbin:/usr/sbin:$PATH dockerd-rootless.sh
[INFO] Creating CLI context "rootless"
Successfully created context "rootless"
[INFO] Make sure the following environment variables
are set (or add them to ~/.bashrc):
export PATH=/usr/bin:$PATH
export DOCKER_HOST=unix:///run/user/3201/docker.sock
user@server ~ $ >
So what does it mean to start it manually? After reading this bug report, I decide to try to run it with the experimental tag and specifying the storage driver. This is my output, as usual, edited for a proper reading. Important messages in blue, comments in cursive, errors in red.
user@server ~ $ > dockerd-rootless.sh
--experimental --storage-driver overlay2
+ case "$1" in
+ '[' -w /run/user/USERID ']'
+ '[' -w /home/user ']'
--> some user-dependent messages...
+ exec dockerd --experimental --storage-driver overlay2
INFO[] Starting up
WARN[] Running experimental build
WARN[] Running in rootless mode.
This mode has feature limitations.
INFO[] Running with RootlessKit integration
...more messages here, loading plugins...
INFO[] skip loading plugin "io.containerd.snapshotter.v1.aufs"...
error="aufs is not supported: skip plugin"
type=io.containerd.snapshotter.v1
INFO[] loading plugin "io.containerd.snapshotter.v1.devmapper"...
type=io.containerd.snapshotter.v1
WARN[] failed to load plugin e
rror="devmapper not configured"
INFO[] loading plugins..
INFO[] skip loading plugin "io.containerd.snapshotter.v1.zfs"...
error="path must be a zfs : skip plugin"
type=io.containerd.snapshotter.v1
WARN[] could not use snapshotter devmapper
in metadata plugin
error="devmapper not configured"
INFO[] metadata content store policy set policy=shared
INFO[] loading a lot of plugins sucessfully...
...more messages here, loading plugins...
INFO[] serving... address=/run/user/USERID/docker/containerd/sockets
INFO[] serving...
INFO[] containerd successfully booted in 0.033234s
WARN[] Could not set may_detach_mounts kernel parameter
error="error opening may_detach_mounts kernel config file:
open /proc/sys/fs/may_detach_mounts: permission denied"
INFO[] parsed scheme: "unix" module=grpc
...more messages here...
INFO[] ClientConn switching balancer to "pick_first" module=grpc
ERRO[] failed to mount overlay:
operation not permitted storage-driver=overlay2
INFO[] stopping event stream following graceful shutdown
error="context canceled"
module=libcontainerd namespace=plugins.moby
failed to start daemon:
error initializing graphdriver: driver not supported
[rootlesskit:child ] error:
command [/usr/bin/dockerd-rootless.sh
--experimental --storage-driver overlay2]
exited: exit status 1
[rootlesskit:parent] error: child exited: exit status 1
What do I get from the above run? There are warnings on zfs, aufs, and finally overlay2 so it looks like there’s some kind of problem with the storage driver. You can get also a dark failed to register layer message or an error creating overlay mount. It makes sense, since I’m coming from a fully working root install. I try once more without the storage driver option, and this is the (interesting part of) the output.
ERRO[] failed to mount overlay:
operation not permitted storage-driver=overlay2
ERRO[] AUFS cannot be used in non-init user namespace
storage-driver=aufs
ERRO[] failed to mount overlay: operation not permitted
storage-driver=overlay
INFO[] Attempting next endpoint for pull after error:
failed to register layer:
ApplyLayer exit status 1 stdout:
stderr: open /root/.bash_logout: permission denied
So if you don’t give a storage option, it tries them all. Mystery solved, I guess. You can have a look to the available overlayfs documentation (covering overlay and overlay2). But in short, the docker daemon running under user doesn’t manage to access to the storage drivers. Let’s have a look onto the docker storage options. Some documentation first. We know how to change the directory to store containers and images in Docker. In my CentOS 7.X, I see my daemon runs overlay2 and indeed the images after dowloaded are stored on /var/lib/docker/overlay2. I can change the docker image installation directory by editing /etc/docker/daemon.json. I add something like this.
{
"data-root": "/extrahd/docker",
"storage-driver": "overlay2"
}
I then clean up by docker system prune -a and restart my docker daemon still as root to be sure the newly downloaded images end up on /extrahd/docker. As expected. 😉. In principle the given location cannot be a GPFS or a CIFS mounted folder, or I end up getting all the driver errors.
Will this work for on rootless mode? In my case,it was not possible until I did the same trick as for root. So one needs to configure the user docker daemon. For my user user, it should be located on
/home/user/.config/docker/daemon.json
Remember, of course, that the storage must be writable by the user user 😉😉. I hope now you can run docker rootless on CentOS 7.X as I can! 🤘😊.
Bonus: a docker storage using a bind mount (not tested) and how to control the docker with systemd, and the full docker daemon configuration file documentation.