LXC cluster: failing to start the first node after reboot Unicorn Meta Zoo #1: Why another...
Are there existing rules/lore for MTG planeswalkers?
A journey... into the MIND
What to do with someone that cheated their way though university and a PhD program?
Is there a verb for listening stealthily?
What's called a person who works as someone who puts products on shelves in stores?
In search of the origins of term censor, I hit a dead end stuck with the greek term, to censor, λογοκρίνω
What *exactly* is electrical current, voltage, and resistance?
What is a 'Key' in computer science?
Is it accepted to use working hours to read general interest books?
Why is arima in R one time step off?
Is a self contained air-bullet cartridge feasible?
Philosophers who were composers?
France's Public Holidays' Puzzle
Does using the Inspiration rules for character defects encourage My Guy Syndrome?
Will I be more secure with my own router behind my ISP's router?
Marquee sign letters
Did war bonds have better investment alternatives during WWII?
Raising a bilingual kid. When should we introduce the majority language?
How do I deal with an erroneously large refund?
Is it OK if I do not take the receipt in Germany?
What is the purpose of the side handle on a hand ("eggbeater") drill?
How can I wire a 9-position switch so that each position turns on one more LED than the one before?
Why do people think Winterfell crypts is the safest place for women, children & old people?
Will I lose my paid in full property
LXC cluster: failing to start the first node after reboot
Unicorn Meta Zoo #1: Why another podcast?
Announcing the arrival of Valued Associate #679: Cesar ManaraPractical number of concurrent LXC limit on one nodelxc-start: failed to spawnunable to start lxc containerlxc 2.0.0 failed to start after upgradingError while mounting root in node of a clusterservices failing to start - control process exited with error codeWacom crashes computer after 18.04 upgradeWhy does Apparmor fail after I install MySQL on 18.04.1 LTS?MariaDB 10.3.3 Galera cluster adding new node in existing cluster # WSREP: Waiting for SST to completelxc-start: controller: lxccontainer.c: wait_on_daemonized_start: 842 Received container state “STOPPING” instead of “RUNNING”
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty{ margin-bottom:0;
}
I have a LXC cluster with three nodes that are all KVM virtual machines with Kubuntu 18.04 with LXD 3.0.3 installed via apt
Also the host machine is a Kubuntu 18.04
I installed LXD on the first node (kvmnode1) and then I joined to the cluster the other two nodes (kvmnode2, kvmnode3)
Then I verified that starting the nodes from the first to the last (waiting for every node to complete the startup) and closing the nodes from the last to the first always gives a fully oerational cluster
Today I forgot to wait for the complete startup of every node and I ended up with a cluster with the first node non starting LXD end the other 2 nodes working correctly
On the failing first node I see
sysop@kvmnode1:~/Scaricati$ sudo systemctl status lxd
[sudo] password di sysop:
● lxd.service - LXD - main daemon
Loaded: loaded (/lib/systemd/system/lxd.service; indirect; vendor preset: enabled)
Active: activating (start-post) (Result: exit-code) since Mon 2019-04-22 10:36:10 CEST; 1min 47s ago
Docs: man:lxd(1)
Process: 1592 ExecStart=/usr/bin/lxd --group lxd --logfile=/var/log/lxd/lxd.log (code=exited, status=1/FAILURE)
Process: 1560 ExecStartPre=/usr/lib/x86_64-linux-gnu/lxc/lxc-apparmor-load (code=exited, status=0/SUCCESS)
Main PID: 1592 (code=exited, status=1/FAILURE); Control PID: 1593 (lxd)
Tasks: 8
CGroup: /system.slice/lxd.service
└─1593 /usr/lib/lxd/lxd waitready --timeout=600
apr 22 10:36:10 kvmnode1 systemd[1]: Starting LXD - main daemon...
apr 22 10:36:10 kvmnode1 lxd[1592]: t=2019-04-22T10:36:10+0200 lvl=warn msg="CGroup memory swap accounting is disabled, swap limits will be ignored."
apr 22 10:36:19 kvmnode1 lxd[1592]: t=2019-04-22T10:36:19+0200 lvl=eror msg="Failed to start the daemon: failed to open cluster database: failed to ensure schema: failed to update node version info: upda
apr 22 10:36:19 kvmnode1 lxd[1592]: Error: failed to open cluster database: failed to ensure schema: failed to update node version info: updated 0 rows instead of 1
apr 22 10:36:19 kvmnode1 systemd[1]: lxd.service: Main process exited, code=exited, status=1/FAILURE
sysop@kvmnode1:~/Scaricati$
And on the second node I see
sysop@kvmnode2:~$ lxc cluster list
+----------+-----------------------------+----------+---------+----------------------------------------+
| NAME | URL | DATABASE | STATE | MESSAGE |
+----------+-----------------------------+----------+---------+----------------------------------------+
| kvmnode1 | https://192.168.201.11:8443 | YES | OFFLINE | no heartbeat since 134h36m2.926365228s |
+----------+-----------------------------+----------+---------+----------------------------------------+
| kvmnode2 | https://192.168.201.12:8443 | YES | ONLINE | fully operational |
+----------+-----------------------------+----------+---------+----------------------------------------+
| kvmnode3 | https://192.168.201.13:8443 | YES | ONLINE | fully operational |
+----------+-----------------------------+----------+---------+----------------------------------------+
sysop@kvmnode2:~$
What can I do to have kvmnode1 starting correctly and join the cluster?
P.S. I tried to start/stop the three nodes in every sequence but I always end in the same status
UPPDATE
I tried to start lxd in debug mode and it report the same failure
sysop@kvmnode1:~$ sudo lxd --debug --group lxd
DBUG[04-22|12:19:42] Connecting to a local LXD over a Unix socket
DBUG[04-22|12:19:42] Sending request to LXD method=GET url=http://unix.socket/1.0 etag=
INFO[04-22|12:19:42] LXD 3.0.3 is starting in normal mode path=/var/lib/lxd
INFO[04-22|12:19:42] Kernel uid/gid map:
INFO[04-22|12:19:42] - u 0 0 4294967295
INFO[04-22|12:19:42] - g 0 0 4294967295
INFO[04-22|12:19:42] Configured LXD uid/gid map:
INFO[04-22|12:19:42] - u 0 165536 65536
INFO[04-22|12:19:42] - g 0 165536 65536
WARN[04-22|12:19:42] CGroup memory swap accounting is disabled, swap limits will be ignored.
INFO[04-22|12:19:42] Kernel features:
INFO[04-22|12:19:42] - netnsid-based network retrieval: no
INFO[04-22|12:19:42] - unprivileged file capabilities: yes
INFO[04-22|12:19:42] Initializing local database
DBUG[04-22|12:19:42] Initializing database gateway
DBUG[04-22|12:19:42] Connecting to a local LXD over a Unix socket
DBUG[04-22|12:19:42] Sending request to LXD method=GET url=http://unix.socket/1.0 etag=
DBUG[04-22|12:19:42] Detected stale unix socket, deleting
INFO[04-22|12:19:42] Starting /dev/lxd handler:
INFO[04-22|12:19:42] - binding devlxd socket socket=/var/lib/lxd/devlxd/sock
INFO[04-22|12:19:42] REST API daemon:
INFO[04-22|12:19:42] - binding Unix socket socket=/var/lib/lxd/unix.socket
INFO[04-22|12:19:42] - binding TCP socket socket=[::]:8443
INFO[04-22|12:19:42] Initializing global database
DBUG[04-22|12:19:42] Found cert k=0
DBUG[04-22|12:19:42] Dqlite: server connection failed err=failed to establish network connection: some nodes are behind this node's version address=192.168.201.11:8443 attempt=0
DBUG[04-22|12:19:42] Dqlite: connected address=192.168.201.12:8443 attempt=0
DBUG[04-22|12:19:42] Database error: failed to update node version info: updated 0 rows instead of 1
EROR[04-22|12:19:42] Failed to start the daemon: failed to open cluster database: failed to ensure schema: failed to update node version info: updated 0 rows instead of 1
INFO[04-22|12:19:42] Starting shutdown sequence
INFO[04-22|12:19:42] Stopping REST API handler:
INFO[04-22|12:19:42] - closing socket socket=[::]:8443
INFO[04-22|12:19:42] - closing socket socket=/var/lib/lxd/unix.socket
INFO[04-22|12:19:42] Stopping /dev/lxd handler
INFO[04-22|12:19:42] - closing socket socket=/var/lib/lxd/devlxd/sock
DBUG[04-22|12:19:42] Stop database gateway
INFO[04-22|12:19:42] Stopping REST API handler:
INFO[04-22|12:19:42] Stopping /dev/lxd handler
INFO[04-22|12:19:42] Stopping REST API handler:
INFO[04-22|12:19:42] Stopping /dev/lxd handler
DBUG[04-22|12:19:42] Not unmounting temporary filesystems (containers are still running)
INFO[04-22|12:19:42] Saving simplestreams cache
INFO[04-22|12:19:42] Saved simplestreams cache
Error: failed to open cluster database: failed to ensure schema: failed to update node version info: updated 0 rows instead of 1
sysop@kvmnode1:~$
Any hint?
18.04 lxc cluster
asked yesterday
Mirto BusicoMirto Busico
1971620
add a comment |
I have a LXC cluster with three nodes that are all KVM virtual machines with Kubuntu 18.04 with LXD 3.0.3 installed via apt
Also the host machine is a Kubuntu 18.04
I installed LXD on the first node (kvmnode1) and then I joined to the cluster the other two nodes (kvmnode2, kvmnode3)
Then I verified that starting the nodes from the first to the last (waiting for every node to complete the startup) and closing the nodes from the last to the first always gives a fully oerational cluster
Today I forgot to wait for the complete startup of every node and I ended up with a cluster with the first node non starting LXD end the other 2 nodes working correctly
On the failing first node I see
sysop@kvmnode1:~/Scaricati$ sudo systemctl status lxd
[sudo] password di sysop:
● lxd.service - LXD - main daemon
Loaded: loaded (/lib/systemd/system/lxd.service; indirect; vendor preset: enabled)
Active: activating (start-post) (Result: exit-code) since Mon 2019-04-22 10:36:10 CEST; 1min 47s ago
Docs: man:lxd(1)
Process: 1592 ExecStart=/usr/bin/lxd --group lxd --logfile=/var/log/lxd/lxd.log (code=exited, status=1/FAILURE)
Process: 1560 ExecStartPre=/usr/lib/x86_64-linux-gnu/lxc/lxc-apparmor-load (code=exited, status=0/SUCCESS)
Main PID: 1592 (code=exited, status=1/FAILURE); Control PID: 1593 (lxd)
Tasks: 8
CGroup: /system.slice/lxd.service
└─1593 /usr/lib/lxd/lxd waitready --timeout=600
apr 22 10:36:10 kvmnode1 systemd[1]: Starting LXD - main daemon...
apr 22 10:36:10 kvmnode1 lxd[1592]: t=2019-04-22T10:36:10+0200 lvl=warn msg="CGroup memory swap accounting is disabled, swap limits will be ignored."
apr 22 10:36:19 kvmnode1 lxd[1592]: t=2019-04-22T10:36:19+0200 lvl=eror msg="Failed to start the daemon: failed to open cluster database: failed to ensure schema: failed to update node version info: upda
apr 22 10:36:19 kvmnode1 lxd[1592]: Error: failed to open cluster database: failed to ensure schema: failed to update node version info: updated 0 rows instead of 1
apr 22 10:36:19 kvmnode1 systemd[1]: lxd.service: Main process exited, code=exited, status=1/FAILURE
sysop@kvmnode1:~/Scaricati$
And on the second node I see
sysop@kvmnode2:~$ lxc cluster list
+----------+-----------------------------+----------+---------+----------------------------------------+
| NAME | URL | DATABASE | STATE | MESSAGE |
+----------+-----------------------------+----------+---------+----------------------------------------+
| kvmnode1 | https://192.168.201.11:8443 | YES | OFFLINE | no heartbeat since 134h36m2.926365228s |
+----------+-----------------------------+----------+---------+----------------------------------------+
| kvmnode2 | https://192.168.201.12:8443 | YES | ONLINE | fully operational |
+----------+-----------------------------+----------+---------+----------------------------------------+
| kvmnode3 | https://192.168.201.13:8443 | YES | ONLINE | fully operational |
+----------+-----------------------------+----------+---------+----------------------------------------+
sysop@kvmnode2:~$
What can I do to have kvmnode1 starting correctly and join the cluster?
P.S. I tried to start/stop the three nodes in every sequence but I always end in the same status
UPPDATE
I tried to start lxd in debug mode and it report the same failure
sysop@kvmnode1:~$ sudo lxd --debug --group lxd
DBUG[04-22|12:19:42] Connecting to a local LXD over a Unix socket
DBUG[04-22|12:19:42] Sending request to LXD method=GET url=http://unix.socket/1.0 etag=
INFO[04-22|12:19:42] LXD 3.0.3 is starting in normal mode path=/var/lib/lxd
INFO[04-22|12:19:42] Kernel uid/gid map:
INFO[04-22|12:19:42] - u 0 0 4294967295
INFO[04-22|12:19:42] - g 0 0 4294967295
INFO[04-22|12:19:42] Configured LXD uid/gid map:
INFO[04-22|12:19:42] - u 0 165536 65536
INFO[04-22|12:19:42] - g 0 165536 65536
WARN[04-22|12:19:42] CGroup memory swap accounting is disabled, swap limits will be ignored.
INFO[04-22|12:19:42] Kernel features:
INFO[04-22|12:19:42] - netnsid-based network retrieval: no
INFO[04-22|12:19:42] - unprivileged file capabilities: yes
INFO[04-22|12:19:42] Initializing local database
DBUG[04-22|12:19:42] Initializing database gateway
DBUG[04-22|12:19:42] Connecting to a local LXD over a Unix socket
DBUG[04-22|12:19:42] Sending request to LXD method=GET url=http://unix.socket/1.0 etag=
DBUG[04-22|12:19:42] Detected stale unix socket, deleting
INFO[04-22|12:19:42] Starting /dev/lxd handler:
INFO[04-22|12:19:42] - binding devlxd socket socket=/var/lib/lxd/devlxd/sock
INFO[04-22|12:19:42] REST API daemon:
INFO[04-22|12:19:42] - binding Unix socket socket=/var/lib/lxd/unix.socket
INFO[04-22|12:19:42] - binding TCP socket socket=[::]:8443
INFO[04-22|12:19:42] Initializing global database
DBUG[04-22|12:19:42] Found cert k=0
DBUG[04-22|12:19:42] Dqlite: server connection failed err=failed to establish network connection: some nodes are behind this node's version address=192.168.201.11:8443 attempt=0
DBUG[04-22|12:19:42] Dqlite: connected address=192.168.201.12:8443 attempt=0
DBUG[04-22|12:19:42] Database error: failed to update node version info: updated 0 rows instead of 1
EROR[04-22|12:19:42] Failed to start the daemon: failed to open cluster database: failed to ensure schema: failed to update node version info: updated 0 rows instead of 1
INFO[04-22|12:19:42] Starting shutdown sequence
INFO[04-22|12:19:42] Stopping REST API handler:
INFO[04-22|12:19:42] - closing socket socket=[::]:8443
INFO[04-22|12:19:42] - closing socket socket=/var/lib/lxd/unix.socket
INFO[04-22|12:19:42] Stopping /dev/lxd handler
INFO[04-22|12:19:42] - closing socket socket=/var/lib/lxd/devlxd/sock
DBUG[04-22|12:19:42] Stop database gateway
INFO[04-22|12:19:42] Stopping REST API handler:
INFO[04-22|12:19:42] Stopping /dev/lxd handler
INFO[04-22|12:19:42] Stopping REST API handler:
INFO[04-22|12:19:42] Stopping /dev/lxd handler
DBUG[04-22|12:19:42] Not unmounting temporary filesystems (containers are still running)
INFO[04-22|12:19:42] Saving simplestreams cache
INFO[04-22|12:19:42] Saved simplestreams cache
Error: failed to open cluster database: failed to ensure schema: failed to update node version info: updated 0 rows instead of 1
sysop@kvmnode1:~$
Any hint?
18.04 lxc cluster
asked yesterday
Mirto BusicoMirto Busico
1971620
add a comment |
I have a LXC cluster with three nodes that are all KVM virtual machines with Kubuntu 18.04 with LXD 3.0.3 installed via apt
Also the host machine is a Kubuntu 18.04
I installed LXD on the first node (kvmnode1) and then I joined to the cluster the other two nodes (kvmnode2, kvmnode3)
Then I verified that starting the nodes from the first to the last (waiting for every node to complete the startup) and closing the nodes from the last to the first always gives a fully oerational cluster
Today I forgot to wait for the complete startup of every node and I ended up with a cluster with the first node non starting LXD end the other 2 nodes working correctly
On the failing first node I see
sysop@kvmnode1:~/Scaricati$ sudo systemctl status lxd
[sudo] password di sysop:
● lxd.service - LXD - main daemon
Loaded: loaded (/lib/systemd/system/lxd.service; indirect; vendor preset: enabled)
Active: activating (start-post) (Result: exit-code) since Mon 2019-04-22 10:36:10 CEST; 1min 47s ago
Docs: man:lxd(1)
Process: 1592 ExecStart=/usr/bin/lxd --group lxd --logfile=/var/log/lxd/lxd.log (code=exited, status=1/FAILURE)
Process: 1560 ExecStartPre=/usr/lib/x86_64-linux-gnu/lxc/lxc-apparmor-load (code=exited, status=0/SUCCESS)
Main PID: 1592 (code=exited, status=1/FAILURE); Control PID: 1593 (lxd)
Tasks: 8
CGroup: /system.slice/lxd.service
└─1593 /usr/lib/lxd/lxd waitready --timeout=600
apr 22 10:36:10 kvmnode1 systemd[1]: Starting LXD - main daemon...
apr 22 10:36:10 kvmnode1 lxd[1592]: t=2019-04-22T10:36:10+0200 lvl=warn msg="CGroup memory swap accounting is disabled, swap limits will be ignored."
apr 22 10:36:19 kvmnode1 lxd[1592]: t=2019-04-22T10:36:19+0200 lvl=eror msg="Failed to start the daemon: failed to open cluster database: failed to ensure schema: failed to update node version info: upda
apr 22 10:36:19 kvmnode1 lxd[1592]: Error: failed to open cluster database: failed to ensure schema: failed to update node version info: updated 0 rows instead of 1
apr 22 10:36:19 kvmnode1 systemd[1]: lxd.service: Main process exited, code=exited, status=1/FAILURE
sysop@kvmnode1:~/Scaricati$
And on the second node I see
sysop@kvmnode2:~$ lxc cluster list
+----------+-----------------------------+----------+---------+----------------------------------------+
| NAME | URL | DATABASE | STATE | MESSAGE |
+----------+-----------------------------+----------+---------+----------------------------------------+
| kvmnode1 | https://192.168.201.11:8443 | YES | OFFLINE | no heartbeat since 134h36m2.926365228s |
+----------+-----------------------------+----------+---------+----------------------------------------+
| kvmnode2 | https://192.168.201.12:8443 | YES | ONLINE | fully operational |
+----------+-----------------------------+----------+---------+----------------------------------------+
| kvmnode3 | https://192.168.201.13:8443 | YES | ONLINE | fully operational |
+----------+-----------------------------+----------+---------+----------------------------------------+
sysop@kvmnode2:~$
What can I do to have kvmnode1 starting correctly and join the cluster?
P.S. I tried to start/stop the three nodes in every sequence but I always end in the same status
UPPDATE
I tried to start lxd in debug mode and it report the same failure
sysop@kvmnode1:~$ sudo lxd --debug --group lxd
DBUG[04-22|12:19:42] Connecting to a local LXD over a Unix socket
DBUG[04-22|12:19:42] Sending request to LXD method=GET url=http://unix.socket/1.0 etag=
INFO[04-22|12:19:42] LXD 3.0.3 is starting in normal mode path=/var/lib/lxd
INFO[04-22|12:19:42] Kernel uid/gid map:
INFO[04-22|12:19:42] - u 0 0 4294967295
INFO[04-22|12:19:42] - g 0 0 4294967295
INFO[04-22|12:19:42] Configured LXD uid/gid map:
INFO[04-22|12:19:42] - u 0 165536 65536
INFO[04-22|12:19:42] - g 0 165536 65536
WARN[04-22|12:19:42] CGroup memory swap accounting is disabled, swap limits will be ignored.
INFO[04-22|12:19:42] Kernel features:
INFO[04-22|12:19:42] - netnsid-based network retrieval: no
INFO[04-22|12:19:42] - unprivileged file capabilities: yes
INFO[04-22|12:19:42] Initializing local database
DBUG[04-22|12:19:42] Initializing database gateway
DBUG[04-22|12:19:42] Connecting to a local LXD over a Unix socket
DBUG[04-22|12:19:42] Sending request to LXD method=GET url=http://unix.socket/1.0 etag=
DBUG[04-22|12:19:42] Detected stale unix socket, deleting
INFO[04-22|12:19:42] Starting /dev/lxd handler:
INFO[04-22|12:19:42] - binding devlxd socket socket=/var/lib/lxd/devlxd/sock
INFO[04-22|12:19:42] REST API daemon:
INFO[04-22|12:19:42] - binding Unix socket socket=/var/lib/lxd/unix.socket
INFO[04-22|12:19:42] - binding TCP socket socket=[::]:8443
INFO[04-22|12:19:42] Initializing global database
DBUG[04-22|12:19:42] Found cert k=0
DBUG[04-22|12:19:42] Dqlite: server connection failed err=failed to establish network connection: some nodes are behind this node's version address=192.168.201.11:8443 attempt=0
DBUG[04-22|12:19:42] Dqlite: connected address=192.168.201.12:8443 attempt=0
DBUG[04-22|12:19:42] Database error: failed to update node version info: updated 0 rows instead of 1
EROR[04-22|12:19:42] Failed to start the daemon: failed to open cluster database: failed to ensure schema: failed to update node version info: updated 0 rows instead of 1
INFO[04-22|12:19:42] Starting shutdown sequence
INFO[04-22|12:19:42] Stopping REST API handler:
INFO[04-22|12:19:42] - closing socket socket=[::]:8443
INFO[04-22|12:19:42] - closing socket socket=/var/lib/lxd/unix.socket
INFO[04-22|12:19:42] Stopping /dev/lxd handler
INFO[04-22|12:19:42] - closing socket socket=/var/lib/lxd/devlxd/sock
DBUG[04-22|12:19:42] Stop database gateway
INFO[04-22|12:19:42] Stopping REST API handler:
INFO[04-22|12:19:42] Stopping /dev/lxd handler
INFO[04-22|12:19:42] Stopping REST API handler:
INFO[04-22|12:19:42] Stopping /dev/lxd handler
DBUG[04-22|12:19:42] Not unmounting temporary filesystems (containers are still running)
INFO[04-22|12:19:42] Saving simplestreams cache
INFO[04-22|12:19:42] Saved simplestreams cache
Error: failed to open cluster database: failed to ensure schema: failed to update node version info: updated 0 rows instead of 1
sysop@kvmnode1:~$
Any hint?
18.04 lxc cluster
asked yesterday
Mirto BusicoMirto Busico
1971620
I have a LXC cluster with three nodes that are all KVM virtual machines with Kubuntu 18.04 with LXD 3.0.3 installed via apt
Also the host machine is a Kubuntu 18.04
I installed LXD on the first node (kvmnode1) and then I joined to the cluster the other two nodes (kvmnode2, kvmnode3)
Then I verified that starting the nodes from the first to the last (waiting for every node to complete the startup) and closing the nodes from the last to the first always gives a fully oerational cluster
Today I forgot to wait for the complete startup of every node and I ended up with a cluster with the first node non starting LXD end the other 2 nodes working correctly
On the failing first node I see
sysop@kvmnode1:~/Scaricati$ sudo systemctl status lxd
[sudo] password di sysop:
● lxd.service - LXD - main daemon
Loaded: loaded (/lib/systemd/system/lxd.service; indirect; vendor preset: enabled)
Active: activating (start-post) (Result: exit-code) since Mon 2019-04-22 10:36:10 CEST; 1min 47s ago
Docs: man:lxd(1)
Process: 1592 ExecStart=/usr/bin/lxd --group lxd --logfile=/var/log/lxd/lxd.log (code=exited, status=1/FAILURE)
Process: 1560 ExecStartPre=/usr/lib/x86_64-linux-gnu/lxc/lxc-apparmor-load (code=exited, status=0/SUCCESS)
Main PID: 1592 (code=exited, status=1/FAILURE); Control PID: 1593 (lxd)
Tasks: 8
CGroup: /system.slice/lxd.service
└─1593 /usr/lib/lxd/lxd waitready --timeout=600
apr 22 10:36:10 kvmnode1 systemd[1]: Starting LXD - main daemon...
apr 22 10:36:10 kvmnode1 lxd[1592]: t=2019-04-22T10:36:10+0200 lvl=warn msg="CGroup memory swap accounting is disabled, swap limits will be ignored."
apr 22 10:36:19 kvmnode1 lxd[1592]: t=2019-04-22T10:36:19+0200 lvl=eror msg="Failed to start the daemon: failed to open cluster database: failed to ensure schema: failed to update node version info: upda
apr 22 10:36:19 kvmnode1 lxd[1592]: Error: failed to open cluster database: failed to ensure schema: failed to update node version info: updated 0 rows instead of 1
apr 22 10:36:19 kvmnode1 systemd[1]: lxd.service: Main process exited, code=exited, status=1/FAILURE
sysop@kvmnode1:~/Scaricati$
And on the second node I see
sysop@kvmnode2:~$ lxc cluster list
+----------+-----------------------------+----------+---------+----------------------------------------+
| NAME | URL | DATABASE | STATE | MESSAGE |
+----------+-----------------------------+----------+---------+----------------------------------------+
| kvmnode1 | https://192.168.201.11:8443 | YES | OFFLINE | no heartbeat since 134h36m2.926365228s |
+----------+-----------------------------+----------+---------+----------------------------------------+
| kvmnode2 | https://192.168.201.12:8443 | YES | ONLINE | fully operational |
+----------+-----------------------------+----------+---------+----------------------------------------+
| kvmnode3 | https://192.168.201.13:8443 | YES | ONLINE | fully operational |
+----------+-----------------------------+----------+---------+----------------------------------------+
sysop@kvmnode2:~$
What can I do to have kvmnode1 starting correctly and join the cluster?
P.S. I tried to start/stop the three nodes in every sequence but I always end in the same status
UPPDATE
I tried to start lxd in debug mode and it report the same failure
sysop@kvmnode1:~$ sudo lxd --debug --group lxd
DBUG[04-22|12:19:42] Connecting to a local LXD over a Unix socket
DBUG[04-22|12:19:42] Sending request to LXD method=GET url=http://unix.socket/1.0 etag=
INFO[04-22|12:19:42] LXD 3.0.3 is starting in normal mode path=/var/lib/lxd
INFO[04-22|12:19:42] Kernel uid/gid map:
INFO[04-22|12:19:42] - u 0 0 4294967295
INFO[04-22|12:19:42] - g 0 0 4294967295
INFO[04-22|12:19:42] Configured LXD uid/gid map:
INFO[04-22|12:19:42] - u 0 165536 65536
INFO[04-22|12:19:42] - g 0 165536 65536
WARN[04-22|12:19:42] CGroup memory swap accounting is disabled, swap limits will be ignored.
INFO[04-22|12:19:42] Kernel features:
INFO[04-22|12:19:42] - netnsid-based network retrieval: no
INFO[04-22|12:19:42] - unprivileged file capabilities: yes
INFO[04-22|12:19:42] Initializing local database
DBUG[04-22|12:19:42] Initializing database gateway
DBUG[04-22|12:19:42] Connecting to a local LXD over a Unix socket
DBUG[04-22|12:19:42] Sending request to LXD method=GET url=http://unix.socket/1.0 etag=
DBUG[04-22|12:19:42] Detected stale unix socket, deleting
INFO[04-22|12:19:42] Starting /dev/lxd handler:
INFO[04-22|12:19:42] - binding devlxd socket socket=/var/lib/lxd/devlxd/sock
INFO[04-22|12:19:42] REST API daemon:
INFO[04-22|12:19:42] - binding Unix socket socket=/var/lib/lxd/unix.socket
INFO[04-22|12:19:42] - binding TCP socket socket=[::]:8443
INFO[04-22|12:19:42] Initializing global database
DBUG[04-22|12:19:42] Found cert k=0
DBUG[04-22|12:19:42] Dqlite: server connection failed err=failed to establish network connection: some nodes are behind this node's version address=192.168.201.11:8443 attempt=0
DBUG[04-22|12:19:42] Dqlite: connected address=192.168.201.12:8443 attempt=0
DBUG[04-22|12:19:42] Database error: failed to update node version info: updated 0 rows instead of 1
EROR[04-22|12:19:42] Failed to start the daemon: failed to open cluster database: failed to ensure schema: failed to update node version info: updated 0 rows instead of 1
INFO[04-22|12:19:42] Starting shutdown sequence
INFO[04-22|12:19:42] Stopping REST API handler:
INFO[04-22|12:19:42] - closing socket socket=[::]:8443
INFO[04-22|12:19:42] - closing socket socket=/var/lib/lxd/unix.socket
INFO[04-22|12:19:42] Stopping /dev/lxd handler
INFO[04-22|12:19:42] - closing socket socket=/var/lib/lxd/devlxd/sock
DBUG[04-22|12:19:42] Stop database gateway
INFO[04-22|12:19:42] Stopping REST API handler:
INFO[04-22|12:19:42] Stopping /dev/lxd handler
INFO[04-22|12:19:42] Stopping REST API handler:
INFO[04-22|12:19:42] Stopping /dev/lxd handler
DBUG[04-22|12:19:42] Not unmounting temporary filesystems (containers are still running)
INFO[04-22|12:19:42] Saving simplestreams cache
INFO[04-22|12:19:42] Saved simplestreams cache
Error: failed to open cluster database: failed to ensure schema: failed to update node version info: updated 0 rows instead of 1
sysop@kvmnode1:~$
Any hint?
18.04 lxc cluster
18.04 lxc cluster
asked yesterday
Mirto BusicoMirto Busico
1971620
asked yesterday
Mirto BusicoMirto Busico
1971620
edited 9 hours ago
Mirto Busico
asked yesterday
Mirto BusicoMirto Busico
1971620
asked yesterday
Mirto BusicoMirto Busico
1971620
asked yesterday
Mirto BusicoMirto Busico
1971620
1971620
add a comment |
add a comment |
0
active
oldest
votes
Your Answer
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "89"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2faskubuntu.com%2fquestions%2f1136012%2flxc-cluster-failing-to-start-the-first-node-after-reboot%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
0
active
oldest
votes
0
active
oldest
votes
active
oldest
votes
active
oldest
votes
Thanks for contributing an answer to Ask Ubuntu!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2faskubuntu.com%2fquestions%2f1136012%2flxc-cluster-failing-to-start-the-first-node-after-reboot%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
