Discussion:
[PATCH] Patch for bug 13414: MSG_SMB_UNLOCK bounces between cleanupds in a cluster
Ralph Böhme via samba-technical
2018-05-02 10:00:43 UTC
Permalink
Hi!

I noticed an issue with messages bouncing between cleanupds in a cluster running
master. Killing a single smbd (with SIGKILL) in a cluster results in
MSG_SMB_UNLOCK messages bouncing between nodes:

Node 0:

[2018/04/30 15:44:57.824893, 1, pid=1646] ../source3/smbd/server.c:881(remove_child_pid)
Scheduled cleanup of brl and lock database after unclean shutdown

...

[2018/04/30 15:45:17.836225, 1, pid=1654] ../source3/smbd/smbd_cleanupd.c:99(smbd_cleanupd_unlock)
smbd_cleanupd_unlock: Cleaning up brl and lock database after unclean shutdown
[2018/04/30 15:45:17.843968, 1, pid=1654] ../source3/smbd/smbd_cleanupd.c:99(smbd_cleanupd_unlock)
smbd_cleanupd_unlock: Cleaning up brl and lock database after unclean shutdown
[2018/04/30 15:45:17.847836, 1, pid=1654] ../source3/smbd/smbd_cleanupd.c:99(smbd_cleanupd_unlock)
smbd_cleanupd_unlock: Cleaning up brl and lock database after unclean shutdown
[2018/04/30 15:45:17.851000, 1, pid=1654] ../source3/smbd/smbd_cleanupd.c:99(smbd_cleanupd_unlock)
smbd_cleanupd_unlock: Cleaning up brl and lock database after unclean shutdown
[2018/04/30 15:45:17.854174, 1, pid=1654] ../source3/smbd/smbd_cleanupd.c:99(smbd_cleanupd_unlock)
smbd_cleanupd_unlock: Cleaning up brl and lock database after unclean shutdown
...

Node 1:

[2018/04/30 15:45:17.837851, 1, pid=19632] ../source3/smbd/smbd_cleanupd.c:99(smbd_cleanupd_unlock)
smbd_cleanupd_unlock: Cleaning up brl and lock database after unclean shutdown
[2018/04/30 15:45:17.844705, 1, pid=19632] ../source3/smbd/smbd_cleanupd.c:99(smbd_cleanupd_unlock)
smbd_cleanupd_unlock: Cleaning up brl and lock database after unclean shutdown
[2018/04/30 15:45:17.848739, 1, pid=19632] ../source3/smbd/smbd_cleanupd.c:99(smbd_cleanupd_unlock)
smbd_cleanupd_unlock: Cleaning up brl and lock database after unclean shutdown
[2018/04/30 15:45:17.853906, 1, pid=19632] ../source3/smbd/smbd_cleanupd.c:99(smbd_cleanupd_unlock)
smbd_cleanupd_unlock: Cleaning up brl and lock database after unclean shutdown
[2018/04/30 15:45:17.859117, 1, pid=19632] ../source3/smbd/smbd_cleanupd.c:99(smbd_cleanupd_unlock)
smbd_cleanupd_unlock: Cleaning up brl and lock database after unclean shutdown
...

This was introduced by the patches around
6423ca4bf293cac5e2f84b1a37bb29b06b5c05ed, because now the messaging send all
primitive broadcasts in the cluster which it didn't before afaict.

The attached patch fixes the issue by simply moving the cleanup trigger
notification from smbd to cleanupd to send a (currently unused)
MSG_SMB_BRL_VALIDATE message type, cf the commit message for details.

Please review carefully and push if ok. Thanks!

-slow
--
Ralph Boehme, Samba Team https://samba.org/
Samba Developer, SerNet GmbH https://sernet.de/en/samba/
GPG Key Fingerprint: FAE2 C608 8A24 2520 51C5
59E4 AA1E 9B71 2639 9E46
Ralph Böhme via samba-technical
2018-05-02 10:59:15 UTC
Permalink
Post by Ralph Böhme via samba-technical
I noticed an issue with messages bouncing between cleanupds in a cluster running
master. Killing a single smbd (with SIGKILL) in a cluster results in
fwiw, I hit this when testing a patch that added the use of ctdb_watch_us() in
my persistent handles branch. To my surprise, after some investigation it turned
out that my patch wasn't at fault. :)

-slow
--
Ralph Boehme, Samba Team https://samba.org/
Samba Developer, SerNet GmbH https://sernet.de/en/samba/
GPG Key Fingerprint: FAE2 C608 8A24 2520 51C5
59E4 AA1E 9B71 2639 9E46
Jeremy Allison via samba-technical
2018-05-03 21:21:05 UTC
Permalink
Post by Ralph Böhme via samba-technical
Post by Ralph Böhme via samba-technical
I noticed an issue with messages bouncing between cleanupds in a cluster running
master. Killing a single smbd (with SIGKILL) in a cluster results in
fwiw, I hit this when testing a patch that added the use of ctdb_watch_us() in
my persistent handles branch. To my surprise, after some investigation it turned
out that my patch wasn't at fault. :)
Really nice catch Ralph, thanks ! RB+.

Loading...