[PATCH: Domain backup samba-tool command]

[Aaron Haslett via samba-technical]

The exists shell script for backing up a domain doesn't lock things
properly while doing the backup and could end up with a corrupt backup
or cause a lockup.  Here's a new python script that actually works,
along with tests and required fixes.

[Stefan Metzmacher via samba-technical]

Post by Aaron Haslett via samba-technical
The exists shell script for backing up a domain doesn't lock things
properly while doing the backup and could end up with a corrupt backup
or cause a lockup.  Here's a new python script that actually works,
along with tests and required fixes.

I haven't looked into this in detail, but I have a few questions/comments:

- Can you write down in words would the new command is supposed to do?

- The most important part of a backup is always the restore!
I come across a few sites already, which tried to restore
DCs from a VM snapshot and corrupted the replication state.

I think we really need a corresponding restore command
and make it relatively hard to restore the backup without
using the restore command.

The restore command should also do this on the backup databases:
- reset highestCommittedUSN to 1 and invent a new invocationID
that will be used for further replPropertyMetaData stamps
- samba-tool domain demote --remove-other-dead-server for all
servers
- create a new machine account and NTDSDsa object (with the new
invocationID)
- samba-tool fsmo seize for all roles
- change the krbtgt passwords twice
So that the restored domain will never replicate with any existing
DC, as it's only a last resort if really all DCs are broken.

Can you please read through the C related patches and fix
tab vs. whitespaces or missing whitespaces.

Thanks!
metze

[Rowland Penny via samba-technical]

On Fri, 23 Mar 2018 17:59:42 +1300

Post by Aaron Haslett via samba-technical
The exists shell script for backing up a domain doesn't lock things
properly while doing the backup and could end up with a corrupt backup
or cause a lockup.  Here's a new python script that actually works,
along with tests and required fixes.

Before we get carried away here, can I ask a couple of questions ?

Does 'python tarfile' backup acls and xattrs ?

Though the big question is, where does the '-r' option to tdbbackup
come from ?

I am using tdb-tools 1.3.15-1.1~deb9+1 on Devuan ascii and running:

tdbbackup -h

Gets me:

Usage: tdbbackup [options] <fname...>

-h this help message
-s suffix set the backup suffix
-v verify mode (restore if corrupt)
-n hashsize set the new hash size for the backup
-l open without locking to back up mutex dbs

No '-r'

Rowland

[William Brown via samba-technical]

On Fri, 2018-03-23 at 17:59 +1300, Aaron Haslett via samba-technical

Post by Aaron Haslett via samba-technical
The exists shell script for backing up a domain doesn't lock things
properly while doing the backup and could end up with a corrupt backup
or cause a lockup. Here's a new python script that actually works,
along with tests and required fixes.

Hey there,

At a quick glance this looks like a useful feature.

My concern is that you can't assert a backup is succesful unless you
can *restore* from it. So I think if this is to be submitted a proper
restore tool is required as well. The restore tool should indicate the
risks of restoring a DC also (mainly that you have to reset every DC in
the topology to be a new replica from the restore).

I think that a matching restore tool is needed to make sure that your
backup command really does work. Else it's sadly just hopes and dreams.

With this in mind, I don't think I'm willing to give a "review" until a
matching restore tool is added. Remember, backups don't exist unless
they are tested as operational.

Sorry, :(

William

[Andrew Bartlett via samba-technical]

On Mon, 2018-05-14 at 12:50 +1000, William Brown via samba-technical

Post by William Brown via samba-technical
On Fri, 2018-03-23 at 17:59 +1300, Aaron Haslett via samba-technical

Post by Aaron Haslett via samba-technical
The exists shell script for backing up a domain doesn't lock things
properly while doing the backup and could end up with a corrupt backup
or cause a lockup. Here's a new python script that actually works,
along with tests and required fixes.

Hey there,
At a quick glance this looks like a useful feature.
My concern is that you can't assert a backup is succesful unless you
can *restore* from it.

Metze raised essentially the same issue and the restore tool to those
specifications has been built.  

We are now just trying to dribble in the patches one at a time as they
parts become ready and as review resources are available.

Thanks,

Andrew Bartlett

[William Brown via samba-technical]

Post by Andrew Bartlett via samba-technical
On Mon, 2018-05-14 at 12:50 +1000, William Brown via samba-technical

Post by William Brown via samba-technical
On Fri, 2018-03-23 at 17:59 +1300, Aaron Haslett via samba-
technical

Post by Aaron Haslett via samba-technical
The exists shell script for backing up a domain doesn't lock things
properly while doing the backup and could end up with a corrupt backup
or cause a lockup. Here's a new python script that actually works,
along with tests and required fixes.

Hey there,
At a quick glance this looks like a useful feature.
My concern is that you can't assert a backup is succesful unless you
can *restore* from it.

Metze raised essentially the same issue and the restore tool to those
specifications has been built.
We are now just trying to dribble in the patches one at a time as they
parts become ready and as review resources are available.

Yep. I'd be hesitant to provide a "reviewed" until a restore tool
exists too. Can those patches be added to this same set, and be
integrated to the backup tests?

Don't let this detract from the work though, I think it's great to have
a backup/restore tool, and it's awesome work to have achieved this.
Thanks Aaron!

Maybe I missed it but is this added as part of quicktest? I think it's
important to test backup/restore to guarantee it works (we did this
every build of 389 to at least give confidence this was working as a
minimum baseline), and I think that level of paranoia would be healthy
for s4 dc (It would certainly make me as an admin much happier :) )

Thanks again,

William

[Andrew Bartlett via samba-technical]

On Mon, 2018-05-14 at 15:24 +1000, William Brown via samba-technical

Post by William Brown via samba-technical
Maybe I missed it but is this added as part of quicktest? I think it's
important to test backup/restore to guarantee it works (we did this
every build of 389 to at least give confidence this was working as a
minimum baseline), and I think that level of paranoia would be healthy
for s4 dc (It would certainly make me as an admin much happier :) )

quicktest is a bit of a bad joke, and fixing that is beyond the scope
of this task.

The task for quicktest is to identify automatically the largest number
of tests that can run within a 10min window. Those tests then need to
be listed in selftest/quick, replacing the current regular expressions.

Thanks,

Andrew Bartlett