Cron job to check for changed upstream versions of diverted conffiles

[andersk]

We're currently just hoping that the upstream versions of diverted conffiles never change. It would be safer to have a cron job to check new packages to see when this happens.

[phurst]

A shell script that does this is attached and in /mit/phurst/Public/; it notably breaks on stuff like kexec-tools that contains diverted files that don't actually get generated until installation. I don't have the technical knowledge, time, or particular inclination to fix this at the moment. Currently the sample output is just like md5sum's output: each line is an md5sum followed by a filename.

[geofft]

There are a couple of things that this doesn't account for:

• We should differentiate between diverted/removed files and transformed files. Diverted or removed files need someone to manually determine that the change is okay; transformed files just need a rebuild (unless the transform script fails and the rebuild fails), so that can be automated. Unfortunately, I don't think any approach based on looking at the binary and decoding the diverts-usr++bin++foo Provides is going to be able to differentiate these cases, and I don't think we want binaries to start providing transforms-usr++bin++foo, so we may need to give up on this current approach and start looking at source packages.
• We shouldn't automatically run an invasive unpack of everything as a cronjob on a regular basis -- in the normal case, none of the build-dependency package versions changed, so we can just skip that. I think this optimization is important enough given how many packages we have and how time-consuming a full rebuild is (and this basically amounts to a fraction of a full rebuild).

I'm currently investigating a solution based on adding a c-p-d Makefile target to print out the list of files diverted, transformed, and removed by each source package, and working from there.

[geofft]

I'm currently investigating a solution based on adding a c-p-d Makefile target to print out the list of files diverted, transformed, and removed by each source package, and working from there.

There's some partial work in /mit/geofft/debathena/cpdwatch, notably that e.g. debian/rules -f /mit/geofft/debathena/cpdwatch/list.mk -s list-divert is useful. Untaking this for now, though, since I'm not actively working on it and don't have plans to.

[jdreed]

I thought about this a bit more. The CDBS solution proposed by geofft obviously no longer works with c-p-d 5.x. There's no real good way to do this programmatically, because we'd have to keep track of what we've "approved" (e.g. /etc/foo.conf changes, but not in a way we care about, but we still want to find out if it changes again). As noted, transformations will fail if the upstream file changes (significantly), but nothing will tell us if the transformation is no longer _necessary_. I don't think there's a good way to do that in any automated fashion. We need to read through changelogs and determine if things are still necessary. We've done a reasonably good job of this where upstream has been responsive. I'm tempted to WONTFIX this and just add some documentation.

[achernya]

I'm not sure I fully agree. Although it's true we can't catch all corner cases where a transformation is no longer needed because of a default behavior change, I think it's valuable to detect when a transformation becomes a no-op. Additionally, we support a pile of distributions, which means a transformation is only a no-op if and only if it's a no-op on all platforms.

I would instead like to propose we build a test-suite that unpacks the upstream deb, snapshots it (in git or something) and then applies the c-p-d package, and then complains loudly if the files that were expected to be transformed "weren't", in that it's a no-op. We'd still need to check defaults by-hand.