Jeremy Harris [Tue, 28 Feb 2017 18:24:40 +0000 (18:24 +0000)]
Fix child-address counting.
When a new address was created by a routing step it was possible for the parent address in the tree to
be marked as having zero children, despite the new child having a pointer to the parent. When the child
was then delivered, the count on the parent could go negative or, if other children had been added which
correctly incremented the count, arrive at zero while some children were outstanding. Fix this to
maintin the invariant. While there, make the counter unsigned.
Jeremy Harris [Mon, 27 Feb 2017 13:38:53 +0000 (13:38 +0000)]
Testsuite: reduce trailing 0372 subtest to queue-only
Suboptimal, but sufficient to regression-test the reason it was added (a live acl_m_
variable being store_reset over, after reception (and ACLs) and then referenced
later. The problem was, child procs of the daemon were still running as the following
0373 started up, and left unexpected output files there.
Phil Pennock [Sun, 26 Feb 2017 01:18:08 +0000 (20:18 -0500)]
Point at readline fix on macOS
dlopen() without a qualified path searches env vars and paths stamped
into the binary; Exim is usually setuid, so this becomes just the paths
stamped into the binary. On some platforms (*cough* macOS) libreadline
is not in the usual places.
So show how to stamp the Homebrew location of libreadline into the Exim
binary.
Phil Pennock [Sun, 19 Feb 2017 22:27:11 +0000 (17:27 -0500)]
releng: able to use gnupg default keys for signing
Git and our previous "must specify one keyid" approach is more
constraining than GnuPG allows; cleanest and simplest way, without
breaking support for people with multiple keys and such like, is to just
provide a way to break out of our logic and say "use the configured
default GnuPG keys".
My PGP key has multiple signing subkeys, one RSA and one Ed25519; I
think I might try a dual-signature in an upcoming RC to see how many
people scream with broken OpenPGP clients.
Jeremy Harris [Sat, 18 Feb 2017 23:07:14 +0000 (23:07 +0000)]
Testsuite: add option to insert delays, for very slow test platforms
An emulated PPC, running as a VM on x86_64, was seeing testcase fails where
the trailing few log line were missing in munged output, despite on inspection
being present in the spool. Adding a delay before the munge-and-compare
made the issue go away.
util/chunking_fixqueue_finalnewlines.pl walks the queue, fixing any
affected messages; see README.UPDATING.
We're extremely cautious about operation failure.
We do one check without locking messages, so that we can quickly skip
past before trying to lock and contending with an actual delivery. Then
we lock and do another fix.
Note that we use flock, not fcntl, because that's what Perl makes
readily available; we use an OS-guard to barf if the OS is not handled.
Phil Pennock [Mon, 13 Feb 2017 02:22:02 +0000 (21:22 -0500)]
Handle PKG_CONFIG_PATH in Local/Makefile
Handle PKG_CONFIG_PATH, stripping whitespace expanding globs, collecting
multiple sets and just build one variable, and use it in environment at
configure time so that the libraries are found.
Phil Pennock [Sun, 12 Feb 2017 11:52:36 +0000 (06:52 -0500)]
FreeBSD: only assume iconv for FreeBSD >= 10
Since FreeBSD 10 is the oldest version of the OS supported by the
FreeBSD Project, we shouldn't need this. But people are still using
older versions. On closer examination, it's only been 6 weeks since 9.3
stopped being supported. People ignoring the status are playing with
fire, getting no security updates, but let's not make that _our_
problem.
Guard the "use system iconv" #define for the libiconv package with an OS
version #ifdef.
Phil Pennock [Sun, 12 Feb 2017 00:37:04 +0000 (19:37 -0500)]
Unbreak test_dbfn make-target
doc/dbm.discuss.txt describes how to make and use `test_dbfn` for
testing DB functionality.
Commit cf0812d5 adds a call to assert_no_variables into store.c which
depends upon expand.c functionality and we can't link that in for
test_dbfn without pulling in half of Exim.
So adjust the test_dbfn target to rebuild store.o in COMPILE_UTILITY
mode and link against that variant, then remove the custom-built store.o
after the executable has been linked.
* In hash.c, for the OpenSSL case, use assert() to guard the paths which
can't happen, instead of just assuming that the calling code never has
a mistake
* Fix some signed/unsigned issues
* Be explicit about some ignored return values
* Some parens around bit-twiddling
* Use our os_getcwd with its extra guards in one place where getcwd was
called
* FreeBSD: use system iconv, safely, always
Phil Pennock [Thu, 2 Feb 2017 20:38:14 +0000 (15:38 -0500)]
Release packaging & scripting improvements.
* Make the .xz tarball variant too, and work harder on compressing our
files for distribution.
+ The .xz files have gained more positive feedback than any other part
of the 4.89 release.
* Drop usercodes from tarball
+ We shouldn't be embedding own-system-specifc ownership information
into software release tarballs. That's for local system backups,
not distribution.
* Script for the size/checksums
+ We include checksums in the mail; this gets the format fixed and not
including checksums-of-signatures, etc. I've also experimented with
including the size, so let's script that to be portably generated.
* Better tarball signing script
+ Automatically find the signing directory (if not already in it)
+ Sign all files, properly skipping existing .asc files
+ Find the signing key from git config, if available, else error out
(Nigel is not on the hook as the default victim now)
+ Show what we're doing as we do it
All changes made on the original `release_4_89` branch with
`RELEASE EXPERIMENT` subject tags.
Testsuite: tidy generation and sorting of exim -bp output
The root cause is, that exim -bp doesn't always return the message
ids in the order they were created, but sorted. The 2nd
part of the message id (PID) can be random on *BSD.
Phil Pennock [Wed, 1 Feb 2017 17:34:52 +0000 (12:34 -0500)]
bug-fix test-driving input
The client driver is a little restrictive in the escape sequences it
handles; two octets here were missing the `x` after the `\`, so `\05` is
two octets, a 0 and then a 5, where `\x05` would be one octet.
So we were sending two more octets than expected, not catching that Exim
was parsing the wrong IP/port at the end, and now that Exim only reads
as much of the proxy protocol header as belongs in it, instead of "up to
the largest it could be", this test-bug has been exposed.
Phil Pennock [Wed, 1 Feb 2017 03:15:55 +0000 (22:15 -0500)]
Handle Proxy Protocol v2 safely as well.
We had test suite failures (test suite success!) because Proxy Protocol
v2 (PPv2) wasn't being detected; by only reading 12 octets, the >= 16
check was failing. But in fact I had previously only fixed reading
"only enough" for PPv1.
Handling both PPv1 and PPv2 is complicated because the minimum valid
length for PPv1 is 15 octets but for PPv2 the size to read is in the
15th and 16th octets.
So refactored a little and we now use a total of 3 reads for the PPv2
case (assuming no fragmentation, etc; we'll actually keep reading now
instead of aborting) to get the entire PPv2 header of exactly the right
size, so that TLS handshake immediately following the PP header is not
also swallowed.
Fixes: 2018
Tested: manually, TLS and non-TLS, PPv1 and PPv2, all ways.
Release: should be cherry-picked into 4.89RC series
Jeremy Harris [Sun, 29 Jan 2017 18:03:40 +0000 (18:03 +0000)]
CHUNKING: Reject messages with malformed line ending. Bug 2000
Actually test only the first header line, but still do full line-ending canonicalisation on the
remainder of the message in case a Evil Person slips past that.