Phil Pennock [Sun, 12 Feb 2017 00:37:04 +0000 (19:37 -0500)]
Unbreak test_dbfn make-target
doc/dbm.discuss.txt describes how to make and use `test_dbfn` for
testing DB functionality.
Commit cf0812d5 adds a call to assert_no_variables into store.c which
depends upon expand.c functionality and we can't link that in for
test_dbfn without pulling in half of Exim.
So adjust the test_dbfn target to rebuild store.o in COMPILE_UTILITY
mode and link against that variant, then remove the custom-built store.o
after the executable has been linked.
* In hash.c, for the OpenSSL case, use assert() to guard the paths which
can't happen, instead of just assuming that the calling code never has
a mistake
* Fix some signed/unsigned issues
* Be explicit about some ignored return values
* Some parens around bit-twiddling
* Use our os_getcwd with its extra guards in one place where getcwd was
called
* FreeBSD: use system iconv, safely, always
Phil Pennock [Thu, 2 Feb 2017 20:38:14 +0000 (15:38 -0500)]
Release packaging & scripting improvements.
* Make the .xz tarball variant too, and work harder on compressing our
files for distribution.
+ The .xz files have gained more positive feedback than any other part
of the 4.89 release.
* Drop usercodes from tarball
+ We shouldn't be embedding own-system-specifc ownership information
into software release tarballs. That's for local system backups,
not distribution.
* Script for the size/checksums
+ We include checksums in the mail; this gets the format fixed and not
including checksums-of-signatures, etc. I've also experimented with
including the size, so let's script that to be portably generated.
* Better tarball signing script
+ Automatically find the signing directory (if not already in it)
+ Sign all files, properly skipping existing .asc files
+ Find the signing key from git config, if available, else error out
(Nigel is not on the hook as the default victim now)
+ Show what we're doing as we do it
All changes made on the original `release_4_89` branch with
`RELEASE EXPERIMENT` subject tags.
Testsuite: tidy generation and sorting of exim -bp output
The root cause is, that exim -bp doesn't always return the message
ids in the order they were created, but sorted. The 2nd
part of the message id (PID) can be random on *BSD.
Phil Pennock [Wed, 1 Feb 2017 17:34:52 +0000 (12:34 -0500)]
bug-fix test-driving input
The client driver is a little restrictive in the escape sequences it
handles; two octets here were missing the `x` after the `\`, so `\05` is
two octets, a 0 and then a 5, where `\x05` would be one octet.
So we were sending two more octets than expected, not catching that Exim
was parsing the wrong IP/port at the end, and now that Exim only reads
as much of the proxy protocol header as belongs in it, instead of "up to
the largest it could be", this test-bug has been exposed.
Phil Pennock [Wed, 1 Feb 2017 03:15:55 +0000 (22:15 -0500)]
Handle Proxy Protocol v2 safely as well.
We had test suite failures (test suite success!) because Proxy Protocol
v2 (PPv2) wasn't being detected; by only reading 12 octets, the >= 16
check was failing. But in fact I had previously only fixed reading
"only enough" for PPv1.
Handling both PPv1 and PPv2 is complicated because the minimum valid
length for PPv1 is 15 octets but for PPv2 the size to read is in the
15th and 16th octets.
So refactored a little and we now use a total of 3 reads for the PPv2
case (assuming no fragmentation, etc; we'll actually keep reading now
instead of aborting) to get the entire PPv2 header of exactly the right
size, so that TLS handshake immediately following the PP header is not
also swallowed.
Fixes: 2018
Tested: manually, TLS and non-TLS, PPv1 and PPv2, all ways.
Release: should be cherry-picked into 4.89RC series
Jeremy Harris [Sun, 29 Jan 2017 18:03:40 +0000 (18:03 +0000)]
CHUNKING: Reject messages with malformed line ending. Bug 2000
Actually test only the first header line, but still do full line-ending canonicalisation on the
remainder of the message in case a Evil Person slips past that.
Josh Soref [Wed, 18 Jan 2017 23:20:12 +0000 (18:20 -0500)]
spelling: Mavrogiannopoulos
Committer note: the name was spelt as was used by Nikos at the time, but
he's since switched to the other latinization form and is using it
everywhere these days. Part of his response was "Feel free to use the
Mavrogiannopoulos variant everywhere.", so I'm merging this commit too.
Josh Soref [Wed, 18 Jan 2017 17:58:52 +0000 (17:58 +0000)]
replace keept with rotation
Log rotate documentation does not actually give a term for this portion
of a filename, but to the extent that I can find a term, a number of
places call it a "rotation number".
Replacing keept which is inaccurate and misleading with rotation makes
the code a little easier to read.
Phil Pennock [Wed, 18 Jan 2017 16:39:24 +0000 (11:39 -0500)]
Recording merge technique in git history
This is an empty commit which exists for this commit message,
documenting how I handled GitHub PR 52, which was 228 separate commits,
each fixing the spelling of one word. The submitter's approach made it
easy to consider and approve/reject each independently, so was valuable,
but I didn't want so many commits in our history.
A few aspects of the shell commands rely upon Zsh: `read -q` for
getting a single Y/N response; `$IFS` containing ASCII NUL (and
builtins handling NUL inside strings) for parsing `.git/MERGE_RR`;
anonymous function calls so that I could abort cleanly if I wanted to.
git log --pretty=tformat:%h master..github/pr/52 > ../1.consider
touch ../2.keep
for F in $(<../1.consider); do git show $F | cat -v; read -q "keep?Keep $F ? " && echo $F >> ../2.keep; echo; echo =============================; done
That let me iterate through each, selecting 214/228 commits to apply in
one pass. Two PR commits were held for a separate commit, because they
fixed behavioural bugs. So 216/228 were accepted. A couple warranted
minor post-fixing as part of the first PR.
for F in $(<../2.keep ) ; do (){ git cherry-pick -n $F && continue; for junk fn in $(<.git/MERGE_RR); do [[ -n $fn ]] || break; if vi $fn; then git add $fn; else return 1; fi; done } || break; done
vi src/src/filter.c && git add src/src/filter.c
vi src/src/dns.c && git add src/src/dns.c
GIT_AUTHOR_NAME='Josh Soref' GIT_AUTHOR_EMAIL='jsoref@users.noreply.github.com' git commit
and similarly for the second commit.
One more commit from the PR requires chasing with a contributor whose
name is natively in a non-Roman alphabet and who appears to have changed
the Romanisation, to check how they'd like it handled. I will chase
under separate cover.
Phil Pennock [Fri, 13 Jan 2017 04:37:50 +0000 (23:37 -0500)]
Provide alternative Heimdal pkg-config example
Building Exim against Heimdal 1.5, heimdal-gssapi.pc was needed.
There's been a major version bump in Heimdal, and against 7.1 that
doesn't work; using heimdal-krb5.pc fails on missing `gss_*` functions.
I can find no build documentation for Heimdal which describes what
should be needed. heimdal-gssapi.pc does reference heimdal-krb5.pc
in `Requires.private` but it's not being used by FreeBSD pkgconf in
such a way that it's available when building on FreeBSD 10.3.
Fortunately, our `*_PC` logic works with multiple packages listed,
so provide that example.