revlog: break up compression of large deltas
Python's zlib apparently makes an internal copy of strings passed to
compress(). To avoid this, compress strings 1M at a time, then join
them at the end if the result would be smaller than the original.
For initial commits of large but compressible files, this cuts peak
memory usage nearly in half.
#!/bin/sh
#
# Corrupt an hg repo with two pulls.
#
# create one repo with a long history
hg init source1
cd source1
touch foo
hg add foo
for i in 1 2 3 4 5 6 7 8 9 10; do
echo $i >> foo
hg ci -m $i
done
cd ..
# create a third repo to pull both other repos into it
hg init version2
hg -R version2 pull source1 &
sleep 1
hg clone --pull -U version2 corrupted
wait
hg -R corrupted verify
hg -R version2 verify