Mercurial > hg > mercurial-crew-with-dirclash

annotate mercurial/appendfile.py @ 2568:52ce0d6bc375

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
HTTPS: fix python2.3, persistent connections, don't explode if SSL is not available The urllib2 differences between python 2.3 and 2.4 are hidden by using keepalive.py, which also gives us support for persistent connections. Support for HTTPS is enabled only if there's a HTTPSHandler class in urllib2. It's not possible to have separate classes as handlers for HTTP and HTTPS: to support persistent HTTPS connections, we need a class that inherits from both keepalive.HTTPHandler and urllib2.HTTPSHandler. If we try to pass (an instance of) this class and (an instance of) the httphandler class to urllib2.build_opener, this function ends up getting confused, since both classes are subclasses of the HTTPHandler default handler, and raises an exception.
author Alexis S. L. Carvalho <alexis@cecm.usp.br>
date Thu, 06 Jul 2006 03:14:55 -0300
parents d7f866789d7e
children abaee83ce0a6
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
1999
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
1 # appendfile.py - special classes to make repo updates atomic
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
2 #
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
3 # Copyright 2006 Vadim Gelfer <vadim.gelfer@gmail.com>
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
4 #
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
5 # This software may be used and distributed according to the terms
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
6 # of the GNU General Public License, incorporated herein by reference.
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
7
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
8 from demandload import *
2176
9b42304d9896 fix file handling bugs on windows.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2102
diff changeset
9 demandload(globals(), "cStringIO changelog errno manifest os tempfile util")
1999
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
10
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
11 # writes to metadata files are ordered. reads: changelog, manifest,
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
12 # normal files. writes: normal files, manifest, changelog.
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
13
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
14 # manifest contains pointers to offsets in normal files. changelog
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
15 # contains pointers to offsets in manifest. if reader reads old
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
16 # changelog while manifest or normal files are written, it has no
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
17 # pointers into new parts of those files that are maybe not consistent
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
18 # yet, so will not read them.
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
19
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
20 # localrepo.addchangegroup thinks it writes changelog first, then
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
21 # manifest, then normal files (this is order they are available, and
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
22 # needed for computing linkrev fields), but uses appendfile to hide
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
23 # updates from readers. data not written to manifest or changelog
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
24 # until all normal files updated. write manifest first, then
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
25 # changelog.
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
26
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
27 # with this write ordering, readers cannot see inconsistent view of
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
28 # repo during update.
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
29
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
30 class appendfile(object):
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
31 '''implement enough of file protocol to append to revlog file.
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
32 appended data is written to temp file. reads and seeks span real
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
33 file and temp file. readers cannot see appended data until
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
34 writedata called.'''
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
35
2089
cb99c711c59f make appendfile simpler so it does not break with revlogng on windows.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2082
diff changeset
36 def __init__(self, fp, tmpname):
cb99c711c59f make appendfile simpler so it does not break with revlogng on windows.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2082
diff changeset
37 if tmpname:
cb99c711c59f make appendfile simpler so it does not break with revlogng on windows.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2082
diff changeset
38 self.tmpname = tmpname
2176
9b42304d9896 fix file handling bugs on windows.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2102
diff changeset
39 self.tmpfp = util.posixfile(self.tmpname, 'ab+')
2089
cb99c711c59f make appendfile simpler so it does not break with revlogng on windows.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2082
diff changeset
40 else:
2165
d821918e3bee Use better names (hg-{usage}-{random}.{suffix}) for temporary files.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 2102
diff changeset
41 fd, self.tmpname = tempfile.mkstemp(prefix="hg-appendfile-")
2176
9b42304d9896 fix file handling bugs on windows.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2102
diff changeset
42 os.close(fd)
9b42304d9896 fix file handling bugs on windows.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2102
diff changeset
43 self.tmpfp = util.posixfile(self.tmpname, 'ab+')
1999
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
44 self.realfp = fp
2010
db1eb0de286a fix appendfile problem on macos.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 1999
diff changeset
45 self.offset = fp.tell()
1999
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
46 # real file is not written by anyone else. cache its size so
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
47 # seek and read can be fast.
2176
9b42304d9896 fix file handling bugs on windows.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2102
diff changeset
48 self.realsize = util.fstat(fp).st_size
9b42304d9896 fix file handling bugs on windows.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2102
diff changeset
49 self.name = fp.name
1999
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
50
2075
343aeefb553b Make the appendfile class inline-data index friendly
mason@suse.com
parents: 2027
diff changeset
51 def end(self):
343aeefb553b Make the appendfile class inline-data index friendly
mason@suse.com
parents: 2027
diff changeset
52 self.tmpfp.flush() # make sure the stat is correct
2176
9b42304d9896 fix file handling bugs on windows.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2102
diff changeset
53 return self.realsize + util.fstat(self.tmpfp).st_size
2089
cb99c711c59f make appendfile simpler so it does not break with revlogng on windows.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2082
diff changeset
54
cb99c711c59f make appendfile simpler so it does not break with revlogng on windows.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2082
diff changeset
55 def tell(self):
cb99c711c59f make appendfile simpler so it does not break with revlogng on windows.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2082
diff changeset
56 return self.offset
cb99c711c59f make appendfile simpler so it does not break with revlogng on windows.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2082
diff changeset
57
cb99c711c59f make appendfile simpler so it does not break with revlogng on windows.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2082
diff changeset
58 def flush(self):
cb99c711c59f make appendfile simpler so it does not break with revlogng on windows.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2082
diff changeset
59 self.tmpfp.flush()
cb99c711c59f make appendfile simpler so it does not break with revlogng on windows.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2082
diff changeset
60
cb99c711c59f make appendfile simpler so it does not break with revlogng on windows.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2082
diff changeset
61 def close(self):
cb99c711c59f make appendfile simpler so it does not break with revlogng on windows.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2082
diff changeset
62 self.realfp.close()
cb99c711c59f make appendfile simpler so it does not break with revlogng on windows.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2082
diff changeset
63 self.tmpfp.close()
2075
343aeefb553b Make the appendfile class inline-data index friendly
mason@suse.com
parents: 2027
diff changeset
64
343aeefb553b Make the appendfile class inline-data index friendly
mason@suse.com
parents: 2027
diff changeset
65 def seek(self, offset, whence=0):
1999
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
66 '''virtual file offset spans real file and temp file.'''
2075
343aeefb553b Make the appendfile class inline-data index friendly
mason@suse.com
parents: 2027
diff changeset
67 if whence == 0:
343aeefb553b Make the appendfile class inline-data index friendly
mason@suse.com
parents: 2027
diff changeset
68 self.offset = offset
343aeefb553b Make the appendfile class inline-data index friendly
mason@suse.com
parents: 2027
diff changeset
69 elif whence == 1:
343aeefb553b Make the appendfile class inline-data index friendly
mason@suse.com
parents: 2027
diff changeset
70 self.offset += offset
343aeefb553b Make the appendfile class inline-data index friendly
mason@suse.com
parents: 2027
diff changeset
71 elif whence == 2:
343aeefb553b Make the appendfile class inline-data index friendly
mason@suse.com
parents: 2027
diff changeset
72 self.offset = self.end() + offset
343aeefb553b Make the appendfile class inline-data index friendly
mason@suse.com
parents: 2027
diff changeset
73
2089
cb99c711c59f make appendfile simpler so it does not break with revlogng on windows.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2082
diff changeset
74 if self.offset < self.realsize:
1999
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
75 self.realfp.seek(self.offset)
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
76 else:
2089
cb99c711c59f make appendfile simpler so it does not break with revlogng on windows.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2082
diff changeset
77 self.tmpfp.seek(self.offset - self.realsize)
1999
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
78
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
79 def read(self, count=-1):
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
80 '''only trick here is reads that span real file and temp file.'''
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
81 fp = cStringIO.StringIO()
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
82 old_offset = self.offset
2089
cb99c711c59f make appendfile simpler so it does not break with revlogng on windows.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2082
diff changeset
83 if self.offset < self.realsize:
1999
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
84 s = self.realfp.read(count)
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
85 fp.write(s)
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
86 self.offset += len(s)
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
87 if count > 0:
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
88 count -= len(s)
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
89 if count != 0:
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
90 if old_offset != self.offset:
2089
cb99c711c59f make appendfile simpler so it does not break with revlogng on windows.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2082
diff changeset
91 self.tmpfp.seek(self.offset - self.realsize)
1999
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
92 s = self.tmpfp.read(count)
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
93 fp.write(s)
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
94 self.offset += len(s)
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
95 return fp.getvalue()
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
96
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
97 def write(self, s):
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
98 '''append to temp file.'''
2027
94d3170399e1 work around python bug on solaris 10.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2010
diff changeset
99 self.tmpfp.seek(0, 2)
1999
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
100 self.tmpfp.write(s)
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
101 # all writes are appends, so offset must go to end of file.
2089
cb99c711c59f make appendfile simpler so it does not break with revlogng on windows.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2082
diff changeset
102 self.offset = self.realsize + self.tmpfp.tell()
1999
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
103
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
104 class appendopener(object):
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
105 '''special opener for files that only read or append.'''
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
106
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
107 def __init__(self, opener):
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
108 self.realopener = opener
2089
cb99c711c59f make appendfile simpler so it does not break with revlogng on windows.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2082
diff changeset
109 # key: file name, value: appendfile name
cb99c711c59f make appendfile simpler so it does not break with revlogng on windows.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2082
diff changeset
110 self.tmpnames = {}
1999
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
111
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
112 def __call__(self, name, mode='r'):
2089
cb99c711c59f make appendfile simpler so it does not break with revlogng on windows.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2082
diff changeset
113 '''open file.'''
1999
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
114
2075
343aeefb553b Make the appendfile class inline-data index friendly
mason@suse.com
parents: 2027
diff changeset
115 assert mode in 'ra+'
2089
cb99c711c59f make appendfile simpler so it does not break with revlogng on windows.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2082
diff changeset
116 try:
cb99c711c59f make appendfile simpler so it does not break with revlogng on windows.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2082
diff changeset
117 realfp = self.realopener(name, 'r')
cb99c711c59f make appendfile simpler so it does not break with revlogng on windows.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2082
diff changeset
118 except IOError, err:
cb99c711c59f make appendfile simpler so it does not break with revlogng on windows.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2082
diff changeset
119 if err.errno != errno.ENOENT: raise
cb99c711c59f make appendfile simpler so it does not break with revlogng on windows.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2082
diff changeset
120 realfp = self.realopener(name, 'w+')
cb99c711c59f make appendfile simpler so it does not break with revlogng on windows.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2082
diff changeset
121 tmpname = self.tmpnames.get(name)
cb99c711c59f make appendfile simpler so it does not break with revlogng on windows.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2082
diff changeset
122 fp = appendfile(realfp, tmpname)
cb99c711c59f make appendfile simpler so it does not break with revlogng on windows.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2082
diff changeset
123 if tmpname is None:
cb99c711c59f make appendfile simpler so it does not break with revlogng on windows.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2082
diff changeset
124 self.tmpnames[name] = fp.tmpname
cb99c711c59f make appendfile simpler so it does not break with revlogng on windows.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2082
diff changeset
125 return fp
1999
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
126
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
127 def writedata(self):
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
128 '''copy data from temp files to real files.'''
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
129 # write .d file before .i file.
2089
cb99c711c59f make appendfile simpler so it does not break with revlogng on windows.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2082
diff changeset
130 tmpnames = self.tmpnames.items()
cb99c711c59f make appendfile simpler so it does not break with revlogng on windows.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2082
diff changeset
131 tmpnames.sort()
cb99c711c59f make appendfile simpler so it does not break with revlogng on windows.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2082
diff changeset
132 for name, tmpname in tmpnames:
2236
d7f866789d7e reduce memory used when appendfile appends to real file.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2232
diff changeset
133 ifp = open(tmpname, 'rb')
d7f866789d7e reduce memory used when appendfile appends to real file.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2232
diff changeset
134 ofp = self.realopener(name, 'a')
d7f866789d7e reduce memory used when appendfile appends to real file.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2232
diff changeset
135 for chunk in util.filechunkiter(ifp):
d7f866789d7e reduce memory used when appendfile appends to real file.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2232
diff changeset
136 ofp.write(chunk)
d7f866789d7e reduce memory used when appendfile appends to real file.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2232
diff changeset
137 ifp.close()
2102
c62112815801 remove appendfile data as soon as not needed.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2089
diff changeset
138 os.unlink(tmpname)
2232
ef3c039e7ab8 Always remove appendopener tmp files (fixes issue235).
Thomas Arendsen Hein <thomas@intevation.de>
parents: 2177
diff changeset
139 del self.tmpnames[name]
2236
d7f866789d7e reduce memory used when appendfile appends to real file.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2232
diff changeset
140 ofp.close()
2089
cb99c711c59f make appendfile simpler so it does not break with revlogng on windows.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2082
diff changeset
141
2232
ef3c039e7ab8 Always remove appendopener tmp files (fixes issue235).
Thomas Arendsen Hein <thomas@intevation.de>
parents: 2177
diff changeset
142 def cleanup(self):
ef3c039e7ab8 Always remove appendopener tmp files (fixes issue235).
Thomas Arendsen Hein <thomas@intevation.de>
parents: 2177
diff changeset
143 '''delete temp files (this discards unwritten data!)'''
ef3c039e7ab8 Always remove appendopener tmp files (fixes issue235).
Thomas Arendsen Hein <thomas@intevation.de>
parents: 2177
diff changeset
144 for tmpname in self.tmpnames.values():
ef3c039e7ab8 Always remove appendopener tmp files (fixes issue235).
Thomas Arendsen Hein <thomas@intevation.de>
parents: 2177
diff changeset
145 os.unlink(tmpname)
ef3c039e7ab8 Always remove appendopener tmp files (fixes issue235).
Thomas Arendsen Hein <thomas@intevation.de>
parents: 2177
diff changeset
146
1999
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
147 # files for changelog and manifest are in different appendopeners, so
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
148 # not mixed up together.
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
149
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
150 class appendchangelog(changelog.changelog, appendopener):
2082
856f0ba200bc Additional appendfile fixes for interleaved data/index files
mason@suse.com
parents: 2075
diff changeset
151 def __init__(self, opener, version):
1999
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
152 appendopener.__init__(self, opener)
2082
856f0ba200bc Additional appendfile fixes for interleaved data/index files
mason@suse.com
parents: 2075
diff changeset
153 changelog.changelog.__init__(self, self, version)
2075
343aeefb553b Make the appendfile class inline-data index friendly
mason@suse.com
parents: 2027
diff changeset
154 def checkinlinesize(self, fp, tr):
343aeefb553b Make the appendfile class inline-data index friendly
mason@suse.com
parents: 2027
diff changeset
155 return
1999
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
156
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
157 class appendmanifest(manifest.manifest, appendopener):
2082
856f0ba200bc Additional appendfile fixes for interleaved data/index files
mason@suse.com
parents: 2075
diff changeset
158 def __init__(self, opener, version):
1999
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
159 appendopener.__init__(self, opener)
2082
856f0ba200bc Additional appendfile fixes for interleaved data/index files
mason@suse.com
parents: 2075
diff changeset
160 manifest.manifest.__init__(self, self, version)
2075
343aeefb553b Make the appendfile class inline-data index friendly
mason@suse.com
parents: 2027
diff changeset
161 def checkinlinesize(self, fp, tr):
343aeefb553b Make the appendfile class inline-data index friendly
mason@suse.com
parents: 2027
diff changeset
162 return