Mercurial > hg > mercurial-crew-with-dirclash

annotate mercurial/appendfile.py @ 2082:856f0ba200bc

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Additional appendfile fixes for interleaved data/index files The appendfile code was not passing default version info to the changelog or manifest classes, and so they were always being created as version 0. revlog.checkinlinesize had to be corrected to seek to the end of the index file when no index file was passed (only clone does this)
author mason@suse.com
date Sat, 08 Apr 2006 20:08:06 -0400
parents 343aeefb553b
children cb99c711c59f
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
1999
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
1 # appendfile.py - special classes to make repo updates atomic
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
2 #
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
3 # Copyright 2006 Vadim Gelfer <vadim.gelfer@gmail.com>
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
4 #
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
5 # This software may be used and distributed according to the terms
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
6 # of the GNU General Public License, incorporated herein by reference.
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
7
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
8 from demandload import *
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
9 demandload(globals(), "cStringIO changelog manifest os tempfile")
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
10
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
11 # writes to metadata files are ordered. reads: changelog, manifest,
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
12 # normal files. writes: normal files, manifest, changelog.
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
13
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
14 # manifest contains pointers to offsets in normal files. changelog
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
15 # contains pointers to offsets in manifest. if reader reads old
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
16 # changelog while manifest or normal files are written, it has no
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
17 # pointers into new parts of those files that are maybe not consistent
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
18 # yet, so will not read them.
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
19
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
20 # localrepo.addchangegroup thinks it writes changelog first, then
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
21 # manifest, then normal files (this is order they are available, and
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
22 # needed for computing linkrev fields), but uses appendfile to hide
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
23 # updates from readers. data not written to manifest or changelog
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
24 # until all normal files updated. write manifest first, then
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
25 # changelog.
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
26
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
27 # with this write ordering, readers cannot see inconsistent view of
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
28 # repo during update.
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
29
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
30 class appendfile(object):
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
31 '''implement enough of file protocol to append to revlog file.
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
32 appended data is written to temp file. reads and seeks span real
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
33 file and temp file. readers cannot see appended data until
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
34 writedata called.'''
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
35
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
36 def __init__(self, fp):
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
37 fd, self.tmpname = tempfile.mkstemp()
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
38 self.tmpfp = os.fdopen(fd, 'ab+')
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
39 self.realfp = fp
2010
db1eb0de286a fix appendfile problem on macos.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 1999
diff changeset
40 self.offset = fp.tell()
1999
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
41 # real file is not written by anyone else. cache its size so
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
42 # seek and read can be fast.
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
43 self.fpsize = os.fstat(fp.fileno()).st_size
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
44
2075
343aeefb553b Make the appendfile class inline-data index friendly
mason@suse.com
parents: 2027
diff changeset
45 def end(self):
343aeefb553b Make the appendfile class inline-data index friendly
mason@suse.com
parents: 2027
diff changeset
46 self.tmpfp.flush() # make sure the stat is correct
343aeefb553b Make the appendfile class inline-data index friendly
mason@suse.com
parents: 2027
diff changeset
47 return self.fpsize + os.fstat(self.tmpfp.fileno()).st_size
343aeefb553b Make the appendfile class inline-data index friendly
mason@suse.com
parents: 2027
diff changeset
48
343aeefb553b Make the appendfile class inline-data index friendly
mason@suse.com
parents: 2027
diff changeset
49 def seek(self, offset, whence=0):
1999
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
50 '''virtual file offset spans real file and temp file.'''
2075
343aeefb553b Make the appendfile class inline-data index friendly
mason@suse.com
parents: 2027
diff changeset
51 if whence == 0:
343aeefb553b Make the appendfile class inline-data index friendly
mason@suse.com
parents: 2027
diff changeset
52 self.offset = offset
343aeefb553b Make the appendfile class inline-data index friendly
mason@suse.com
parents: 2027
diff changeset
53 elif whence == 1:
343aeefb553b Make the appendfile class inline-data index friendly
mason@suse.com
parents: 2027
diff changeset
54 self.offset += offset
343aeefb553b Make the appendfile class inline-data index friendly
mason@suse.com
parents: 2027
diff changeset
55 elif whence == 2:
343aeefb553b Make the appendfile class inline-data index friendly
mason@suse.com
parents: 2027
diff changeset
56 self.offset = self.end() + offset
343aeefb553b Make the appendfile class inline-data index friendly
mason@suse.com
parents: 2027
diff changeset
57
1999
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
58 if self.offset < self.fpsize:
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
59 self.realfp.seek(self.offset)
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
60 else:
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
61 self.tmpfp.seek(self.offset - self.fpsize)
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
62
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
63 def read(self, count=-1):
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
64 '''only trick here is reads that span real file and temp file.'''
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
65 fp = cStringIO.StringIO()
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
66 old_offset = self.offset
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
67 if self.offset < self.fpsize:
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
68 s = self.realfp.read(count)
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
69 fp.write(s)
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
70 self.offset += len(s)
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
71 if count > 0:
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
72 count -= len(s)
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
73 if count != 0:
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
74 if old_offset != self.offset:
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
75 self.tmpfp.seek(self.offset - self.fpsize)
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
76 s = self.tmpfp.read(count)
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
77 fp.write(s)
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
78 self.offset += len(s)
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
79 return fp.getvalue()
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
80
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
81 def write(self, s):
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
82 '''append to temp file.'''
2027
94d3170399e1 work around python bug on solaris 10.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 2010
diff changeset
83 self.tmpfp.seek(0, 2)
1999
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
84 self.tmpfp.write(s)
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
85 # all writes are appends, so offset must go to end of file.
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
86 self.offset = self.fpsize + self.tmpfp.tell()
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
87
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
88 def writedata(self):
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
89 '''copy data from temp file to real file.'''
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
90 self.tmpfp.seek(0)
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
91 s = self.tmpfp.read()
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
92 self.tmpfp.close()
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
93 self.realfp.seek(0, 2)
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
94 # small race here. we write all new data in one call, but
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
95 # reader can see partial update due to python or os. file
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
96 # locking no help: slow, not portable, not reliable over nfs.
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
97 # only safe thing is write to temp file every time and rename,
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
98 # but performance bad when manifest or changelog gets big.
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
99 self.realfp.write(s)
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
100 self.realfp.close()
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
101
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
102 def __del__(self):
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
103 '''delete temp file even if exception raised.'''
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
104 try: os.unlink(self.tmpname)
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
105 except: pass
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
106
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
107 class sharedfile(object):
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
108 '''let file objects share a single appendfile safely. each
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
109 sharedfile has own offset, syncs up with appendfile offset before
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
110 read and after read and write.'''
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
111
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
112 def __init__(self, fp):
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
113 self.fp = fp
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
114 self.offset = 0
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
115
2075
343aeefb553b Make the appendfile class inline-data index friendly
mason@suse.com
parents: 2027
diff changeset
116 def tell(self):
343aeefb553b Make the appendfile class inline-data index friendly
mason@suse.com
parents: 2027
diff changeset
117 return self.offset
343aeefb553b Make the appendfile class inline-data index friendly
mason@suse.com
parents: 2027
diff changeset
118
343aeefb553b Make the appendfile class inline-data index friendly
mason@suse.com
parents: 2027
diff changeset
119 def seek(self, offset, whence=0):
343aeefb553b Make the appendfile class inline-data index friendly
mason@suse.com
parents: 2027
diff changeset
120 if whence == 0:
343aeefb553b Make the appendfile class inline-data index friendly
mason@suse.com
parents: 2027
diff changeset
121 self.offset = offset
343aeefb553b Make the appendfile class inline-data index friendly
mason@suse.com
parents: 2027
diff changeset
122 elif whence == 1:
343aeefb553b Make the appendfile class inline-data index friendly
mason@suse.com
parents: 2027
diff changeset
123 self.offset += offset
343aeefb553b Make the appendfile class inline-data index friendly
mason@suse.com
parents: 2027
diff changeset
124 elif whence == 2:
343aeefb553b Make the appendfile class inline-data index friendly
mason@suse.com
parents: 2027
diff changeset
125 self.offset = self.fp.end() + offset
1999
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
126
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
127 def read(self, count=-1):
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
128 try:
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
129 if self.offset != self.fp.offset:
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
130 self.fp.seek(self.offset)
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
131 return self.fp.read(count)
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
132 finally:
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
133 self.offset = self.fp.offset
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
134
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
135 def write(self, s):
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
136 try:
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
137 return self.fp.write(s)
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
138 finally:
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
139 self.offset = self.fp.offset
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
140
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
141 def close(self):
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
142 # revlog wants this.
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
143 pass
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
144
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
145 def flush(self):
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
146 # revlog wants this.
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
147 pass
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
148
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
149 def writedata(self):
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
150 self.fp.writedata()
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
151
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
152 class appendopener(object):
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
153 '''special opener for files that only read or append.'''
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
154
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
155 def __init__(self, opener):
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
156 self.realopener = opener
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
157 # key: file name, value: appendfile object
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
158 self.fps = {}
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
159
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
160 def __call__(self, name, mode='r'):
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
161 '''open file. return same cached appendfile object for every
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
162 later call.'''
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
163
2075
343aeefb553b Make the appendfile class inline-data index friendly
mason@suse.com
parents: 2027
diff changeset
164 assert mode in 'ra+'
1999
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
165 fp = self.fps.get(name)
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
166 if fp is None:
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
167 fp = appendfile(self.realopener(name, 'a+'))
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
168 self.fps[name] = fp
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
169 return sharedfile(fp)
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
170
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
171 def writedata(self):
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
172 '''copy data from temp files to real files.'''
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
173 # write .d file before .i file.
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
174 fps = self.fps.items()
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
175 fps.sort()
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
176 for name, fp in fps:
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
177 fp.writedata()
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
178
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
179 # files for changelog and manifest are in different appendopeners, so
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
180 # not mixed up together.
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
181
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
182 class appendchangelog(changelog.changelog, appendopener):
2082
856f0ba200bc Additional appendfile fixes for interleaved data/index files
mason@suse.com
parents: 2075
diff changeset
183 def __init__(self, opener, version):
1999
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
184 appendopener.__init__(self, opener)
2082
856f0ba200bc Additional appendfile fixes for interleaved data/index files
mason@suse.com
parents: 2075
diff changeset
185 changelog.changelog.__init__(self, self, version)
2075
343aeefb553b Make the appendfile class inline-data index friendly
mason@suse.com
parents: 2027
diff changeset
186 def checkinlinesize(self, fp, tr):
343aeefb553b Make the appendfile class inline-data index friendly
mason@suse.com
parents: 2027
diff changeset
187 return
1999
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
188
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
189 class appendmanifest(manifest.manifest, appendopener):
2082
856f0ba200bc Additional appendfile fixes for interleaved data/index files
mason@suse.com
parents: 2075
diff changeset
190 def __init__(self, opener, version):
1999
fdb3a72ef664 forgot to add new module.
Vadim Gelfer <vadim.gelfer@gmail.com>
parents:
diff changeset
191 appendopener.__init__(self, opener)
2082
856f0ba200bc Additional appendfile fixes for interleaved data/index files
mason@suse.com
parents: 2075
diff changeset
192 manifest.manifest.__init__(self, self, version)
2075
343aeefb553b Make the appendfile class inline-data index friendly
mason@suse.com
parents: 2027
diff changeset
193 def checkinlinesize(self, fp, tr):
343aeefb553b Make the appendfile class inline-data index friendly
mason@suse.com
parents: 2027
diff changeset
194 return