annotate mercurial/util.py @ 1270:fc3b41570082

Switch to new syntax for .hgignore files. Here is the new syntax, in summary. Trailing white space is dropped. The escape character is "\". Comments start with #. Empty lines are skipped. Lines can be of the following formats: syntax: regexp # defaults following lines to non-rooted regexps syntax: glob # defaults following lines to non-rooted globs re:pattern # non-rooted regular expression glob:pattern # non-rooted glob pattern # pattern of the current default type The default pattern type is regexp, which is completely backwards compatible with the old hgignore syntax. In the dirstate class, the ignore method has been reworked to be based on the util.matcher function, by way of a new dirstate.hgignore method.
author Bryan O'Sullivan <bos@serpentine.com>
date Sat, 17 Sep 2005 00:27:27 -0700
parents 1945754e466b
children 1546c2aa6b30
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
1082
ce96e316278a Update util.py docstrings, fix walk test
mpm@selenic.com
parents: 1081
diff changeset
1 """
ce96e316278a Update util.py docstrings, fix walk test
mpm@selenic.com
parents: 1081
diff changeset
2 util.py - Mercurial utility functions and platform specfic implementations
ce96e316278a Update util.py docstrings, fix walk test
mpm@selenic.com
parents: 1081
diff changeset
3
ce96e316278a Update util.py docstrings, fix walk test
mpm@selenic.com
parents: 1081
diff changeset
4 Copyright 2005 K. Thananchayan <thananck@yahoo.com>
ce96e316278a Update util.py docstrings, fix walk test
mpm@selenic.com
parents: 1081
diff changeset
5
ce96e316278a Update util.py docstrings, fix walk test
mpm@selenic.com
parents: 1081
diff changeset
6 This software may be used and distributed according to the terms
ce96e316278a Update util.py docstrings, fix walk test
mpm@selenic.com
parents: 1081
diff changeset
7 of the GNU General Public License, incorporated herein by reference.
ce96e316278a Update util.py docstrings, fix walk test
mpm@selenic.com
parents: 1081
diff changeset
8
ce96e316278a Update util.py docstrings, fix walk test
mpm@selenic.com
parents: 1081
diff changeset
9 This contains helper routines that are independent of the SCM core and hide
ce96e316278a Update util.py docstrings, fix walk test
mpm@selenic.com
parents: 1081
diff changeset
10 platform-specific details from the core.
ce96e316278a Update util.py docstrings, fix walk test
mpm@selenic.com
parents: 1081
diff changeset
11 """
419
28511fc21073 [PATCH] file seperator handling for the other 'OS'
mpm@selenic.com
parents:
diff changeset
12
704
5ca319a641e1 Make makelock and readlock work on filesystems without symlink support.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 698
diff changeset
13 import os, errno
724
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
14 from demandload import *
1258
1945754e466b Add file encoding/decoding support
mpm@selenic.com
parents: 1241
diff changeset
15 demandload(globals(), "re cStringIO shutil popen2 threading")
1945754e466b Add file encoding/decoding support
mpm@selenic.com
parents: 1241
diff changeset
16
1945754e466b Add file encoding/decoding support
mpm@selenic.com
parents: 1241
diff changeset
17 def filter(s, cmd):
1945754e466b Add file encoding/decoding support
mpm@selenic.com
parents: 1241
diff changeset
18 "filter a string through a command that transforms its input to its output"
1945754e466b Add file encoding/decoding support
mpm@selenic.com
parents: 1241
diff changeset
19 (pout, pin) = popen2.popen2(cmd, -1, 'b')
1945754e466b Add file encoding/decoding support
mpm@selenic.com
parents: 1241
diff changeset
20 def writer():
1945754e466b Add file encoding/decoding support
mpm@selenic.com
parents: 1241
diff changeset
21 pin.write(s)
1945754e466b Add file encoding/decoding support
mpm@selenic.com
parents: 1241
diff changeset
22 pin.close()
1945754e466b Add file encoding/decoding support
mpm@selenic.com
parents: 1241
diff changeset
23
1945754e466b Add file encoding/decoding support
mpm@selenic.com
parents: 1241
diff changeset
24 # we should use select instead on UNIX, but this will work on most
1945754e466b Add file encoding/decoding support
mpm@selenic.com
parents: 1241
diff changeset
25 # systems, including Windows
1945754e466b Add file encoding/decoding support
mpm@selenic.com
parents: 1241
diff changeset
26 w = threading.Thread(target=writer)
1945754e466b Add file encoding/decoding support
mpm@selenic.com
parents: 1241
diff changeset
27 w.start()
1945754e466b Add file encoding/decoding support
mpm@selenic.com
parents: 1241
diff changeset
28 f = pout.read()
1945754e466b Add file encoding/decoding support
mpm@selenic.com
parents: 1241
diff changeset
29 pout.close()
1945754e466b Add file encoding/decoding support
mpm@selenic.com
parents: 1241
diff changeset
30 w.join()
1945754e466b Add file encoding/decoding support
mpm@selenic.com
parents: 1241
diff changeset
31 return f
419
28511fc21073 [PATCH] file seperator handling for the other 'OS'
mpm@selenic.com
parents:
diff changeset
32
1015
22571b8d35d3 Add automatic binary file detection to diff and export
mpm@selenic.com
parents: 917
diff changeset
33 def binary(s):
1082
ce96e316278a Update util.py docstrings, fix walk test
mpm@selenic.com
parents: 1081
diff changeset
34 """return true if a string is binary data using diff's heuristic"""
1015
22571b8d35d3 Add automatic binary file detection to diff and export
mpm@selenic.com
parents: 917
diff changeset
35 if s and '\0' in s[:4096]:
22571b8d35d3 Add automatic binary file detection to diff and export
mpm@selenic.com
parents: 917
diff changeset
36 return True
22571b8d35d3 Add automatic binary file detection to diff and export
mpm@selenic.com
parents: 917
diff changeset
37 return False
22571b8d35d3 Add automatic binary file detection to diff and export
mpm@selenic.com
parents: 917
diff changeset
38
556
f6c6fa15ff70 Move dirstate.uniq to util.unique
mpm@selenic.com
parents: 521
diff changeset
39 def unique(g):
1082
ce96e316278a Update util.py docstrings, fix walk test
mpm@selenic.com
parents: 1081
diff changeset
40 """return the uniq elements of iterable g"""
556
f6c6fa15ff70 Move dirstate.uniq to util.unique
mpm@selenic.com
parents: 521
diff changeset
41 seen = {}
f6c6fa15ff70 Move dirstate.uniq to util.unique
mpm@selenic.com
parents: 521
diff changeset
42 for f in g:
f6c6fa15ff70 Move dirstate.uniq to util.unique
mpm@selenic.com
parents: 521
diff changeset
43 if f not in seen:
f6c6fa15ff70 Move dirstate.uniq to util.unique
mpm@selenic.com
parents: 521
diff changeset
44 seen[f] = 1
f6c6fa15ff70 Move dirstate.uniq to util.unique
mpm@selenic.com
parents: 521
diff changeset
45 yield f
f6c6fa15ff70 Move dirstate.uniq to util.unique
mpm@selenic.com
parents: 521
diff changeset
46
870
a82eae840447 Teach walk code about absolute paths.
Bryan O'Sullivan <bos@serpentine.com>
parents: 869
diff changeset
47 class Abort(Exception):
a82eae840447 Teach walk code about absolute paths.
Bryan O'Sullivan <bos@serpentine.com>
parents: 869
diff changeset
48 """Raised if a command needs to print an error and exit."""
508
42a660abaf75 [PATCH] Harden os.system
mpm@selenic.com
parents: 464
diff changeset
49
724
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
50 def always(fn): return True
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
51 def never(fn): return False
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
52
1062
6d5a62a549fa pep-0008 cleanup
benoit.boissinot@ens-lyon.fr
parents: 1031
diff changeset
53 def globre(pat, head='^', tail='$'):
724
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
54 "convert a glob pattern into a regexp"
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
55 i, n = 0, len(pat)
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
56 res = ''
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
57 group = False
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
58 def peek(): return i < n and pat[i]
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
59 while i < n:
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
60 c = pat[i]
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
61 i = i+1
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
62 if c == '*':
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
63 if peek() == '*':
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
64 i += 1
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
65 res += '.*'
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
66 else:
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
67 res += '[^/]*'
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
68 elif c == '?':
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
69 res += '.'
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
70 elif c == '[':
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
71 j = i
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
72 if j < n and pat[j] in '!]':
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
73 j += 1
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
74 while j < n and pat[j] != ']':
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
75 j += 1
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
76 if j >= n:
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
77 res += '\\['
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
78 else:
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
79 stuff = pat[i:j].replace('\\','\\\\')
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
80 i = j + 1
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
81 if stuff[0] == '!':
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
82 stuff = '^' + stuff[1:]
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
83 elif stuff[0] == '^':
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
84 stuff = '\\' + stuff
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
85 res = '%s[%s]' % (res, stuff)
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
86 elif c == '{':
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
87 group = True
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
88 res += '(?:'
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
89 elif c == '}' and group:
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
90 res += ')'
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
91 group = False
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
92 elif c == ',' and group:
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
93 res += '|'
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
94 else:
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
95 res += re.escape(c)
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
96 return head + res + tail
1c0c413cccdd Get add and locate to use new repo and dirstate walk code.
Bryan O'Sullivan <bos@serpentine.com>
parents: 705
diff changeset
97
812
b65af904d6d7 Reduce the amount of stat traffic generated by a walk.
Bryan O'Sullivan <bos@serpentine.com>
parents: 782
diff changeset
98 _globchars = {'[': 1, '{': 1, '*': 1, '?': 1}
b65af904d6d7 Reduce the amount of stat traffic generated by a walk.
Bryan O'Sullivan <bos@serpentine.com>
parents: 782
diff changeset
99
884
087771ebe2e6 Fix walk code for files that do not exist anywhere, and unhandled types.
Bryan O'Sullivan <bos@serpentine.com>
parents: 878
diff changeset
100 def pathto(n1, n2):
886
509de8ab6f31 Fix walk path handling on Windows
Bryan O'Sullivan <bos@serpentine.com>
parents: 884
diff changeset
101 '''return the relative path from one place to another.
509de8ab6f31 Fix walk path handling on Windows
Bryan O'Sullivan <bos@serpentine.com>
parents: 884
diff changeset
102 this returns a path in the form used by the local filesystem, not hg.'''
509de8ab6f31 Fix walk path handling on Windows
Bryan O'Sullivan <bos@serpentine.com>
parents: 884
diff changeset
103 if not n1: return localpath(n2)
509de8ab6f31 Fix walk path handling on Windows
Bryan O'Sullivan <bos@serpentine.com>
parents: 884
diff changeset
104 a, b = n1.split('/'), n2.split('/')
884
087771ebe2e6 Fix walk code for files that do not exist anywhere, and unhandled types.
Bryan O'Sullivan <bos@serpentine.com>
parents: 878
diff changeset
105 a.reverse(), b.reverse()
087771ebe2e6 Fix walk code for files that do not exist anywhere, and unhandled types.
Bryan O'Sullivan <bos@serpentine.com>
parents: 878
diff changeset
106 while a and b and a[-1] == b[-1]:
087771ebe2e6 Fix walk code for files that do not exist anywhere, and unhandled types.
Bryan O'Sullivan <bos@serpentine.com>
parents: 878
diff changeset
107 a.pop(), b.pop()
087771ebe2e6 Fix walk code for files that do not exist anywhere, and unhandled types.
Bryan O'Sullivan <bos@serpentine.com>
parents: 878
diff changeset
108 b.reverse()
087771ebe2e6 Fix walk code for files that do not exist anywhere, and unhandled types.
Bryan O'Sullivan <bos@serpentine.com>
parents: 878
diff changeset
109 return os.sep.join((['..'] * len(a)) + b)
087771ebe2e6 Fix walk code for files that do not exist anywhere, and unhandled types.
Bryan O'Sullivan <bos@serpentine.com>
parents: 878
diff changeset
110
1081
8b7d63489db3 Change canonpath to not know about repo objects
mpm@selenic.com
parents: 1075
diff changeset
111 def canonpath(root, cwd, myname):
1082
ce96e316278a Update util.py docstrings, fix walk test
mpm@selenic.com
parents: 1081
diff changeset
112 """return the canonical path of myname, given cwd and root"""
1081
8b7d63489db3 Change canonpath to not know about repo objects
mpm@selenic.com
parents: 1075
diff changeset
113 rootsep = root + os.sep
870
a82eae840447 Teach walk code about absolute paths.
Bryan O'Sullivan <bos@serpentine.com>
parents: 869
diff changeset
114 name = myname
a82eae840447 Teach walk code about absolute paths.
Bryan O'Sullivan <bos@serpentine.com>
parents: 869
diff changeset
115 if not name.startswith(os.sep):
1081
8b7d63489db3 Change canonpath to not know about repo objects
mpm@selenic.com
parents: 1075
diff changeset
116 name = os.path.join(root, cwd, name)
870
a82eae840447 Teach walk code about absolute paths.
Bryan O'Sullivan <bos@serpentine.com>
parents: 869
diff changeset
117 name = os.path.normpath(name)
a82eae840447 Teach walk code about absolute paths.
Bryan O'Sullivan <bos@serpentine.com>
parents: 869
diff changeset
118 if name.startswith(rootsep):
886
509de8ab6f31 Fix walk path handling on Windows
Bryan O'Sullivan <bos@serpentine.com>
parents: 884
diff changeset
119 return pconvert(name[len(rootsep):])
1081
8b7d63489db3 Change canonpath to not know about repo objects
mpm@selenic.com
parents: 1075
diff changeset
120 elif name == root:
870
a82eae840447 Teach walk code about absolute paths.
Bryan O'Sullivan <bos@serpentine.com>
parents: 869
diff changeset
121 return ''
a82eae840447 Teach walk code about absolute paths.
Bryan O'Sullivan <bos@serpentine.com>
parents: 869
diff changeset
122 else:
1081
8b7d63489db3 Change canonpath to not know about repo objects
mpm@selenic.com
parents: 1075
diff changeset
123 raise Abort('%s not under root' % myname)
897
fe30f5434b51 Fix bug with empty inc and exc
mpm@selenic.com
parents: 896
diff changeset
124
1270
fc3b41570082 Switch to new syntax for .hgignore files.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1258
diff changeset
125 def matcher(canonroot, cwd='', names=['.'], inc=[], exc=[], head=''):
1082
ce96e316278a Update util.py docstrings, fix walk test
mpm@selenic.com
parents: 1081
diff changeset
126 """build a function to match a set of file patterns
ce96e316278a Update util.py docstrings, fix walk test
mpm@selenic.com
parents: 1081
diff changeset
127
ce96e316278a Update util.py docstrings, fix walk test
mpm@selenic.com
parents: 1081
diff changeset
128 arguments:
ce96e316278a Update util.py docstrings, fix walk test
mpm@selenic.com
parents: 1081
diff changeset
129 canonroot - the canonical root of the tree you're matching against
ce96e316278a Update util.py docstrings, fix walk test
mpm@selenic.com
parents: 1081
diff changeset
130 cwd - the current working directory, if relevant
ce96e316278a Update util.py docstrings, fix walk test
mpm@selenic.com
parents: 1081
diff changeset
131 names - patterns to find
ce96e316278a Update util.py docstrings, fix walk test
mpm@selenic.com
parents: 1081
diff changeset
132 inc - patterns to include
ce96e316278a Update util.py docstrings, fix walk test
mpm@selenic.com
parents: 1081
diff changeset
133 exc - patterns to exclude
ce96e316278a Update util.py docstrings, fix walk test
mpm@selenic.com
parents: 1081
diff changeset
134 head - a regex to prepend to patterns to control whether a match is rooted
ce96e316278a Update util.py docstrings, fix walk test
mpm@selenic.com
parents: 1081
diff changeset
135
ce96e316278a Update util.py docstrings, fix walk test
mpm@selenic.com
parents: 1081
diff changeset
136 a pattern is one of:
1270
fc3b41570082 Switch to new syntax for .hgignore files.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1258
diff changeset
137 'glob:<rooted glob>'
fc3b41570082 Switch to new syntax for .hgignore files.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1258
diff changeset
138 're:<rooted regexp>'
fc3b41570082 Switch to new syntax for .hgignore files.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1258
diff changeset
139 'path:<rooted path>'
fc3b41570082 Switch to new syntax for .hgignore files.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1258
diff changeset
140 'relglob:<relative glob>'
1082
ce96e316278a Update util.py docstrings, fix walk test
mpm@selenic.com
parents: 1081
diff changeset
141 'relpath:<relative path>'
1270
fc3b41570082 Switch to new syntax for .hgignore files.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1258
diff changeset
142 'relre:<relative regexp>'
fc3b41570082 Switch to new syntax for .hgignore files.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1258
diff changeset
143 '<rooted path or regexp>'
1082
ce96e316278a Update util.py docstrings, fix walk test
mpm@selenic.com
parents: 1081
diff changeset
144
ce96e316278a Update util.py docstrings, fix walk test
mpm@selenic.com
parents: 1081
diff changeset
145 returns:
ce96e316278a Update util.py docstrings, fix walk test
mpm@selenic.com
parents: 1081
diff changeset
146 a 3-tuple containing
ce96e316278a Update util.py docstrings, fix walk test
mpm@selenic.com
parents: 1081
diff changeset
147 - list of explicit non-pattern names passed in
ce96e316278a Update util.py docstrings, fix walk test
mpm@selenic.com
parents: 1081
diff changeset
148 - a bool match(filename) function
ce96e316278a Update util.py docstrings, fix walk test
mpm@selenic.com
parents: 1081
diff changeset
149 - a bool indicating if any patterns were passed in
ce96e316278a Update util.py docstrings, fix walk test
mpm@selenic.com
parents: 1081
diff changeset
150
ce96e316278a Update util.py docstrings, fix walk test
mpm@selenic.com
parents: 1081
diff changeset
151 todo:
ce96e316278a Update util.py docstrings, fix walk test
mpm@selenic.com
parents: 1081
diff changeset
152 make head regex a rooted bool
ce96e316278a Update util.py docstrings, fix walk test
mpm@selenic.com
parents: 1081
diff changeset
153 """
ce96e316278a Update util.py docstrings, fix walk test
mpm@selenic.com
parents: 1081
diff changeset
154
820
89985a1b3427 Clean up walk and changes code to use normalised names properly.
Bryan O'Sullivan <bos@serpentine.com>
parents: 814
diff changeset
155 def patkind(name):
1270
fc3b41570082 Switch to new syntax for .hgignore files.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1258
diff changeset
156 for prefix in 're', 'glob', 'path', 'relglob', 'relpath', 'relre':
fc3b41570082 Switch to new syntax for .hgignore files.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1258
diff changeset
157 if name.startswith(prefix + ':'): return name.split(':', 1)
812
b65af904d6d7 Reduce the amount of stat traffic generated by a walk.
Bryan O'Sullivan <bos@serpentine.com>
parents: 782
diff changeset
158 for c in name:
820
89985a1b3427 Clean up walk and changes code to use normalised names properly.
Bryan O'Sullivan <bos@serpentine.com>
parents: 814
diff changeset
159 if c in _globchars: return 'glob', name
89985a1b3427 Clean up walk and changes code to use normalised names properly.
Bryan O'Sullivan <bos@serpentine.com>
parents: 814
diff changeset
160 return 'relpath', name
89985a1b3427 Clean up walk and changes code to use normalised names properly.
Bryan O'Sullivan <bos@serpentine.com>
parents: 814
diff changeset
161
888
e7a943e8c52b Fix up handling of regexp paths.
Bryan O'Sullivan <bos@serpentine.com>
parents: 886
diff changeset
162 def regex(kind, name, tail):
742
092937de2ad7 Refactor matchpats and walk
mpm@selenic.com
parents: 740
diff changeset
163 '''convert a pattern into a regular expression'''
820
89985a1b3427 Clean up walk and changes code to use normalised names properly.
Bryan O'Sullivan <bos@serpentine.com>
parents: 814
diff changeset
164 if kind == 're':
89985a1b3427 Clean up walk and changes code to use normalised names properly.
Bryan O'Sullivan <bos@serpentine.com>
parents: 814
diff changeset
165 return name
89985a1b3427 Clean up walk and changes code to use normalised names properly.
Bryan O'Sullivan <bos@serpentine.com>
parents: 814
diff changeset
166 elif kind == 'path':
888
e7a943e8c52b Fix up handling of regexp paths.
Bryan O'Sullivan <bos@serpentine.com>
parents: 886
diff changeset
167 return '^' + re.escape(name) + '(?:/|$)'
1270
fc3b41570082 Switch to new syntax for .hgignore files.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1258
diff changeset
168 elif kind == 'relglob':
fc3b41570082 Switch to new syntax for .hgignore files.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1258
diff changeset
169 return head + globre(name, '(?:|.*/)', tail)
888
e7a943e8c52b Fix up handling of regexp paths.
Bryan O'Sullivan <bos@serpentine.com>
parents: 886
diff changeset
170 elif kind == 'relpath':
e7a943e8c52b Fix up handling of regexp paths.
Bryan O'Sullivan <bos@serpentine.com>
parents: 886
diff changeset
171 return head + re.escape(name) + tail
1270
fc3b41570082 Switch to new syntax for .hgignore files.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1258
diff changeset
172 elif kind == 'relre':
fc3b41570082 Switch to new syntax for .hgignore files.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1258
diff changeset
173 if name.startswith('^'):
fc3b41570082 Switch to new syntax for .hgignore files.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1258
diff changeset
174 return name
fc3b41570082 Switch to new syntax for .hgignore files.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1258
diff changeset
175 return '.*' + name
742
092937de2ad7 Refactor matchpats and walk
mpm@selenic.com
parents: 740
diff changeset
176 return head + globre(name, '', tail)
092937de2ad7 Refactor matchpats and walk
mpm@selenic.com
parents: 740
diff changeset
177
092937de2ad7 Refactor matchpats and walk
mpm@selenic.com
parents: 740
diff changeset
178 def matchfn(pats, tail):
092937de2ad7 Refactor matchpats and walk
mpm@selenic.com
parents: 740
diff changeset
179 """build a matching function from a set of patterns"""
092937de2ad7 Refactor matchpats and walk
mpm@selenic.com
parents: 740
diff changeset
180 if pats:
888
e7a943e8c52b Fix up handling of regexp paths.
Bryan O'Sullivan <bos@serpentine.com>
parents: 886
diff changeset
181 pat = '(?:%s)' % '|'.join([regex(k, p, tail) for (k, p) in pats])
742
092937de2ad7 Refactor matchpats and walk
mpm@selenic.com
parents: 740
diff changeset
182 return re.compile(pat).match
092937de2ad7 Refactor matchpats and walk
mpm@selenic.com
parents: 740
diff changeset
183
820
89985a1b3427 Clean up walk and changes code to use normalised names properly.
Bryan O'Sullivan <bos@serpentine.com>
parents: 814
diff changeset
184 def globprefix(pat):
89985a1b3427 Clean up walk and changes code to use normalised names properly.
Bryan O'Sullivan <bos@serpentine.com>
parents: 814
diff changeset
185 '''return the non-glob prefix of a path, e.g. foo/* -> foo'''
89985a1b3427 Clean up walk and changes code to use normalised names properly.
Bryan O'Sullivan <bos@serpentine.com>
parents: 814
diff changeset
186 root = []
89985a1b3427 Clean up walk and changes code to use normalised names properly.
Bryan O'Sullivan <bos@serpentine.com>
parents: 814
diff changeset
187 for p in pat.split(os.sep):
89985a1b3427 Clean up walk and changes code to use normalised names properly.
Bryan O'Sullivan <bos@serpentine.com>
parents: 814
diff changeset
188 if patkind(p)[0] == 'glob': break
89985a1b3427 Clean up walk and changes code to use normalised names properly.
Bryan O'Sullivan <bos@serpentine.com>
parents: 814
diff changeset
189 root.append(p)
886
509de8ab6f31 Fix walk path handling on Windows
Bryan O'Sullivan <bos@serpentine.com>
parents: 884
diff changeset
190 return '/'.join(root)
820
89985a1b3427 Clean up walk and changes code to use normalised names properly.
Bryan O'Sullivan <bos@serpentine.com>
parents: 814
diff changeset
191
870
a82eae840447 Teach walk code about absolute paths.
Bryan O'Sullivan <bos@serpentine.com>
parents: 869
diff changeset
192 pats = []
a82eae840447 Teach walk code about absolute paths.
Bryan O'Sullivan <bos@serpentine.com>
parents: 869
diff changeset
193 files = []
a82eae840447 Teach walk code about absolute paths.
Bryan O'Sullivan <bos@serpentine.com>
parents: 869
diff changeset
194 roots = []
a82eae840447 Teach walk code about absolute paths.
Bryan O'Sullivan <bos@serpentine.com>
parents: 869
diff changeset
195 for kind, name in map(patkind, names):
a82eae840447 Teach walk code about absolute paths.
Bryan O'Sullivan <bos@serpentine.com>
parents: 869
diff changeset
196 if kind in ('glob', 'relpath'):
1081
8b7d63489db3 Change canonpath to not know about repo objects
mpm@selenic.com
parents: 1075
diff changeset
197 name = canonpath(canonroot, cwd, name)
870
a82eae840447 Teach walk code about absolute paths.
Bryan O'Sullivan <bos@serpentine.com>
parents: 869
diff changeset
198 if name == '':
a82eae840447 Teach walk code about absolute paths.
Bryan O'Sullivan <bos@serpentine.com>
parents: 869
diff changeset
199 kind, name = 'glob', '**'
888
e7a943e8c52b Fix up handling of regexp paths.
Bryan O'Sullivan <bos@serpentine.com>
parents: 886
diff changeset
200 if kind in ('glob', 'path', 're'):
e7a943e8c52b Fix up handling of regexp paths.
Bryan O'Sullivan <bos@serpentine.com>
parents: 886
diff changeset
201 pats.append((kind, name))
870
a82eae840447 Teach walk code about absolute paths.
Bryan O'Sullivan <bos@serpentine.com>
parents: 869
diff changeset
202 if kind == 'glob':
a82eae840447 Teach walk code about absolute paths.
Bryan O'Sullivan <bos@serpentine.com>
parents: 869
diff changeset
203 root = globprefix(name)
a82eae840447 Teach walk code about absolute paths.
Bryan O'Sullivan <bos@serpentine.com>
parents: 869
diff changeset
204 if root: roots.append(root)
a82eae840447 Teach walk code about absolute paths.
Bryan O'Sullivan <bos@serpentine.com>
parents: 869
diff changeset
205 elif kind == 'relpath':
888
e7a943e8c52b Fix up handling of regexp paths.
Bryan O'Sullivan <bos@serpentine.com>
parents: 886
diff changeset
206 files.append((kind, name))
870
a82eae840447 Teach walk code about absolute paths.
Bryan O'Sullivan <bos@serpentine.com>
parents: 869
diff changeset
207 roots.append(name)
897
fe30f5434b51 Fix bug with empty inc and exc
mpm@selenic.com
parents: 896
diff changeset
208
820
89985a1b3427 Clean up walk and changes code to use normalised names properly.
Bryan O'Sullivan <bos@serpentine.com>
parents: 814
diff changeset
209 patmatch = matchfn(pats, '$') or always
89985a1b3427 Clean up walk and changes code to use normalised names properly.
Bryan O'Sullivan <bos@serpentine.com>
parents: 814
diff changeset
210 filematch = matchfn(files, '(?:/|$)') or always
897
fe30f5434b51 Fix bug with empty inc and exc
mpm@selenic.com
parents: 896
diff changeset
211 incmatch = always
fe30f5434b51 Fix bug with empty inc and exc
mpm@selenic.com
parents: 896
diff changeset
212 if inc:
fe30f5434b51 Fix bug with empty inc and exc
mpm@selenic.com
parents: 896
diff changeset
213 incmatch = matchfn(map(patkind, inc), '(?:/|$)')
fe30f5434b51 Fix bug with empty inc and exc
mpm@selenic.com
parents: 896
diff changeset
214 excmatch = lambda fn: False
fe30f5434b51 Fix bug with empty inc and exc
mpm@selenic.com
parents: 896
diff changeset
215 if exc:
fe30f5434b51 Fix bug with empty inc and exc
mpm@selenic.com
parents: 896
diff changeset
216 excmatch = matchfn(map(patkind, exc), '(?:/|$)')
742
092937de2ad7 Refactor matchpats and walk
mpm@selenic.com
parents: 740
diff changeset
217
1031
503aaf19a040 Rewrite log command. New version is faster and more featureful.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1015
diff changeset
218 return (roots,
503aaf19a040 Rewrite log command. New version is faster and more featureful.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1015
diff changeset
219 lambda fn: (incmatch(fn) and not excmatch(fn) and
503aaf19a040 Rewrite log command. New version is faster and more featureful.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1015
diff changeset
220 (fn.endswith('/') or
503aaf19a040 Rewrite log command. New version is faster and more featureful.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1015
diff changeset
221 (not pats and not files) or
503aaf19a040 Rewrite log command. New version is faster and more featureful.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1015
diff changeset
222 (pats and patmatch(fn)) or
503aaf19a040 Rewrite log command. New version is faster and more featureful.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1015
diff changeset
223 (files and filematch(fn)))),
503aaf19a040 Rewrite log command. New version is faster and more featureful.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1015
diff changeset
224 (inc or exc or (pats and pats != [('glob', '**')])) and True)
742
092937de2ad7 Refactor matchpats and walk
mpm@selenic.com
parents: 740
diff changeset
225
521
0fb8ade0f756 [PATCH] Fix use of util.CommandError
mpm@selenic.com
parents: 515
diff changeset
226 def system(cmd, errprefix=None):
508
42a660abaf75 [PATCH] Harden os.system
mpm@selenic.com
parents: 464
diff changeset
227 """execute a shell command that must succeed"""
42a660abaf75 [PATCH] Harden os.system
mpm@selenic.com
parents: 464
diff changeset
228 rc = os.system(cmd)
42a660abaf75 [PATCH] Harden os.system
mpm@selenic.com
parents: 464
diff changeset
229 if rc:
521
0fb8ade0f756 [PATCH] Fix use of util.CommandError
mpm@selenic.com
parents: 515
diff changeset
230 errmsg = "%s %s" % (os.path.basename(cmd.split(None, 1)[0]),
0fb8ade0f756 [PATCH] Fix use of util.CommandError
mpm@selenic.com
parents: 515
diff changeset
231 explain_exit(rc)[0])
0fb8ade0f756 [PATCH] Fix use of util.CommandError
mpm@selenic.com
parents: 515
diff changeset
232 if errprefix:
0fb8ade0f756 [PATCH] Fix use of util.CommandError
mpm@selenic.com
parents: 515
diff changeset
233 errmsg = "%s: %s" % (errprefix, errmsg)
870
a82eae840447 Teach walk code about absolute paths.
Bryan O'Sullivan <bos@serpentine.com>
parents: 869
diff changeset
234 raise Abort(errmsg)
508
42a660abaf75 [PATCH] Harden os.system
mpm@selenic.com
parents: 464
diff changeset
235
421
43b8da7420a9 [PATCH] rename under the other OS
mpm@selenic.com
parents: 419
diff changeset
236 def rename(src, dst):
1082
ce96e316278a Update util.py docstrings, fix walk test
mpm@selenic.com
parents: 1081
diff changeset
237 """forcibly rename a file"""
421
43b8da7420a9 [PATCH] rename under the other OS
mpm@selenic.com
parents: 419
diff changeset
238 try:
43b8da7420a9 [PATCH] rename under the other OS
mpm@selenic.com
parents: 419
diff changeset
239 os.rename(src, dst)
43b8da7420a9 [PATCH] rename under the other OS
mpm@selenic.com
parents: 419
diff changeset
240 except:
43b8da7420a9 [PATCH] rename under the other OS
mpm@selenic.com
parents: 419
diff changeset
241 os.unlink(dst)
43b8da7420a9 [PATCH] rename under the other OS
mpm@selenic.com
parents: 419
diff changeset
242 os.rename(src, dst)
43b8da7420a9 [PATCH] rename under the other OS
mpm@selenic.com
parents: 419
diff changeset
243
1241
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
244 def copyfiles(src, dst, hardlink=None):
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
245 """Copy a directory tree using hardlinks if possible"""
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
246
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
247 if hardlink is None:
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
248 hardlink = (os.stat(src).st_dev ==
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
249 os.stat(os.path.dirname(dst)).st_dev)
698
df78d8ccac4c Use python function instead of external 'cp' command when cloning repos.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 667
diff changeset
250
1207
a7b8812973d9 Rewrite copytree as copyfiles
mpm@selenic.com
parents: 1200
diff changeset
251 if os.path.isdir(src):
a7b8812973d9 Rewrite copytree as copyfiles
mpm@selenic.com
parents: 1200
diff changeset
252 os.mkdir(dst)
a7b8812973d9 Rewrite copytree as copyfiles
mpm@selenic.com
parents: 1200
diff changeset
253 for name in os.listdir(src):
a7b8812973d9 Rewrite copytree as copyfiles
mpm@selenic.com
parents: 1200
diff changeset
254 srcname = os.path.join(src, name)
a7b8812973d9 Rewrite copytree as copyfiles
mpm@selenic.com
parents: 1200
diff changeset
255 dstname = os.path.join(dst, name)
1241
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
256 copyfiles(srcname, dstname, hardlink)
1207
a7b8812973d9 Rewrite copytree as copyfiles
mpm@selenic.com
parents: 1200
diff changeset
257 else:
1241
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
258 if hardlink:
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
259 try:
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
260 os_link(src, dst)
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
261 except:
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
262 hardlink = False
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
263 shutil.copy2(src, dst)
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
264 else:
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
265 shutil.copy2(src, dst)
698
df78d8ccac4c Use python function instead of external 'cp' command when cloning repos.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 667
diff changeset
266
1090
1bca39b85615 Move opener to utils
mpm@selenic.com
parents: 1082
diff changeset
267 def opener(base):
1bca39b85615 Move opener to utils
mpm@selenic.com
parents: 1082
diff changeset
268 """
1bca39b85615 Move opener to utils
mpm@selenic.com
parents: 1082
diff changeset
269 return a function that opens files relative to base
1bca39b85615 Move opener to utils
mpm@selenic.com
parents: 1082
diff changeset
270
1bca39b85615 Move opener to utils
mpm@selenic.com
parents: 1082
diff changeset
271 this function is used to hide the details of COW semantics and
1bca39b85615 Move opener to utils
mpm@selenic.com
parents: 1082
diff changeset
272 remote file access from higher level code.
1bca39b85615 Move opener to utils
mpm@selenic.com
parents: 1082
diff changeset
273 """
1bca39b85615 Move opener to utils
mpm@selenic.com
parents: 1082
diff changeset
274 p = base
1bca39b85615 Move opener to utils
mpm@selenic.com
parents: 1082
diff changeset
275 def o(path, mode="r"):
1bca39b85615 Move opener to utils
mpm@selenic.com
parents: 1082
diff changeset
276 f = os.path.join(p, path)
1bca39b85615 Move opener to utils
mpm@selenic.com
parents: 1082
diff changeset
277
1bca39b85615 Move opener to utils
mpm@selenic.com
parents: 1082
diff changeset
278 mode += "b" # for that other OS
1bca39b85615 Move opener to utils
mpm@selenic.com
parents: 1082
diff changeset
279
1bca39b85615 Move opener to utils
mpm@selenic.com
parents: 1082
diff changeset
280 if mode[0] != "r":
1bca39b85615 Move opener to utils
mpm@selenic.com
parents: 1082
diff changeset
281 try:
1241
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
282 nlink = nlinks(f)
1090
1bca39b85615 Move opener to utils
mpm@selenic.com
parents: 1082
diff changeset
283 except OSError:
1bca39b85615 Move opener to utils
mpm@selenic.com
parents: 1082
diff changeset
284 d = os.path.dirname(f)
1bca39b85615 Move opener to utils
mpm@selenic.com
parents: 1082
diff changeset
285 if not os.path.isdir(d):
1bca39b85615 Move opener to utils
mpm@selenic.com
parents: 1082
diff changeset
286 os.makedirs(d)
1bca39b85615 Move opener to utils
mpm@selenic.com
parents: 1082
diff changeset
287 else:
1241
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
288 if nlink > 1:
1090
1bca39b85615 Move opener to utils
mpm@selenic.com
parents: 1082
diff changeset
289 file(f + ".tmp", "wb").write(file(f, "rb").read())
1bca39b85615 Move opener to utils
mpm@selenic.com
parents: 1082
diff changeset
290 rename(f+".tmp", f)
1bca39b85615 Move opener to utils
mpm@selenic.com
parents: 1082
diff changeset
291
1bca39b85615 Move opener to utils
mpm@selenic.com
parents: 1082
diff changeset
292 return file(f, mode)
1bca39b85615 Move opener to utils
mpm@selenic.com
parents: 1082
diff changeset
293
1bca39b85615 Move opener to utils
mpm@selenic.com
parents: 1082
diff changeset
294 return o
1bca39b85615 Move opener to utils
mpm@selenic.com
parents: 1082
diff changeset
295
704
5ca319a641e1 Make makelock and readlock work on filesystems without symlink support.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 698
diff changeset
296 def _makelock_file(info, pathname):
5ca319a641e1 Make makelock and readlock work on filesystems without symlink support.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 698
diff changeset
297 ld = os.open(pathname, os.O_CREAT | os.O_WRONLY | os.O_EXCL)
5ca319a641e1 Make makelock and readlock work on filesystems without symlink support.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 698
diff changeset
298 os.write(ld, info)
5ca319a641e1 Make makelock and readlock work on filesystems without symlink support.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 698
diff changeset
299 os.close(ld)
5ca319a641e1 Make makelock and readlock work on filesystems without symlink support.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 698
diff changeset
300
5ca319a641e1 Make makelock and readlock work on filesystems without symlink support.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 698
diff changeset
301 def _readlock_file(pathname):
5ca319a641e1 Make makelock and readlock work on filesystems without symlink support.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 698
diff changeset
302 return file(pathname).read()
5ca319a641e1 Make makelock and readlock work on filesystems without symlink support.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 698
diff changeset
303
1241
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
304 def nlinks(pathname):
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
305 """Return number of hardlinks for the given file."""
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
306 return os.stat(pathname).st_nlink
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
307
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
308 if hasattr(os, 'link'):
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
309 os_link = os.link
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
310 else:
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
311 def os_link(src, dst):
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
312 raise OSError(0, "Hardlinks not supported")
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
313
1082
ce96e316278a Update util.py docstrings, fix walk test
mpm@selenic.com
parents: 1081
diff changeset
314 # Platform specific variants
419
28511fc21073 [PATCH] file seperator handling for the other 'OS'
mpm@selenic.com
parents:
diff changeset
315 if os.name == 'nt':
461
9ae0034f2772 [PATCH] /dev/null for other OS
mpm@selenic.com
parents: 441
diff changeset
316 nulldev = 'NUL:'
9ae0034f2772 [PATCH] /dev/null for other OS
mpm@selenic.com
parents: 441
diff changeset
317
1241
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
318 try: # ActivePython can create hard links using win32file module
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
319 import win32file
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
320
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
321 def os_link(src, dst): # NB will only succeed on NTFS
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
322 win32file.CreateHardLink(dst, src)
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
323
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
324 def nlinks(pathname):
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
325 """Return number of hardlinks for the given file."""
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
326 try:
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
327 fh = win32file.CreateFile(pathname,
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
328 win32file.GENERIC_READ, win32file.FILE_SHARE_READ,
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
329 None, win32file.OPEN_EXISTING, 0, None)
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
330 res = win32file.GetFileInformationByHandle(fh)
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
331 fh.Close()
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
332 return res[7]
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
333 except:
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
334 return os.stat(pathname).st_nlink
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
335
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
336 except ImportError:
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
337 pass
3b4f05ff3130 Add support for cloning with hardlinks on windows.
Stephen Darnell
parents: 1207
diff changeset
338
441
e8af362cfb01 Permission handling for the other OS
mpm@selenic.com
parents: 422
diff changeset
339 def is_exec(f, last):
e8af362cfb01 Permission handling for the other OS
mpm@selenic.com
parents: 422
diff changeset
340 return last
e8af362cfb01 Permission handling for the other OS
mpm@selenic.com
parents: 422
diff changeset
341
e8af362cfb01 Permission handling for the other OS
mpm@selenic.com
parents: 422
diff changeset
342 def set_exec(f, mode):
e8af362cfb01 Permission handling for the other OS
mpm@selenic.com
parents: 422
diff changeset
343 pass
515
03f27b1381f9 Whitespace cleanups
mpm@selenic.com
parents: 508
diff changeset
344
419
28511fc21073 [PATCH] file seperator handling for the other 'OS'
mpm@selenic.com
parents:
diff changeset
345 def pconvert(path):
28511fc21073 [PATCH] file seperator handling for the other 'OS'
mpm@selenic.com
parents:
diff changeset
346 return path.replace("\\", "/")
422
10c43444a38e [PATCH] Enables lock work under the other 'OS'
mpm@selenic.com
parents: 421
diff changeset
347
886
509de8ab6f31 Fix walk path handling on Windows
Bryan O'Sullivan <bos@serpentine.com>
parents: 884
diff changeset
348 def localpath(path):
509de8ab6f31 Fix walk path handling on Windows
Bryan O'Sullivan <bos@serpentine.com>
parents: 884
diff changeset
349 return path.replace('/', '\\')
509de8ab6f31 Fix walk path handling on Windows
Bryan O'Sullivan <bos@serpentine.com>
parents: 884
diff changeset
350
509de8ab6f31 Fix walk path handling on Windows
Bryan O'Sullivan <bos@serpentine.com>
parents: 884
diff changeset
351 def normpath(path):
509de8ab6f31 Fix walk path handling on Windows
Bryan O'Sullivan <bos@serpentine.com>
parents: 884
diff changeset
352 return pconvert(os.path.normpath(path))
509de8ab6f31 Fix walk path handling on Windows
Bryan O'Sullivan <bos@serpentine.com>
parents: 884
diff changeset
353
704
5ca319a641e1 Make makelock and readlock work on filesystems without symlink support.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 698
diff changeset
354 makelock = _makelock_file
5ca319a641e1 Make makelock and readlock work on filesystems without symlink support.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 698
diff changeset
355 readlock = _readlock_file
461
9ae0034f2772 [PATCH] /dev/null for other OS
mpm@selenic.com
parents: 441
diff changeset
356
782
cdb9e95b2fab Provided platform dependent implementations for explain_exit
thananck@yahoo.com
parents: 742
diff changeset
357 def explain_exit(code):
cdb9e95b2fab Provided platform dependent implementations for explain_exit
thananck@yahoo.com
parents: 742
diff changeset
358 return "exited with status %d" % code, code
cdb9e95b2fab Provided platform dependent implementations for explain_exit
thananck@yahoo.com
parents: 742
diff changeset
359
419
28511fc21073 [PATCH] file seperator handling for the other 'OS'
mpm@selenic.com
parents:
diff changeset
360 else:
461
9ae0034f2772 [PATCH] /dev/null for other OS
mpm@selenic.com
parents: 441
diff changeset
361 nulldev = '/dev/null'
9ae0034f2772 [PATCH] /dev/null for other OS
mpm@selenic.com
parents: 441
diff changeset
362
441
e8af362cfb01 Permission handling for the other OS
mpm@selenic.com
parents: 422
diff changeset
363 def is_exec(f, last):
1082
ce96e316278a Update util.py docstrings, fix walk test
mpm@selenic.com
parents: 1081
diff changeset
364 """check whether a file is executable"""
441
e8af362cfb01 Permission handling for the other OS
mpm@selenic.com
parents: 422
diff changeset
365 return (os.stat(f).st_mode & 0100 != 0)
e8af362cfb01 Permission handling for the other OS
mpm@selenic.com
parents: 422
diff changeset
366
e8af362cfb01 Permission handling for the other OS
mpm@selenic.com
parents: 422
diff changeset
367 def set_exec(f, mode):
e8af362cfb01 Permission handling for the other OS
mpm@selenic.com
parents: 422
diff changeset
368 s = os.stat(f).st_mode
e8af362cfb01 Permission handling for the other OS
mpm@selenic.com
parents: 422
diff changeset
369 if (s & 0100 != 0) == mode:
e8af362cfb01 Permission handling for the other OS
mpm@selenic.com
parents: 422
diff changeset
370 return
e8af362cfb01 Permission handling for the other OS
mpm@selenic.com
parents: 422
diff changeset
371 if mode:
e8af362cfb01 Permission handling for the other OS
mpm@selenic.com
parents: 422
diff changeset
372 # Turn on +x for every +r bit when making a file executable
e8af362cfb01 Permission handling for the other OS
mpm@selenic.com
parents: 422
diff changeset
373 # and obey umask.
e8af362cfb01 Permission handling for the other OS
mpm@selenic.com
parents: 422
diff changeset
374 umask = os.umask(0)
e8af362cfb01 Permission handling for the other OS
mpm@selenic.com
parents: 422
diff changeset
375 os.umask(umask)
e8af362cfb01 Permission handling for the other OS
mpm@selenic.com
parents: 422
diff changeset
376 os.chmod(f, s | (s & 0444) >> 2 & ~umask)
e8af362cfb01 Permission handling for the other OS
mpm@selenic.com
parents: 422
diff changeset
377 else:
e8af362cfb01 Permission handling for the other OS
mpm@selenic.com
parents: 422
diff changeset
378 os.chmod(f, s & 0666)
e8af362cfb01 Permission handling for the other OS
mpm@selenic.com
parents: 422
diff changeset
379
419
28511fc21073 [PATCH] file seperator handling for the other 'OS'
mpm@selenic.com
parents:
diff changeset
380 def pconvert(path):
28511fc21073 [PATCH] file seperator handling for the other 'OS'
mpm@selenic.com
parents:
diff changeset
381 return path
28511fc21073 [PATCH] file seperator handling for the other 'OS'
mpm@selenic.com
parents:
diff changeset
382
886
509de8ab6f31 Fix walk path handling on Windows
Bryan O'Sullivan <bos@serpentine.com>
parents: 884
diff changeset
383 def localpath(path):
509de8ab6f31 Fix walk path handling on Windows
Bryan O'Sullivan <bos@serpentine.com>
parents: 884
diff changeset
384 return path
509de8ab6f31 Fix walk path handling on Windows
Bryan O'Sullivan <bos@serpentine.com>
parents: 884
diff changeset
385
509de8ab6f31 Fix walk path handling on Windows
Bryan O'Sullivan <bos@serpentine.com>
parents: 884
diff changeset
386 normpath = os.path.normpath
509de8ab6f31 Fix walk path handling on Windows
Bryan O'Sullivan <bos@serpentine.com>
parents: 884
diff changeset
387
422
10c43444a38e [PATCH] Enables lock work under the other 'OS'
mpm@selenic.com
parents: 421
diff changeset
388 def makelock(info, pathname):
704
5ca319a641e1 Make makelock and readlock work on filesystems without symlink support.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 698
diff changeset
389 try:
5ca319a641e1 Make makelock and readlock work on filesystems without symlink support.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 698
diff changeset
390 os.symlink(info, pathname)
5ca319a641e1 Make makelock and readlock work on filesystems without symlink support.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 698
diff changeset
391 except OSError, why:
5ca319a641e1 Make makelock and readlock work on filesystems without symlink support.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 698
diff changeset
392 if why.errno == errno.EEXIST:
5ca319a641e1 Make makelock and readlock work on filesystems without symlink support.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 698
diff changeset
393 raise
5ca319a641e1 Make makelock and readlock work on filesystems without symlink support.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 698
diff changeset
394 else:
5ca319a641e1 Make makelock and readlock work on filesystems without symlink support.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 698
diff changeset
395 _makelock_file(info, pathname)
422
10c43444a38e [PATCH] Enables lock work under the other 'OS'
mpm@selenic.com
parents: 421
diff changeset
396
10c43444a38e [PATCH] Enables lock work under the other 'OS'
mpm@selenic.com
parents: 421
diff changeset
397 def readlock(pathname):
704
5ca319a641e1 Make makelock and readlock work on filesystems without symlink support.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 698
diff changeset
398 try:
5ca319a641e1 Make makelock and readlock work on filesystems without symlink support.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 698
diff changeset
399 return os.readlink(pathname)
5ca319a641e1 Make makelock and readlock work on filesystems without symlink support.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 698
diff changeset
400 except OSError, why:
5ca319a641e1 Make makelock and readlock work on filesystems without symlink support.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 698
diff changeset
401 if why.errno == errno.EINVAL:
5ca319a641e1 Make makelock and readlock work on filesystems without symlink support.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 698
diff changeset
402 return _readlock_file(pathname)
5ca319a641e1 Make makelock and readlock work on filesystems without symlink support.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 698
diff changeset
403 else:
5ca319a641e1 Make makelock and readlock work on filesystems without symlink support.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 698
diff changeset
404 raise
782
cdb9e95b2fab Provided platform dependent implementations for explain_exit
thananck@yahoo.com
parents: 742
diff changeset
405
cdb9e95b2fab Provided platform dependent implementations for explain_exit
thananck@yahoo.com
parents: 742
diff changeset
406 def explain_exit(code):
cdb9e95b2fab Provided platform dependent implementations for explain_exit
thananck@yahoo.com
parents: 742
diff changeset
407 """return a 2-tuple (desc, code) describing a process's status"""
cdb9e95b2fab Provided platform dependent implementations for explain_exit
thananck@yahoo.com
parents: 742
diff changeset
408 if os.WIFEXITED(code):
cdb9e95b2fab Provided platform dependent implementations for explain_exit
thananck@yahoo.com
parents: 742
diff changeset
409 val = os.WEXITSTATUS(code)
cdb9e95b2fab Provided platform dependent implementations for explain_exit
thananck@yahoo.com
parents: 742
diff changeset
410 return "exited with status %d" % val, val
cdb9e95b2fab Provided platform dependent implementations for explain_exit
thananck@yahoo.com
parents: 742
diff changeset
411 elif os.WIFSIGNALED(code):
cdb9e95b2fab Provided platform dependent implementations for explain_exit
thananck@yahoo.com
parents: 742
diff changeset
412 val = os.WTERMSIG(code)
cdb9e95b2fab Provided platform dependent implementations for explain_exit
thananck@yahoo.com
parents: 742
diff changeset
413 return "killed by signal %d" % val, val
cdb9e95b2fab Provided platform dependent implementations for explain_exit
thananck@yahoo.com
parents: 742
diff changeset
414 elif os.WIFSTOPPED(code):
912
302f83b85054 Minor tweak: os.STOPSIG -> os.WSTOPSIG. Pychecker spotted this one.
mark.williamson@cl.cam.ac.uk
parents: 897
diff changeset
415 val = os.WSTOPSIG(code)
782
cdb9e95b2fab Provided platform dependent implementations for explain_exit
thananck@yahoo.com
parents: 742
diff changeset
416 return "stopped by signal %d" % val, val
cdb9e95b2fab Provided platform dependent implementations for explain_exit
thananck@yahoo.com
parents: 742
diff changeset
417 raise ValueError("invalid exit code")
1199
78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an
Eric Hopper <hopper@omnifarious.org>
parents: 1169
diff changeset
418
78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an
Eric Hopper <hopper@omnifarious.org>
parents: 1169
diff changeset
419 class chunkbuffer(object):
78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an
Eric Hopper <hopper@omnifarious.org>
parents: 1169
diff changeset
420 """Allow arbitrary sized chunks of data to be efficiently read from an
78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an
Eric Hopper <hopper@omnifarious.org>
parents: 1169
diff changeset
421 iterator over chunks of arbitrary size."""
1200
333de1d53846 Minor cleanups.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1199
diff changeset
422
1199
78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an
Eric Hopper <hopper@omnifarious.org>
parents: 1169
diff changeset
423 def __init__(self, in_iter, targetsize = 2**16):
78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an
Eric Hopper <hopper@omnifarious.org>
parents: 1169
diff changeset
424 """in_iter is the iterator that's iterating over the input chunks.
78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an
Eric Hopper <hopper@omnifarious.org>
parents: 1169
diff changeset
425 targetsize is how big a buffer to try to maintain."""
78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an
Eric Hopper <hopper@omnifarious.org>
parents: 1169
diff changeset
426 self.in_iter = iter(in_iter)
78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an
Eric Hopper <hopper@omnifarious.org>
parents: 1169
diff changeset
427 self.buf = ''
78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an
Eric Hopper <hopper@omnifarious.org>
parents: 1169
diff changeset
428 self.targetsize = int(targetsize)
1200
333de1d53846 Minor cleanups.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1199
diff changeset
429 if self.targetsize <= 0:
333de1d53846 Minor cleanups.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1199
diff changeset
430 raise ValueError("targetsize must be greater than 0, was %d" %
333de1d53846 Minor cleanups.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1199
diff changeset
431 targetsize)
1199
78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an
Eric Hopper <hopper@omnifarious.org>
parents: 1169
diff changeset
432 self.iterempty = False
1200
333de1d53846 Minor cleanups.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1199
diff changeset
433
1199
78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an
Eric Hopper <hopper@omnifarious.org>
parents: 1169
diff changeset
434 def fillbuf(self):
1200
333de1d53846 Minor cleanups.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1199
diff changeset
435 """Ignore target size; read every chunk from iterator until empty."""
1199
78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an
Eric Hopper <hopper@omnifarious.org>
parents: 1169
diff changeset
436 if not self.iterempty:
78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an
Eric Hopper <hopper@omnifarious.org>
parents: 1169
diff changeset
437 collector = cStringIO.StringIO()
78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an
Eric Hopper <hopper@omnifarious.org>
parents: 1169
diff changeset
438 collector.write(self.buf)
78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an
Eric Hopper <hopper@omnifarious.org>
parents: 1169
diff changeset
439 for ch in self.in_iter:
78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an
Eric Hopper <hopper@omnifarious.org>
parents: 1169
diff changeset
440 collector.write(ch)
78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an
Eric Hopper <hopper@omnifarious.org>
parents: 1169
diff changeset
441 self.buf = collector.getvalue()
78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an
Eric Hopper <hopper@omnifarious.org>
parents: 1169
diff changeset
442 self.iterempty = True
78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an
Eric Hopper <hopper@omnifarious.org>
parents: 1169
diff changeset
443
78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an
Eric Hopper <hopper@omnifarious.org>
parents: 1169
diff changeset
444 def read(self, l):
1200
333de1d53846 Minor cleanups.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1199
diff changeset
445 """Read L bytes of data from the iterator of chunks of data.
333de1d53846 Minor cleanups.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1199
diff changeset
446 Returns less than L bytes if the iterator runs dry."""
1199
78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an
Eric Hopper <hopper@omnifarious.org>
parents: 1169
diff changeset
447 if l > len(self.buf) and not self.iterempty:
78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an
Eric Hopper <hopper@omnifarious.org>
parents: 1169
diff changeset
448 # Clamp to a multiple of self.targetsize
78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an
Eric Hopper <hopper@omnifarious.org>
parents: 1169
diff changeset
449 targetsize = self.targetsize * ((l // self.targetsize) + 1)
78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an
Eric Hopper <hopper@omnifarious.org>
parents: 1169
diff changeset
450 collector = cStringIO.StringIO()
78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an
Eric Hopper <hopper@omnifarious.org>
parents: 1169
diff changeset
451 collector.write(self.buf)
78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an
Eric Hopper <hopper@omnifarious.org>
parents: 1169
diff changeset
452 collected = len(self.buf)
78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an
Eric Hopper <hopper@omnifarious.org>
parents: 1169
diff changeset
453 for chunk in self.in_iter:
78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an
Eric Hopper <hopper@omnifarious.org>
parents: 1169
diff changeset
454 collector.write(chunk)
78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an
Eric Hopper <hopper@omnifarious.org>
parents: 1169
diff changeset
455 collected += len(chunk)
78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an
Eric Hopper <hopper@omnifarious.org>
parents: 1169
diff changeset
456 if collected >= targetsize:
78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an
Eric Hopper <hopper@omnifarious.org>
parents: 1169
diff changeset
457 break
78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an
Eric Hopper <hopper@omnifarious.org>
parents: 1169
diff changeset
458 if collected < targetsize:
78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an
Eric Hopper <hopper@omnifarious.org>
parents: 1169
diff changeset
459 self.iterempty = True
78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an
Eric Hopper <hopper@omnifarious.org>
parents: 1169
diff changeset
460 self.buf = collector.getvalue()
1200
333de1d53846 Minor cleanups.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1199
diff changeset
461 s, self.buf = self.buf[:l], buffer(self.buf, l)
1199
78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an
Eric Hopper <hopper@omnifarious.org>
parents: 1169
diff changeset
462 return s
78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an
Eric Hopper <hopper@omnifarious.org>
parents: 1169
diff changeset
463
78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an
Eric Hopper <hopper@omnifarious.org>
parents: 1169
diff changeset
464 def filechunkiter(f, size = 65536):
1200
333de1d53846 Minor cleanups.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1199
diff changeset
465 """Create a generator that produces all the data in the file size
333de1d53846 Minor cleanups.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1199
diff changeset
466 (default 65536) bytes at a time. Chunks may be less than size
333de1d53846 Minor cleanups.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1199
diff changeset
467 bytes if the chunk is the last chunk in the file, or the file is a
333de1d53846 Minor cleanups.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1199
diff changeset
468 socket or some other type of file that sometimes reads less data
333de1d53846 Minor cleanups.
Bryan O'Sullivan <bos@serpentine.com>
parents: 1199
diff changeset
469 than is requested."""
1199
78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an
Eric Hopper <hopper@omnifarious.org>
parents: 1169
diff changeset
470 s = f.read(size)
78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an
Eric Hopper <hopper@omnifarious.org>
parents: 1169
diff changeset
471 while len(s) >= 0:
78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an
Eric Hopper <hopper@omnifarious.org>
parents: 1169
diff changeset
472 yield s
78ceaf83f28f Created a class in util called chunkbuffer that buffers reads from an
Eric Hopper <hopper@omnifarious.org>
parents: 1169
diff changeset
473 s = f.read(size)